Should Algorithms Decide?

14 August 2018

Our topic this week, the ethics and morality of algorithms. Strictly speaking, an algorithm is not a moral agent. It’s just a step-by-step, foolproof, mechanical procedure for computing some mathematical function. Think, for example, of long division as a case in point. From this perspective it may seem a bit like a category mistake or contradiction in terms to talk about the ethics or morality of algorithms.

But what’s really relevant isn’t the strict mathematical notion of an algorithm, but a broader notion of an algorithm, which is roughly coextensive to whatever it is that computers do. And as it happens, we are delegating more and more morally fraught decisions to computers and their algorithms. In the strict sense of that term ‘algorithm’ there is no algorithm that would allow us to precisely compute the value of a human life in a mechanical, step-by-step, foolproof manner.

But that doesn’t stop us from programming a computer to assigned weights to various factors like say age or income or race, health status, performing some calculations, and spitting out a number that tell us whether we ought or ought not give a person a potentially life-saving treatment. Many find the prospect of such a thing truly alarming. It’s hard to blame them for that. After all, how many of us would be willing to trust our own lives to a computer algorithm?

But in fact, we do so all the time—every time we fly on an airplane, for example. The air traffic control system is basically the domain of the computer. It’s a domain in which, for example, human pilots are called upon to act only on an as needed basis. And thanks to the wonders of modern computation, human intervention is, in fact, rarely needed. And the remarkable thing is that air travel is much safer than it otherwise would be as a result.

现在有人可能会反对说空中交通管制是一个相对容易的例子。基本上,计算机所要做的就是让两架飞机保持足够的距离,让它们从A点到B点,而不会相互碰撞。但我们正在接近算法决策的新前沿,这比空中交通管制的计算难度和道德担忧都要大得多。想想即将到来的自动驾驶汽车的冲击。在那些拥挤的城市街道上,汽车、行人和骑自行车的人来往的距离比飞机要近得多。我们真的想让电脑来决定一辆汽车什么时候应该转向,杀死乘客来拯救行人吗?

Unfortunately, I doubt that it matters what any of us want. The day is coming, and fast, when the degree of computer automation in what we might call the ground traffic control system will rival or exceed the degree of automation in the air traffic control system. Some will say that the day is coming much too fast and in too many spheres. Computers are already in almost complete control of the stock market. They’re gradually taking over medical diagnosis. Some even want to turn sentencing decisions over to them. Perhaps things are getting out of control.

But let’s not forget that humans aren’t exactly infallible decision makers ourselves. Think of the mess that our human centered sentencing system has become. Given the mess that judges have made, why should we trust them over a well programmed computer? I grant that human judges have something that computers lack. Judges are living, breathing, human beings, with a sense of duty, and responsibility, and empathy. But those very same judges can also be full of racial biases, hidden political agendas, and overblown emotional reactions. With a computer, we just encode the sentencing guidelines specified by the law into your algorithm and let the computer decide without fear or favoritism.

如果真的有那么简单就好了。不幸的是,事实并非如此。在我们开发出完全自主、完全自主编程的计算机之前,我们一直被人类所束缚,被我们的种族偏见和隐藏的议程所束缚,去做大量的编程。尽管一些程序员可能认为自己是年轻的斯波克——全是逻辑,没有感情——但最终他们也是人。不幸的是,他们和我们其他人一样容易产生偏见和盲目。我们也不能轻易地训练他们,让他们避免将自己的偏见写入算法。大多数人甚至没有意识到自己的偏见。

In the old days, the days of so-called good-old fashioned AI, this might not have been such a big deal. Even if we couldn’t eliminate the biases from the programmers, we could still test, debug and tweak their programs. By contrast, you can’t rewrite a judge’s neural code when you discover he’s got this thing against black people. If you think about it that way, who wouldn’t still take the computer over the judge any day?

不幸的是,老式的AI编程(GOFAI)——那种你必须明确地编写每一行代码,以便通过“蛮力”将类似人类的知识塞进计算机的编程方式——正在迅速成为过去。在许多领域,GOFAI正在迅速让位于机器学习算法。有了这种计算架构,你就不用把知识一行一行地塞进计算机里,你基本上给计算机一个问题,让它自己想办法解决问题。特别是,你给机器一堆训练数据,它会尝试给出正确答案。如果它得到了错误的答案,您(或整个世界)就会给它一个错误信号作为回应。然后网络会自发地调整它的网络,基本上不需要人工干预,直到它对训练集中的所有数据都得到正确的答案。然后我们把它放到世界上,去面对这个问题类别的全新实例,而不是在原来的训练集中。这是一种美丽而有力的技术。

但现在假设你有一些硅谷的科技兄弟在训练一台机器去做,比如说,人脸识别。也许他们会选择一群朋友作为训练集。我们可以相当肯定的是,训练集不能代表总体!这意味着,至少,如果我们不想在问题域的网络表示中引入偏见,我们必须确保使用统计上合理的方法来设计我们的训练集。但正如我们在这一集讨论的那样,说起来容易做起来难,至少在一般情况下是这样。这是因为我们能得到的唯一合理的数据,例如,量刑决定的案例,可能充满了历史偏见的影响。

But there’s an even harder problem. These networks can sometimes be inscrutable black boxes. That’s because the network basically decides on its own how to partition and represent the data and what weights to assign to what factors. And these “decisions” may be totally opaque to their human “teachers.” That means that if something goes wrong, we can’t even get in there and debug and tweak the network, as with old-fashioned AI. At least with those systems, we knew exactly what the algorithm was supposed to be doing.

Now I don’t want to sound like a luddite. I recognize the decided upsides of moving beyond human decision to automated decision making. And even though I still have a soft spot for old-style knowledge representation from the heyday of GOFAI, I appreciate the amazing success of newfangled machine learning architectures. Still, I’m not really in a hurry to farm out too much of our moral agency to machines. I think before rushing pell-mell into the breach, we need to slow down and think this through much more systematically. Perhaps you can help us out.

Comments(5)


Michael Haddon's picture

Michael Haddon

Wednesday, August 15, 2018 -- 9:50 PM

During your presentation on

During your presentation on the Ethics of Algorithms, reference was made to the difference in arrests for drugs between blacks and whites, while rates of drug use are approximately equal. This was claimed to demonstrate a bias in the way drug arrests occur. Having recently read Thomas Sowell’s Discrimination and Disparities, I recalled purchasing drugs in my youth. Most purchases were made in someone’s house, or perhaps in their parent’s garage. ‘Someone,’ in this case being a white dealer. I never heard of a white dealer selling on the streets. Black drug dealers on the sidewalks were at the time rather obvious in certain parts of Oakland, where I grew up. I presume you’re much more likely to get arrested selling on the street than in your parent’s garage. (The statute of limitations has long since run on my youthful indiscretions.)

I’m not sure y’all talk to the folks at the Hoover Institution, but a presentation regarding how you determine bias with Thomas Sowell as a guest would certainly be lively.

Beloved'lil'king's picture

Beloved'lil'king

Monday, August 20, 2018 -- 11:05 AM

The greatest concern I have

The greatest concern I have regarding the implementation of algorithms or algorithmic mechanisms to expedite the current criminal justice system (CJS) is for at least two parts:
a . 1(1)和-1(-1):无论你构建的任何算法多么出色,或者多么不受影响,你都可能构建一个工具来使用完美无缺的算法,甚至使用一些有意的双盲机制来提供不受限制的约束;even in that, there are always a minimum of two separate variables required to compute anything:
1. Therefore, even if for this example, the social wisdom or intellectual value of the algorithm was 100,000,000,000 (100 Billion) times the value of anything it judged/digested to social interpretation to achieve a better more balanced CJS; the ultimate end equals solution will be also and equally contingent upon then second variable, that is the thing being judged/examined.
2. Because the notion of the algorithm in construct is to alleviate the burden upon CJS to expedite ‘more’ or somehow ‘better’ justice upon the people(s), whatever the concern in question to be judged, the total second variable value is not an ‘absolute value’ because people are always hurting out of hurt and criminalized by crimes so the second variable is essentially negative.
3. While the algorithm may not see bias in the way people see bias, because of the inherently flawed dataset where all people will develop at slightly different rates and their experiences, even those of equal social force/value, will cause varying impressions which may cause differing effects; any algorithmic mechanisms applied will observe and apply a simple value to each circumstance without the understanding to know which evolutionary processes have or have not been impressioned upon.
4. Talent is fashioned to a world lottery system but opportunity is always cultivated by the more just society. For this reason, people selected from a group of peers to offer reasoned and compassionate verdict to then be further examined and sentence/justice applied is the best solution for equitable fair-minded CJS.
5. Variable 1 cannot be infinite fashioned from the metaphysical realm despite near-infinite capability. Variable 2 always drawn from a broken/negative/selfish/criminal world.
B. The current burden of CJS was cultivated by racial and discriminatory prejudice(s) e.g. “redlining” and/or lack of social mobility for all peoples/creeds/nations/cultures being under the jurisdiction of the CJS and moreover this discrimination has been taught and/or instilled by the very educational institutions established to protect and perpetuate a notion of free-indiscriminate CJS. Turning to/utilization of any algorithmic mechanism or machined learning to expedite the solution is as though cheating on a test or turning to the cat in the hat for some magical non-logical solution rather that apologizing and putting in the more difficult work of social-equality for all under the jurisdictional umbrella. Analogous to taking hydroxy-cut instead of practiced exercise and then wondering why after it seemed to be working your heart suddenly fails. Trial/Temptation to Perseverance to Proven Character to Hope. Everyone needs Hope.

Beloved'lil'king's picture

Beloved'lil'king

Tuesday, August 21, 2018 -- 8:03 AM

IF, bent upon the application

IF, bent upon the application of algorithms to vacate personal responsibility: You must first account for;
A. Physiologically dormant hibernation disorder(s) and the varying rates under which those disorders are prompted for expression based upon measured social interaction.
B. Generationally dormant psychologically dormant disorder(s) which may only be expressed or ‘activated’ every other or third or sometimes even more subtle down to the fourth generation and be able to quantify their correlated intermittent social interactions.
C. Determining social values and implementation of the attributed or ‘perfected’ person to become cannot be referenced by any singularity or steady control because with each passing generation the history, carved in battle and selfish ambition usually almost always, reset(s) and complicates the variables in question by manipulating the control through the progression of time. In other words, no one society ever lasts long enough to establish a proper control to identify the ‘perfected’ person.
D. No one generation is ever ‘perfect’, not a single one. There is no ah-ha and now all past history is irrelevant. Every culture/tongue/creed/people(s) have value. The perfect was created and then because of selfish illogical behavior was marred and defaced beyond recognition from what purity of blessed character was originally established by your Maker. For this reason, what appears as petty or under the guise of pride to be wasteful by some, that is to Love all people indiscriminately without measure and help one another in loving-kindness and compassionate support; this is actually the accurate metric for reestablishing the foundation of humanity by imitating Jesus, who is the Christ, Son of God and Son of Man. He, Jesus, brought the essential road map and compass down to teach those, both male and female, He Loved how to restore the Joy of Salvation and Peace inexplainable which overflows from a life Honoring the Love of God.
E. Virtue: The nature of all virtue is divine. Manipulated or warped virtue tainted by selfishness is unholy and cannot serve any benevolent purpose. In other words, generosity is not generous if you carve for yourself some personal (in)tangible gain. But when you seek to love one another in effort to Honor the Love of God do so in secret no showing or boasting but work so that others do not notice because God the Father, who sent us his only Son of God, Jesus, watches and takes note of all things even to reward each according to what he/she/they have done according to His Mercy.
E1. The virtues were provided for to resonate within person(s) in symmetrical extension from the chief virtue, Love, as it has been made intelligible to us. Within Love there is Justice, Righteousness, Loving-Kindness, Compassion, Faithfulness established in Holy promise by God, your God.
E2. By extension from the Perfect Holy Pious Love down to the created people(s), to ascend into the divine order you must first harness at least two virtuous faculties and ‘back’ into their higher parent virtue. This is as though climbing a latter not by sight but by faith in trust for what there is good reason to hold is all Goodness, Righteousness and Truth. The lens of algorithm(s) is mathematics and cannot accurately quantify the condition of the heart.

Tim Smith's picture

Tim Smith

Saturday, March 27, 2021 -- 2:15 PM

Algorithms can and do

Algorithms can and do quantify conditions of the heart. Virtue is enacted by humans. Previous generations have never faced machine learning or artificial intelligence. Perfection is not the standard. Genetic or epigenetic disorders can be screened by algorithms.

Tim Smith's picture

Tim Smith

Saturday, April 10, 2021 -- 9:14 AM

Algorithms are not the

Algorithms are not the problem, nor are they unethical by nature or use. It is the height of rationality to use an algorithm to solve uncertainty. Much of what we consider inevitable without proof is likely not certain at all. Though algorithms often don't provide certainty in their answers, they can prove the best solutions possible to philosophical questions that themselves defy logical proof. These are some of the most interesting questions in philosophy. What is rational? What is the nature of our thought? How to Live?

COMPAS, the public safety assessment (PSA) program featured in Holly's report, was featured in a story by ProPublica in 2016. This specific program was accurate and precise in predicting recidivism across Black and White defendants in 61% of cases. However, in the 39% of the cases where it failed, COMPAS was twice as likely to classify White defendants as low risk – who then later committed new crimes, while Black defendants were twice as likely not to recidivate when rated as high risk. This injustice is a huge problem. I'm not sure how the judicial system did before using the program, but it seems this reflects the training set. If algorithms in the best case only point this out we shouldn't use them to reinforce these models.

The case of Loomis v. Wisconsin is similarly discomfiting as the judge in this case, using the same PSA system, which was intended for bail hearings only, instead uses COMPAS as a sentencing tool.

No one is going to deny the goodness of a computer. We are all too immersed in the standard of living computers provide for us all. Something is alluring about algorithms and computational artifice.

Iris Berent's 'The Blind Storyteller' (she was a guest on the PT: The Examined Year: 2020 ) posits dualism as innate human behavior. Berent's argument is based on her own and cited studies reflecting experimental philosophy at its best. I would extend this innate dualism to an uninterrogated and unhealthy surrender of humanity to the power of models, both internal and external, as suggested and validated by algorithms.

In addition to exposing existing models of bias and truth, algorithms can also create false models beyond the human minds' capacity to understand. Inaccurate AI-driven models of thought and action are the best argument not to surrender matters of justice, ethics, and morality to an algorithm. I wish Ken and Josh had discussed this along with the capacity for algorithms to derive truth beyond human understanding.

I share Ken's resignation that this human surrender to the coming singularity is unavoidable. A few brains are working on algorithmic safety as a project. The keys to this safety are transparency, fairness, and accurate representation of human qualia. Built on these three pillars – I see a safe path for humanity. Barring this, we need to bow down to the one we serve. Regardless, we are going to get the one we deserve.

Cross post with the 2021 show -//m.f8r7.com/shows/ethics-algorithms