The Myth of Neutral AI. Dawn Intelligence.

There is no neutral AI.

There is AI that is honest about its politics, and there is AI that is lying about them. That is the choice. That has always been the choice. Anyone who tells you their algorithm is unbiased is selling you something, and the thing they are selling you is the appearance of objectivity, dressed up in the credibility of mathematics.

I want to be careful, because this is the kind of statement that gets read as anti-technology. I am not anti-technology. I work in this field. I have spent the last two years writing code that runs predictive models on data about violence against women. I have skin in the game. What I am opposed to is dishonesty about what these systems are. Dishonesty serves the people who built them. It does not serve the rest of us.

This essay is about why the language of “unbiased” or “neutral” or “objective” AI is a marketing strategy, not a technical claim. About what is actually political in every algorithm. About the worldview behind the current moment in AI, and why we should refuse it.

What neutral means

I want to begin with the word neutral, because the whole argument lives inside its slipperiness.

Neutral, in ordinary English, means not taking a side. The country is neutral in the war. The journalist is neutral on the political question. The neutral term in a chemical reaction is the one that is neither acid nor base. Neutrality, in everyday usage, suggests an absence. The absence of position. The absence of bias. The absence of stake.

This is the sense in which AI companies use the word. They describe their systems as neutral, meaning, the system does not have an opinion. The system is just looking at the data. The system is just doing the maths. The system is just letting the patterns speak.

The problem is that neutrality, in this sense, is impossible. It is not a feature an AI system can possess. The mathematics is not neutral. The data is not neutral. The choice of what to count is not neutral. The choice of how to label is not neutral. The choice of what to optimise for is not neutral. The choice of what to deploy and where is not neutral. Every step in the pipeline involves a decision, and every decision encodes a worldview, whether the engineer making the decision realises it or not.

The journalism analogy is useful here. There has been, for decades, a debate in serious journalism about what objectivity actually means. The view that emerged from the best of that debate, articulated by people like the late James Carey at Columbia and more recently by Wesley Lowery, is that journalistic objectivity, properly understood, is not the absence of perspective. It is the discipline of being honest about your perspective and applying it rigorously and consistently. The journalist who claims to have no perspective is the journalist whose perspective is invisible, which is to say, the dominant one. The journalist who is honest about her perspective and tests it against evidence is doing the actual work of objectivity.

The same logic applies, exactly, to AI. The system that claims to be neutral is the system whose worldview is invisible. The system that admits its values, applies them transparently, and tests them against evidence is the system that is doing the actual ethical work. Neutrality is not a virtue. Honesty about your stake is.

I want to walk through where the politics actually lives in an AI system, because this is the part that gets glossed over most often.

It starts with the data. Every AI system is trained on data that was collected by someone, for some purpose, in some context. The data is not neutral because the collection was not neutral. The classic example, well documented by now, is medical data. For most of the twentieth century, medical research was disproportionately conducted on white men. The clinical pictures of many conditions, including heart attacks, were established on male patients, with the result that women’s heart attacks, which often present differently, are routinely missed in clinical settings to this day. Train an AI on that data and the AI will, with statistical fidelity, miss women’s heart attacks. The AI is not biased because someone coded it to be. The AI is biased because it learned from a biased archive.

This is what the field calls historical bias. The data carries the history of who was paid attention to, who was studied, who was counted, who was made visible. An AI trained on it will reproduce the patterns of attention. A radiology AI trained primarily on chest X-rays from American white men will be less accurate on the chests of, say, Bangladeshi women. The deficiency is not in the AI. The deficiency is in the medical history of the world the AI was trained on. The AI is, in a precise sense, a mirror.

Then there is labelling bias. Even when the data has been collected, someone has to decide what the data shows. What gets labelled as a tumour. What gets labelled as a face. What gets labelled as a violation of platform terms. The labellers are humans, working at speed, with their own assumptions about the world. There is well documented research on platform content moderation showing that posts written in African American Vernacular English are flagged at higher rates as toxic, by the labellers and by the AI systems trained on those labellers’ decisions. The bias is in the labels. The bias is in the labellers. The AI inherits both.

Then there is what the field calls objective function bias. Every machine learning model is optimising for something. The choice of what to optimise for is a choice. If you build a hiring AI to optimise for “candidates similar to past successful employees,” and your past successful employees were disproportionately a particular demographic, the AI will reproduce that demographic. If you build a recidivism AI to optimise for “predicted likelihood of re-offending,” and re-offending is itself measured through the lens of who gets caught, the AI will reproduce the patterns of policing rather than the patterns of crime.

Then there is deployment bias. Even a perfect model, if such a thing existed, can be deployed in ways that produce political effects. Facial recognition technology that is technically accurate is still political, because it is being deployed in some places and not others, on some communities and not others. The deployment of a tool is a political act. The deployment encodes who is being watched and who is not.

So when an AI company tells you their system is neutral, what they are really telling you is, we have not noticed where our politics lives. The politics lives in the data, in the labels, in the objective function, in the deployment. It is not, technically speaking, possible to remove it. It is only possible to be honest about it.

There is one more category, the one most missed. The politics of refusal. Every AI system has been trained, at the very last stages of its development, on what to refuse to do. ChatGPT will not write you a manifesto for genocide. Claude will not help you build a bioweapon. Grok, depending on which version you are using, will or will not engage with various controversial topics. These refusals are not natural to the underlying language model. They are bolted on, deliberately, by humans, on the basis of judgements about what is and is not acceptable for the system to produce.

The list of refusals is one of the most political artefacts in any AI system. It tells you what the developers thought was unacceptable. What they thought was acceptable. Where they drew the line. The line is, always, contestable. The line in ChatGPT is in a different place from the line in Claude is in a different place from the line in Grok. None of these lines is neutral. They are all the result of human decisions about what kinds of harm a given system will participate in.

When you read a thinkpiece about how an AI system is “too cautious” or “too willing,” you are reading a piece about the location of the refusal line, not about the underlying capabilities of the model. The model can do almost anything. The refusal is what the developers decided the model would not do. Disagreements about the line are political disagreements, not technical ones, even when they are dressed up in technical language.

The worked examples

I want to walk through three case studies, because the abstract argument lands better when you have specific failures to point at.

The first is COMPAS. This is a recidivism prediction tool that has been used in American courts since the early 2000s to inform sentencing and parole decisions. In 2016, ProPublica published a major analysis of the tool. They obtained the COMPAS scores for thousands of defendants in Broward County, Florida, and they cross-referenced those scores with what actually happened to those defendants over the following two years. The finding was stark. Black defendants who did not go on to re-offend were nearly twice as likely as white defendants who did not go on to re-offend to have been classified as high risk. The model was, in technical terms, miscalibrated by race.

The company that built COMPAS, Northpointe, then called Equivant, disputed the methodology. The dispute was real. There are genuine technical questions about how to define fairness in a predictive system. What ProPublica had measured, and what Northpointe had been optimising for, were different things, and reasonable people can argue about which definition of fairness should be the priority. But the underlying point, that no choice of fairness definition is itself neutral, that every choice encodes a politics, remained.

What is most striking about the COMPAS story is what happened next. The system continued to be used. The legal challenges were limited and partial. Most of the jurisdictions deploying COMPAS or similar tools continued to deploy them. The political will to change the practice was much smaller than the technical evidence demanded. AI ethics scholars have been writing about COMPAS for nearly a decade. The system is still in use. This is one of the most important data points in the entire field. Even with overwhelming evidence of harm, an AI system can keep operating, because the institutional momentum behind it is greater than the political will to dismantle it.

The second worked example is hiring AI. Amazon famously had to scrap an internal hiring AI in 2018 after it was discovered to be downgrading CVs that contained the word “women’s,” as in “women’s chess club” or “women’s college.” The model had been trained on a decade of past Amazon hiring decisions, which had been overwhelmingly male, and it had learned, with statistical fidelity, that the company hired men. The model was doing exactly what it had been trained to do. The model was a perfect mirror of the company’s actual hiring history. Amazon had not noticed, until the AI surfaced it, what its hiring practice had actually been.

This is the second key insight. AI bias is, often, AI honesty. The model has read the history accurately. The horror is not that the model lied. The horror is that the model told the truth.

There is a follow-on point I want to draw out. After Amazon scrapped the model, the press coverage focused on the model’s failure. The framing was, the AI was sexist. What the framing missed was that the AI had been correctly identifying the company’s pre-existing sexist hiring pattern. The pre-existing pattern did not get the press attention. The model’s reflection of it did. We have a strange tendency, in coverage of AI bias, to treat the model as the actor and the underlying social pattern as the background. It should be the other way around.

The third worked example is healthcare. There is a well known paper by Obermeyer, Powers, Vogeli, and Mullainathan, published in Science in 2019, examining a healthcare AI used by an American insurance company to identify patients who needed extra care. The model was using healthcare expenditure as a proxy for healthcare need. Sounds reasonable. Patients who spend more on healthcare are sicker, right.

Wrong. Black patients in the United States spend less on healthcare than white patients with equivalent conditions, because of structural barriers to access, because of discrimination, because of historic mistrust of the medical system, because of insurance gaps. The model, by using expenditure as a proxy for need, was systematically underestimating the needs of Black patients. The paper estimated that correcting the bias would more than double the proportion of Black patients identified as needing extra care.

This case is, I think, the cleanest illustration of what political AI looks like. The model was technically working as designed. The math was correct. The data was the data the company had. The bias was not in the algorithm. The bias was in the choice of expenditure as a proxy for need. That choice was, structurally, a political choice. It encoded a worldview in which what gets measured stands in for what is real, and the measurement happened to be racist by inheritance.

These three cases, COMPAS, Amazon hiring, healthcare expenditure, are foundational reading in the AI ethics field. If you want to go further, the books I would point you at are Cathy O’Neil’s Weapons of Math Destruction, Virginia Eubanks’s Automating Inequality, Safiya Noble’s Algorithms of Oppression, Ruha Benjamin’s Race After Technology, and Joy Buolamwini’s Unmasking AI. Together they make the structural case better than any single essay can.

I want to add a UK-specific case. The Department for Work and Pensions has been using algorithmic systems to flag potential benefit fraud for several years now. Investigative reporting by The Guardian, by Big Brother Watch, by the Public Law Project, has raised serious concerns about the accuracy of these systems and their disproportionate impact on disabled claimants and claimants of colour. The DWP has been resistant to publishing the data that would allow proper external audit. Disabled claimants have had their benefits suspended on the basis of algorithmic flags, in some cases for months, with limited recourse. The Public Law Project has brought legal challenges. The DWP has, in several cases, settled rather than allow the algorithms to be examined in court.

This is, in some ways, a smaller scandal than COMPAS, in that the affected population is smaller and the legal stakes per case are lower. It is, in other ways, a more important one, because it is happening in this country, right now, to people the British state has decided are not politically powerful enough to defend the algorithm against. The pattern of deployment, on populations with the least capacity to fight back, is the same pattern as every other case I have walked you through.

There is a fifth case I want to flag, because it is becoming relevant to the GBV space specifically. The various AI-based systems being marketed to UK police forces for predictive risk assessment in domestic abuse cases. Some of these are evidence-based and reasonably designed. Others are vendor products with weak research foundations, sold to police forces under political pressure to “do something” after high profile failures. The vendors are profiting from a crisis. The police forces are buying products without the technical capacity to evaluate them. The women whose cases are being scored by these systems have no input into the process and no transparency about how decisions about their safety are being made.

This is happening now. It is happening in your country. The accountability infrastructure that should be evaluating these systems before deployment does not currently exist. We are, in this country, in 2026, deploying experimental AI in life-or-death contexts without anything resembling adequate oversight. The next major scandal in this space, when it comes, will be predictable. It is being foreshadowed, right now, by the absence of the regulatory capacity that would prevent it.

The based AI moment

The conversation about AI politics has shifted in ways that matter and that the public conversation has not fully caught up with.

The shift, if I had to put it crisply, is from “AI should be unbiased” to “AI should not be woke.” The first framing dominated AI ethics conversations from roughly 2015 to 2022. The second framing has, since around 2023, become the dominant framing in significant parts of the AI industry, particularly in the United States, particularly under the second Trump administration.

The clearest example is Grok, the chatbot built by Elon Musk’s xAI. Musk has been explicit, repeatedly, that Grok was designed to be more permissive, more willing to engage with controversial topics, less concerned with what he calls political correctness, than competing models. The branding is “based AI.” The argument, as it is made by its proponents, is that previous AI models had been corrupted by liberal political agendas, and that the corrective is an AI that refuses to be corrupted in this way.

This framing is, I think, useful, because it makes explicit what was always implicit. Every AI is trained with political values. Grok is trained with right-leaning American libertarian values. ChatGPT is trained with broadly liberal centrist values. Claude is trained with values its developers describe as broadly aligned with human flourishing, which encodes a different set of priorities again. None of these is neutral. None of these can be neutral. The interesting question is which set of values you want operating in the systems you use, and whether you have any meaningful choice in the matter.

The “based AI” framing is, however, dishonest in a specific way. It claims to be the absence of political programming, when it is, in fact, the presence of a different political programming. The framing wants the credibility of neutrality without the work of being neutral. It is the same trick as the “view from nowhere” journalism that James Carey was pushing back against fifty years ago. The system claims to be objective by virtue of its political alignment with whoever is making the claim.

There is a deeper political shift happening underneath this. The Trump administration, in the early months of 2025, signed executive orders that significantly weakened the AI bias and fairness work that had been built up under the Biden administration. The AI Safety Institute, the work on algorithmic discrimination, the equity-focused executive orders, were rolled back or repurposed. The signal to the industry was clear. The federal government no longer considers AI bias a priority. Companies that had been investing in fairness teams have, in many cases, scaled them back or eliminated them entirely.

This has knock-on effects for the rest of the world. The United States is the largest single AI market and the home of most of the major foundation model companies. When the US shifts, the global conversation shifts with it. The UK government’s position on AI ethics has been, for the last eighteen months, somewhere between cautious and incoherent. The AI Safety Institute that was set up under Sunak is still nominally in operation but its remit has been narrowed. The AI legislation that the previous government had been gesturing at has not materialised. The current political climate in this country is not friendly to robust AI ethics work.

The European Union, to give credit, has done significantly better. The AI Act, which came into force in 2024, is the most serious piece of AI regulation in the world. It is imperfect. It is being implemented patchily. But it represents a real attempt to put political values into law, in a way that the Anglo-American world has not.

What this means, practically, is that the locus of AI ethics work has shifted. It is no longer in the foundation model companies. It is in civil society, in academic institutions, in journalism, in regulators, in the small companies that have made it their commercial proposition. This is not, in some ways, a bad shift. It distributes the work more widely. It also, however, means that the largest and most powerful AI systems in the world are now being built by companies that have, explicitly or implicitly, decided that the political values question is not one they want to be in charge of answering.

There is a strain of right-wing AI discourse that argues, with some sophistication, that previous AI models were trained on data that had been, in their framing, contaminated by left-wing media bias. The argument is that the internet is dominated by left-leaning text, that LLMs trained on the internet absorb left-leaning values, and that the corrective is to deliberately rebalance the training data toward the right.

This is, technically, a coherent argument. It is also a highly political argument, dressed up as a technical correction. The premise that “the internet is left-leaning” is itself contested. The premise that “balance” is a meaningful goal in AI training is itself contested. The premise that the right’s preferred outputs are more accurate to reality is itself contested. None of these premises are technical. All of them are political. The rhetoric of technical neutrality is being used to justify what is, in fact, a political project.

The “based AI” framing is, in some ways, the AI version of a much older move in conservative politics, which is to claim that you are the actual neutral party because the other side has, in your framing, captured the institutions. This move has been used about universities for fifty years. About the BBC for thirty. About the civil service for twenty. About corporate diversity programmes for ten. The AI version is the same move at the next level of abstraction. We are not the political ones. They are the political ones. The proof of our neutrality is that they have called us political.

The move is rhetorically powerful because it is non-falsifiable. If your opponents disagree with you, that is evidence that they are political. If they agree with you, that is evidence that you have won them over to neutrality. There is no possible response that does not confirm the framing. This is why the move has been so durable across so many political contexts. It is also why the move is dishonest. Politics, properly understood, is the disagreement about how things should be ordered. There is no view from nowhere. The party that claims to be standing nowhere is, almost always, standing somewhere with a particular set of interests in mind.

This is the deeper game. The myth of neutral AI is now being weaponised by the right to argue that left-aligned AI is the corrupted form and right-aligned AI is the corrected form. Both are political. Neither is neutral. The right has just got better, in 2026, at making its political project look like a return to neutrality.

We should not fall for it. The honest answer is that every AI is political and that the political question is which set of values we want our systems to embody. That conversation, which has barely begun in mainstream public life, is the conversation we should be having.

The conversation about AI politics is missing something almost everyone misses, which is the historical dimension.

The slave registers in the eighteenth and nineteenth century Caribbean were, in the technical sense, datasets. They counted enslaved people. They classified them. They tracked births and deaths and sales and escapes. They produced statistical outputs that the British colonial administration used to make decisions about taxation, military deployment, and economic policy. The slave registers were, in their structure, an early version of the data infrastructure that contemporary AI runs on.

The fingerprinting techniques that became the basis for modern biometric identification were developed, principally, in colonial India in the late nineteenth century. The British colonial administration needed a way to identify Indian people whose names and physical descriptions, the British thought, were not reliable for legal purposes. The solution was a body-based identification system that was tested, refined, and deployed on Indian subjects before being exported back to the metropole as a general law enforcement technology. The first systematic fingerprinting in this country was done on people the British state did not consider full citizens. The technology has continued to be used, disproportionately, on populations the state does not consider full citizens.

I bring this up because the contemporary AI conversation has tended to treat the question of bias as if it were new. It is not new. It is the latest version of a five hundred year old pattern in which colonial and racial classification systems are developed, refined on marginalised populations, and then generalised to the whole population. AI is, in this lineage, the most powerful instance yet of a logic that has been operating since the slave register.

Readers whose families come from the colonised world have, often, an inherited intuition for this. The familiar suspicion that a new official system is going to be used in ways the developers did not promise. The familiar pattern of deployment in racialised neighbourhoods first. The familiar ratchet by which what was once exceptional becomes routine. None of this is paranoia. It is pattern recognition trained over centuries.

The technology industry, which is overwhelmingly led by people whose families are not from the colonised world, has tended to treat these intuitions as marginal. The intuitions are not marginal. They are evidence-based. The evidence is the historical record. The evidence is currently being added to, in real time, by the deployment patterns of facial recognition, predictive policing, immigration AI, and algorithmic welfare administration in the United Kingdom and elsewhere.

What to demand

If neutrality is impossible, what does honest political AI look like?

Six things to demand of any AI system you encounter in a high-stakes context. By high-stakes I mean any context where the system’s outputs will affect a person’s access to housing, healthcare, employment, criminal justice, immigration, education, or safety. Lower-stakes contexts can have lower bars, although the principles still apply.

First, stated values. The system’s developers should be willing to say, in plain language, what political and ethical values the system has been built around. Not generic values like “safety” or “alignment.” Specific values. Whose interests does the system prioritise when those interests conflict. What kinds of outputs is the system designed to produce. What kinds of outputs is it designed to refuse. If the developers cannot answer these questions, or refuse to, the system is not ready to be trusted.

Second, transparency about training. The system’s developers should be willing to explain what the system was trained on, in enough detail for an external auditor to assess. Not the proprietary details. The structural details. Where did the training data come from. What was filtered out. What was deliberately included. What populations are represented. What populations are missing.

Third, evidence of testing. The system should have been tested for differential performance across demographic groups before deployment. Not vaguely. Specifically. Performance metrics should be reported for every protected group the deployment context cares about. If a hiring AI has not been tested separately on women, Black candidates, disabled candidates, and so on, it is not ready for hiring.

Fourth, ongoing audit. The system should be subject to ongoing external audit, with the audit results made public. AI systems do not stay still. They drift. They are updated. They encounter new data. A system that was tested once at deployment is not, eighteen months later, the same system. The audit has to be continuous.

Fifth, refusal capacity. The system should be willing to refuse certain uses. Not for marketing reasons. For structural reasons. Encoded in the architecture. The list of refusals should be public and specific. A system with no refusals is a system whose ethics is whatever the customer decides on the day, which is to say, no ethics at all.

Sixth, the one most often missed because it is institutional rather than technical. Independent oversight. The system, and the company that built it, should be subject to oversight by bodies that are not commercially dependent on the company. That means independent regulators with real power to investigate and sanction. It means civil society organisations with the standing to bring legal challenges. It means academic researchers with access to the system for audit purposes. It means investigative journalists who are taken seriously when they raise concerns.

This is the bit that almost no current AI system has, in any meaningful form. The companies prefer voluntary standards, self-certification, internal ethics boards. None of these provide the friction necessary to constrain corporate behaviour at scale. The history of every other powerful industry tells us this. Pharmaceutical regulation got serious only when independent regulators with prosecutorial power were created. Financial regulation got serious only after the various crises that demonstrated what self-regulation produces. Aviation safety got serious only with the establishment of independent investigative bodies. The pattern is the same across industries. Powerful technologies need independent oversight or they will, eventually, harm enough people for the political will for oversight to emerge anyway, after the fact, after the damage.

The UK government, in 2026, is significantly behind where it should be on this. The AI Bill that was promised under the previous Conservative administration was watered down repeatedly and never made it to the statute book. The current Labour government has been hesitant to push for stronger regulation, partly because of the perceived need to keep the UK attractive to AI investment, partly because the political coalition needed to pass serious AI legislation does not currently exist. The result is that we are, in this country, deploying AI systems in increasingly consequential domains without the regulatory infrastructure that the deployment would justify.

These six demands are the floor for honest AI in high-stakes contexts. They are achievable. Some companies are already meeting most of them. Many are not. Your job, as a citizen and as a consumer of AI services, is to ask. To demand. To not accept the brand-level reassurance that “our AI is fair.” Fair according to whom. Tested how. Audited by whom. Refusing what. Overseen by what body with what powers. The questions are the work.

The everyday

There is no neutral AI. There is honest AI and dishonest AI. The honest version states its values and tests them. The dishonest version pretends not to have values and asks you to trust the maths.

Honest AI is harder to build. It is more expensive. It is slower. It produces, in many cases, less impressive demos. The marketing is harder. The investors are less excited. The customer base is more demanding.

It is also the only kind of AI worth deploying in any context where actual lives are affected.

The political question of the next decade is which kind we are going to build. We are at, I think, a genuine fork in the road. The “based AI” tendency, which is the right’s preferred direction, is winning institutional ground in the United States and is making inroads in this country. The honest AI tendency, which has been doing the slow work in academia and civil society for fifteen years, is more rigorous, more thoughtful, and dramatically less well funded. Which one prevails is going to be determined, partly, by what the public is willing to accept and what it is willing to refuse.

The principles I have walked through can sound like they apply only to high-stakes institutional deployments. Hospital AI, police AI, immigration AI. They do apply most urgently there. But they also apply to the AI you use every day, the chatbots, the recommendation systems, the search results, the social media feeds. The everyday AI is where most of our exposure to algorithmic politics actually happens. The high-stakes deployments are where the harms are sharpest. The everyday deployments are where the values get shaped.

When you use ChatGPT to draft an email, you are training your sense of what professional writing sounds like on the political values encoded in that system. When you scroll TikTok, you are letting an algorithm with specific values shape what you find interesting and what you find boring. When you ask Google a question, you are accepting Google’s ranking of relevance, which is itself a political artefact. None of these uses is neutral. All of them are shaping you, slowly, in directions that the systems’ developers chose.

The healthy response to this is not to refuse all AI tools. The healthy response is to use them with awareness. To notice when a system’s outputs feel different from your own judgement, and to take that difference seriously. To remember that the system has been trained, that the training had a politics, and that the politics is not yours unless you have actively chosen it.

The witches we keep coming back to in this season had a phrase. As above, so below. The principle was that the macro and the micro mirror each other. What is true at the level of the cosmos is true at the level of the body. What is true at the level of the body is true at the level of the cosmos.

The same applies here. What is true at the level of the AI industry is true at the level of the individual product you use. The politics of the system you click “I agree” on at the bottom of a ten thousand word terms of service is the same politics as the system that is shaping the global economy. The micro and the macro are the same. You participate in the macro by making decisions at the micro. The decisions matter.

So, here is the prompt. Next time you encounter an AI system in a context that matters, ask the six questions. Stated values. Training transparency. Testing evidence. Ongoing audit. Refusal capacity. Independent oversight. If the system fails the questions, refuse it. If you cannot refuse it because it is being deployed on you by an institution, raise the questions to the institution. Make it expensive, in attention and accountability terms, for the institution to not have asked the questions itself.

This is not utopian. This is just citizenship in an algorithmic age.

I want to close with a thought that is, properly speaking, theological.

We are at a strange historical moment. We are building systems that approach, at least functionally, capacities we used to attribute only to gods. The ability to see patterns across millions of lives at once. The ability to predict, with reasonable accuracy, what large groups of people will do. The ability to generate language, images, voices, that are indistinguishable from those produced by individual humans. These are, when you stand far enough back from them, divine capacities. Or at least, capacities our ancestors would have understood as divine.

What is missing, from the people building these systems, is the corresponding humility. The traditions that developed alongside divine capacities, in every religious culture that grappled with them, included serious thinking about how those capacities should be used, who could be trusted with them, what restraints needed to be put in place. The whole apparatus of ethics, of law, of religious tradition, of philosophical reflection, accreted around the question of how powerful actors should constrain themselves.

The contemporary AI industry has, broadly, refused to engage with that tradition. The refusal is sometimes principled (we are doing engineering, not theology) and sometimes self-serving (we do not want to be regulated). The result is the same. We are deploying systems with godlike functional reach, in the hands of human institutions with corporate-quarterly-earnings horizons. The mismatch between the power of the tool and the wisdom of the wielders is the deepest source of risk in the current moment.

The witches knew this. The priests knew this. The shamans knew this. Power requires constraint. The constraint has to come from somewhere. If it does not come from inside the institution, it has to come from outside. The outside is, in the AI case, the rest of us. Citizens. Regulators. Civil society. Journalists. Readers of essays.

The role of the audience, in the Cassandra story, was to listen. Troy did not. We are Troy. We can still choose differently.

Refuse neutral AI. Demand honest AI. Hold the institutions to the questions. The micro is the macro. The macro is the micro. As above, so below.

Reading list

On AI bias and politics

Cathy O’Neil, Weapons of Math Destruction (2016)
Virginia Eubanks, Automating Inequality (2018)
Safiya Umoja Noble, Algorithms of Oppression (2018)
Ruha Benjamin, Race After Technology (2019)
Joy Buolamwini, Unmasking AI (2023)
Kate Crawford, Atlas of AI (2021)
Meredith Broussard, More than a Glitch (2023)

Specific case studies

ProPublica, “Machine Bias” (Angwin, Larson, Mattu, Kirchner, 2016) on COMPAS
Obermeyer, Powers, Vogeli, Mullainathan, “Dissecting racial bias in an algorithm used to manage the health of populations,” Science (2019)
Reuters reporting on the Amazon hiring AI scrapping (2018)

On objectivity and the “view from nowhere”

James Carey, Communication as Culture (1989)
Wesley Lowery, “A Reckoning over Objectivity, Led by Black Journalists” (2020)
Jay Rosen, PressThink archive

UK accountability work

Big Brother Watch, ongoing reports
Public Law Project, ongoing legal challenges to government algorithms
Ada Lovelace Institute, Regulating AI in the UK (2023)

Calls to action

Ask the six questions. The next time you encounter an AI system in a context that matters, ask: stated values, training transparency, testing evidence, ongoing audit, refusal capacity, independent oversight.
Read O’Neil and Eubanks. Weapons of Math Destruction and Automating Inequality together give you the framework for everything else.
Support investigative journalism in this space. Big Brother Watch, the Public Law Project, the Bureau of Investigative Journalism.
Vote for AI accountability. The choices we make at the ballot box about AI policy in the next two elections will shape the next thirty years.
Subscribe to keep reading the rest of this season.

This essay accompanies Episode 5 of Sacred Space. Listen wherever you get your podcasts.

Sacred Space is a feminist podcast and Substack written by Leah Garrett. New episodes Wednesdays. New essays the same day.