Redistributed

Book: Virginia Eubanks. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin's Press, 2018. Link: Macmillan

Why This Book Matters

The AI industry talks about bias as a bug — an unintended consequence to be engineered away. Virginia Eubanks documents something different. In welfare offices, homeless services, and child protective agencies, automated systems don't malfunction when they harm the poor. They function exactly as designed.

Eubanks introduces a concept that has become foundational: the "digital poorhouse." Just as nineteenth-century poorhouses used moral judgment and physical confinement to manage poverty, today's automated systems use algorithmic profiling and data surveillance. The technology is new; the function is old.

This matters for redistribution because it shows automation serving a particular kind of redistribution — downward. Costs, burdens, and risks flow to those with the least power to resist. "Efficiency" becomes a mechanism for reducing public services while expanding public surveillance. The poor don't benefit from these systems. They are managed by them.

The Argument

Three Case Studies

Eubanks examines three automated systems in detail. Each reveals how algorithms inherit and intensify existing inequalities.

Indiana's Welfare Automation (2006-2009): In 2006, Indiana outsourced welfare eligibility to IBM for a $1.3 billion contract. The system was designed to reduce fraud and increase efficiency. What it did was deny benefits at scale.

The automated system rejected applications for minor errors — a wrong checkmark, a missed deadline, a document misfiled. Before automation, caseworkers would catch these problems and help applicants fix them. The algorithm simply denied claims. Appeals were difficult; the system was opaque.

During three years of operation, the denial rate for disability benefits increased by 54%. One million applications were denied. Many of those denials were wrong, reversed on appeal months later — but the damage was done. People lost housing, went hungry, died.

The case reveals automation's real function. It didn't catch more fraud (fraud rates didn't change). It processed denials faster. The system was efficient at saying no.

Los Angeles Coordinated Entry System (VI-SPDAT): LA's homeless services use an algorithm called VI-SPDAT to prioritize people for housing. The premise sounds reasonable: scarce housing should go to those most in need. The algorithm scores vulnerability to determine who gets help first.

But the scoring encodes particular assumptions. It asks about drug use, criminal history, mental health — questions that penalize the survival strategies homeless people develop. It creates categories ("chronic homeless," "transitional homeless") that determine who is deserving of help. It turns a political question — who deserves housing? — into a technical one.

The system also creates a surveillance infrastructure. To be scored, you must be surveyed. Your life becomes data points in a system you can't see or contest. The algorithm doesn't explain its decisions. People don't know why they scored low or how to score higher. They just wait, sometimes for years, while the algorithm decides their fate.

Eubanks documents the human toll. A man named Gary, sixty-eight years old and diabetic, sleeping in a broken-down car, scored "moderate" vulnerability — not high enough for housing. He died waiting.

Allegheny County Child Welfare (AFST): Allegheny County, Pennsylvania uses an algorithm to predict which children are at risk of abuse or neglect. Child welfare calls come in; the algorithm scores the risk; caseworkers decide whether to investigate.

The Allegheny Family Screening Tool (AFST) uses data from multiple public systems — welfare, mental health, juvenile justice, housing. It identifies patterns that correlate with child welfare involvement. The idea is to catch at-risk families before harm occurs.

But the data reflects existing inequalities. Poor families interact with public systems more often because they depend on public services. This generates data that the algorithm interprets as risk. Using welfare becomes a risk factor for losing your children.

The algorithm can't distinguish correlation from causation. It finds that mental health treatment correlates with child welfare involvement — but is that because treatment causes risk, or because families who seek treatment are more likely to be reported? The algorithm doesn't know. It just calculates a score.

And once the score exists, it shapes decisions. A high-scoring family faces more scrutiny. More scrutiny means more opportunities to find problems. The prediction becomes self-fulfilling.

The Digital Poorhouse

Eubanks's central concept connects historical and contemporary systems for managing poverty.

The original poorhouses (almshouses, workhouses) emerged in the nineteenth century as responses to urbanization and immigration. They served multiple functions: providing minimal aid to the "deserving poor," punishing the "undeserving," deterring people from seeking help, and segregating the poor from public view.

Poorhouses combined moral judgment with physical confinement. Entering one meant surrendering autonomy — what you ate, when you slept, whether you could leave. It meant being categorized, surveilled, controlled. The conditions were deliberately harsh to discourage reliance on public aid.

The digital poorhouse performs similar functions through different means:

Moral categorization: Automated systems classify the poor into deserving and undeserving. Algorithms score risk, predict behavior, and determine eligibility. The categories are technical ("high-risk," "moderate vulnerability") but they carry moral weight.

Surveillance: To receive aid, you must be watched. Systems collect data from every interaction — welfare applications, homeless surveys, child welfare calls. This data follows you, shapes your options, determines your future access to services.

Deterrence: Automation doesn't just deny benefits more efficiently; it makes seeking benefits harder. Navigating the system requires time, knowledge, and persistence. Many give up. The complexity is a feature, not a bug — it reduces demand for services.

Containment: Digital systems don't confine bodies but they constrain lives. A bad score limits housing options. A denied application means no healthcare. A child welfare investigation means surveillance for years. The walls are invisible but real.

The poorhouse didn't end. It digitized.

Automation Doesn't Remove Bias — It Encodes and Scales It

A common defense of automated systems: algorithms are neutral. Remove human discretion and you remove human bias.

Eubanks demolishes this argument. Algorithms learn from historical data that reflects historical inequalities. They're designed by humans with particular assumptions. They optimize for objectives that encode values. Every technical choice is a political choice.

Consider the Allegheny Family Screening Tool. It was trained on historical child welfare data — data reflecting which families were investigated and which children were removed. But those investigations reflected biased decisions. Poor families, Black families, families in surveilled neighborhoods were investigated more often. The algorithm learned to reproduce those patterns.

The tool's designers knew this. They tried to exclude some data (race, neighborhood) to reduce bias. But correlated data remained. Welfare receipt correlates with poverty; poverty correlates with race. Remove one variable and the pattern persists through others.

Worse, automation scales bias. A biased caseworker can only harm so many families. A biased algorithm affects everyone who enters the system. The efficiency that advocates promise is real — the system processes bias faster and more consistently than any human could.

And automation obscures responsibility. When a caseworker makes a bad decision, someone can be held accountable. When an algorithm makes a bad decision, who's responsible? The programmer? The agency that deployed it? The policymaker who funded it? The diffusion of responsibility is itself a feature — it protects institutions from accountability.

The Poor as Testing Ground

Eubanks identifies a pattern: surveillance systems are tested on the poor before expanding upward.

Electronic benefit cards, which track every purchase, were developed for welfare recipients. Now everyone uses debit cards that generate similar data. Drug testing was pioneered in welfare programs. Now it's common in employment. Digital identity verification started with public benefits. Now it's everywhere.

The poor are "a testing ground for the technologies that will eventually be used on everyone," Eubanks writes. Systems developed to manage poverty become normalized, then generalized.

This pattern matters because it shapes what's politically possible. When surveillance affects only the poor, middle-class people don't care. By the time it reaches them, it's too late — the infrastructure exists, the precedents are set.

It also reveals who has power. The poor can't resist surveillance because they need the benefits. Their weakness makes them ideal test subjects. Systems refined on them are then deployed on populations who might have objected earlier.

Efficiency Claims vs. Actual Functions

Automated welfare systems are sold as efficiency improvements. They'll reduce fraud. They'll speed processing. They'll save money. They'll help people faster.

Eubanks examines these claims against reality:

Fraud reduction: Indiana's system didn't catch more fraud. Fraud rates were unchanged. What increased was denial of legitimate claims. The "efficiency" was in saying no, not in identifying wrongdoing.

Cost savings: The IBM contract cost Indiana $1.3 billion over ten years. When the contract was cancelled (after lawsuits and scandal), the state returned to its previous system. Where were the savings?

Faster service: Automated systems are faster at processing — but processing what? VI-SPDAT scores homeless people quickly, but they still wait years for housing. AFST assesses risk instantly, but investigations drag on for months. Speed at the wrong point in the process doesn't help anyone.

Better outcomes: LA's homeless population hasn't decreased since VI-SPDAT was implemented. Child welfare outcomes in Allegheny County haven't notably improved. The systems optimize for metrics, not for human wellbeing.

The real function is different. Automated systems:

Reduce caseworker discretion (and caseworker costs)
Shift burden to applicants (who must navigate complexity)
Generate data for future systems
Create legal and political cover ("the algorithm decided")
Allow cuts to be presented as modernization

Efficiency is the rhetoric. Management of the poor is the reality.

Historical Continuity

Eubanks's historical analysis connects digital systems to centuries of poverty management.

The Elizabethan Poor Laws (1601) distinguished "deserving" from "undeserving" poor. The deserving (widows, orphans, the disabled) received aid; the undeserving (able-bodied unemployed) were punished. This moral categorization persists in algorithmic scoring.

The poorhouse movement (1820s-1930s) centralized and institutionalized poverty management. It combined aid with surveillance, punishment with help. Digital systems reproduce this combination.

Scientific charity (late 1800s) applied systematic methods to distinguish worthy from unworthy recipients. Charity workers collected data, made assessments, shared information across agencies. The coordinated data systems of VI-SPDAT and AFST are direct descendants.

New Deal welfare (1930s) embedded racial exclusion by design. Programs were structured to exclude agricultural and domestic workers — predominantly Black — from benefits. The discretion given to local administrators allowed discrimination. Today's algorithms encode similar exclusions through correlates of race.

Welfare reform (1996) added work requirements, time limits, and intensive surveillance. It framed poverty as a behavioral problem to be managed rather than an economic condition to be addressed. Automated systems implement this framing.

The technology changes. The function persists: classifying the poor, controlling their behavior, deterring their reliance on public support.

What the Book Gets Right

Empirical depth: Eubanks spent years embedded in these systems, talking to people affected by them. The book isn't theoretical — it's documented. The human stories give the argument weight.

The poorhouse concept: "Digital poorhouse" is a powerful frame that connects present to past. It reveals that we're not seeing new problems — we're seeing old problems with new tools.

Mechanism focus: Eubanks doesn't just claim these systems are harmful. She shows how they harm — the specific design choices, data sources, and institutional pressures that produce bad outcomes.

Affected people's voice: The book centers people harmed by these systems. They're not abstractions or data points. Gary waiting for housing, Sophie navigating Indiana's system, Patrick fighting for his children — their experiences ground the analysis.

Historical grounding: The connections to poorhouses, scientific charity, and welfare reform aren't rhetorical flourishes. They reveal structural continuity beneath technological change.

Critiques and Limitations

Geographic scope: The case studies are US-specific. Automated welfare systems exist globally with different dynamics. The book's focus on American poverty management limits its applicability elsewhere.

Alternatives underexplored: Eubanks is clear about what's wrong but less clear about alternatives. What would non-punitive automated systems look like? Are there cases where algorithms help? The critique is stronger than the constructive vision. Readers looking for constructive counterparts might explore Democratic AI Governance for participatory models, or AI Cooperatives for ownership structures that align incentives differently. The Collective Intelligence Project offers one model for involving affected communities in AI design decisions.

Implementation variation: Not all automated systems are identical. Some are worse than others. Some have more safeguards. The book's broad critique sometimes obscures this variation.

Evolution since 2018: The systems Eubanks documented have changed. Some problems have been addressed; others have worsened. New systems have emerged (pandemic benefit automation, AI fraud detection). The landscape is dynamic.

Technical depth: The book is written for general audiences, which limits technical detail. Readers wanting to understand exactly how these algorithms work will need additional sources.

Engaging the Broader Literature

Automating Inequality belongs to a critical literature on technology and power:

Ruha Benjamin's Race After Technology (2019) extends Eubanks's analysis to race specifically. Benjamin's "New Jim Code" concept describes technology that encodes racial inequality while appearing neutral. The two books complement each other — Eubanks on class, Benjamin on race (though both recognize the intersection).

Safiya Noble's Algorithms of Oppression (2018) examines commercial search algorithms with similar conclusions. Google's search results don't just reflect bias; they construct it. Noble and Eubanks both show that technical neutrality is a myth.

Kate Crawford's Atlas of AI (2021) maps AI's material infrastructure — mines, data centers, labor. Where Eubanks focuses on how AI affects the poor, Crawford shows how AI production depends on extraction and exploitation globally.

Mary Gray and Siddharth Suri's Ghost Work (2019) documents the hidden labor behind AI systems. The workers who label data and moderate content are another population made invisible by technology. Both books reveal the human costs hidden by automation rhetoric.

Cathy O'Neil's Weapons of Math Destruction (2016) provides mathematical grounding for similar arguments. O'Neil, a data scientist, explains how algorithmic systems encode and amplify inequality. She offers more technical detail than Eubanks but less ethnographic depth.

What's Changed Since 2018

Pandemic automation: COVID-19 forced rapid automation of benefit systems. Unemployment insurance, stimulus payments, and emergency aid all required digital processing. The results were mixed — faster distribution in some cases, massive fraud and denial in others. The patterns Eubanks identified persisted.

AI fraud detection: New AI systems claim to identify benefit fraud algorithmically. Early results suggest high false-positive rates that harm legitimate recipients. The Michigan unemployment fraud scandal — where an automated system falsely accused thousands of fraud — illustrated the risks.

Algorithmic accountability movements: Eubanks's book contributed to growing demands for algorithmic accountability. Some jurisdictions now require impact assessments for automated decision systems. Progress is slow but real.

"AI for Good" discourse: The tech industry has responded to critics like Eubanks with claims about beneficial AI. These claims should be evaluated skeptically — "AI for good" often reproduces the same dynamics under friendlier branding. That said, some institutional alternatives are emerging that take the critique seriously: the DAIR Institute, founded by Timnit Gebru, conducts AI research centered on affected communities. The AI Now Institute continues to document harms while advocating for structural reform. The existence of these alternatives doesn't resolve the problems Eubanks documents, but it demonstrates that different institutional models are possible.

Surveillance expansion: The testing-ground pattern Eubanks identified continues. Digital identity systems, biometric verification, and behavioral prediction have expanded from welfare to general population. The digital poorhouse is becoming the digital city.

Implications for Redistribution

The book maps redistribution along multiple dimensions:

Burden redistribution: Automated systems shift costs from institutions to individuals. Navigating complex application processes, appealing wrongful denials, gathering documentation — all this work falls on people seeking benefits. The efficiency gained by agencies is extracted from applicants.

Risk redistribution: The poor bear the downside of algorithmic errors. When Indiana's system wrongly denied benefits, people lost housing and healthcare. IBM faced no equivalent risk. Automation transfers uncertainty from powerful to powerless.

Surveillance redistribution: To receive public benefits, you must surrender privacy. Your purchases, your medical history, your family relationships become data. This information asymmetry compounds other inequalities.

Power redistribution: Algorithmic systems concentrate decision-making power. Caseworkers lose discretion; agencies gain control; affected people lose voice. The distribution of power shifts upward even as the distribution of harm shifts downward.

Recognition redistribution: Automation obscures what's happening. When benefits are denied, it's "the system." When families are surveilled, it's "the algorithm." The human decisions behind these outcomes become invisible. Responsibility diffuses; accountability evaporates.

This is redistribution — not the kind that flows from rich to poor, but the kind that flows from weak to strong.

How to Use This Book

When evaluating "efficient" AI systems: Ask: efficient for whom? At whose cost? What does the system optimize for, and what does it ignore?

When hearing "algorithms are neutral": Remember Indiana, LA, Allegheny County. Algorithms encode choices. Neutrality is a fiction that obscures power.

When told surveillance protects the vulnerable: Ask who's being surveilled and who's protected. The subjects and beneficiaries of surveillance are rarely the same people.

When designing systems that affect vulnerable populations: Eubanks provides concrete examples of how good intentions produce bad outcomes. The patterns are predictable and avoidable.

When building political coalitions: The testing-ground argument connects poor people's struggles to everyone's. Today's welfare surveillance becomes tomorrow's general surveillance. Solidarity has practical grounds.

Source

Eubanks, Virginia. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin's Press, 2018. Macmillan

General

Book Review: Automating Inequality