Redistributed

General

Content Moderation's Mental Health Crisis

Published March 11, 2026

laborcontent-moderationmental-healthoutsourcingexploitation

Every AI platform you use has been made safer by someone else's suffering.

This is not metaphor. It is the operating model. The systems that prevent you from seeing child sexual abuse material on Facebook, beheading videos on TikTok, or racist slurs in ChatGPT responses exist because human beings sat in front of screens and absorbed that content first — eight hours a day, five days a week, for wages that wouldn't cover rent in the cities where the companies profiting from their labor are headquartered.

Content moderation is the hidden human infrastructure behind AI safety. It is also one of the most direct mechanisms of harm redistribution in the technology industry: psychological damage flows downward, to the world's lowest-paid workers, so that users in the Global North can experience the internet as a tolerable place. This workforce is one part of the broader hidden labor supply chain that makes AI products function — alongside data labelers and RLHF trainers who face overlapping conditions and hazards.

The Scale of the Workforce

The content moderation industry is vast, though companies work hard to keep exact figures opaque. Meta employs roughly 15,000 moderators, the vast majority through third-party contractors. TikTok reportedly engages over 10,000. YouTube, X (formerly Twitter), and every other major platform maintain their own armies of reviewers. Beyond social media, AI companies like OpenAI, Anthropic, and Google DeepMind use thousands more workers for Reinforcement Learning from Human Feedback (RLHF) — training models to avoid generating harmful outputs by having humans label what "harmful" looks like.

Add up the platforms, the AI labs, the outsourcing firms, and the freelance microtask workers, and the global content moderation workforce likely numbers in the hundreds of thousands. These are not niche roles. This is a major industry — one that deliberately avoids being recognized as such.

Where the work happens: Overwhelmingly, it is outsourced to the Global South. Kenya, the Philippines, India, Colombia, Morocco. Companies like Sama (formerly Samasource), Majorel (now absorbed into Teleperformance), Accenture, and Cognizant operate moderation centers in these countries, marketing access to cheap, English-speaking labor forces. The geographic distance is functional — it makes the workers invisible to the users they protect, and it ensures their wages reflect local labor markets rather than the value they create for trillion-dollar companies.

What the Work Actually Involves

Content moderation is not skimming flagged posts and clicking "remove." Moderators review the worst material the internet produces — systematically, at volume, under time pressure.

The categories include:

  • Child sexual abuse material (CSAM) — images and videos of child exploitation, sometimes involving infants
  • Graphic violence — murders, torture, dismemberment, including real footage
  • Terrorist and extremist content — propaganda, recruitment videos, execution footage
  • Self-harm and suicide content — detailed depictions and instructions
  • Hate speech — targeted racial, ethnic, religious, and gender-based abuse
  • Sexual exploitation — non-consensual imagery, trafficking-related content

Moderators typically face a queue of such material for their entire shift. Casey Newton's 2019 investigation for The Verge documented conditions at Cognizant's moderation facility in Phoenix, Arizona, where workers reviewed Facebook content. They described watching people die on camera, then being expected to return to their queue within seconds. Breaks were timed. Quotas were enforced. The volume was relentless — some moderators processed hundreds of items per hour (Newton, 2019).

For RLHF workers, the task is subtly different but no less damaging. They read AI-generated text describing violence, abuse, and exploitation, then rate it or label it, so models learn what not to produce. OpenAI contracted Sama to perform this work in Nairobi, Kenya. Workers described reading graphic descriptions of child abuse, bestiality, and torture for up to nine hours a day. They were paid between $1.32 and $2 per hour (Perrigo, 2023).

The Psychological Toll

The clinical evidence is stark. Moderators develop post-traumatic stress disorder at rates comparable to combat veterans and first responders — except veterans and first responders are generally recognized as having hazardous occupations that merit specialized support. Content moderators are classified as customer service workers.

Sarah T. Roberts documented the psychological damage comprehensively in Behind the Screen (2019). Her research, drawn from interviews with moderators across multiple companies and countries, found pervasive symptoms: intrusive thoughts, nightmares, hypervigilance, emotional numbing, difficulty maintaining personal relationships, and altered perception of other people and the world. Moderators reported becoming unable to be around children without intrusive images. Some could no longer consume any media — films, news, social media — without triggering flashbacks.

The specific psychiatric outcomes documented across multiple studies and investigations include:

  • PTSD and complex trauma — repeated exposure to graphic material produces effects comparable to direct trauma exposure
  • Depression and anxiety disorders — at rates significantly above population baselines
  • Substance abuse — workers self-medicating with alcohol and drugs to manage symptoms
  • Vicarious traumatization — a cumulative effect where exposure to others' trauma reshapes the moderator's own worldview
  • Sleep disorders — insomnia, nightmares, and disrupted sleep architecture
  • Suicidal ideation — documented among former moderators in multiple investigations

Newton's reporting found that Cognizant's Phoenix office had moderators develop PTSD symptoms within months of starting. Some began espousing conspiracy theories after prolonged exposure to fringe content. Others couldn't stop watching the violent material during breaks, a recognized symptom of trauma exposure. The on-site "wellness" resources consisted of a counselor available for limited hours and a room where workers could go to cry (Newton, 2019).

In Kenya, Sama's moderators working on OpenAI's RLHF training reported similar symptoms, compounded by inadequate support and the stigma of discussing traumatic content in conservative social contexts. Workers described hallucinations, relationship breakdowns, and lasting psychological changes. One moderator told TIME that the work "ichaguliwa na shetani" — was chosen by the devil (Perrigo, 2023).

The Structural Problem

The psychological damage is not incidental. It is produced by specific structural choices that AI companies and platforms have made — choices that could be made differently.

Contractor classification. The vast majority of content moderators are not employees of the companies whose platforms they protect. They are contractors, employed by outsourcing firms, which are in turn contracted by the platforms. This layered structure is deliberate. It shifts liability away from the platform, reduces costs (no benefits, no long-term healthcare, no pension), and makes workers disposable. When a moderator develops PTSD, they are the outsourcing firm's problem — or, more often, nobody's problem.

Meta's moderators at Cognizant in Phoenix earned $28,800 per year — in one of the most expensive regions in the United States — while Meta's median employee salary exceeded $200,000. The moderators received no equity, limited healthcare, and mental health support that Roberts and others have described as wholly inadequate for the nature of the work.

Inadequate mental health support. The "wellness" programs companies provide are typically cosmetic. A counselor available for a few hours a week. An employee assistance program offering a handful of therapy sessions. No long-term psychological monitoring. No coverage for treatment after the employment ends — which matters, because PTSD symptoms often emerge months or years after exposure.

No exposure limits. Unlike occupations such as radiology, where exposure to harmful material (radiation) is tightly regulated, content moderation has no industry-standard limits on how much graphic content a worker can be exposed to in a shift, a week, or a career. Some companies have implemented internal guidelines — four hours of graphic content per day, for instance — but these are voluntary, unaudited, and frequently overridden when content queues back up.

Inadequate pay for hazardous work. Even setting aside the Global South outsourcing, moderation wages do not reflect the occupational hazard. Hazardous work in other industries — mining, oil rigs, bomb disposal — commands premium pay. Content moderation pays minimum wage or below.

The Automation Paradox

Here is the knot at the center of AI safety: you cannot train a model to avoid harmful content without showing humans the harmful content first.

Automated classifiers can catch some material. Hash-matching databases like PhotoDNA identify known CSAM images. Language models can flag obvious hate speech. But novel content, edge cases, context-dependent material, and evolving forms of abuse all require human judgment. The moderators and RLHF workers are not a stopgap until automation catches up. They are a permanent feature of the system, because the frontier of harmful content moves at least as fast as the technology designed to detect it.

This creates a grim equilibrium. The better AI safety systems become, the more novel and difficult the content that reaches human reviewers — because the easy cases get filtered automatically. Workers are left with the hardest, most disturbing material. Technological progress doesn't reduce their burden; it concentrates it.

And every label those workers produce feeds back into the training data that makes the automated systems marginally better — until the next category of harm emerges, and humans are needed again.

The legal landscape is shifting, if slowly.

In 2023, former Cognizant moderators filed a lawsuit against Meta in Kenya, alleging that the company failed to protect them from psychological harm and that the outsourcing structure was designed to avoid liability. Daniel Motaung, a former Sama moderator in Nairobi, led a parallel legal effort arguing that the working conditions violated Kenyan labor law and that Meta bore direct responsibility for the harm (Mureithi, 2022).

In the United States, a 2020 class action settlement saw Facebook pay $52 million to current and former content moderators — a figure that works out to roughly $1,000 per eligible worker, a sum that barely covers a few therapy sessions. The settlement acknowledged that moderators could develop PTSD but required no structural changes to how the work is organized (Dwoskin, 2020).

These cases matter not because the settlements are adequate — they manifestly are not — but because they establish a principle: companies bear some responsibility for the psychological harm their content supply chains produce. The collective bargaining movement is building on this principle: the African Content Moderators Union, formed in 2023, represents a concrete effort to organize these workers and demand enforceable standards rather than voluntary gestures. The question is whether that principle translates into meaningful change or remains a manageable cost of doing business.

The Redistribution

Strip away the corporate structure, the outsourcing layers, and the euphemistic job titles, and the economics of content moderation are straightforward.

AI companies and social media platforms generate enormous value by providing services that users experience as safe. That safety requires human beings to absorb graphic, traumatic content at industrial scale. The companies locate this work where labor is cheapest — predominantly the Global South — and structure employment to minimize their obligations to the workers.

The result is a direct transfer of harm. Users in wealthy countries get platforms scrubbed of the worst content. Shareholders get profitable companies. AI labs get safety-trained models. Workers in Nairobi, Manila, and Bogota get $1.50 an hour and PTSD.

This is not an accident or an externality. It is the system working as designed. The mental health costs of AI safety are not borne by the companies that profit from it, or by the users who benefit from it. They are borne by the people with the least power to refuse the work or demand better conditions.

What Would Change This

The interventions are not mysterious. They are the same protections that other industries with hazardous working conditions have developed over decades — applied to an industry that has avoided them.

Mandatory mental health support. Not an employee assistance hotline. Comprehensive, ongoing psychological monitoring and treatment, fully funded by the contracting company, continuing for years after employment ends. The cost should be built into the contract price, not treated as optional overhead.

Exposure limits. Regulated, audited caps on how much graphic content a worker can review per day, per week, and over the course of a career. Enforced by regulation, not company policy.

Minimum pay standards. Content moderation is hazardous work and should be compensated as such. If a company's AI safety depends on human reviewers, those reviewers should be paid wages that reflect both the value they create and the harm they absorb — regardless of the country they work in.

Employee classification. Moderators should be employees, not contractors. This means benefits, healthcare, job protections, and — critically — the right to organize. The African Content Moderators Union, formed in 2023, represents a beginning, but collective bargaining requires an employment relationship to function. The EU AI Act and the EU Platform Work Directive are creating legal frameworks that could support reclassification, though implementation remains uneven.

Supply chain liability. AI companies should bear legal responsibility for the mental health outcomes of workers in their content moderation supply chains, whether those workers are employed directly or through contractors. The outsourcing structure should not function as a liability shield.

None of these changes would eliminate the fundamental problem — that AI safety requires human beings to engage with harmful content. But they would redistribute the costs more fairly. Right now, the people who bear the greatest harm receive the least compensation and the fewest protections. That is a policy choice, and it can be changed.

There are models of what better looks like. Some companies — notably in the EU, where regulatory pressure is strongest — have implemented genuine exposure limits, provided long-term psychological monitoring, and paid hazard premiums. These measures are more expensive, but they are not prohibitively so when measured against the revenues of the companies that depend on this labor. The cost of proper mental health support for content moderators is rounding error on Meta's quarterly earnings. The choice not to provide it is exactly that: a choice.

The question, as always with redistribution, is whether those who benefit from the current arrangement have any incentive to change it — and whether the rest of us will insist that they do. The history of occupational health regulation — from coal mines to asbestos to chemical exposure — suggests that industries rarely reform voluntarily. Change comes from organizing, litigation, and legislation. All three are now underway in content moderation. The outcome will depend on whether they build enough momentum to overcome the structural advantages of the companies they challenge.

Sources

  • Dwoskin, Elizabeth. "Facebook to Pay $52 Million in Settlement with Moderators Who Developed PTSD on the Job." The Washington Post, 2020. Link
  • Mureithi, Carlos. "Kenyan Workers Sue Meta Over Alleged Exploitation." Rest of World, 2022. Link
  • Newton, Casey. "The Trauma Floor: The Secret Lives of Facebook Moderators in America." The Verge, 2019. Link
  • Perrigo, Billy. "OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic." TIME, 2023. Link
  • Roberts, Sarah T. Behind the Screen: Content Moderation in the Shadows of Social Media. Yale University Press, 2019. Publisher
← Back to all articles
On this page