The RLHF Workforce: Who Trains the Trainers

When you ask an AI model a question and it gives you a clear, helpful, non-toxic answer, you're experiencing the product of human judgment — thousands of individual decisions made by workers you'll never see, working for wages you'd find hard to live on.

Reinforcement Learning from Human Feedback — RLHF — is the process that transforms a raw language model into something useful. The model starts as a statistical engine that predicts the next word. RLHF is what makes it helpful. Human workers compare model outputs, rank responses by quality, flag harmful content, and evaluate reasoning. Their collective judgment becomes the model's behavior. Understanding How Language Models Actually Work makes this clearer: without RLHF, these systems are autocomplete at scale. With it, they become products worth billions. The gap between those two states is measured in human labor.

This is not a minor refinement. It's the difference between a model that generates plausible text and one that people actually pay for. RLHF is where commercial value is created. And the workers who create that value capture almost none of it.

How RLHF Works

The technical process has three stages, each dependent on human labor.

Stage one: supervised fine-tuning. Human annotators write ideal responses to prompts. The model learns to imitate these demonstrations. This requires workers who can write clearly, reason carefully, and understand what a "good" answer looks like across domains — from coding to history to medical questions.

Stage two: comparison and ranking. The model generates multiple responses to the same prompt. Human workers read them all and rank them from best to worst. Which answer is more accurate? More complete? More appropriately cautious? These judgments require genuine expertise — not just literacy, but the ability to evaluate factual claims, detect subtle errors, and assess reasoning quality.

Stage three: reward modeling and reinforcement learning. The human rankings train a "reward model" that scores outputs automatically. The language model is then optimized against this reward signal. The reward model is, in effect, a compressed version of thousands of human judgments. The workers' expertise is distilled, abstracted, and deployed at scale — without ongoing compensation.

Each stage requires different skills, but all demand what the industry insists on calling "annotation" — a word that minimizes the cognitive work involved. Evaluating whether a model's explanation of quantum entanglement is accurate, or whether its summary of a legal case captures the key issues, is not annotation. It's expert review, paid like data entry. As What Fine-Tuning Really Does explains, this human feedback is what distinguishes a commercial product from a research artifact — the Training Pipeline depends on it entirely.

Who Does This Work

The RLHF workforce is globally distributed but concentrated in specific corridors. Companies like Scale AI, Surge AI, Appen, and Remotasks recruit workers in Kenya, the Philippines, India, Pakistan, and across Latin America. AI labs including OpenAI, Anthropic, Google DeepMind, and Meta also contract RLHF labor, both directly and through intermediaries.

The geography is not accidental. It follows the established logic of business process outsourcing: find educated, English-speaking workers in countries where prevailing wages are low. Kenya produces university graduates who earn a few dollars a day in local job markets. The Philippines has a massive BPO sector with millions of English-proficient workers accustomed to serving American clients. These are pools of skilled labor that can be accessed cheaply.

As Kate Crawford documents in Atlas of AI, this pattern repeats across AI's supply chain. The computational infrastructure depends on mineral extraction in the Global South. The training data is labeled by workers in the Global South. And the behavioral refinement — RLHF — follows the same extractive geography. "The cloud is not weightless," Crawford writes. Neither is the labor that shapes it.

The numbers tell the story of geographic wage arbitrage:

In 2023, TIME reported that Sama, an outsourcing firm contracted by OpenAI, paid Kenyan workers between $1.32 and $2 per hour to label toxic content and evaluate model outputs for ChatGPT (Perrigo, 2023). Workers described reviewing descriptions of child abuse, bestiality, and violence — content the model needed to learn to refuse. The psychological toll was severe. The contract was terminated early, but the model retained every judgment those workers made.

At the other end of the spectrum, US-based RLHF workers recruited through platforms like Scale AI or directly by AI labs report rates of $15 to $25 per hour — sometimes higher for specialized domains like medicine or law. This is better, but still modest for work that requires expert-level judgment and directly shapes products worth billions.

The gap between $1.50 in Nairobi and $20 in San Francisco is not a market inefficiency. It is the market working exactly as designed. The same task, the same cognitive demand, the same contribution to model quality — valued at ten to fifteen times more depending on which side of the global labor market you sit on. Abeba Birhane's framework of "algorithmic colonialism" applies directly: data and labor are extracted from the Global South while value accumulates in the North (Birhane, 2020). The broader pattern of data labeling in the Global South follows the same extractive geography, as does the content moderation workforce.

The Skill Mismatch

There is a persistent contradiction at the heart of RLHF labor. The work requires genuine expertise. The compensation treats it as unskilled.

Consider what an RLHF worker actually does. They read a prompt asking about the French Revolution, then evaluate three model responses for historical accuracy, completeness, and clarity. They assess whether a code snippet actually solves the stated problem. They judge whether a model's medical information is safe to follow. They identify subtle logical fallacies in an argument about economic policy.

This is knowledge work. It requires education, domain familiarity, reading comprehension, critical thinking, and careful judgment. Many RLHF workers hold university degrees. Some hold advanced degrees. They are not clicking buttons on a mechanical task — they are making intellectual evaluations that directly determine how well a multibillion-dollar product performs.

Yet the structure of the work ensures it's compensated as piecework. Tasks are atomized: evaluate this pair of responses, rate this output on these dimensions, flag any safety issues in this generation. Each microtask takes minutes and pays cents or small dollars. The intellectual contribution is fragmented until it looks like unskilled labor on a spreadsheet.

Mary Gray and Siddharth Suri identified this dynamic in Ghost Work: platform-mediated labor is designed to be invisible. The fragmentation serves two purposes — it makes the work manageable at scale, and it obscures the skill involved. When a task takes three minutes and pays forty cents, it doesn't look like expert review. It looks like clicking (Gray & Suri, 2019).

Scale AI and the Intermediary Problem

Scale AI, valued at $13.8 billion as of its 2024 funding round, sits at the center of the RLHF labor market. The company contracts with major AI labs — OpenAI, Meta, the US Department of Defense — and recruits a global workforce to perform data labeling, RLHF evaluation, and other human-in-the-loop tasks.

The business model is straightforward intermediation. Scale AI charges AI labs premium rates for high-quality human feedback. It pays workers a fraction of those rates. The difference — a substantial margin — accrues to Scale AI and its investors.

This is not unique to Scale AI. Appen, Surge AI, Labelbox, and other data labeling companies operate similarly. But Scale AI's valuation makes the arithmetic stark. The company is worth $13.8 billion. Its value derives entirely from organizing human labor to produce training data and feedback. The workers who perform that labor have no equity in Scale AI, no profit-sharing arrangement, no royalties on the models their work improves. They are, in economic terms, the means of production — compensated as variable costs.

The intermediary structure also insulates AI labs from accountability. When OpenAI's Kenyan workers reported trauma and low pay, the contractor — Sama — absorbed the reputational damage. OpenAI could distance itself: we didn't employ those workers, we contracted with a company that did. This is the same subcontracting logic that insulates brands from sweatshop conditions in garment manufacturing. The supply chain is designed to distribute responsibility until no one is responsible.

The Data Labeling Pipeline

Understanding how RLHF labor is organized reveals why workers have so little leverage.

Task design: AI labs or their contractors define the annotation schema — what workers should evaluate, what dimensions matter, what scales to use. Workers have no input into task design, even though their on-the-ground experience often reveals flaws in the schema.

Recruitment: Workers are recruited through platforms, often with minimal screening. Qualification tasks — unpaid — filter candidates. Many workers spend hours on qualification tasks they don't pass, receiving no compensation for that time. The surplus of available labor in target countries means workers are easily replaceable.

Annotation: Workers complete tasks in a platform interface. They have limited context about what the data is for, which model it will train, or how their judgments will be used. The work is isolated — no collaboration, no collective deliberation, just individual workers making individual judgments.

Quality review: Automated systems and spot-checks assess worker quality. Workers whose judgments diverge too far from consensus are flagged, retrained, or removed. This creates pressure toward conformity, which may not always serve quality — minority viewpoints and careful dissent get coded as error.

Integration: The aggregated human judgments are integrated into the training pipeline. At this point, the labor is fully abstracted. It exists as data points in a dataset — anonymous, undifferentiated, disconnected from the humans who produced it. Workers have no ownership claim on the training data they create, no attribution, no mechanism to trace their contribution through to the final product.

This pipeline is efficient. It is also designed to minimize worker power at every stage. Workers don't choose what to evaluate, can't negotiate rates, have no visibility into how their work is used, and can be replaced without notice. The platform mediates everything.

The Automation Paradox

Here is the cruel irony of RLHF work: the workers are training the systems that may replace them.

AI-assisted labeling is already reducing demand for some forms of human annotation. Models can now generate initial labels that human workers verify — faster and cheaper than human-only annotation. Constitutional AI, developed by Anthropic, uses AI-generated feedback to partially substitute for human RLHF. Synthetic data generation reduces the need for human-created training examples.

This doesn't eliminate the need for human judgment — edge cases, novel domains, and quality assurance still require people. But it reduces the volume of human labor needed per unit of model improvement. The workers whose judgments made these advances possible are the first to be displaced by them.

Daron Acemoglu's concept of "so-so automation" is relevant here. Automating RLHF tasks doesn't necessarily improve quality — AI-generated feedback may be cheaper but less nuanced than human judgment. It's automation that reduces costs without clear productivity gains. The savings come from replacing human workers, not from doing the work better (Acemoglu, 2021).

For RLHF workers, this creates a perverse dynamic. Perform well, and your judgments train a system that needs you less. The better the models get at evaluating their own outputs, the fewer human evaluators are needed. Unlike workers who build durable physical infrastructure, RLHF workers are building their own obsolescence.

What Workers Are Organizing For

Despite the structural barriers, RLHF and data labeling workers are beginning to organize.

The African Content Moderators Union, formed in Nairobi in 2023, represents workers who perform content moderation and RLHF tasks for major AI companies. Their demands are concrete: a minimum hourly wage of $10, access to mental health support, transparent task allocation, the right to know which company and which product their labor serves, and recognition as skilled workers rather than disposable contractors.

In Kenya, Daniel Motaung, a former Sama content moderator, filed suit against Sama and Meta, alleging exploitative conditions and inadequate support for workers exposed to traumatic content. The case brought international attention to the working conditions behind AI development (Mureithi, 2022).

Workers on platforms like Remotasks and Scale AI have used forums and social media to share information about pay rates, task availability, and platform policies — building informal solidarity networks that the platforms' design actively discourages.

These efforts face significant obstacles. Workers are distributed across countries, employed through intermediaries, and classified as independent contractors. Traditional labor organizing assumes a shared workplace, an identifiable employer, and legal protections for collective action. RLHF workers often have none of these.

What Fair Compensation Would Look Like

If RLHF workers are creating the value that makes AI models commercially viable — and they are — then compensation should reflect that contribution. Several models have been proposed:

Profit-sharing. Workers whose feedback trains a specific model could receive a percentage of that model's revenue. This is technically feasible — platforms already track which workers contributed to which datasets. It would require AI companies to treat labor as a form of investment rather than an operating expense.

Minimum pay standards. Industry-wide minimums, pegged to the cost of living in workers' locations or to a global floor, would prevent the race to the bottom. The Partnership on AI published voluntary guidelines for data enrichment work in 2024, including minimum compensation standards. Adoption has been limited.

Equity stakes. Workers could receive equity in the companies they serve, directly or through trusts. This would give RLHF workers a stake in the long-term value they help create. It would also align incentives: workers who benefit from model success have reason to invest in quality.

Attribution and data rights. Workers could retain rights over the training data they produce, licensing it to AI companies on terms that include ongoing compensation. This mirrors how musicians receive royalties when their recordings are used — the contribution continues to generate value, so compensation should be continuous.

Mental health infrastructure. Workers exposed to harmful content — a routine part of safety-focused RLHF — deserve employer-funded mental health support, exposure limits, and hazard premiums. This is not a benefit; it's an occupational safety requirement.

None of these proposals are radical. Profit-sharing, equity compensation, occupational health standards, and attribution rights are established features of other industries. What makes them seem radical in AI is the assumption, embedded in the industry's structure, that RLHF labor is a commodity to be purchased at the lowest possible price rather than a skilled contribution to be fairly compensated.

The most promising model operating today is Karya, a nonprofit data annotation platform in India that pays workers approximately 20 times the prevailing local wage, shares profits from data licensing, and lets workers retain rights over the data they produce. It demonstrates that dignified compensation is not just morally necessary — it is economically feasible. The challenge is making the exception the norm, which requires the kind of industry-wide standards that the collective bargaining movement and policy frameworks like the EU AI Act are beginning to establish.

The Redistribution Question

The story of RLHF labor is a redistribution story in miniature.

Value flows upward. Workers in Nairobi, Manila, and Lahore generate the human judgments that make AI models useful. Those models generate billions in revenue for companies headquartered in San Francisco. The workers receive subsistence wages. The companies receive historic valuations. Scale AI, the intermediary, captures $13.8 billion in value for the act of organizing the labor. The workers who perform the labor capture almost nothing.

Risk flows downward. Workers bear the psychological costs of exposure to toxic content, the economic precarity of gig-style employment, and the looming threat of automation. Companies bear minimal risk — contractors absorb labor disputes, liability, and reputational damage.

Credit is invisible. No one using ChatGPT, Claude, or Gemini sees the human labor that shaped its responses. The marketing says "AI." The reality is thousands of people making millions of judgments, then being erased from the story.

This pattern is not new. Ursula Franklin observed that prescriptive technologies — systems that divide labor into small, manageable steps — tend to concentrate control while distributing the work. RLHF is a prescriptive technology par excellence: the process is designed so that no individual worker has enough context to exercise real autonomy, while the companies that design and deploy the system retain full control over the product (Franklin, 1989).

What would redistribution look like? Workers with a stake in the models they train. Compensation that reflects the value created, not the local minimum wage. Transparency about who does this work and under what conditions. Legal frameworks that hold AI companies accountable for labor conditions throughout their supply chains, not just within their corporate offices.

The models are getting better. The companies are getting richer. The question is whether the workers who make both possible will share in any of it.

Related

• Hidden Labor
• Ghost Work
• Manager Is Algorithm
• Data Labeling Global South
• Content Moderation Mental Health
• Training Pipeline
• What Fine-Tuning Really Does
• How Language Models Actually Work
• Atlas of AI
• AI Cooperatives
• Collective Bargaining

Sources

• Acemoglu, Daron. "Harms of AI." Cambridge Journal of Economics, 2021. Cambridge
• Birhane, Abeba. "Algorithmic Colonization of Africa." SCRIPTed 17, no. 2 (2020). SCRIPTed
• Crawford, Kate. Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Yale University Press, 2021. Publisher
• Franklin, Ursula. The Real World of Technology. House of Anansi Press, 1989. Publisher
• Gray, Mary L. and Siddharth Suri. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt, 2019. Publisher
• Mureithi, Carlos. "Kenyan Workers Sue Meta Over Alleged Exploitation." Rest of World, 2022. Link
• Partnership on AI. "Responsible Practices for Synthetic Media." 2024. PAI
• Perrigo, Billy. "OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic." TIME, January 2023. Link
• Tubaro, Paola, Antonio A. Casilli, and Marion Coville. "The Trainer, the Verifier, the Imitator: Three Ways in Which Human Platform Workers Support Artificial Intelligence." Big Data & Society, 2020. Sage