Redistributed

Standard language models generate text one token at a time, optimising for the most plausible next word. Reasoning models do something different: they spend compute thinking before they answer. The distinction sounds subtle. The consequences — for performance, for cost, and for who gets access to capable AI — are not.

The Core Mechanism

If you've read How Language Models Actually Work, you know that a standard LLM is a next-token prediction machine. It takes your prompt, processes it through billions of parameters, and generates a response word by word. It does this in a single forward pass — one shot at getting the answer right.

Reasoning models add a step. Before producing the final answer, they generate an internal chain of thought: a sequence of intermediate reasoning steps that break the problem down, check partial results, and sometimes backtrack when a line of reasoning fails. OpenAI's o1 and o3, DeepSeek's R1, and Anthropic's extended thinking for Claude all implement variations of this approach.

Think of the difference this way. A standard model answering a maths problem is like a student writing their answer immediately. A reasoning model is like a student who works through the problem on scratch paper first — trying one approach, checking if it makes sense, trying another if it doesn't — and only then writing the final answer.

That scratch paper is the reasoning trace.

Test-Time Compute vs Training-Time Compute

This distinction matters technically, but it also matters economically.

Traditional improvements to AI came from training-time compute — bigger datasets, more parameters, longer training runs. You spend more upfront, and every user benefits from the smarter model. The cost is borne once and amortised across millions of queries.

Reasoning models shift significant compute to test-time — also called inference time. Every individual query triggers its own chain of thought, consuming tokens and GPU cycles that scale with the difficulty of the problem. A simple factual question might generate a brief reasoning trace. A hard maths proof might generate thousands of internal tokens before producing an answer.

This is a fundamentally different cost structure. Training-time compute is a fixed investment. Test-time compute is a variable cost that scales with usage and difficulty. As Snell et al. (2024) demonstrated, optimally allocating compute at test time can outperform simply scaling up model size — but someone has to pay for every reasoning step.

How the Thinking Actually Works

The internal mechanics vary across implementations, but the common elements are:

Chain-of-thought generation. The model produces a sequence of reasoning steps, often in natural language. These might include restating the problem, identifying sub-problems, working through calculations, or considering alternative approaches. Research by Wei et al. (2022) showed that simply prompting large models to "think step by step" dramatically improved performance on arithmetic, commonsense, and symbolic reasoning tasks — a finding that launched an entire paradigm.

Self-verification. Reasoning models don't just generate a chain of thought — they check their work. The model may re-examine a step, notice an inconsistency, and revise. This is where process reward models (PRMs) come in: during training, the model learns to evaluate not just whether the final answer is correct, but whether each intermediate step is valid (Lightman et al., 2023). This is analogous to a teacher grading the working, not just the answer.

Backtracking. When a reasoning path hits a dead end, the model can abandon it and try a different approach. This is a significant departure from standard autoregressive generation, where each token commits the model further down a single path. Backtracking gives reasoning models a form of search — exploring multiple solution strategies rather than committing to the first one.

Reinforcement learning. Models like OpenAI's o1 and DeepSeek-R1 are trained using reinforcement learning, where the model is rewarded for producing correct final answers through valid reasoning chains. DeepSeek-R1 is notable for demonstrating that large-scale reinforcement learning alone — without supervised fine-tuning on human-written chains of thought — can produce emergent reasoning behaviours including self-verification and reflection (DeepSeek-AI, 2025). The model essentially learned how to think by being rewarded for thinking well. (For context on how reinforcement learning fits into the broader model development process, see The Training Pipeline and What Fine-Tuning Really Does.)

The Performance Gains Are Real

Reasoning models represent a genuine capability jump in specific domains:

Mathematics. OpenAI's o1 scored in the 89th percentile on competitive maths problems (AMC), up from roughly the 50th percentile for GPT-4. On the notoriously difficult AIME exam, o3 achieved scores that would place it among top human competitors.

Coding. On programming competition benchmarks like Codeforces, reasoning models moved from amateur to expert-level ratings. The ability to plan, debug, and iterate through a solution — rather than generating code in a single pass — mirrors how human programmers actually work.

Science. On the GPQA benchmark of graduate-level science questions (physics, chemistry, biology), reasoning models surpassed the average performance of domain PhD holders.

These aren't marginal improvements. They represent the difference between a system that can assist with routine tasks and one that can engage with genuinely hard problems. Worth noting: benchmarks have known limitations, and performance on competition problems does not straightforwardly predict reliability in real-world applications. But the trajectory is clear.

The Cost: 5-50x More Tokens Per Query

Here's where the redistribution question bites.

A standard GPT-4 query might generate a few hundred tokens. A reasoning query on the same problem might generate 5,000 to 50,000 tokens internally — most of which the user never sees — before producing a final answer. Each of those tokens costs compute.

The practical result: reasoning models are dramatically more expensive to run. OpenAI's o1-pro, available only to $200/month ChatGPT Pro subscribers, costs roughly 6x more per token than standard GPT-4o. For API users, reasoning model pricing can be 10-30x higher depending on the complexity of the task and the length of the reasoning trace.

This isn't a temporary premium that will disappear with scale. The cost is structural. Test-time compute scales with query difficulty, and the hardest problems — the ones where reasoning models shine brightest — are precisely the ones that consume the most resources.

The Redistribution Angle: A Two-Tier AI Is Forming

This cost structure is producing a familiar pattern: stratification by ability to pay.

Major AI providers now operate tiered systems. The base tier offers standard language models — fast, cheap, good enough for many tasks. The premium tier offers reasoning models — slower, expensive, dramatically better at complex work. OpenAI's pricing makes this explicit: free users get GPT-4o, paying users get o1, and $200/month subscribers get o1-pro with "more compute" per query.

This mirrors stratification we've seen in every essential service that becomes infrastructure (a pattern explored in How Technology Gets Distributed):

Healthcare. The difference between a rushed 10-minute GP appointment and a specialist who takes time to reason through a complex diagnosis.

Education. The difference between a 200-student lecture and a tutorial with a professor who asks probing follow-up questions and checks your working.

Legal services. The difference between a template contract and a lawyer who anticipates edge cases, stress-tests arguments, and thinks through consequences.

In each case, the premium version involves more thinking time applied to your specific problem. Reasoning models are the AI equivalent. And as AI increasingly mediates access to expertise — in coding, analysis, research, medical information, legal understanding — the gap between "AI that generates plausible text" and "AI that actually reasons through your problem" becomes a gap in real-world capability.

As Virginia Eubanks documented in Automating Inequality, technological systems that appear neutral tend to encode and amplify existing disparities. The reasoning tier doesn't create inequality from nothing. It maps onto an existing landscape where enterprises can afford $200/month seats and API costs while individuals and small organisations often cannot.

Open Models as Partial Equaliser

There's a counterweight — and it's an important one. DeepSeek-R1, released in January 2025 with open weights, demonstrated that reasoning capability doesn't have to be locked behind proprietary APIs (see DeepSeek and the Price of Open AI for the broader implications). The model performs competitively with OpenAI's o1 on many benchmarks, and its weights can be downloaded and run by anyone with sufficient hardware.

"Sufficient hardware" is doing real work in that sentence. Running a full-scale reasoning model locally requires high-end GPUs — thousands of dollars of equipment at minimum. But distilled versions of R1, fine-tuned into smaller models, can run on more modest setups. And the existence of open reasoning models means that hosted alternatives — from cloud providers, from academic institutions, from community projects — can emerge without depending on a single vendor's pricing decisions.

This matters. The history of technology infrastructure shows that openness doesn't automatically produce equity, but proprietary lock-in reliably produces concentration. Open reasoning models are necessary, even if not sufficient, for preventing a permanent two-tier system.

What This Means for You

If you're deciding whether to use a reasoning model, the calculus is straightforward:

Use reasoning models for problems with verifiable right answers — maths, coding, logic, analysis with clear evaluation criteria. These are the domains where the extra compute pays off.

Standard models are fine for drafting, summarisation, brainstorming, conversation, and tasks where "good enough" is genuinely good enough. Most writing tasks don't benefit from 10x the compute.

Watch the cost. If you're building applications that call reasoning models via API, a single complex query can cost 50-100x what a standard query costs. Design systems with fallbacks: use a standard model for triage, and route only genuinely hard problems to the reasoning tier.

Consider open alternatives. If you're running workloads at scale, DeepSeek-R1 and its derivatives offer reasoning capability without per-query API costs — though you'll pay in infrastructure instead. Running AI Locally covers the practical details of self-hosted inference.

The deeper question isn't technical. It's structural. As reasoning capability becomes the dividing line between AI that assists and AI that genuinely augments human capability, we should pay attention to who falls on which side of that line — and whether the division is an engineering necessity or a business decision.

Sources

Wei, Jason, et al. "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." NeurIPS, 2022. arXiv
Lightman, Hunter, et al. "Let's Verify Step by Step." arXiv, 2023. arXiv
Snell, Charlie, et al. "Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters." arXiv, 2024. arXiv
DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." arXiv, 2025. arXiv
Eubanks, Virginia. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin's Press, 2018. Publisher

General

How Reasoning Models Actually Work