Redistributed

General

Doc Review: DeepSeek's Technical Report

Published March 11, 2026

doc-reviewdeepseekchinaopen-sourcetransparency

Documents: DeepSeek-V3 Technical Report (December 2024); DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (January 2025) Source: DeepSeek-V3 Technical Report; DeepSeek-R1 Technical Report

In January 2025, a Chinese AI lab most people had never heard of dropped two technical reports that sent tremors through Silicon Valley. Nvidia lost nearly $600 billion in market value in a single day. OpenAI's CEO called it a "wake-up call." Venture capitalists scrambled to recalculate their assumptions about how much money frontier AI actually requires.

The reports describe DeepSeek-V3, a 671-billion-parameter language model, and DeepSeek-R1, a reasoning model that matches OpenAI's o1 on major benchmarks. What made the AI industry sit up was not just the performance. It was the price tag — $5.576 million in compute costs for V3 — and the fact that DeepSeek published the details at all.

These reports contain more architectural specificity than anything OpenAI, Google, or Anthropic has released for their flagship models. That fact alone deserves attention — not because transparency is bad, but because selective transparency is always a strategy. Understanding what DeepSeek chose to show — and what it chose to withhold — tells you something important about how power operates in the global AI landscape. It also offers genuine lessons about what better technical disclosure could look like.

What These Documents Claim

The core claims across both reports:

DeepSeek-V3 is a Mixture of Experts (MoE) model with 671 billion total parameters, of which only 37 billion are active for any given token. It was trained on 14.8 trillion tokens. The team reports the full training process — pre-training, context extension, and supervised fine-tuning — consumed approximately 2.788 million H800 GPU hours, yielding a compute cost of roughly $5.576 million.

DeepSeek-R1 is a reasoning model trained primarily through reinforcement learning, with minimal supervised fine-tuning. It achieves performance competitive with OpenAI's o1 on mathematics, coding, and scientific reasoning benchmarks. DeepSeek released R1 with open weights under an MIT license.

Both models were trained on clusters of 2,048 Nvidia H800 GPUs — the China-export variant of the H100, throttled by U.S. export restrictions on interconnect bandwidth. For context on how these restrictions shape the global AI landscape, see AI Chip Export Controls.

The implicit claim threading through both reports: frontier AI does not require the capital expenditure that Western labs insist it does. If true, this challenges assumptions explored in Economics of Training a Frontier Model and has direct implications for the scaling debate.

What's Actually In Them

Architecture: Genuine Innovation, Clearly Documented

The V3 report is unusually detailed on architecture. Two innovations stand out.

Multi-head Latent Attention (MLA): A modification to the standard attention mechanism that compresses key-value pairs into a latent space, reducing the memory footprint during inference. The report explains the mathematical formulation, the dimensionality choices, and the performance tradeoffs. For readers unfamiliar with the underlying mechanisms, How Language Models Actually Work provides the foundation. This is the kind of architectural detail that Western labs typically reserve for separate research papers, if they publish it at all.

DeepSeekMoE with auxiliary-loss-free load balancing: Mixture of Experts models route each input to a subset of specialized "expert" networks rather than running the entire model. The perennial challenge is load balancing — ensuring tokens are distributed evenly across experts. DeepSeek introduces a method that avoids the auxiliary loss terms other approaches require, using a bias-based dynamic adjustment instead. The mechanism is explained with enough precision that other researchers could reimplement it.

The V3 report also documents a multi-token prediction objective — training the model to predict multiple future tokens simultaneously rather than just the next one — which improved training signal efficiency. The paper specifies exactly how this was implemented, including the number of additional prediction heads and the training schedule.

The Cost Breakdown: Specific, But Narrow

The $5.576 million figure comes with a clear methodology. DeepSeek reports 2.788 million H800 GPU hours at an assumed rental cost of $2 per GPU hour. They break down the training phases: 2,664K hours for pre-training on 14.8T tokens, 119K hours for context length extension, and 5K hours for post-training. The numbers are internally consistent and the accounting is transparent.

This is more cost transparency than any Western frontier lab has provided for a comparable model. Meta's Llama 3.1 paper, for instance, describes its training infrastructure but does not provide a dollar figure. OpenAI and Anthropic disclose nothing of the sort. This level of cost reporting represents a genuine best practice — one that the rest of the industry could adopt without compromising safety or competitive position.

The R1 Report: A Different Kind of Disclosure

The R1 report focuses on the reinforcement learning training pipeline. The central finding — that RL alone, without supervised fine-tuning on chain-of-thought data, can produce strong reasoning behavior — is documented with ablation studies and staged training results. The report shows R1-Zero (pure RL, no SFT) developing chain-of-thought reasoning spontaneously, then describes how a small amount of cold-start supervised data improved coherence.

The training recipe is described in enough detail to be broadly reproducible: the reward model design, the RL algorithm (Group Relative Policy Optimization), and the distillation process for creating smaller models from R1's outputs. For more on how reasoning models work and why this approach matters, see How Reasoning Models Actually Work.

What's Missing

What DeepSeek chose not to disclose is as significant as what it did. The gaps do not erase the value of what was published — but they define its limits.

Training Data: The Black Box

Neither report provides meaningful information about training data composition. V3 was trained on 14.8 trillion tokens. Of what? The report mentions "a diverse, high-quality dataset" and leaves it there.

This is not a minor omission. Training data determines a model's capabilities, biases, and legal exposure — as Who Owns Training Data explores in depth. For a model trained in China and released globally, the questions multiply. Does the training corpus include copyrighted material from Western publishers? How were Chinese-language datasets curated? Were politically sensitive topics filtered during data preparation or only during post-training alignment? The reports do not say. To be fair, Western labs are no better on this front — but "everyone does it" is an explanation, not a justification.

Alignment and Safety: Conspicuously Thin

The R1 report describes a reinforcement learning process with rule-based and human-evaluated rewards. But the details of human evaluation are absent. Who were the annotators? What guidelines did they follow? How were edge cases adjudicated? What content was deemed off-limits, and by whose standards?

This matters because DeepSeek models demonstrably refuse to engage with topics sensitive to the Chinese Communist Party — Tiananmen Square, Taiwan's sovereignty, Xinjiang. This censorship is not documented in the technical reports. It exists as a known property of the deployed models, visible to any user, yet absent from the documentation that claims to describe how these models were built and trained.

A technical report that documents architecture down to the dimensionality of latent attention vectors but says nothing about political content filtering is making a choice about what "transparency" means. This is worth naming without losing sight of the fact that DeepSeek's architectural disclosure still represents a contribution to the field. The goal should be to hold all labs — Chinese and Western — to standards that include both technical and institutional transparency. See China's AI Strategy for the broader regulatory context.

The True Cost Question

The $5.576 million figure covers the final training run. It does not include the cost of prior research iterations, failed experiments, architecture search, data curation, or the development of the training infrastructure itself. As several researchers noted after the reports' release, the marginal cost of a final training run understates the total investment by potentially an order of magnitude.

DeepSeek is a subsidiary of High-Flyer, a Chinese quantitative hedge fund. The full extent of High-Flyer's investment in DeepSeek's research program is not publicly disclosed. The GPU cluster, reportedly numbering around 10,000 A100s acquired before the tightening of U.S. export controls in addition to the H800s used for training, represents a capital investment the reports never acknowledge.

The $5.576 million is a real number describing a real thing. But framing it as "the cost to train DeepSeek-V3" is like quoting the cost of the final print run of a book while omitting the years of writing, editing, and typesetting. The Economics of Training a Frontier Model explainer examines why total cost accounting matters and how marginal cost claims can mislead.

Governance and Political Context

The reports contain no information about DeepSeek's organizational structure, its relationship to the Chinese government, its compliance with China's Interim Measures for the Management of Generative AI Services, or how Chinese regulatory requirements shaped the model's development. These are not unreasonable omissions for an academic-style technical report — Western labs don't typically address their regulatory environment in research papers either. But Western labs also aren't subject to a regulatory regime that mandates content alignment with "core socialist values."

The absence matters differently when the political context is different.

The Redistribution Question

DeepSeek's reports rearrange assumptions about who can build frontier AI and on what terms. Three redistribution dynamics are in play.

Capital redistribution. If the cost claims hold — even directionally — they undermine the narrative that frontier AI is a game only playable with tens of billions in capital expenditure. That narrative serves incumbents. Every dollar the cost threshold drops, the number of potential competitors rises. DeepSeek's architectural innovations, particularly the MoE efficiency gains and MLA, are now available to anyone reading the paper. The knowledge has been redistributed even if the compute hasn't.

The transparency paradox. DeepSeek published more architectural detail than its Western competitors. A Chinese lab, operating under an authoritarian government with active censorship requirements, produced documentation that is in several measurable respects more open than what comes out of San Francisco. This does not make DeepSeek more trustworthy overall — the governance gaps are real. But it does complicate the self-serving narrative, common among Western labs, that their opacity is necessary for safety. DeepSeek's reports suggest that detailed technical disclosure is possible without the sky falling.

Open weights as power shift. R1's release under an MIT license means any organization — a university lab in Nairobi, a startup in Bangalore, a government agency in Brasilia — can download, modify, and deploy a reasoning model that performs near frontier levels. Projects like EleutherAI, Masakhane, and Hugging Face have already demonstrated how open weights can enable communities that closed models exclude. This is a concrete redistribution of capability. The question — explored in Open Source vs Open Weights and Open (For Business) — is whether the terms of that redistribution serve the recipients' interests, or whether open-weight release is primarily a competitive strategy to commoditize the complement and undermine Western labs' Pricing Power.

What This Tells You

Read these reports the way you'd read a company's annual financial filing: for what's disclosed, for what's omitted, and for the strategy behind the boundary between the two.

DeepSeek's technical reports are genuinely impressive documents. The architecture sections would be strong publications on their own. The cost transparency exceeds industry norms. The open-weight release of R1 is a material contribution to the global AI research ecosystem.

But "more transparent than OpenAI" is not the same as "transparent." The reports are silent on training data provenance, silent on censorship and political content filtering, silent on alignment methodology, silent on governance. These silences are not accidental. They reflect what DeepSeek — and by extension, its operating environment — considers appropriate to disclose.

The lesson is not that DeepSeek is unusually secretive. Most of these gaps exist in Western labs' documentation too — as our reviews of Claude's Model Card, Gemini's Safety Documentation, and OpenAI's Terms of Service demonstrate. The lesson is that technical transparency and institutional transparency are different things, and the AI industry has become skilled at offering the former as a substitute for the latter.

What would a genuinely complete technical report look like? It would combine DeepSeek's architectural detail with training data provenance (per the Datasheets for Datasets standard), labor documentation, environmental accounting, and honest disclosure of content governance policies. No lab currently meets that bar. DeepSeek shows that parts of it are achievable — and that the rest is a matter of will, not capability.

When someone shows you the engine in exquisite detail but won't tell you where the fuel comes from or who's steering, appreciate the engineering while asking about the rest.

Sources

  • DeepSeek-AI. "DeepSeek-V3 Technical Report." arXiv:2412.19437, December 2024. arXiv
  • DeepSeek-AI. "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning." arXiv:2501.12948, January 2025. arXiv
  • Bommasani, Rishi, et al. "The Foundation Model Transparency Index (v1.1)." Stanford HAI, May 2024. Stanford HAI
  • Mitchell, Margaret, et al. "Model Cards for Model Reporting." Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019. ACM
  • State Council of the People's Republic of China. "Interim Measures for the Management of Generative Artificial Intelligence Services." July 2023. China Law Translate
← Back to all articles
On this page