Redistributed

General

Open Weights Are Not Open Source

Published March 11, 2026

pitfallopen-sourceopen-weightsmisconceptionpower

When Meta releases Llama, the press writes "Meta open-sources its latest AI model." When Mistral drops a new set of weights, the headline reads "another open-source challenger to GPT." The language is consistent, familiar, and wrong.

What these companies release is open weights — the numerical parameters that define a trained model's behavior. That's a meaningful thing to share. But calling it "open source" borrows credibility from a movement with a precise definition and a forty-year history, then applies it to something that meets almost none of the criteria.

The distinction isn't pedantic. It determines who actually controls the technology, who can audit it, and who can build on it independently. In redistribution terms: open weights share utility while retaining power. Understanding the spectrum of openness helps users, policymakers, and developers make more informed choices — and helps genuinely open projects get the recognition they deserve.

What Open Source Actually Means

The term has a specific definition, formalized by the Open Source Initiative in 1998 and rooted in the Free Software Foundation's four freedoms articulated by Richard Stallman in the 1980s:

  1. Freedom to run the program for any purpose.
  2. Freedom to study how the program works, and change it.
  3. Freedom to redistribute copies.
  4. Freedom to distribute modified versions to others.

These freedoms require access to source code — the human-readable instructions from which the software is built. With source code, you can understand what a program does, verify its behavior, fix its flaws, and build something new. Without it, you're a consumer, not a participant.

The open-source software movement produced Linux, Apache, Firefox, Python, and most of the infrastructure the internet runs on. The model works because source code is the complete blueprint. If you have it, you can reproduce the artifact.

What Open Weights Actually Provides

A machine learning model is defined by its weights — billions of numerical values that encode patterns learned during training. When a company releases "open weights," it typically provides:

  • The trained model weights (often in a format like safetensors or GGUF)
  • Architecture details (how the layers are structured)
  • Some documentation on capabilities and intended use
  • Sometimes a model card describing evaluation results

With these, you can run the model, generate outputs, fine-tune it for specific tasks, and deploy it in your own applications. This is genuinely useful — and worth celebrating as a meaningful step. Developers worldwide build products on Llama and Mistral weights. Researchers use them to study model behavior. Projects like Ollama and Running AI Locally depend on open-weight releases. The release creates real value.

But it does not create the conditions that made open-source software transformative.

What's Missing

Training data

You don't know what the model was trained on. Meta has never fully disclosed Llama's training dataset. You know it includes "publicly available online data" — a phrase that conceals more than it reveals.

Without training data, you cannot:

  • Reproduce the model. You can't retrain it from scratch to verify the results.
  • Audit for bias. You can't check whether the training data overrepresents certain demographics, geographies, or viewpoints.
  • Assess copyright exposure. You can't determine whether the model memorized copyrighted text, personal data, or other material with legal implications.
  • Improve the foundation. You can fine-tune the surface, but the deep patterns baked in during pretraining remain opaque.

This is roughly equivalent to releasing a compiled binary and calling it open source. You can run it, but you can't see how it was built.

Training code and process

Even with the data, reproducing a model requires the full training recipe: hyperparameters, data preprocessing pipelines, curriculum design (what data is shown in what order), the training infrastructure, and the tricks that made optimization converge. Most of this remains proprietary. Training a frontier model costs tens of millions of dollars in compute. The recipe is as valuable as the ingredients.

Alignment and safety data

After pretraining, models go through reinforcement learning from human feedback (RLHF) or similar alignment processes. The human preference data that shapes a model's behavior — what it refuses, how it hedges, which values it expresses — is rarely released. This is arguably the most consequential part of The Training Pipeline, the layer where human judgments get encoded into system behavior, and it's largely opaque. (Some labs, including Anthropic, have published details about their alignment approach — a step toward transparency, though far from complete.)

Actual freedom

Llama's license — the Llama Community License — imposes significant restrictions. If your product has more than 700 million monthly active users, you need a separate commercial agreement from Meta. Certain use cases are explicitly prohibited. You cannot use the model to train other models that compete with Meta's offerings.

These are reasonable business decisions. They are not open source. The Open Source Definition's first clause requires "free redistribution" with no restriction on selling or giving away the software. Llama fails this on its face.

Why Companies Do This Anyway

In a 2023 paper, David Widder, Sarah West, and Meredith Whittaker analyzed the strategic logic behind corporate "openness" in AI. Their conclusion is blunt: open release strategies by large technology companies primarily serve corporate interests, not public ones (Widder et al., 2023).

Meta's calculus with Llama is legible. By releasing capable model weights for free, Meta:

  • Commoditizes the model layer that competitors like OpenAI charge for, undermining their business model
  • Drives adoption of Meta's ecosystem — developers who build on Llama become dependent on Meta's releases, fine-tuning tools, and optimization work
  • Generates goodwill and talent attraction from a research community that values openness
  • Gets free labor from the community in the form of fine-tunes, benchmarks, bug reports, and deployment testing

This is what critics call open-washing: using the language and social capital of open source to serve a strategy that concentrates rather than distributes power. The weights are free. The control is not. (For more on how market structure shapes these dynamics, see Frontier Lab Oligopoly and Open (For Business).)

The OSI's Attempt at a Definition

Recognizing the confusion, the Open Source Initiative published its Open Source AI Definition (OSAID) in late 2024, attempting to establish what "open source" should mean for AI systems. The definition requires sufficient information to allow a "skilled person to substantially recreate" the system, including data information, code, and model parameters.

The response was polarized. Critics, including some within the OSI's own process, argued the definition was too permissive — particularly because it does not strictly require the release of training data, only "sufficiently detailed" information about it. Stefano Maffulli, OSI's executive director, defended the pragmatic compromise. Others saw it as a capitulation to corporate interests that would never agree to full data transparency (Maffulli, 2024).

By the OSI's own definition, most "open" model releases — including Llama — don't qualify. But the definition's compromises mean that even meeting it wouldn't guarantee the kind of reproducibility and auditability that open-source software provides.

Who Benefits, Who Pays

Who benefits from the confusion:

  • Meta and similar companies gain ecosystem control and competitive advantage while claiming the moral high ground of openness.
  • Developers and companies building products on open-weight models get powerful capabilities at zero licensing cost. This is real value — but it comes with dependency.
  • Researchers get access to capable models for study and experimentation, accelerating certain kinds of work.

Who pays:

  • The open-source movement loses definitional clarity. When "open source" can mean anything from full GPL-licensed code to a restricted-license binary blob of weights, the term's power to demand transparency erodes.
  • Workers and creators whose data trained these models — scraped from the public internet, often without knowledge or consent — get no say in how the resulting systems are used or who profits.
  • Genuine open efforts like EleutherAI, BLOOM, and OLMo, which release training data, code, and documentation alongside weights, get lumped in with corporate open-washing, making it harder to distinguish real openness from strategic marketing. These projects demonstrate that full openness is technically feasible — the barriers are commercial, not engineering constraints.

The Better Frame

Openness in AI isn't binary. It's a spectrum with distinct dimensions, and each dimension has different implications for power and accountability.

Rather than asking "is this model open source?" — a question that invites a misleading yes — ask specific questions:

  • Are the weights available? (Can you run and fine-tune the model?)
  • Is the training data disclosed? (Can you audit what shaped its behavior?)
  • Is the training code released? (Can you reproduce or improve the process?)
  • Is the license permissive? (Can you actually use it freely, or are there strings?)
  • Is the alignment process documented? (Can you understand whose values are embedded?)

A model can score high on some dimensions and low on others. Llama scores well on weight availability and poorly on everything else. BLOOM and OLMo score well across the board. GPT-4 scores zero on all counts.

This granularity matters because it tracks where power actually sits. Releasing weights distributes capability. Releasing data and code distributes understanding. Releasing everything with a permissive license distributes control. These are different things, and conflating them serves the interests of those who want credit for the first while withholding the rest.

Next time you read that a company has "open-sourced" its AI model, check what they actually released. The gap between the headline and the reality is where power hides. And when you find projects that genuinely open everything — support them. They are building the infrastructure for a more public AI ecosystem.

Sources

  • Widder, David Gray, Sarah West, and Meredith Whittaker. "Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI." Social Science Research Network, August 2023. SSRN
  • Open Source Initiative. "The Open Source AI Definition — 1.0." October 2024. OSI
  • Groeneveld, Dirk, et al. "OLMo: Accelerating the Science of Language Models." Allen Institute for AI, February 2024. arXiv
  • Maffulli, Stefano. "The Open Source AI Definition is Out." Open Source Initiative Blog, October 2024. OSI Blog
  • Stallman, Richard. "What is Free Software?" GNU Project, Free Software Foundation. GNU
← Back to all articles
On this page