A grassroots collective that trained and released GPT-Neo and GPT-J — proving open research could match corporate labs. The original insurgency against closed AI, born from a Discord server.
What It Is
EleutherAI started in a Discord server in 2020, born from frustration that GPT-3 was inaccessible. A group of volunteers decided to replicate it openly. They succeeded — releasing GPT-Neo, GPT-J, and GPT-NeoX as fully open alternatives.
The collective operates without formal hierarchy or employment structure. Contributors volunteer time. Compute comes from donations and partnerships. Models, code, and datasets are released with permissive licenses.
Beyond model training, EleutherAI created The Pile — a 825GB text dataset assembled from diverse sources specifically for language model training. This dataset became a foundation for other open projects.
Why It Matters for Redistribution
EleutherAI proved a point: frontier AI research doesn't require a billion-dollar company.
Who benefits? Researchers get fully open models to study, modify, and build on — not just weights behind an API. Startups get foundations without the licensing complexity of corporate "open" models. The research community gets reproducibility that closed labs can't provide.
Who pays? Contributors donate time. Compute sponsors (CoreWeave, Stability AI at various points) donate resources. The model is donation-funded, which creates dependencies but avoids the capture that comes with conventional funding.
What changes? EleutherAI demonstrated an alternative to corporate AI labs. When they released GPT-J-6B in 2021, it was the largest open language model available. This created pressure on commercial labs and established that "open" was viable at scale.
The collective also pioneered transparency practices that commercial labs now follow. Model cards, training documentation, and dataset disclosure became expectations partly because EleutherAI proved they were possible.
EleutherAI's story illuminates a recurring tension in Open Source vs. Open Weights: they released genuinely open models (weights, training code, data), but the subsequent wave of corporate "open" releases (Meta's Llama, for instance) adopted the aesthetics of openness without the substance. EleutherAI set the standard; the industry moved the goalposts. As Open (For Business) examines, the difference between open-for-research and open-for-marketing matters enormously for redistribution.
The Pile dataset also forced a conversation about Who Owns the Training Data — assembled from diverse sources, it made explicit what corporate training runs obscured: language models are built from other people's work.
Current Status
Active but evolved. The early urgency of "we need open alternatives to GPT-3" succeeded — open models now exist at every scale. EleutherAI shifted toward research tooling (lm-evaluation-harness is widely used), technical papers, and supporting the broader ecosystem.
Key alumni went on to found or join other AI labs, spreading the open research ethos. The model of volunteer-driven AI research proved influential beyond EleutherAI itself.
How to Get Involved
- Use the models: Available through Hugging Face
- Join the community: Discord server — active discussions, open to newcomers
- Contribute to research: Working groups tackle specific projects
- Use the tools: lm-evaluation-harness for model evaluation
Related
- Hugging Face
- BLOOM
- Open Source vs. Open Weights
- Open Weights Are Not Open Source
- Open (For Business)
- The Frontier Lab Oligopoly
- Who Owns the Training Data
- Compute Is the New Oil