Common Pitfalls

One misconception at a time, dismantled carefully.

pitfalls

AI Benchmarks Don't Measure What You Think

Model X scores 90.1% on MMLU. Model Y scores 88.7%. Therefore Model X is smarter, more capable, the better choice. This is how AI labs sell their products, how journalists rank models, and how many users pick their tools.

pitfallbenchmarksevaluation
pitfalls

No, AI Doesn't Learn Like a Brain

The "neural" in neural network was a marketing decision from the 1940s. As Gary Marcus argues in his critical appraisal, the resemblance to biological brains is superficial at best (Marcus, 2018). The analogy has been productive in some ways — it inspired architectures that genuinely work — but it misleads in consequential ways when taken literally. Understanding where the metaphor breaks down helps us reason more clearly about what these systems can and cannot do.

pitfalllearningneuroscience
pitfalls

AI Won't Solve Climate Change by Itself

The pitch is seductive: AI will optimize our power grids, discover new battery materials, model climate systems with unprecedented accuracy, and make agriculture efficient enough to feed ten billion people on a warming planet. The technology that reshapes everything will reshape our relationship with the atmosphere too. Microsoft, Google, and a parade of climate-tech startups have built entire narratives around this premise.

pitfallclimateenvironment
pitfalls

The Disruption Narrative Hides Who Gets Hurt

"Disruption" is the most successful euphemism in modern business. It takes a process that destroys livelihoods, hollows out communities, and concentrates wealth, and recasts it as the natural metabolism of progress. Painful, sure — but ultimately beneficial. The old gives way to the new. It all works out.

pitfalldisruptionnarrative
pitfalls

Open Weights Are Not Open Source

When Meta releases Llama, the press writes "Meta open-sources its latest AI model." When Mistral drops a new set of weights, the headline reads "another open-source challenger to GPT." The language is consistent, familiar, and wrong.

pitfallopen-sourceopen-weights
pitfalls

The Scaling Laws Plateau

Since 2020, the AI industry has operated on a seductive premise: just make it bigger. More parameters, more data, more compute, better results. The scaling laws published by Kaplan et al. at OpenAI seemed to confirm this with mathematical precision — model performance improves as a smooth, predictable function of scale (Kaplan et al., 2020). Two years later, DeepMind's Chinchilla paper refined the ratios, arguing that training data should scale proportionally with parameters (Hoffmann et al., 2022). The message to the industry was clear: spend more, get more.

pitfallscalingmodels
pitfalls

The Autonomy Illusion

When AI systems produce surprising outputs, we often describe them as having "chosen" or "decided" something. This language is convenient but misleading. It obscures where decisions actually get made — and who's accountable when they go wrong.

pitfallautonomyagency
pitfalls

The Two-Species Fallacy: Why Treating AI as a Life Form Obscures Power

A new metaphor is colonizing the AI discourse. We are told that humanity no longer lives alone — that a second species has arrived, computational and cognitive, and that our central challenge is learning to coexist with it. The framing shows up in policy papers, TED talks, and boardroom decks. It sounds profound. It is profoundly misleading.

pitfallai-ontologypower
pitfalls

Why Removing Bias From Data Doesn't Remove Bias

The most common response to AI bias problems is "we need to fix the training data." This sounds reasonable — and data quality genuinely matters. But treating it as the complete solution is insufficient, sometimes counterproductive, and often a distraction from harder, more productive questions. Understanding why helps us build fairer systems, not just cleaner datasets.

pitfallbiasfairness