Training data is the raw material fed into a foundation model during pre-training — terabytes of text, code, images, and other content that determine what the model knows and, critically, what it doesn't.

The problem is opacity. OpenAI, Anthropic, and Google don't publish full training corpora. Audits of public datasets like C4 and The Pile found real gaps: underrepresented languages, shallow coverage of specialized domains, and a heavy skew toward the English-language web. When a model fumbles your industry's terminology or misses a regulatory nuance, you're usually looking at a training data gap, not a reasoning failure. The model isn't confused — it was never taught.

This reframes model selection. "Which model is smartest?" matters less than "which model was trained on data most relevant to my problem?" A model with deep legal text in its training mix can outperform a nominally stronger general model on contract analysis. Domain fit beats benchmark scores more often than teams expect.

There's also legal exposure. The New York Times, Getty Images, and thousands of authors have filed suits arguing that training on copyrighted material without permission is infringement. These cases remain unresolved, but the direction of travel is clear: if licensing mandates follow, training costs rise and flow downstream. Your vendor's data sourcing practices are now part of your risk profile — even if you'll never see the receipts.