Pre-training is the initial, industrial-scale phase where a foundation model learns language patterns by ingesting terabytes of text. It is the most expensive step in building a machine intelligence system — hundreds of millions of dollars and weeks of compute at frontier scale.

Most companies will never pre-train a model. The cost and infrastructure put it out of reach for everyone except a handful of labs. That makes your foundation model choice a genuine strategic dependency. The architectural decisions, data mix, and tradeoffs baked in during pre-training are locked in. You inherit them.

This matters more than it looks. Fine-tuning and prompt engineering feel cheap precisely because pre-training already paid the bill. And because training costs keep rising, this layer of the stack is concentrating further — fewer providers, more leverage over the ecosystem. When you pick a model vendor, you're picking a long-term dependency on decisions you had no hand in making.