Does a Bigger Model Always Mean a Smarter AI?

Lately, it feels impossible to escape headlines about data centers. Every other day brings news of a company pouring tens of billions of dollars into new data centers, or building nuclear plants just to secure a stable supply of electricity. There is even talk of putting data centers in space.

If you squint a little, it can sometimes look as if the AI industry is no longer in the business of building models - it is in the business of building giant machines that eat electricity.

Which raises an obvious question.

Why have data centers become so important to building better AI?

The easy answer is that training an AI model requires an enormous amount of computation. But dig one layer deeper, and the question becomes more complicated than it first appears.

Say you have a fixed amount of compute. Where should you spend it? Do you make the model bigger? Feed it more data? Or just train the same model for longer?

In an era when AI labs are racing to secure hundreds of thousands of GPUs, this might sound like a question that should have an obvious answer. But just a few years ago, there was no clear answer at all.

Then, in 2020, OpenAI published a paper that tried to answer it mathematically:

Scaling Laws for Neural Language Models.

This wasn’t just a paper proving that “bigger models are better.”

It opened the door to predicting how performance changes as models scale - and to reasoning quantitatively about how a limited compute budget should be allocated.

But here’s the twist.

Two years later, another study revisited the same problem and arrived at a different conclusion:

We’d been building our models too big.

That paper became known as Chinchilla.

---

Why nobody had a clear answer before

Even before Scaling Laws, researchers already knew that bigger models tended to perform better.

The real question was: how much bigger?

If you double the model size, how much does performance improve? What if you double the amount of training data instead? Or spend twice as much compute on training?

Which of these is the most efficient place to put your resources?

At the time, the only practical way to find out was largely through experimentation. Build a small model, train it, measure its performance. Then build a bigger one and do it all again.

But once models began growing into the billions and tens of billions of parameters, that approach became increasingly impractical. A single training run could cost an enormous amount of money.

If you spend hundreds of millions of dollars training a model and the results disappoint, that is no longer just an experiment that failed. It is an extremely expensive mistake.

What was needed was not intuition or experience, but a predictable rule.

What if you could run experiments at a smaller scale and use the results to predict what would happen at a much larger one?

That was exactly the problem Scaling Laws set out to solve.

What actually determines a model’s performance?

The paper’s basic question is surprisingly simple: What determines the performance of a language model?

The authors focused on several factors:

• Model architecture
• Model size - the number of parameters
• Amount of training data
• Total compute used for training

At the time, it was reasonable to assume that architecture itself would be a major variable.

How deep should the Transformer be? How wide should it be? How many attention heads should it use?

But the experiments revealed something interesting. Within the same overall parameter budget, substantially changing the balance between depth and width had a relatively small effect on performance.

Something else appeared to matter more.

How big is the model? How much data does it train on? How much compute does it use?

That does not mean architecture is irrelevant.

Rather, the result suggested that across sufficiently large language models, scale could be a stronger driver of performance than many architectural details within the range the researchers tested.

That helped shift attention toward scale itself as a major factor in language-model performance.

Performance turned out to be surprisingly predictable

The most important finding in Scaling Laws was not simply that bigger models perform better.

It was that the improvement followed a consistent pattern.

As the researchers independently increased model size, the amount of training data, and training compute, language-model loss decreased according to relationships that were well approximated by power laws.

In plain terms, increasing scale tended to improve performance.

But doubling the model size does not mean doubling the performance. As models grow, increasingly large amounts of resources are needed to keep pushing loss lower.

The crucial point, however, was that the relationship was not random.

The curves remained remarkably smooth and predictable across the scales the researchers studied.

That had major implications.

It meant that experiments on smaller models could potentially be used to estimate how much larger models would improve.

Before this, building very large models involved a great deal of experience and trial and error. After Scaling Laws, questions like these could begin to be approached quantitatively:

If I scale the model up by 10x, how much might the loss decrease? Would it be better to train the same model on more data, or to make the model itself bigger?

AI model performance was beginning to look like a predictable engineering problem.

So where should you actually spend your compute?

This is where the paper’s most practical question appears.

If your compute budget is fixed, how should you spend it? Suppose you have $100 million worth of compute available. Should you train a smaller model for a very long time? Use more training data? Or make the model itself larger?

Scaling Laws approached this as a problem of compute-optimal training.

Within the experimental regime studied in the paper, the conclusion was relatively clear: as the available compute budget increased, the compute-optimal model size grew rapidly.

Training data and training duration also increased, but more slowly than model size under the paper’s estimated scaling relationships.

And there was another interesting implication. You do not necessarily want to train every model all the way to convergence.

The intuitive approach might be to build a model and keep training it until it has extracted as much as possible from the available data.

But if your actual goal is to achieve the best possible performance under a fixed compute budget, the calculation changes.

Instead of fully training a smaller model, it can be more efficient to build a larger model and train it for fewer steps.

Put simply:

If the goal is to achieve the best possible performance within a fixed compute budget, training a single model all the way to convergence is not always optimal. Under the same compute constraint, a larger model trained for comparatively fewer steps can perform better.

This provided a powerful direction for the development of increasingly large language models.

How should model size and data scale together?

But there is an obvious problem.

You cannot simply keep making the model larger forever.

If a model has an enormous number of parameters but not enough training data, it may not have enough information to make effective use of all the capacity.

Turn the situation around, and a model with huge amounts of data but insufficient capacity may not be able to take full advantage of the data either.

Scaling Laws found that model size and training data were connected through predictable scaling relationships.

The important idea was simple:

Bigger models need more data.

But under the compute-optimal scaling relationships estimated in the paper, model size increased much faster than the amount of training data.

That particular result becomes important later in the story.

Because it is exactly the relationship Chinchilla would revisit.

And so, models kept getting bigger

Scaling Laws appeared around the same time GPT-3 was demonstrating the potential of large language models on a new scale. Researchers already knew that larger models tended to perform better. But now, that relationship no longer looked like just a rule of thumb.

It appeared to follow a predictable pattern. The transition from GPT-2 to GPT-3 made that change dramatically visible. GPT-3 had 175 billion parameters. Seen through this same lens, it becomes easier to understand why GPT-3 arrived at such an enormous scale.

What mattered was not simply that the number was large. GPT-3 came to symbolize a broader idea about how AI systems could improve: Scale itself could become a strategy.

More compute. Bigger models. Larger data centers. AI progress was no longer purely a question of algorithms.

The ability to secure massive amounts of computation was becoming increasingly tied to the ability to train more capable models.

And the data center race we see today is part of that same trajectory. But one question remained.

Was this actually optimal?

Except, maybe we had been building our models too big

The conclusions of Scaling Laws were powerful. But like any empirical scaling law, they came with an important condition:

Over what range was the relationship measured?

Scaling Laws estimated the relationship between model size, training data, and compute within the experimental range that was feasible at the time.

Within that range, aggressively increasing model size appeared to be compute-efficient.

Then, in 2022, researchers at DeepMind revisited the problem. Their question was almost identical:

Given a fixed compute budget, how should model size and training data be balanced to achieve the best performance?

This time, the answer was different.

Making the model bigger was not enough.

From a compute-optimal perspective, many large language models at the time were being trained on too few tokens relative to their parameter count.

In other words, they were undertrained for their size. The paper was called Training Compute-Optimal Large Language Models.

It became much better known by the name of the model used to demonstrate its findings: Chinchilla.

What did Chinchilla actually recalculate?

Chinchilla’s contribution was not a new architecture. It did not replace the Transformer or introduce an entirely new kind of language model. Instead, it revisited the same fundamental question posed by Scaling Laws:

As compute increases, how should model size and the amount of training data increase together?

Chinchilla arrived at a substantially different scaling prescription. Under the earlier Scaling Laws estimates, compute-optimal model size increased rapidly while the amount of training data increased more slowly.

Chinchilla found that model size and training tokens should grow in much closer balance as the compute budget increases. In other words, simply making the model larger was not enough.

If you build a larger model, you also need enough additional training data for that model to make effective use of its increased capacity.

The difference was significant.

The earlier scaling prescription favored rapidly increasing model size while training data grew comparatively slowly. Chinchilla’s result pointed toward a different principle:

Bigger models need substantially more training data than the earlier scaling estimates suggested.

Given the same compute, a bigger model isn’t always the better choice

This is one of Chinchilla’s most important implications.

When compute is limited, continuing to increase parameter count is not always the best use of that compute.

A smaller model trained on substantially more data can sometimes perform better. Chinchilla demonstrated this directly.

The Chinchilla model had 70 billion parameters - far fewer than several leading large language models at the time - but was trained on far more tokens.

Under a comparable training-compute budget, the 70B-parameter Chinchilla outperformed substantially larger models, including DeepMind’s own 280B-parameter Gopher.

This was more than a minor hyperparameter adjustment. It challenged a basic assumption behind the way large language models were being scaled. The question could no longer simply be: “How large a model can we build?”

It also had to be: “How much data do we need to train that model properly?”

A new relationship between model size and data

Chinchilla’s findings are often summarized with a simple rule of thumb: Roughly 20 training tokens per model parameter.

But treating that number as an absolute law would be a mistake. The optimal point can shift with data quality, model architecture, training methods, and other assumptions.

The more important takeaway is not the number itself. It is the direction.

Under the compute-optimal scaling described in the earlier Scaling Laws work, model size grew relatively quickly compared with training data.

Chinchilla shifted the emphasis toward jointly scaling model size and training data in a much more balanced way.

That has direct implications for the AI industry. Building a larger model is not simply a matter of buying more GPUs. You also need enough high-quality data to train that model effectively.

If compute is the bottleneck, GPUs matter. If data is the bottleneck, adding more GPUs alone will not solve the problem.

The scaling problem for large language models is more complicated than a competition over parameter count.

It is a problem of balancing model, data, and compute.

Was Scaling Laws wrong, then?

That naturally raises another question.

Was Scaling Laws wrong?

I don’t think that’s the most useful way to look at it.

Chinchilla is better understood not simply as a rebuttal, but as a continuation of the question Scaling Laws helped establish.

Scaling Laws matters not because it produced one set of numbers that would remain correct forever. It matters because it changed how researchers could think about AI performance.

Instead of treating scale largely as a matter of experimentation and intuition, researchers could begin asking quantitative questions:

How does performance change as a model gets bigger?
What is the relationship between data and compute?
Can we calculate a more efficient training strategy?

Chinchilla returned to those same questions under a broader experimental setup and showed that the previously estimated optimum could be improved. That is not unusual in science. One study helps define the question. Another tests it under different or broader conditions and revises the answer. That revised answer then becomes a starting point for the research that follows. Scaling Laws and Chinchilla can be understood in much the same way.

In the end, AI became a resource problem

Looking at the two papers together makes it easier to understand why the AI industry looks the way it does today.

Scaling Laws showed that language-model performance could improve predictably with scale. Chinchilla showed that scale should not be understood simply as parameter count.

Model. Data. Compute.

All three have to be considered together. And in the AI industry of 2026, compute has become one of the most significant real-world constraints among them.

So companies race to secure GPUs. They build data centers. They secure electricity. And at the same time, they look for larger quantities of high-quality data.

AI progress is no longer just a matter of designing a clever model inside a research lab. It has become an industrial problem connected to power grids, data centers, semiconductor supply chains, and access to data.

Which brings us back to where this piece started. Companies are not building enormous data centers simply because AI models happen to be large.

They are doing it because we learned that AI can scale.

Scaling Laws and Chinchilla offered two different answers - at two different points in time - to the question of how that scaling should work.

Closing

Scaling Laws posed a powerful question:

To make AI smarter, what should we scale, and by how much?

Its first answer pointed strongly toward larger models.

Chinchilla went back and recalculated the balance. Its answer was different: Don’t just scale the model.

Scale the data alongside it, enough for the model to make effective use of its capacity.

Looking back, the most important thing these two papers left behind is not a specific parameter count or token ratio. It is a way of thinking about AI performance itself.

A good model architecture is not the end of the problem. There is the structure and size of the model. The quantity and quality of the data. And the compute required to train it.

The relationship among those three has to be continually reconsidered as models, datasets, hardware, and training methods change.

If Scaling Laws asked:

How big should we make the model?

Chinchilla pushed the question one step further:

What needs to grow alongside it?

And in 2026, the AI industry is still working out the answer.