GPT-3: when scale became the strategy (2020)
OpenAI's 175-billion-parameter model showed that a big enough language model could learn new tasks from a few examples in the prompt, no retraining required.
In 2020, OpenAI published GPT-3, a large language model with 175 billion parameters, more than a hundred times larger than its predecessor. The paper’s title was modest, “Language Models are Few-Shot Learners”, but the claim inside was not.
Few-shot learning
Earlier models were usually specialised: to make one good at translation or summarisation, you fine-tuned it on a labelled dataset for that task. GPT-3 suggested a different way of working. Because it was a Transformer trained on a vast amount of internet text, you could often just describe the task in the prompt, show it a couple of examples, and it would follow the pattern, with no weight updates at all.
This ability, called in-context or few-shot learning, is why the prompt became such a central idea. Suddenly the way you asked mattered enormously, and a single model could be coaxed into many jobs without the cost of fine-tuning a separate version for each.
The scaling bet
GPT-3’s real message was about scale. Its behaviour largely came from making the same basic recipe much bigger: more parameters, more data, more compute. The paper documented how capabilities tended to improve smoothly as those numbers grew, giving research and industry a shared, almost economic argument: if bigger reliably meant better, then investment could be pointed at scaling.
That argument reshaped the field. It justified the enormous training runs and data-centre budgets that followed, and it set expectations that each new generation would be dramatically more capable than the last.
Why it matters
GPT-3 was not released to the public as a chat product. Access ran through an API aimed at developers, and for a while it was mostly a topic among people who followed the field closely. But it laid the groundwork for what came next. Once it was clear that one large model could handle open-ended language tasks from a prompt, the missing piece was an interface anyone could use.
That interface arrived two years later as ChatGPT, and the mass audience it reached was reacting, whether they knew it or not, to the capabilities GPT-3 had first demonstrated.