Model Parameters
Parameters are the learned numbers inside an AI model, also called weights, that training adjusts and that together store what the model has learned. The parameter count, such as 70 billion, is the usual shorthand for a model's size.
Every neural network is a large set of numbers. Before training they are random; training adjusts them until the model’s predictions are good. Those numbers are the parameters, and in a large language model there are billions of them. When a company releases open weights, the parameters are what it publishes.
Parameter count is a rough guide to a model’s capability and cost. More parameters can store more knowledge and patterns, and among models of similar design they generally take more memory and computing power to run, which raises the cost and latency of each answer. Design choices change the math: a mixture-of-experts model uses only some of its parameters for each token, and quantization shrinks the memory each one takes. Size isn’t everything: on the prompts OpenAI’s InstructGPT research tested, people preferred a 1.3-billion-parameter model trained to follow instructions over the 175-billion-parameter GPT-3. That is why teams often move stable, narrow tasks to a small language model.
