A parameter is one adjustable number in a neural network. Modern language models have billions of them, which is why sizes are quoted as 7B or 70B. Parameter count hints at capacity, but training data quality, training method and architecture matter just as much for real performance.
How it works
A neural network is made of layers of simple units. Each connection between units has a weight, a number saying how strongly one signal affects the next, and many units also have a bias, a number that shifts their output. These weights and biases are the parameters. Before training they are random. During training the model makes a prediction, such as the next token in a sentence, compares it with the right answer, and an algorithm called backpropagation works out how each parameter contributed to the error. Gradient descent then nudges every parameter slightly in the direction that reduces the error. Repeated over huge amounts of data, these tiny changes encode grammar, facts and patterns. Parameters are different from hyperparameters, such as learning rate or temperature, which people set rather than the model learning. Size has practical costs: each parameter is usually stored in 16 bits (2 bytes), so a 7-billion-parameter model needs roughly 14 GB of memory, while quantising to 4 bits cuts that to about a quarter. Some models use a mixture-of-experts design, where only part of the parameters is active for each token.
Why it matters
Parameter count helps you judge what a model needs and roughly what it can do. Bigger models tend to handle complex reasoning and broad knowledge better but cost more to run and respond more slowly. Smaller models can be fast, cheap and good enough for focused tasks like tagging, extracting or short replies. Knowing this helps creators and businesses pick the right model for the job instead of assuming the biggest is always best.
Where you will see it
- Estimating whether a model will fit on a laptop, phone or single GPU.
- Comparing model sizes when choosing between faster, cheaper and more capable options.
- Planning hosting costs, since memory and compute grow with parameter count.
Limits and common misconceptions
- More parameters does not guarantee better answers; a well-trained smaller model can beat a larger, poorly trained one.
- Parameters are not a database of facts; knowledge is spread across them and can be recalled wrongly, causing hallucinations.
- Model parameters are often confused with settings like temperature, which you choose at use time and the model never learns.
How it differs from related terms
- Parameters vs Token Tokens are the chunks of text a model reads and writes; parameters are the learned numbers that process those tokens.
- Parameters vs Temperature Temperature is a setting you choose when generating output; parameters are learned during training and fixed afterwards.
Frequently asked questions
What does 7B or 70B mean in an AI model name?
It is the number of parameters: 7B means about seven billion, 70B about seventy billion. It tells you roughly how large the model is, how much memory it needs and how expensive it is to run. It does not, on its own, tell you how good the model is at a given task.
Are more parameters always better?
No. Larger models usually have more capacity, but results depend heavily on training data, training length and fine-tuning. Smaller models trained well can match or beat bigger ones on specific tasks, while being faster and cheaper. Many companies now release several sizes so users can trade quality for speed.
What is the difference between parameters and hyperparameters?
Parameters are learned by the model during training, such as weights and biases. Hyperparameters are chosen by people to control training or generation, such as learning rate, batch size or number of layers. Temperature is a generation setting of the same kind: you set it, the model does not learn it.
Related terms
- No-code Building apps, websites or automations through visual editors, templates and settings instead of writing programming code.
- Open-weight model An AI model whose trained weights are publicly released, so anyone can download, run and often fine-tune it on their own hardware.
- Prompt The text, files or other input you give an AI model to tell it what to do, including your question, context and instructions.
- Prompt engineering The practice of designing, testing and refining prompts and their surrounding context so an AI model gives accurate, consistent and useful output.
- RAG (retrieval-augmented generation) A method where an AI system first retrieves relevant information from a chosen source, then uses it to generate an answer grounded in that material.