Parameters
What does 7B or 70B parameters mean?
A parameter ≈ one learned numerical value (a weight). More parameters means more numbers to fit — but that alone doesn't guarantee higher "intelligence".
Parameter count doesn't directly equate to intelligence — architecture, data, training method and optimization all matter too.
In more detail
Parameters are all the numbers a model learns: embedding matrices, attention matrices, MLP weights, biases and normalization scales. "7B" means about 7 billion of them.
Size has a concrete cost. If each parameter is stored in 16 bits (2 bytes), 7 billion parameters take about 14 GB and 70 billion take about 140 GB, just for the weights. Techniques such as quantization (Advanced track) reduce this.
More parameters usually give a model more capacity, but quality also depends on the amount and quality of training data, the amount of training computation, the architecture and the later training stages. Studies of scaling laws (Kaplan et al., 2020; Hoffmann et al., 2022) found that the loss falls in a predictable way as parameters, data and computation grow, and that for a fixed budget the model size and the amount of data should be increased together. Some earlier large models were trained on too little data for their size.
In mixture-of-experts models (Advanced track), the total number of parameters is larger than the number used for each token.