Skip to content
Omnikalc

Neural network parameter count calculator

Enter the number of inputs, two hidden layers and the outputs of a fully connected network to get the parameters, the memory for inference and training, and the operations per prediction.

Updated on
Your inputs

Value out of range

Value out of range

Value out of range

Value out of range

Results

Total parameters

109,386

Weights
109,184
Biases
202
Share of the first layer
91.9 %
Memory for inference
0.438 MB
Memory to train with Adam
1.750 MB
Operations per prediction
218,368

The neural network parameter count calculator gives the number of weights and biases, the memory and the operations per prediction of a fully connected network with up to two hidden layers.

How the calculation works

  • Enter the input features, the neurons of the hidden layers and the output neurons.
  • Pick the precision of the weights.
  • The calculator gives the parameters, the weights, the biases, the share of the first layer, the memory and the operations.

Example

A network 784-128-64-10 has 784 × 128 + 128 × 64 + 64 × 10 = 109,184 weights and 128 + 64 + 10 = 202 biases, so 109,386 parameters. In 32-bit it takes 0.438 MB for inference and 1.750 MB to train with Adam. The first layer holds 91.9 % of the weights. One prediction costs about 218,368 operations.

Why it matters

The calculator helps students, hobbyists and engineers who size a small neural network, compare architectures or check that a model fits in the memory of a microcontroller or a phone.

Good to know

The counts are exact for a dense network with biases in every layer. Other layer types differ: convolutions share weights, so their counts do not depend on the image size, and attention layers have four projection matrices. Normalization layers add a few parameters per feature. A rule of thumb is to have more training examples than parameters, but deep networks often work well with more parameters than examples, as long as they are regularized. Quantizing the weights to 8 bits divides the memory by 4 with a small loss of accuracy in many models. The training memory shown excludes activations and the data pipeline, which can dominate for large batches.

The convolution layer calculator counts convolution parameters, the embedding table calculator sizes vocabularies, the LLM memory calculator handles large language models and the linear regression calculator covers the simplest model.

Frequently asked questions

How are the parameters of a dense layer counted?

A layer connecting n inputs to m neurons has n × m weights and m biases. For the whole network, add these up for each layer. The 784-128 layer alone has 100,352 weights, which is most of the network.

Why does training need more memory?

Besides the weights, training stores one gradient per weight and, with the Adam optimizer, two moving averages kept in 32-bit. That is about 4 + 4 + 8 = 16 bytes per parameter in 32-bit, plus the activations, which depend on the batch size and are not counted here.

What do the operations mean?

Each weight takes part in one multiplication and one addition per prediction, so a forward pass costs about 2 operations per weight. The biases and activation functions add a small extra, ignored here.

Was this calculator helpful?

Thanks for your feedback!

Results are estimates provided for information purposes. They do not replace professional advice.