The training steps per epoch calculator gives the effective batch size, the optimizer steps per epoch, the total steps, the samples processed and the training time of a machine learning run.
How the calculation works
- Enter the number of training examples and the batch size per device.
- Enter the number of devices, the gradient accumulation and the epochs.
- Enter the throughput in samples per second.
- The calculator gives the effective batch, the steps per epoch, the total steps, the samples, the last batch and the time.
Example
A data set of 50,000 examples with a batch of 128 on one device has an effective batch of 128. One epoch takes ceil(50,000 / 128) = 391 steps, the last batch holding 80 examples. Over 90 epochs, that makes 35,190 steps and 4,500,000 samples. At 1,500 samples per second, training takes 0.83 hours.
Why it matters
The calculator helps students and engineers who schedule training runs, set learning rate schedules and warm-up lengths in steps, or estimate how long and how costly a training job will be.
Good to know
Learning rate schedules and checkpoints are usually defined in steps, so the total steps are the number to give. If the last incomplete batch is dropped, which some data loaders do by default, the steps per epoch are rounded down instead. With several devices, each processes its own share of the batch and the throughput entered should be the total over all devices. Real throughput varies with data loading, communication between devices and the length of the examples, so measure it on a few steps. Larger batches use the hardware better but may need more epochs to reach the same accuracy.
Related tools
The neural network parameter count calculator sizes the model, the learning curve calculator relates experience and speed, the LLM memory calculator estimates memory and the embedding table calculator sizes vocabularies.