Optimizer Benchmarks
Comprehensive benchmark results from these research papers on different domains.
Research Citation Required
All benchmark results are derived from these research papers. Please cite these work when using these results:
Note: Benchmark results in any research paper are inevitably influenced by factors such as computational hardware and training configurations. Consequently, optimizer performance is highly sensitive to these variables. When evaluating or comparing the efficacy of different optimizers, it is imperative to conduct meticulous hyperparameter tuning within your specific hardware environment to ensure a fair and rigorous comparison.
"Benchmarking Neural Network Training Algorithms"
arXiv:2306.07179"Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning"
arXiv:2410.06373v1"Benchmarking Optimizers for Large Language Model Pretraining"
arXiv:2509.01440"Fantastic Pretraining Optimizers and Where to Find Them"
arXiv:2509.02046Want to Contribute Benchmarks?
Help us expand our benchmark suite! Submit your optimization results and contribute to the community knowledge base.