OmniOpt Benchmark
Explore score history ↗A cross-domain benchmark for comparing optimizers in large-scale model training. It covers 24+ optimizers across two stages: Stage 1 sweeps on C4 with LLaMA-3 architectures (60M to 1B), and Stage 2 transfers to FineWeb-Edu with four architectures (Transformer++, GLA, DeltaNet, Gated DeltaNet) at 340M and 1B scales. Controlled-variable protocol with fixed architecture, data, and schedule settings.
- Released
- 2026-07-04
- Institutions
- Westlake University · Shanghai Artificial Intelligence Laboratory · Shanghai University
- Evaluates
- General AI, Geometric reasoning, Language & Knowledge, cs.LG
- Openness
- unknown
- Importer
- Claire Radar
- Review status
- unreviewed
OmniOpt paper and code
No reported scores are on record for this benchmark yet.