Description
Most training contests assume unlimited data and race for speed, but quality data is increasingly scarce. NanoGPT Slowrun is a benchmark for language modeling algorithms with fixed data and infinite compute.
Using 100M FineWeb tokens with no compute limit, lowest validation loss wins, rewarding data-efficient training.
Fixed data:100M tokens.
Infinite compute:No time cap.
Metric:Lowest loss.
Using 100M FineWeb tokens with no compute limit, lowest validation loss wins, rewarding data-efficient training.
Features
Fixed data:100M tokens.
Infinite compute:No time cap.
Metric:Lowest loss.
