RULER

Benchmarking Context Windows: NIAH, RULER, MRCR, and What They Actually Measure

Needle in a Haystack tests one thing. RULER tests another. MRCR tests yet another. Here's what each benchmark actually measures and what it misses.