Most big launches come with charts showing the new model ahead of its rivals. Those numbers usually come from Benchmark: A standard test used to compare models, like an exam. Useful, but labs tend to show the tests they do well on, so one score never tells the whole story.: standard tests, like exams, where every model answers the same questions and is marked the same way. There are benchmarks for maths, coding, science, general knowledge and long tasks with many steps.
A score is often the share of questions answered correctly, so 80% means four in five. As models improve, old tests get too easy, and researchers keep writing harder ones.
Votes and ratings
Some rankings rely on people instead of answer sheets. On sites such as LMArena, you type a question, two anonymous models answer, and you vote for the better reply before the names are revealed. Millions of these votes are turned into ratings.
The ratings work like chess ratings, where the gap between two players predicts how often one beats the other. A model 100 points ahead wins about 64% of the time, roughly two times in three. That's a real edge, but far from winning every time.
Other rankings blend many tests into a single combined score, a bit like an average across all your school subjects. One lucky or unlucky result matters less, so these scores tend to be steadier than any single test.
Reasons to be careful
Labs choose which results to show at launch, so expect to see the tests they did well on. Popular tests can also leak. Once the questions are online, models may be trained on them or tuned for them, like a student who memorised last year's exam paper.
Small gaps don't mean much. Scores shift a little each time a test is run, and votes depend on who's voting, so models a few points apart are effectively tied. Voters can also be swayed by style, such as longer answers with tidy formatting.
Above all, rankings miss things you'll notice straight away. Does the tone suit you? Does the model follow your instructions every time, or only most of the time? A model can top a chart and still feel wrong for your work.
Use rankings as a shortlist
Treat rankings like restaurant reviews: they tell you where to look, but your own taste decides. Pick two or three models near the top for the kind of task you care about, give them the same real job, and compare.
Use the things you actually do: a tricky email, a question about a spreadsheet, a long document to summarise. Half an hour of trying will tell you more about what suits you than any chart.