Get all your news in one place.
100’s of premium titles.
One app.
Start reading
Tom’s Guide
Tom’s Guide
Technology
Christoph Schwaiger

AI models are getting better at grade school math — but a new study suggests they may be cheating

Adobe Firefly.

Large language models (LLMs) that power chatbots like ChatGPT may be getting better at answering benchmark questions that measure mathematical reasoning. But this may actually be a bad thing.

A pre-print research paper released on Wednesday by researchers at Scale AI detailed how LLMs have been achieving impressive results on math benchmark tests but that there’s growing concern that dataset contamination is fuelling high grades. 

This is when data resembling benchmark questions leaks into training data. The LLM then may end up training in a way that prioritizes passing these standardized tests rather than truly understanding the mathematical problem it’s trying to solve.

This is similar to if you’re preparing for a math quiz by memorizing answers, rather than learning how to solve the problem. This issue is called overfitting.

However, the authors of the paper say their results don’t support this theory, suggesting that it doesn't mean the AI is bad at reasoning, just that it might not be as good as the benchmarks suggest..

Developing a new math benchmark

In the paper the authors wrote: “The fact that a model is overfit does not mean that it is poor at reasoning, merely that it is not as good as the benchmarks might indicate it to be." They found that many of the most overfit models can still reason and solve problems they’ve never encountered before in their training sets.

To run these evaluations, they developed their own math benchmark test (GSM1k) which they say tests the AIs ability to understand the problem, not just the answer.

The fact that a model is overfit does not mean that it is poor at reasoning, merely that it is not as good as the benchmarks might indicate it to be.

Study authors

The questions are at grade school math level and a typical GSM1k question would look like: Jim wants to spend 15% of his monthly earnings on groceries. He makes $2500 a month. How much money will he have left over? The correct answer is $2125.

While such questions closely resemble those in the industry gold standard test (GSM8k) in difficulty, they're different enough to test whether the LLMs can handle math puzzles they haven’t seen before.

Using their new test, the research team at Scale AI reported accuracy drops of up to 13% when they evaluated leading open- and closed-source LLMs. Other models on the frontier such as Gemini, GPT, and Claude showed minimal signs of overfitting.

What's next?

This ‘issue’ may end up resolving itself over time as the authors predict that by 2025 grade school math will likely no longer be difficult enough to benchmark new LLMs. Still, they say that improving reasoning in LLMs “is one of the most important directions of current research.”

Senior Research Scientist at NVIDIA Jim Fan said on X that academic benchmarks are losing their potency.

He said that three types of LLM evaluations that will matter in the future would be privately held tests like that of Scale AI, public comparative benchmarks like Chatbot Arena where you can test models side-by-side, and privately curated benchmarks for each company’s own use cases.

More from Tom's Guide

Sign up to read this article
Read news from 100’s of titles, curated specifically for you.
Already a member? Sign in here
Related Stories
Top stories on inkl right now
One subscription that gives you access to news from hundreds of sites
Already a member? Sign in here
Our Picks
Fourteen days free
Download the app
One app. One membership.
100+ trusted global sources.