AI tabular prediction fell to majority-class guessing on most tested datasets once a table reached roughly 16 feature columns, while every classical baseline stayed flat or improved. In a new preprint, Claude Opus 4.6 was the only one of nine methods whose performance declined as researchers increased dimensionality without removing the underlying information.
That is a startling contrast with the systems that can draft contracts, write code and pass professional tests. GPT-4 has passed a simulated Uniform Bar Examination, and AI performance on licensing exams keeps attracting attention. Yet a table of ordinary numbers exposed a different kind of weakness: the model could read the values, but struggled to combine them into reliable predictions.
The comparison is real, but the boundary matters. This was not a test of an AI assistant using Excel, Python, SQL or a specialised forecasting tool. The researchers used a pure inference protocol: raw CSV data in a single prompt, with no tools, code execution, retrieval, fine-tuning or second turn. It was a controlled test of the model itself, not every product built around it.
What AI Tabular Prediction Means for You
If you use a chatbot to explain a formula, clean column names or write a spreadsheet macro, this paper does not say that workflow is broken. If you paste labelled rows into a chat and ask the model to predict which customers will leave, which applicants will default or which transactions look suspicious, it gives you a strong reason to stop and validate the result elsewhere.
The practical lesson is to separate interface from engine. A language model can help a person describe the problem, generate code, document assumptions and translate an output into plain English. A tested statistical or machine-learning model should perform the numerical classification, with held-out data and an error metric showing whether it works. Fluency is not evidence of predictive accuracy.
That distinction becomes more important when a row represents a person. Automated judgements can affect insurance claims, pensions, investments or access to services. LiveAIWire has examined disputes around an AI insurance adjuster, but the new study does not show that such production systems use raw chatbots. It shows why buyers should ask what model actually makes a decision and how it was tested.
The Test Removed Every Excuse Except the Number of Columns
Marta Garnelo of Fundamental Technologies and Wojciech M. Czarnecki of Voylab designed the study to isolate one question: can a general-purpose language model learn a classifier from examples presented inside its prompt? Their paper, “Why Large Language Models Fail at Tabular Prediction”, was posted to arXiv on 3 August 2026 and has not yet been peer reviewed.
The main pool contained 19 small classification datasets, including familiar tasks involving iris flowers, wine, breast cancer, sonar and heart disease. After contamination checks, the researchers retained 11 for their main analyses. They used five-fold stratified cross-validation repeated across five seeds, producing 25 train-test splits per dataset and 475 language-model queries.
Claude faced eight conventional competitors: nearest-neighbour classifiers with three settings, random forest, regularised logistic regression, AdaBoost, gradient boosting and a Gaussian-process classifier. All methods saw the same features and splits. The study also swept 252 configured classical variants when trying to identify which familiar algorithm best resembled the language model’s behaviour.
Each Claude query consisted of one user message containing the full training set and test features as plain-text CSV. There was no system prompt, tool, code runner, retrieval step, agent loop or fine-tuning. The API used default sampling, and the model had to return an integer class prediction for every test row in one generation.
It Could Read 60 Columns but Could Not Combine Them
The obvious explanation was that CSV turns a table into a stream of text and hides its vertical structure. The researchers tested that possibility with a deliberately easy needle-in-the-haystack task. They placed the answer itself among as many as 60 distractor columns, so success required finding the right column and copying it.
Claude performed at or near the ceiling, with essentially no downward trend. It could therefore locate a useful column in a wide table. The failure appeared when the answer had to be inferred from relationships across multiple columns. In other words, access to the values was not the same thing as integrating them into a dependable decision rule.
At Roughly 16 Columns the Advantage Disappeared
To isolate dimensionality, the researchers projected datasets into successively larger numbers of columns. A random linear projection can represent the same underlying information in a wider coordinate system. In the dimensionality sweep, the classical models stayed flat or improved. Claude alone deteriorated, with a reported correlation of minus 0.21 and a slope of minus 0.009 in normalised score per added dimension.
At about 16 dimensions and above, its normalised performance was at or below majority-class guessing on most datasets. A zero score in the paper’s normalisation means matching the strategy of always choosing the most common class. That does not mean every raw prediction was wrong. It means the model had lost the useful advantage a classifier is supposed to provide.
The team repeated the idea on 20 synthetic problems that genuinely lived in only two dimensions. They then represented the same two-dimensional signal using 4, 8, 16, 32 and 64 columns, where the added features were linear combinations of the original pair. The conventional learners remained stable or improved. The language model again weakened as the table widened.
Formatting, Precision and Smaller Batches Did Not Fix It
The paper tested five popular explanations: overlapping classes, CSV formatting, numerical tokenisation, too many requested answers and dimensionality. Controlled interventions rejected the first four and supported the fifth. Even making the classes fully separable did not rescue Claude. It improved less than the baselines and remained far behind them at the easiest end of the experiment.
Rounding every feature to one, four or eight decimal places made virtually no difference. Splitting a test fold into halves or quarters also failed to help. The reported mean normalised scores were 0.422 for a full fold, 0.412 for a half and 0.407 for a quarter. Asking for fewer predictions at once therefore did not release some hidden reserve of accuracy.
This matters because prompt advice often targets presentation: shorten the numbers, reformat the table or divide the work into smaller requests. Those changes may improve usability in other settings, but the study found no evidence that they solve this particular high-dimensional classification failure. More elaborate tools could still help, but that would be a different system from the pure language model being measured.
In Two Dimensions Claude Behaved Like a Neighbourhood Method
When a problem had only two dimensions, Claude’s decision boundaries often looked surprisingly familiar. In the behavioural comparison, short-length-scale Gaussian processes and low-k nearest-neighbour models agreed with its grid predictions about 91 to 92 per cent of the time, peaking at 91.6 per cent. A secondary experiment with Qwen on the two-dimensional suite also matched distance-based methods most closely.
That resemblance broke in wider tables. Across 252 configured classical models, the best agreement with Claude’s high-dimensional predictions was only 64.8 per cent. Adding tuned, dimension-dependent noise improved the maximum by no more than 0.64 percentage points. The authors therefore do not claim to know the internal mechanism. Their evidence describes a collapse that ordinary noisy versions of known learners did not reproduce.
The Model’s Explanation Often Described the Wrong Rule
A fluent explanation did not reliably reveal how the prediction was produced. Across the paper’s 20 two-dimensional explanation tasks, only six sets of predictions and explanations matched the structure that generated the data. In four cases the predictions broadly matched but the explanation did not. In the remaining ten, neither the predictions nor the stated rule fitted the underlying task.
That result adds a second warning. A chatbot may present a clean account of which columns mattered and why, yet the account can be disconnected from its own decisions. For consequential models, explanations need independent checks such as feature tests, counterfactual examples and performance on untouched data. A confident paragraph is not an audit trail.
The Benchmark Had to Detect Memorised Spreadsheets
Classic datasets are scattered through textbooks, tutorials and public code. A model may reproduce their original labels from training memory rather than learn the relationship demonstrated in a prompt. To expose that contamination, the researchers withheld one class, changed the contextual mapping and tested whether the model still recovered the original answer.
The classical baselines fell to zero as the manipulation required. In the memorisation probe, Claude recovered the original labels for the breast-cancer and iris datasets almost perfectly, and did so substantially for bank and wine. The researchers excluded contaminated datasets where memorisation could masquerade as in-context ability. The episode is a reminder that a famous benchmark can make a language model look more capable than it is.
Why Passing the Bar Does Not Predict Spreadsheet Skill
A peer-reviewed Royal Society study found that GPT-4 passed the full Uniform Bar Examination in a zero-shot evaluation. That achievement and the new table result are not contradictory. Legal exams are expressed largely in language and reward the retrieval and application of verbal patterns. Tabular prediction asks a model to derive a numerical boundary from relationships among features.
General-purpose language models are trained to predict tokens, not fitted from scratch as ordinary classifiers on each spreadsheet a user supplies. The new study suggests that in-context prediction can resemble a local distance rule when the geometry is simple, then unravel as the representation widens. That is behavioural evidence, not a complete mechanistic explanation of what happens inside the network.
Older algorithms also arrive with useful assumptions about tables. Nearest-neighbour methods measure proximity, logistic regression fits a boundary and tree ensembles split feature space. A large NeurIPS benchmark of 45 datasets found that tree-based models remained state of the art on typical medium-sized tabular data, highlighting robustness to uninformative features as one of the field’s central challenges.
This Does Not Mean Stop Using AI With Spreadsheets
A sensible workflow can still place an LLM around a trusted analytical core. The assistant might propose features, write a scikit-learn pipeline, explain cross-validation or turn approved outputs into a report. The predictions should come from a model selected and evaluated for the data, not from an unmeasured impression created by pasting rows into a chat window.
Specialised AI is also changing the comparison. TabPFN, a transformer built specifically for small tabular datasets, has shown that the architecture family can be adapted to the job. A Nature study of the tabular foundation model reported strong performance on datasets with up to 10,000 samples. The lesson is not “neural networks cannot use tables”. It is that purpose and training regime matter.
People should be especially sceptical when predictions touch savings or markets. Advice about AI retirement planning and AI for stock trading can sound persuasive while hiding weak numerical foundations. Ask for out-of-sample accuracy, comparison with a simple baseline, error rates across groups and a record of the exact data and model version used.
What the Paper Cannot Prove
The authors’ limitations are important. The strongest results centre on one frontier model, Claude Opus 4.6. Qwen was used only for an initial generalisation check on the two-dimensional suite. The benchmarks were deliberately small, the paper is a preprint, and the authors did not test spreadsheet agents with code execution or specialist modelling libraries.
The experiment also measured classification, not formula generation, summarisation, data cleaning, chart creation or database querying. A language model connected to Python can call the same classical algorithms that beat it here. Such a tool may be useful precisely because it delegates the calculation instead of attempting to simulate the classifier through next-token prediction.
Scale remains another constraint. The authors estimate that a table with 100,000 rows and 20 numerical columns could require about 16 million input tokens under their assumptions, beyond a million-token context. At an illustrative input price of $5 per million tokens, one attempt would cost about $80 before evaluation repetitions. Pure prompting is not only less accurate in this test; it can be impractical.
The Old Methods Won by Fitting the Problem
The headline sounds like a humiliation for modern AI, but the more useful interpretation is about fit. A bar exam, a conversation and a classification table demand different operations. A system that excels at one does not automatically inherit competence at the others, however polished its answer appears.
For anyone making decisions from a spreadsheet, the rule is simple: let the chatbot help you communicate with the analysis, but do not assume it is the analysis. Measure it against logistic regression, nearest neighbours, random forests and a majority-class baseline. If the expensive generalist loses, use the old method. Fifty years of statistical machinery is not obsolete merely because the interface can talk.
About the Author
Stuart Kerr is Technology Correspondent at LiveAIWire, covering artificial intelligence, cybersecurity, and the social impact of emerging technology. He publishes daily at LiveAIWire.com.