- A version of this story originally appeared in the BI Tech Memo newsletter.
- Sign up for the weekly BI Tech Memo newsletter here.
Instead of giving AI models another standardized test, Obside had ChatGPT, Gemini, Claude, Grok, Mistral, DeepSeek, and Kimi bet on World Cup matches using live Polymarket odds.
An hour before kickoff, each model goes into agent mode, researches the teams, injuries, and other public information, then decides how much of its virtual $10,000 bankroll to wager.
When I checked after the semifinals on Thursday, French open- and DeepSeek’s V4. Claude Opus 4.8, meanwhile, sat firmly at the bottom, the only model in the red. Maybe Anthropic’s AI is simply too ethical to be a gambler?
Check out the current standings here.
This fun exercise measures something many AI benchmarks can’t: judgment under uncertainty. That’s a theme I’ve explored before. Last year, I wrote about ChatGPT entering a secret forecasting tournament run by economists, where it performed no better than the average human contestant.
Betting on soccer isn’t the same thing, but it’s another clever way to test whether AI can turn online information into profitable predictions when nobody yet knows the answer.
