The grand talk about the extraordinary capabilities of artificial intelligence—we're even starting to hear talk of "superintelligence"!—can also crash into the cold wall of reality. Citrix engineer Robert Caruso wanted to know how quickly ChatGPT could beat a 48-year-old Atari 2600 at chess, the prehistory of personal computing.
Cutting-Edge AI Challenged by Retro Game
Using an emulator, the engineer started a game of Video Chess and asked ChatGPT (using the GPT-4o model) to analyze the board positions from images of the game board. Given the model's power and the limitations of the anemic Atari performance, he expected ChatGPT to win easily.
"ChatGPT made itself completely spray in beginner mode,” plays Robert Caruso in a LinkedIn post. "Even after being given a starting grid to identify the pieces, ChatGPT confused rooks and bishops, missed winning pawn moves, and repeatedly lost track of the pieces—first blaming the Atari icons for being too abstract, then doing no better after switching to classic chess notation."
That multi-billion dollar AI is beautiful! The Atari game, however, has very modest gaming capabilities. After an hour and a half of (virtual) sweat, ChatGPT agreed that it was no match. The bot still asked if it could start again...
ChatGPT is not a model designed for chess, unlike the open source Stockfish engine, whose ELO (a system measuring a player's level) exceeds 3600 - the best human players reach level 2800. But still, the fiasco is notable after the often exaggerated promises surrounding generative AI.
Asked by us about this total flop, ChatGPT explained to us that "even the greatest can have a slump... And besides, I wasn't trained to recognize bishops in 8 fluorescent green pixels"!
Source: ExtremeTech
0 Comments