OpenAI gets caught vibe graphing
During its big GPT-5 livestream on Thursday, OpenAI showed off a few charts that made the model seem quite impressive — but if you look closely, some graphs were a little bit off.
In one, ironically showing how well GPT-5 does in “deception evals across models,” the scale is all over the place. For “coding deception,” for example, GPT-5 apparently gets a 50.0 percent deception rate, but that’s compared to OpenAI’s smaller 47.4 percent o3 score which somehow has a larger bar.
Or this one, where one of GPT-5’s scores is lower than o3’s but is shown with a bigger bar. In this same chart, o3 and GPT-4o’s scores are different but shown with equally-sized bars. That chart was bad enough that CEO Sam Altman commented on it, calling it a “mega chart screwup.” An OpenAI marketing staffer also apologized for the “unintentional chart crime.”
OpenAI didn’t immediately respond to a request for comment. And while it’s unclear if OpenAI used GPT-5 to actually make the charts, it’s still not a great look for the company on its big launch day — especially when it is touting the “significant advances in reducing hallucinations” with its new model.
During its big GPT-5 livestream on Thursday, OpenAI showed off a few charts that made the model seem quite impressive — but if you look closely, some graphs were a little bit off. In one, ironically showing how well GPT-5 does in “deception evals across models,” the scale is all…
Recent Posts
- Best Buy slashes up to $400 off Apple tech in a limited-time sale — get AirPods, MacBooks, iPads and Apple Watches from $99.99
- The Instagram Plus subscription has officially launched
- Cyberdecks used to look like little laptops, but now they’re getting more personal
- Canada Prime Minister Mark Carney announces questionable national AI strategy
- Kevin O’Leary agrees to downsize massive Utah data center
Archives
- June 2026
- May 2026
- April 2026
- March 2026
- February 2026
- January 2026
- December 2025
- November 2025
- October 2025
- September 2025
- August 2025
- July 2025
- June 2025
- May 2025
- April 2025
- March 2025
- February 2025
- January 2025
- December 2024
- November 2024
- October 2024
- September 2024
- August 2024
- July 2024
- June 2024
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023