OpenAI gets caught vibe graphing
During its big GPT-5 livestream on Thursday, OpenAI showed off a few charts that made the model seem quite impressive — but if you look closely, some graphs were a little bit off.
In one, ironically showing how well GPT-5 does in “deception evals across models,” the scale is all over the place. For “coding deception,” for example, GPT-5 apparently gets a 50.0 percent deception rate, but that’s compared to OpenAI’s smaller 47.4 percent o3 score which somehow has a larger bar.
Or this one, where one of GPT-5’s scores is lower than o3’s but is shown with a bigger bar. In this same chart, o3 and GPT-4o’s scores are different but shown with equally-sized bars. That chart was bad enough that CEO Sam Altman commented on it, calling it a “mega chart screwup.” An OpenAI marketing staffer also apologized for the “unintentional chart crime.”
OpenAI didn’t immediately respond to a request for comment. And while it’s unclear if OpenAI used GPT-5 to actually make the charts, it’s still not a great look for the company on its big launch day — especially when it is touting the “significant advances in reducing hallucinations” with its new model.
During its big GPT-5 livestream on Thursday, OpenAI showed off a few charts that made the model seem quite impressive — but if you look closely, some graphs were a little bit off. In one, ironically showing how well GPT-5 does in “deception evals across models,” the scale is all…
Recent Posts
- How to watch the World Cup Final ‘66 In Colour for *FREE*
- ‘Elon Musk said he thinks humanoid robots will be in many homes in three years, and I agree with him.’ I sat down with Jake Dyson to hear his predictions for AI and robotics in your home — and why you shouldn’t throw out your stick vac just yet
- LaCie 8big Pro5 review: I tested LaCie’s huge 256TB DAS solution, and it’s ideal for 8K video editing but it comes with a price tag that’s just as big
- EA’s Star Wars Zero Company drops August 27
- Buying your dad a tech gift or gadget for Father’s Day? You may want to wait until Prime Day, if possible
Archives
- June 2026
- May 2026
- April 2026
- March 2026
- February 2026
- January 2026
- December 2025
- November 2025
- October 2025
- September 2025
- August 2025
- July 2025
- June 2025
- May 2025
- April 2025
- March 2025
- February 2025
- January 2025
- December 2024
- November 2024
- October 2024
- September 2024
- August 2024
- July 2024
- June 2024
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023