AI chip built using ancient Samsung tech is claimed to be as fast as Nvidia A100 GPU — prototype is smaller and much more power efficient but is it just too good to be true?
Scientists from the Korea Advanced Institute of Science and Technology (KAIST) have unveiled an AI chip that they claim can match the speed of Nvidia‘s A100 GPU but with a smaller size and significantly lower power consumption. The chip was developed using Samsung‘s 28-nanometer manufacturing process, a technology considered relatively old in the fast-moving world of semiconductors.
The team, led by Professor Yoo Hoi-jun at KAIST’s processing-in-memory research center, has developed what it says is the world’s first ‘Complementary-Transformer’ (C-Transformer) AI chip. This neuromorphic computing system mimics the structure and workings of the human brain, using a deep learning model often employed in visual data processing.
“Neuromorphic computing is a technology that even companies like IBM and Intel have not been able to implement, and we are proud to be the first in the world to run the LLM with a low-power neuromorphic accelerator,” Yoo said.
Questions remain
This technology learns context and meaning by tracking relationships within data, such as words in a sentence, which is a key technology for generative AI services like ChatGPT.
During a demonstration at the ICT ministry’s headquarters, team member Kim Sang-yeob showcased the chip’s capabilities. On a laptop equipped with the chip he performed tasks such as Q&A sessions, sentence summation, and translations using OpenAI‘s LLM, GPT-2. The tasks were completed at least three times faster and, in some cases, up to nine times faster than when running GPT-2 on an internet-connected laptop.
Implementing LLMs in generative AI tasks typically requires numerous GPUs and 250 watts of power, but the team claims their semiconductor uses only 1/625 of the power of Nvidia’s GPU for the same tasks. In addition, because it is also 1/41 of the size, measuring just 4.5mm by 4.5mm, it could ultimately be used in devices like mobile phones.
Whether the chip can deliver on its promises in real-world applications remains to be seen, however. As Tom’s Hardware reports, “Though we are told that the KAIST C-Transformer chip can do the same LLM processing tasks as one of Nvidia’s beefy A100 GPUs, none of the press nor conference materials we have provided any direct comparative performance metrics. That’s a significant statistic, conspicuous by its absence, and the cynical would probably surmise that a performance comparison doesn’t do the C-Transformer any favors.“
More from TechRadar Pro
Scientists from the Korea Advanced Institute of Science and Technology (KAIST) have unveiled an AI chip that they claim can match the speed of Nvidia‘s A100 GPU but with a smaller size and significantly lower power consumption. The chip was developed using Samsung‘s 28-nanometer manufacturing process, a technology considered relatively…
Recent Posts
- Amazon expands enterprise AI play with wider availability of its Q chatbot
- YouTube TV on Android introduces new Multiview feature – here’s how to use it
- Prime Video movie of the day: Road House is an action-packed remake of the ’80s cult classic
- CBP is interrogating TikTok employees
- The European Union is investigating Meta’s election policies
Archives
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023
- May 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- November 2022
- October 2022
- September 2022
- August 2022
- July 2022
- June 2022
- May 2022
- April 2022
- March 2022
- February 2022
- January 2022
- December 2021
- November 2021
- October 2021
- September 2021
- August 2021
- July 2021
- June 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- December 2020
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
- April 2020
- March 2020
- February 2020
- January 2020
- December 2019
- November 2019
- December 2011