Microsoft-backed AI startup beats Nvidia H100 on key tests with GPU-like card equipped with 256GB RAM
D-Matrix’s unique compute platform, known as the Corsair C8, can stake a huge claim to have displaced Nvidia’s industry-leading H100 GPU – at least according to some staggering test results the startup has published.
Designed specifically for generative AI workloads, the Corsair C8 differs from GPUs in that it uses d-Matrix’s unique digital-in-memory computer (DIMC) architecture.
The result? A nine-times increase in throughput versus the industry-leading Nvidia H100, and a 27-times increase versus the A100.
Corsair C8 power
The startup is one of the most hotly followed in Silicon Valley, raising $110 million from investors in its latest funding round, including funding from Microsoft. This came alongside a $44 million investment round from backers including Microsoft, SK Hynix, and others, in April 2022.
Its flagship Corsair C8 card includes 2,048 DIMC cores with 130 billion transistors and 256GB LPDDR5 RAM. It can boast 2,400 to 9,600 TFLOPS of computing performance, and has a chip-to-chip bandwidth of 1TB/s
These unique cards can produce up to 20 times high throughput for generative inference on large language models (LLMS), up to 20 times lower inference latency for LLMs, and up to 30 times cost savings when compared with traditional GPUs.
With generative AI rapidly expanding, the industry is locked in a race to build increasingly powerful hardware to power future generations of the technology.
The leading components are GPUs and, more specifically, Nvidia’s A100 and newer H100 units. But GPUs aren’t optimized for LLM inference, according to d-Matrix, and too many GPUs are needed to handle AI workloads, leading to excessive energy consumption.
This is because the bandwidth demands of running AI inference lead to GPUs spending a lot of time idle, waiting for data to come in from DRAM. Moving data out of DRAM also means higher energy consumption alongside reduced throughput and added latency. This means cooling demands are then heightened.
The solution, this firm claims, is its specialized DIMC architecture that mitigates many of the issues in GPUs. D-Matrix claims its solution can reduce costs by 10 to 20 times – and in some cases as much as 60 times.
Beyond d-Matrix’s technology, other players are beginning to emerge in the race to outpace Nvidia’s H100. IBM presented a new analog AI chip in August that mimics the human brain and can perform up to 14 times more efficiently.
More from TechRadar Pro
D-Matrix’s unique compute platform, known as the Corsair C8, can stake a huge claim to have displaced Nvidia’s industry-leading H100 GPU – at least according to some staggering test results the startup has published. Designed specifically for generative AI workloads, the Corsair C8 differs from GPUs in that it uses…
Recent Posts
- Google now offers ‘web’ search — and an AI opt-out button
- Ayn’s new gaming handheld looks like a PSP, and it might just fill the hole in your heart left by Sony’s best portable
- Google Search is getting a massive upgrade – including letting you search with video
- Google Project Astra hands-on: Full of potential, but it’s going to be a while
- OpenAI chief scientist Ilya Sutskever is officially leaving
Archives
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023
- May 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- November 2022
- October 2022
- September 2022
- August 2022
- July 2022
- June 2022
- May 2022
- April 2022
- March 2022
- February 2022
- January 2022
- December 2021
- November 2021
- October 2021
- September 2021
- August 2021
- July 2021
- June 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- December 2020
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
- April 2020
- March 2020
- February 2020
- January 2020
- December 2019
- November 2019
- December 2011