AMD lands yet another major cloud deal as Oracle adopts thousands of Instinct MI300X GPUs to power new AI supercluster
AMD’s Instinct MI300X is an incredibly powerful AI accelerator, and major cloud companies are beginning to integrate it into their infrastructure to support intensive AI workloads.
Vultr recently announced that it had ordered “thousands” of MI300X units, and now Oracle Cloud Infrastructure (OCI) says it has adopted AMD’s hardware for its new OCI Compute Supercluster instance, BM.GPU.MI300X.8.
The new supercluster is designed for massive AI models containing billions of parameters and supports up to 16,384 GPUs in a single cluster. This setup leverages the same high-speed technology used by other OCI accelerators, enabling large-scale AI training and inference with the memory capacity and throughput required for the most demanding tasks. The configuration makes it particularly suited for LLMs and complex deep learning operations.
Preproduction testing
“AMD Instinct MI300X and ROCm open software continue to gain momentum as trusted solutions for powering the most critical OCI AI workloads,” said Andrew Dieckmann, corporate vice president and general manager, Data Center GPU Business, AMD. “As these solutions expand further into growing AI-intensive markets, the combination will benefit OCI customers with high performance, efficiency, and greater system design flexibility.”
Oracle says its testing of the MI300X as part of its preproduction efforts validated the GPU’s performance in real-world scenarios. For the Llama 2 70B model, the MI300X achieved a 65 millisecond “time to first token” latency and scaled efficiently to generate 3,643 tokens across 256 concurrent user requests. In another test with 2,048 input and 128 output tokens, it delivered an end-to-end latency of 1.6 seconds, matching closely with AMD’s own benchmarks.
The OCI BM.GPU.MI300X.8 instance features 8 AMD Instinct MI300X accelerators, delivering 1.5TB of HBM3 GPU memory with a bandwidth of 5.3TB/s, paired with 2TB of system memory and 8 x 3.84TB NVMe storage. Oracle will be offering the bare-metal solution for $6 per GPU/hour.
“The inference capabilities of AMD Instinct MI300X accelerators add to OCI’s extensive selection of high-performance bare metal instances to remove the overhead of virtualized compute commonly used for AI infrastructure,” said Donald Lu, senior vice president of software development at Oracle Cloud Infrastructure. “We are excited to offer more choice for customers seeking to accelerate AI workloads at a competitive price point.”
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
More from TechRadar Pro
AMD’s Instinct MI300X is an incredibly powerful AI accelerator, and major cloud companies are beginning to integrate it into their infrastructure to support intensive AI workloads. Vultr recently announced that it had ordered “thousands” of MI300X units, and now Oracle Cloud Infrastructure (OCI) says it has adopted AMD’s hardware for…
Recent Posts
- Arcane season 2 act 1 ending explained: who is [SPOILER], when is episode 4 coming out, and your biggest questions answered
- The Best Veterans Day Mattress Deals (2024)
- This new phishing strategy utilizes GitHub comments to distribute malware
- Arcane season 2 confirms the hit series isn’t just one of the best Netflix shows ever made – it’s an animated legend that’ll stand the test of time
- ICYMI: the week’s 7 biggest tech stories, from Kindle Colorsoft yellowing woes to our PS5 Pro and Mac M4 reviews
Archives
- November 2024
- October 2024
- September 2024
- August 2024
- July 2024
- June 2024
- May 2024
- April 2024
- March 2024
- February 2024
- January 2024
- December 2023
- November 2023
- October 2023
- September 2023
- August 2023
- July 2023
- June 2023
- May 2023
- April 2023
- March 2023
- February 2023
- January 2023
- December 2022
- November 2022
- October 2022
- September 2022
- August 2022
- July 2022
- June 2022
- May 2022
- April 2022
- March 2022
- February 2022
- January 2022
- December 2021
- November 2021
- October 2021
- September 2021
- August 2021
- July 2021
- June 2021
- May 2021
- April 2021
- March 2021
- February 2021
- January 2021
- December 2020
- November 2020
- October 2020
- September 2020
- August 2020
- July 2020
- June 2020
- May 2020
- April 2020
- March 2020
- February 2020
- January 2020
- December 2019
- November 2019
- September 2018
- December 2011