Is there a silver bullet to help data centers cope with AI’s energy demands?
There’s no denying the impact AI is already having on data centers and energy use. If left unchecked, the situation is only going to get worse. A recent IDC report reveals that with AI adoption skyrocketing, the energy needed to support AI workloads is set to soar, with data center electricity consumption expected to more than double between 2023 and 2028. AI-driven workloads alone are projected to grow at a staggering compound annual growth rate (CAGR) of 44.7% through 2027, with energy demands reaching a massive 146.2 TWh. The implications are stark – data centers, which already account for 46% of enterprise energy spend, could soon become unsustainable.
That, of course, cannot happen. But with AI workloads rapidly increasing, data centers have to evolve quickly and manage the prospect of a new energy crisis, with rising electricity prices driven by geopolitical instability in the Middle East. The growing influence of AI tools across industries – from healthcare to financial services – is undeniable. However, an AI-powered search uses 100 times more energy than a non-AI-powered search, while building foundational AI models can consume enough energy to power 20,000 homes for six months.
Director of Systems Engineering at Nutanix.
A solution?
An Atlantic Ventures report, Improving Sustainability in Data Centers 2024, suggests a solution, revealing how next-generation data center architectures, like hyperconverged infrastructure (HCI), can reduce energy consumption, lower carbon emissions, and drive cost savings across the EMEA region. In just six years, the report finds that modernizing data centers with HCI could save up to 19 million tCO2e in the EMEA region, equivalent to the emissions of almost 4.1 million cars. It could also save €25 billion by 2030 from improved energy and operational efficiencies.
As organizations integrate AI across operations and come to terms with the sheer scale of energy consumption, HCI could reduce the risk of spiraling costs and ensure sustainability goals are not missed. But it’s not just about HCI, it’s about how organizations work with AI. Focus should shift to optimising where and how AI workloads are processed, using modernisation to manage workloads more intelligently. This makes so much more sense than to just keep building more energy-efficient data centers.
This is important, as we have to take into account how AI works and where the demands for power are going to increase. While many organizations are captivated by the energy consumption required to train foundational AI models, for example, it is inferencing – the real-time decision-making AI performs – where the bulk of the energy is spent.
Foundational model training happens once but inferencing is a continuous process that happens millions of times, especially with AI-driven applications like fraud detection or predictive maintenance. Optimizing inferencing, particularly at the edge, could be the silver bullet data centers need to manage AI energy demands more efficiently.
Turning to renewable energy
As the IDC report suggests, more data center providers need to turn to renewable energy sources, but they also need to rethink their infrastructure. Hybrid cloud, edge computing, and on-premise systems offer a way to balance AI’s energy demands by distributing workloads more intelligently.
Processing data closer to its source with edge computing, for example, reduces the energy needed to transfer large datasets back and forth from centralized servers. Meanwhile, hybrid cloud computing environments can handle the computationally intense AI training tasks, leaving real-time inferencing to occur on-premise or at the edge.
Edge computing also plays a pivotal role by processing data closer to where it’s generated, such as in retail stores or IoT devices. This not only improves response times but also significantly reduces the energy required for inferencing.
Modern infrastructure is crucial to managing AI’s energy demands and a containerized platform, designed to handle both CPUs and GPUs, is necessary to run AI workloads efficiently. Storage also becomes critical, as AI typically deals with unstructured data like files and objects. By investing in high-performance storage systems and optimized compute stacks, businesses can significantly reduce the energy required to run AI applications.
Moreover, the ability to measure and manage energy consumption is critical. Platforms that provide real-time visibility into energy use enable data centers to optimize every stage of AI processing – from training to inferencing. Even a 10% improvement in energy efficiency, according to the IDC report, can lead to substantial savings.
Real-time decision-making
Rather than focusing solely on the immense energy costs of training foundational models, businesses need to pay attention to how often these models are used in real-time decision-making. Compressing models, refining their structure, and running them on platforms designed for efficiency will be key in reducing AI’s overall energy footprint. For example, we have developed container platforms and high-performance storage solutions specifically tailored for AI inferencing, offering businesses a way to optimize their AI workloads and temper their energy demands.
The true cost of AI is no longer just about performance and innovation, it’s about the energy required to sustain it. As organisations ramp up their AI initiatives, the question isn’t whether they can afford to invest in AI but whether they can afford the energy it consumes. With hybrid infrastructure and a focus on efficient inferencing, businesses have a way to temper this energy surge. Otherwise, those that ignore this reality may soon find their data centers at the mercy of an AI energy crisis.
We’ve featured the best colocation providers.
This article was produced as part of TechRadarPro’s Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro
There’s no denying the impact AI is already having on data centers and energy use. If left unchecked, the situation is only going to get worse. A recent IDC report reveals that with AI adoption skyrocketing, the energy needed to support AI workloads is set to soar, with data center…
