Arm to address the insistent energy demands of AI

Arm’s CEO Rene Haas recently penned a compelling piece on the pivotal role Arm is playing in addressing the energy challenges posed by AI. In the blog, Haas highlights the power that today’s data centers consume, ie. 460 terawatt-hours (TWh) of electricity are needed annually at a global scale. That’s equivalent to the entire country of Germany. The rise in AI is expected to increase this figure 3x by 2030, more than the total power consumption of India, the most populated country in the world. 

The blog outlines Arm’s unique approach to addressing this challenge through its Neoverse CPU, renowned for its high performance and energy efficiency. Haas explains how Arm’s technology enables companies to optimize their silicon for demanding workloads, leading to significant power savings. He cites examples of major players like Amazon, Google, Microsoft, and Oracle adopting Arm’s Neoverse technology for improved performance and energy efficiency in their data centers.

The recent announcements include:

AWS Arm-based Graviton: 25 percent faster performance for Amazon Sagemaker for AI inference, 30 percent faster for web applications, 40 percent faster for databases, and 60 percent more efficient than competition. 

Google Cloud Arm-based Axion: 50 percent more performance and 60 percent better energy efficiency compared to legacy competition architectures, powering CPU-based AI inference and training, YouTube, Google Earth, among others.

Microsoft Azure Arm-based Cobalt: 40 percent performance improvement over competition, powering services such as Microsoft Teams and coupling with Maia accelerators to drive Azure’s end-to end AI architecture. 

Oracle Cloud Arm-based Ampere Altra Max: 2.5 times more performance per rack of servers at 2.8 times less power versus traditional competition and being used for generative AI inference models – summarization, tokenization of data for LLM training, and batched inference use cases. 

Haas points out that Arm Neoverse has enabled vast improvements on performance and power-efficiency for general-purpose compute in the cloud. However, customers are now finding the same benefits for accelerated computing. Large-scale AI training requires unique accelerated computing architectures, like the NVIDIA Grace Blackwell platform (GB200), which combines NVIDIA’s Blackwell GPU architecture with the Arm-based Grace CPU. 

“This Arm-based computing architecture enables system-level design optimizations that reduce energy consumption by 25x and provide a 30x increase in performance per GPU compared to NVIDIA H100 GPUs using competitive architectures for LLMs. These optimizations, which deliver game-changing performance and power savings, are only possible thanks to the unprecedented flexibility for silicon customization that Arm Neoverse enables.”

Haas further assures that with Arm’s broadening deployments, these companies could save upwards of 15% the total data center power. Those enormous savings could then be used to drive additional AI capacity within the same power envelope and not add to the energy problem. 

In conclusion, Haas reiterates Arm’s commitment to driving the future of AI compute while simultaneously contributing to environmental sustainability.

Partner Resources

Popular Right Now

Edgecore Insight Podcast

Ep-1: Navigating the Waters of Sustainability

Others have also read ...