Sven Oehme, Chief Technology Officer (CTO) at DDN, drives innovation across both current and future products.
For decades, the AI infrastructure race was measured by a single equation: more compute, more speed, bigger clusters. But as AI workloads scale and mature, a new metric is emerging as arguably the most important: energy efficiency.
NVIDIA CEO Jensen Huang recently highlighted a stark reality: Doubling performance per watt can effectively double a data center’s revenue potential without consuming a single extra unit of power. Put another way, every watt that doesn’t produce AI output isn’t just wasted electricity—it’s lost economic opportunity. And in today’s AI-driven economy, that opportunity cost can be enormous.
From Speed To Strategic Productivity
The AI era has evolved far beyond occasional research experiments. Enterprises now run AI continuously, powering real-time decision-making, predictive analytics, generative content and even national-scale AI initiatives. This shift has brought new pressures and limitations.
• Enterprises face strict energy budgets and regulatory limits on carbon emissions.
• Hyperscale operators are constrained by local power availability and sustainability mandates.
• National or sovereign clouds must innovate within rigid energy allocations, balancing AI ambition with energy realism.
Traditional metrics like FLOPS, GPU count or memory bandwidth no longer fully capture the value of an AI system. Today, executives need to ask: “How much business value can this infrastructure deliver per megawatt?”
Tokens Per Watt: Measuring Real ROI
A practical way to evaluate AI performance in this energy-conscious world is tokens per watt. Tokens—whether words generated by a language model, images rendered by a generative AI or predictive decisions made by analytics engines—represent tangible units of AI output. By measuring AI productivity against energy use, organizations can directly tie infrastructure performance to economic impact.
This metric matters more than ever. Modern AI workloads are not batch jobs; they are continuous, multi-tenant and increasingly complex. They stress not only compute but also storage, I/O and data movement. Efficiency, in this context, is not optional—it is the foundation for sustainable growth.
Energy-Aware Infrastructure Strategy
Executives looking to scale AI effectively should adopt an energy-first approach:
1. Measure what matters. Track energy consumption in relation to AI output and business outcomes rather than hardware benchmarks alone.
2. Maximize utilization without increasing power. Identify bottlenecks or idle cycles that waste expensive compute resources.
3. Optimize before expanding. Re-architecting pipelines, streamlining data delivery or improving storage efficiency often delivers higher returns per watt than buying more hardware.
4. Gain visibility into energy loss. Power wasted in staging, transfer or idle cycles can often be recovered with smarter orchestration and intelligent scheduling.
5. Design for real-world workloads. AI is now an always-on system. Infrastructure should support low-latency inference, micro-batch processing and continuous retraining without excessive energy use.
Why Energy Efficiency Is A Boardroom Priority
Energy efficiency is no longer a technical concern confined to IT teams—it’s a strategic lever for growth. Organizations that optimize performance per watt can scale faster, reduce operating costs and meet sustainability commitments. They also gain a competitive advantage by delivering AI insights more reliably and efficiently than peers who continue to chase raw performance.
In short, the future of AI isn’t about who can deploy the largest clusters or the most powerful GPUs. It’s about who can extract the most value per unit of energy, translating efficiency into tangible business impact.
Leaders who embed energy efficiency into their AI strategy today will unlock new levels of ROI, scale responsibly and thrive in an era where AI is both pervasive and power-hungry. The companies that master this balance won’t just survive—they will define the sustainable, high-performance infrastructure model for the next decade.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?