Hot Generative AI Market Must ‘Cool Down’

To say that generative artificial intelligence has created a great deal of concern is an understatement. People are worried about everything from losing their jobs to losing their privacy to losing control to AI systems should they run amok. A concern that may not be top of mind — but should be, especially for organizations strategizing the use of generative AI — is the enormous amount of energy that the technology consumes and the resulting need for conservation and cooling. The impact will be widespread, but it will be felt especially keenly in small data centers and edge sites.

From virtual assistants to IoT-enabled healthcare diagnostics to supply-chain optimization and beyond, AI has made huge inroads across industries and holds seemingly limitless promise moving forward. Even taking into account the concerns over the use of AI and machine learning (ML), AI adoption has more than doubled in the past five years, according to a recent McKinsey survey. In addition, 63% of respondents to the survey say they expect their organizations’ AI investment to increase in the next three years.

Generative AI, specifically, uses deep learning models trained on large data sets to generate content, including text and images. AI applications consume more energy than traditional software because they are reading through far more data. In addition, the GPUs used to train generative AI models draw a significant amount of power. Greater demand for power will create greater heat, which will create greater demand for supplemental cooling systems.

“There are already huge resources involved in indexing and searching internet content, but the incorporation of AI requires a different kind of firepower,” said Alan Woodward, professor of cybersecurity at the University of Surrey in the United Kingdom, as reported in a recent Wired article. “It requires processing power as well as storage and efficient search. Every time we see a step change in online processing, we see significant increases in the power and cooling resources required by large processing [centers]. I think this could be such a step.”

While this will be a huge (to say the least) issue for AWS- and Google Cloud-level data centers hosting organizations’ generative AI applications, it will also be a significant concern for companies that run their own data centers and train systems on their own large language models (LLMs).

Partners will play a critical role in supporting these organizations as high-power generative AI systems create more heat than can be dissipated by traditional room cooling systems. Indeed, as a growing number of organizations seek to deploy on-premises generative AI systems for confidentiality, sovereignty or security reasons, partners can help recommend IT-grade cooling solutions with the ability to lower energy costs and increase the lifespan of IT equipment. On-premises AI solutions represent some of the most power-hungry systems found in a data center, so providing additional cooling close to the equipment is the most efficient way to handle this potential hot spot. Solutions suitable for small to midsize data centers, server rooms, edge computing data centers and other high-density zones include in-row precision cooling solutions and rack-mounted air conditioners.

Partnering with Eaton enables you to capture the growth of the emerging generative AI market, providing customers with all of connectivity and power management systems they will need — including racks, cooling, UPSs, rack PDUs, and KVMs —to balance the energy demands of generative AI with its potential.

One recent example of Eaton’s expanded cooling capabilities — the result of its acquisition of Tripp Lite — is the reference architectures the company built in partnership with Dell for its new Helix server line of generative AI systems. The Eaton solution for LLM training systems consists of three racks and four in-row cooling systems, while a second solution for inference systems encompasses two racks and two in-row cooling systems. These MDCs are powered by large three-phase UPSs of 120 kW and 60 kW respectively.

The infrastructure needed for generative AI systems requires careful design and expertise. Now is the time to ensure that your customers are aware of your ability to provide just that, anchored by a broad range of cooling solutions.  Eaton pre-sales support teams are available to help you build these new capabilities and take advantage of incremental revenue and margin opportunities.

This guest blog is part of a Channel Futures sponsorship.