Thirsty work: why cutting-edge AI needs water cooling

The red-hot pace of the AI race is being felt in the financial, corporate, political and social spheres – and across the racks of servers driving the technology forward.

With AI expected to become integral to public- and private-sector operations, fierce competition is making powerful large language models (LLMs) increasingly thirsty for data, energy – and for water itself. This is because AI servers need the natural element, in the form of water cooling, to dissipate the heat generated by ever-increasing computing power. 

With heat a physical barrier to progress in AI, liquid-cooling solutions are becoming a crucial element of the supply chain to augment or replace traditional air-cooling services. The market is growing strongly – and with scaled incumbents benefitting from high barriers to entry, it provides enticing investment opportunities, in our view.

AI will be key to nations’ strategic advantage and future autonomy 

In a rapidly digitising world, AI will have a fundamental role to play in national sovereignty, security and economic productivity. 

From predictive analytics for improving transport, energy distribution, urban planning and defence systems to enhanced diagnostics, resource allocation and personalisation for healthcare and education services, there are use cases for AI across the public sphere. 

Within the private sector, effectively deployed AI can boost productivity, drive market creation, improve data controls and reduce supply-chain vulnerabilities. 

Given AI’s growing importance – in addition to the need to adapt LLMs to accurately reflect cultures, since most are trained on data from the West – national governments are keen to develop proprietary AI capabilities. However, with most lacking the necessary resources, they will need to adopt and adapt ecosystems developed elsewhere. Those countries selling their AI technology to others will potentially capture a share of new markets while also increasing their geopolitical influence.

US vs China: a two-horse race for AI leadership

The US and China are clear leaders in the development and rollout of AI technology and services. With other countries behind or yet to leave the starting gate, the superpowers’ advanced systems make it likely they will dominate AI globally, our Asia High Conviction Equity team believes. 

The US is battling to maintain control over key hardware, particularly the high-powered chips needed for complex inference – the ‘doing’ stage of the process in which a trained AI model applies knowledge to a task. It remains to be seen whether China will be able to develop its own next-gen AI chipsets, but it is ahead of the US in integrating AI into daily life – in areas like motoring, e-commerce, gaming and food delivery.

Given AI’s broad use cases across sectors, demand growth can be expected to be structural and significant, driving momentum for data-centre growth. Bank of America¹ forecasts a compound annual growth rate (CAGR) for total data-centre spending on servers, infrastructure and engineering of 13% between 2024 and 2028.²

However, amid the market fervour and great expectations for the technology, the application of AI for leaps in efficiency is still in its infancy. With Google and Microsoft¹ rolling out AI agents – knowledge-worker-like entities that can autonomously learn and perform task-oriented jobs without prescribed rules – demand for AI will likely increase as firms seek higher productivity.  

our expertise.

Asia High Conviction

learn more

Planetary Transition

learn more

When mixing water and electricity makes sense

Already, AI servers generate 5x more heat than traditional servers and require 10x more cooling per square metre. Although AI algorithms are becoming more efficient, models are being enhanced to handle increasingly complex tasks. And as compute grows, servers will need to stay cool.

With heat a major barrier to improving AI performance, the received wisdom that electricity and water shouldn’t mix is being turned on its head. Server racks are getting more densely packed as computational power increases, making effective cooling crucial to ensure stable performance. However, the heat created by the next generation of graphics processing units (GPUs), such as Nvidia’s¹ B200 chip, overpowers traditional air cooling. 

FIG 1. It’s getting hot in here: key metrics explaining the rise of water cooling⁴

The market for liquid cooling will outpace data centre growth

Due to the significantly greater cooling required by higher rack densities, Bank of America¹ forecasts a CAGR of 24% for the market – almost double the growth in overall data centre infrastructure spending.¹ Water cooling is likely to be a key beneficiary of this additional investment, in our view. 

While liquid leakage issues have proven to be a key bottleneck in the development of Nvidia’s¹ next-generation Blackwell chips, the optimisation of leakage detection and cooling component designs has largely solved this problem. With these issues resolved, the need for high-performance cooling is increasingly being met by liquid solutions. Goldman Sachs estimates a 54% liquid-cooling penetration rate in 2025 for AI training servers, rising to 72% in 2026.⁵

FIG 2. Liquid cooling to penetrate the market for AI training servers⁶

Scaled incumbents on the front foot 

Liquid cooling is more complex and requires very different infrastructure from air cooling, involving a long and sophisticated supply chain. As cooling becomes more critical for performance, semiconductor manufacturers including Nvidia, AMD and Intel¹ are introducing their own rack-level architecture. Major tech firms including Microsoft and Meta¹ have also introduced reference designs. In our view, scaled incumbents in the cooling subsector will be best placed to benefit, leveraging existing partnerships with such companies to collaborate on the design of new rack-level cooling solutions. 

Water will expand and reshape the cooling market

Liquid cooling is critical to the rise of rapidly scalable applications, such as the cloud-computing solutions offered by Google, Meta and Microsoft¹. It will also enable better performance in resource-intensive applications like crypto mining, as well as support the use of AI systems in compact environments where stable operations would be impossible using air cooling.

As well as being more effective, ‘direct-to-chip’ cooling, whereby water is passed through cold plates attached directly to microprocessors, is far more energy efficient than air-cooling solutions. The difference is significant, since about 30–40% of the energy that a data centre consumes is typically used for cooling.⁷ While a long way from commercialisation, a future generation of cooling could be based on systems that fully immerse GPUs in liquid, which would deliver an even more efficient solution.

Investment opportunities: a positive medium-term outlook

Competition in the water-cooling market is set to intensify in the next three to five years, in our view. The market for some key components – such as the control units for liquid-cooling drives – will continue to favour dominant players due to the high barriers to entry. 

In the case studies below, we highlight two companies that we perceive as being illustrative of the potential opportunities for equity investors.

Keeping cool in the AI race

As demand for AI compute intensifies, traditional server cooling methods are proving inadequate, making advanced water-cooling solutions essential and driving strong growth in this part of the data centre supply chain. But the complexity involved in implementing these systems creates significant barriers to entry, giving established digital infrastructure players with scale and tested offerings a clear advantage.
 

Explore the impact of water-cooling technology in AI with Lombard Odier: Quenching computing’s insatiable thirst – the AI water cooling challenge