Shifting loads to smartphones can help meet AI’s ‘insatiable’ demand for power: Panellists

SINGAPORE – Rapid advances in artificial intelligence (AI) are placing more demands on power, and the task of meeting those demands will fall increasingly on end-devices such as smartphones, a panel of business leaders said.

Currently, AI systems rely on data centres to support their processes, with increasing demand for electricity, said Mr Nigel Toon, chief executive of Graphcore, a British semiconductor firm that develops specialised chips for AI.

“You look at the way database centres were built just a few years ago, maybe 20 kilowatts in a (server) rack. Now racks are 400 kilowatts,” he said, referring to the power assigned to a server rack in a data centre. “It’s just mind-blowing the amount of power that needs to be delivered to these sites.”

Eventually, this immense need for energy will strain the power infrastructure and affect the quality of electricity being supplied to other consumers.

Mr Toon was speaking at a panel discussion during Fortune Brainstorm AI Singapore: The Age of Intelligence on July 22.

The global conference is organised by Fortune and Accenture in partnership with Ant International. It is being held at The Ritz-Carlton, Millenia Singapore from July 22 to 23.

Fellow panellist Will Abbey, executive vice-president of Arm, a British semiconductor firm, emphasised energy efficiency as the key to delivering the performance desired of AI models amid a power scarcity.

Describing their “insatiable” power needs, he asked how semiconductors can deliver performance while using less power, since power is a scarce resource.

One solution discussed by the panellists was edge computing.

This would involve chips that distribute AI workloads onto “edge” devices like smartphones instead of having all of the computing done in the data centre. The distribution would thus reduce the amount of centralised power used by data centres.

This would allow AI models to run faster, which is essential for certain applications such as driverless cars, where instantaneous decisions are crucial.

“We’re pushing AI to the edge, and we’re going to see less emphasis on the data centre and more on the smartphone in our pocket, new models, new ways of thinking, as well as workloads,” said Fortune Brainstorm editorial director Andrew Nusca, who served as the panel moderator.

Customised chips could also provide another solution. Tech giants such as Google, Microsoft and Amazon Web Services are developing their own specialised chips designed to guzzle less power while meeting the performance requirements of AI workloads.

But the semiconductor industry is highly outsourced and fragmented. The designing of the chips, manufacturing of chip-making equipment, and the manufacturing of the chips themselves are carried out in different parts of the world.

The industry is now buffeted by geopolitical tensions between the US and China that are driving a “war” over such advanced chips, the panel noted.

US President Donald Trump has introduced export controls on some advanced chips from US firms like Nvidia. These export controls are aimed at restricting China’s access to the chips.

This restricts the customer base and encourages more vertical integration of supply chains – where the process from start (design) to end (packaging) takes place in the same country – an effect which Mr Toon is sceptical about.

A culture of excellence and continuous improvement is needed to manufacture chips at a high standard, he said, making it unlikely a vertical infrastructure for making chips could be easily transplanted from one country to another.

“This is not like, build a factory, plugging in a lock, and here you go,” he remarked.

Mr Toon recommended maintaining the status quo for the global chip supply chain, allowing it to remain fragmented.

Mr Abbey, on the other hand, pointed to the need for nations to think about “controlling their own destiny” – in particular, the need to control the infrastructure that supports advanced technology has become increasingly critical amid geopolitical tensions.

Nonetheless, he observed that there is a collaborative mindset in the AI industry and praised it, saying that this is what will enable companies to solve the problems discussed by the panel.