AI is changing computing, big and small

The implications of these changes will be transformative across a number of critical dimensions, including the speed of technological development, sector leadership, electricity production and consumption, construction and industrial supply chains, environmental considerations, market economics, national security interests, as well as financing and investment. To stay at the forefront of the market, leaders will need to make unprecedented investments in technology infrastructure. If large data centers currently cost between $1 billion and $4 billion, data center costs in five years could be between $10 billion and $25 billion.

Pressure on resources

The energy demand and prices of these large data centers will impose limits on the number of centers that can be built and how quickly they can be built. The rush to acquire AI resources is already creating extreme competition for high-end resources in the market, and growing data center demands will further strain capabilities.

Energy consumption is a critical example. Utilities are already responding to requests from hyperscaler customers to significantly increase their power capacity over the next five years. Their needs will compete with growing demand for electric vehicles and the relocation of manufacturing, putting strain on the electricity grid. Growth in electricity demand has remained virtually stable over the past 15 to 20 years, but investments to expand and strengthen the grid and add new energy sources (including on-site generation and renewable energies) will have to increase considerably.

Infrastructure providers and technology supply chains, including networking, memory and storage, are also investing to meet the high-performance computing demands of hyperscalers, digital services companies and enterprises. Large data centers will push the boundaries and unleash innovation in physical design, advanced liquid cooling, silicon architecture, and highly efficient hardware and software co-design to support the rise of AI.

Cutting-edge innovation

As companies evaluate the tradeoffs between cloud and edge computing for AI, deciding where to manage inference is critical. One consideration is how much to focus on specific areas and tasks, in order to use better organized and more targeted data to create targeted models that reduce the load on the compute infrastructure.

Another issue is how to bring more computing power closer to the edge for AI in low-latency-tolerant environments, such as autonomous driving. The rise of smaller models and specialized compute that can run these models at the edge are important steps in this direction. At the same time, the industry is rapidly developing new form factors for the edge, including edge AI servers, AI PCs, robots, speakers and wearable devices.

Expansion preparations

The changing nature of data centers and edge computing increases the likelihood that AI will reshape the technology sector and establish a new order for the next era. Companies in the sector should examine their market position and rethink their strategic ambitions to ensure they remain competitive in their chosen areas.

• Cloud service and data center providers. The major challenge for large players in this market segment will be finding ways for their AI capabilities to meet the future demands of their customers. Vendors will need to decide what to provide as a service and what to provide as enabling technologies at the industry level. Their efforts will also focus on accelerating model development and collaborating across the supply chain to build large, distributed data centers. This will require the ability to refocus on compelling opportunities, rapidly build capacity, and form partnerships that strengthen the platform. Meta, for example, is competing with OpenAI, Alphabet and others to secure a leadership role in major language models. To support these ambitions, Meta has significantly increased the scale of its computing capacity over the past two years. Meta also released Llama as an open source language model, to serve as a catalyst in the broader ecosystem.
• Infrastructurestructure providers. AI workloads require more specialization than previous generations of computing. Companies that design and manufacture servers, networking, storage, cooling, power, cabling, and all the other elements needed to build a data center will need to design their products to support the AI. They will develop large-scale solutions to optimize the calculation and performance of AI software. These companies also play an important role in providing infrastructure and services to customers. Accelerating the introduction of AI to market represents a significant opportunity for businesses.

• Software providers will continue to integrate AI into its core products to remain competitive. Increasingly, their business will need to focus on capturing and interpreting data insights while optimizing language models to deliver better (and faster) results to customers. These aspects of their business will complement each other as software companies build capabilities to upskill their customers’ workforces.
• Edge device manufacturers will find ways to capitalize on innovation across the ecosystem, testing new form factors and interfaces and using AI to increase personalization across devices. Sorting out user privacy preferences will be key to increasing adoption rates.
• Data Center Supply Chain Suppliers have a formative opportunity to reshape their role in the marketplace as power centers proliferate and edge computing evolves. These players will focus on building scalability capabilities and developing meaningful partnerships with engineering companies that can help address the challenges of large data centers and more sophisticated edge computing.

As hyperscalers and other large companies plan for the large data centers needed to meet the needs of AI, other factors will also need to be considered. Of these, investment requirements are perhaps the most important, as companies compete for funding for many large projects at once. Stresses on the power grid are another area where companies have limited direct control. They may also need to manage the environmental implications of data center expansion and electricity consumption, including the effect on their carbon footprint and emissions reduction pledges. The challenges are vast and complex, but as the global race to win in AI intensifies, no company in this ecosystem can afford to sit back and wait; now is the time to act.