The AI infrastructure bottleneck is becoming a CIO problem - InformationWeek

The AI industry's infrastructure ambitions are beginning to collide with physical reality.

In recent weeks, multiple reports have highlighted delays and constraints affecting the expansion of AI capacity, from data center construction bottlenecks to growing concern over power availability. A recent JPMorgan analysis pointed to mounting pressure on energy infrastructure as AI-related electricity demand accelerates. Our sister publication, Data Center Knowledge, has chronicled the legal disputes, permitting delays and contracting complexity that are increasingly slowing the development of new AI data centers.

At the same time, major technology companies continue to increase their AI infrastructure spending, reinforcing expectations that enterprise demand for compute will also continue to rise sharply.

For CIOs, the issue is becoming harder to ignore. The AI conversation has largely focused on models, applications and productivity gains. Less attention has been paid outside infrastructure circles to the infrastructure required to sustain enterprise AI adoption at scale -- and to what happens if that infrastructure becomes constrained, delayed or regionally uneven.

David Linthicum, a former Deloitte managing director and founder of Linthicum Research, said the industry is already experiencing "a classic mismatch between announced investment and deployable capacity."

The immediate risk is not necessarily a dramatic shortage of AI capacity. More likely is a gradual shift toward a more constrained operating environment, where inference becomes more expensive, access less predictable and prioritization decisions increasingly unavoidable. That possibility is already prompting some technology leaders to rethink the assumptions underpinning their AI roadmaps.

The gap between AI investment, operational capacity

The scale of investment flowing into AI infrastructure remains enormous, with hyperscalers and AI vendors continuing to spend billions in pursuit of future compute supply. But several experts said the industry may be underestimating how difficult it is to convert capital expenditure into operational AI capacity.

The challenge, several experts said, is that physical infrastructure scaled far more slowly than software demand.

"Capital commitments make headlines, but power availability, permitting, grid upgrades, cooling, specialized hardware supply and construction timelines slow real delivery," Linthicum said. "Money is moving faster than infrastructure."

Edward Liebig, CEO and CISO of Yoink Industries and an adjunct professor at Washington University in St. Louis, emphasized that the challenge extends beyond compute availability. "The demand curve for AI infrastructure appears to be outpacing not only data center construction, but also power availability, cooling, interconnect scalability and the operational integration needed to bring these environments online reliably," he said.

Yet Liebig also cautioned against treating infrastructure constraints purely as a supply problem. In his view, the pressure is exposing weaknesses in how enterprises themselves are approaching AI deployment.

"What we're beginning to see is that infrastructure constraints expose whether organizations have a disciplined AI operating strategy or simply an accumulation of disconnected AI initiatives competing for resources," Liebig said.

That distinction may become increasingly important as enterprises scale AI adoption across departments. Many organizations are experimenting simultaneously with copilots, AI-assisted workflows, analytics tools, retrieval systems and agentic systems, often without centralized governance or operational prioritization. Liebig described the result as "AI sprawl," where infrastructure demand grows faster than measurable business value.

"The organizations most affected by AI capacity shortages may not be the ones with the least infrastructure, but the ones with the least operational discipline around AI deployment," he said.

David Linthicum, founder, Linthicum Research

How infrastructure pressure could surface

Not every expert believes enterprises are facing an immediate AI capacity crisis. Donald Farmer, futurist at Tranquilla AI, took a more measured view, arguing that many CIOs may have more time than current headlines suggest.

"We expect agentic AI to be the big driver of enterprise adoption, not GenAI," Farmer said, referencing TDWI research that shows only 31% of businesses think agentic AI adoption is happening now; 49% predict it will take 1-5 years. "So, I suspect there is still time for power production to pick up."

Farmer also pointed to improving efficiency across both models and hardware, which will lessen the compute burden. Even so, several experts agreed that constraints are likely to emerge unevenly, with midsize enterprises potentially facing the greatest pressure during periods of peak demand.

"I suspect training runs are safe," Farmer said. "Hyperscalers, when capacity is tight, will presumably prioritize their own first-party AI workloads and their largest enterprise customers."

Linthicum similarly framed the issue less as outright scarcity and more as intermittent instability. "The biggest risk is not that AI disappears, but that access becomes more expensive, delayed or uneven across regions and providers," he said.

That distinction matters because many enterprise AI strategies currently assume relatively frictionless access to compute. Organizations building roadmaps around rapid experimentation, real-time inference and always-available AI services may need to prepare for a more constrained environment than they initially anticipated.

"One of the emerging risks here is that organizations may unintentionally build business processes that assume infinite AI availability and infinite inference responsiveness," Liebig said. "Physical infrastructure realities may challenge that assumption sooner than many expect."

AI governance becomes an infrastructure issue

The prospect of constrained AI capacity is also beginning to reshape conversations around governance and prioritization.

Liebig argued that enterprises focused on operational assurance and resiliency may ultimately be better positioned during periods of infrastructure pressure because they tend to expand AI more deliberately. These companies tend to prioritize operationally critical use cases and expand incrementally once value, governance and controls are validated.

"Bounded expansion creates resilience because organizations can prioritize the AI functions that matter most when infrastructure conditions tighten," Liebig said.

That approach also changes how CIOs evaluate AI investments internally. The central question becomes less about acquiring additional AI capacity and more about determining which workloads justify priority access to constrained infrastructure.

Linthicum described a similar need for operational discipline. He argued that CIOs should begin separating AI initiatives into tiers -- critical, important and experimental -- so infrastructure allocation becomes intentional, rather than reactive.

"Enterprises without contingency plans are the most exposed," he said.

That shift may also force organizations to become more selective about where frontier AI models are truly necessary. Farmer noted that many enterprises are already finding success with smaller, local models running on commodity hardware, particularly in environments where governance, compliance or cost concerns make cloud dependence less attractive.

"Not everything has to run on the latest and greatest model," Farmer said.

What CIOs should ask vendors now

As infrastructure constraints become more visible, experts said CIOs should also begin treating AI capacity as a resilience and continuity issue rather than simply a procurement concern. In order to get ahead of potential issues, IT leadership needs clarity into their current supply.

Linthicum said enterprises need far more transparency from vendors about how capacity shortages are managed. "They should ask very directly about capacity guarantees, regional availability, queue priority, pricing volatility, failover options and portability between environments," he said.

Farmer similarly argued that conversations should increasingly focus on operational reliability, not feature sets. Among the questions he suggested CIOs ask vendors were the following:

"What is your contractual commitment on capacity availability during peak windows?"
"If I commit to multi-year reserved capacity, what does that purchase me in terms of priority versus on-demand customers?"

Liebig pushed further, arguing that CIOs should demand visibility into how vendors themselves behave under constrained conditions.

"How are workloads prioritized during peak demand?" he asked. "Can services degrade gracefully under infrastructure stress? What dependencies exist on shared GPU pools or third-party model providers?"

Those questions reflect a broader change underway in enterprise AI strategy. Infrastructure availability, once treated largely as an abstract hyperscaler concern, is increasingly becoming an operational dependency. Enterprise AI roadmaps will need to factor in not just what organizations want AI systems to do, but also whether the underlying infrastructure can reliably support those ambitions at scale.

About the Author

Senior Editor, InformationWeek

Madeleine Streets is a senior editor at InformationWeek, where she shapes stories and contributes news analysis through a CIO lens.

She comes to InformationWeek from TechTarget’s Learning Content team, in which she authored explainers and features on a range of enterprise IT topics. Before moving to the field of enterprise technology, Madeleine spent several years covering retail, consumer finance, and ecommerce technology for fashion trade publication Footwear News. She has also been published in Women’s Wear Daily, TIME, Associated Press, SELF, and Observer, among others. The thread that ties her coverage together is a commitment to honest, impactful storytelling -- and insatiable curiosity.

Outside of writing, Madeleine can be found studying wine, singing in her local choir, and working her way towards her annual reading goal of 100 books. She is based in New York City, US.