"Not Seeing AI Productivity" Storyline is Inevitable

It’s inevitable that we will keep seeing headlines, and seeing, hearing and reading stories about how many businesses are not seeing financial returns from their investments in artificial intelligence. 

Important new technologies rarely show up in the bottom line immediately, and the issues are structural.

First of all, business processes have to be recreated to harness the innovations. 

When electricity first entered factories, managers simply replaced their massive central steam engine with one massive electric motor. Productivity didn't move. Only after firms discovered they could put a small motor on every individual machine (the "unit drive") were they able to redesign the factory floor. 

In 2026 companies are using AI to "chat with docs" or "summarize emails" (overlaying tech on old habits) rather than redesigning their entire supply chains. That will take some time. 

Also, firms must retrain workers and staffs. That imposes real costs (time and money) while possibly lowering productivity in the short run as time and effort is diverted to such training. So a "J-curve" of productivity will happen: lower productivity in the near term, with the benefits in the future. .

Then there are measurement issues, such as how to quantify the impact of quality, variety and speed. If an AI helps a legal team finish a contract in two hours instead of 10, but the firm still charges a flat fee, the "productivity" is invisible to the GDP, even though the human cost has plummeted.

Study	Technology Period	Key Finding	Duration of Lag
Paul David (1990)	Electricity (1890–1920)	Factories had to be physically demolished and rebuilt to utilize "unit drive" motors before TFP spiked.	~30–40 Years
Robert Solow (1987)	Computing (1970–1990)	The "Solow Paradox": You could see the computer age everywhere except in the productivity statistics.	~20 Years
Brynjolfsson et al. (2021)	AI & Software (2010–2021)	Formulated the "Productivity J-Curve"; firms must invest in unmeasured "intangible assets" that initially depress earnings.	Ongoing
NBER / Juhász et al. (2020)	19th Century France	Productivity in mechanized spinning was initially lower and more dispersed than hand-spinning due to the need for factory reorganization.	~15–20 Years
Man Group / Bara (2026)	Generative AI (2023–2026)	80% of firms report no macro productivity impact yet, despite task-level gains of 15-55%, due to "workflow friction."	Projected 5-10 Years

In the meantime, leaders will have to try and come up with some quantifiable metrics (directly related or not) to justify the investments. It won’t take too much imagination to realize that headcount reductions are one such way to “show” outcomes, even if AI and headcount are indirectly, loosely or even unrelated in the short term. 

In 1900 the "electricity bubble" looked real to everyone still using steam. By 1920, the steam users were bankrupt. 

So “productivity proxies” must be developed.

The most immediate impact of AI is the compression of time. 

Firms can measure the "distance" between an idea and its execution.Time-to-prototype can show how many days it takes to move from a natural language prompt or requirement to a functional, testable version.

Draft-to-final ratio might be used by marketing and legal firms to measure the time spent on the "first 80 percent" of a task versus the "final 20 percent" of human polishing.

For engineering teams, the metric isn't just "lines of code," but the number of successful pushes to production per developer per week. 

Larger firms might try to assess the reduction in total "human hours" spent in meetings.

Query-to-find latency is a measurement of how long it takes an employee to retrieve a specific piece of internal tribal knowledge. AI should reduce that latency. 

Admin-to-maker ratio tracks whether the percentage of an employee's day spent on "coordination" is shrinking in favor of "creation." 

“Agents” also will need new metrics that quantify AI outcomes as though it were a digital employee rather than a software tool.

Autonomous completion rate is the percentage of workflows that an AI agent initiates and completes without a human "click" or intervention.

Human-in-the-loop friction measures how often an agent has to "hand back" a task to a human because it hit a reasoning wall. A falling HITL rate is a leading indicator of future productivity.

Token efficiency per outcome calculates the cost of AI "thinking" (API/Compute costs) relative to a successful business outcome. 

Business Function	Traditional Metric (Lagging)	AI Proxy Metric (Leading)	Why it Matters
Software Engineering	Lines of Code, Story Points	PR Cycle Time	Measures how fast code is reviewed and merged, not just typed.
Legal, Compliance	Billable Hours	Review Velocity per Page	Shows the acceleration of document ingestion and risk flagging.
Customer Support	First Response Time	Resolution via Zero-Touch	Measures the percentage of issues solved entirely by agents.
R&D	Patents Filed, Products Launched	Iteration Cycles per Quarter	Shows how many "failed fast" experiments the firm can run.
Human Resources	Headcount Growth	Talent Density (Revenue/FTE)	Measures if the firm is scaling output without scaling people.

The “productivity lag” is entirely predictable. So are the storylines about it. Sure, it is a significant practical problem for those firms making the investments. But the “lag” storyline is entirely predictable.

What AI Changes in the Area of Connectivity

Year in and year out, it is always safe to predict that connectivity bandwidth demand will grow. But we might always ask how specific innovations drive growth in different parts of the connectivity fabric.

Broadly speaking, artificial intelligence computing workloads and need for connectivity occur across several parts of the network:

• Intra-Data Center: GPUs within a single cluster must constantly synchronize "weights" and "gradients" during training. This requires 800G (and soon 1.6T or 3.2T) optical links.

• Data Center to Data Center (DCI): Large models are increasingly trained across distributed clusters in different geographic regions to tap into available power grids. So data center to data center capacity must be reinforced.

• Data Center to End User: While inference uses less bandwidth than training, the content density is increasing, moving from text-based AI to real-time video. This is primarily access network augmentation and already is happening for other reasons.

But it isn’t simply the data volume that AI might be changing. 

AI traffic, being machine traffic, has different characteristics than human-generated traffic, which has fairly well-defined hourly patterns across any 24-hour period. AI traffic, on the other hand, is less predictable. 

Redundancy and load-shifting are traditional ways of dealing with temporary spikes in compute demand, but AI demand spikes might paradoxically also cascade, if additional resources in other regions are insufficient, or if the connectivity fabric cannot support spikey load shifting. 

So some might say the architecture of internet traffic is evolving from human-shaped to machine-shaped. 

Connectivity Segment	Growth Driver	Primary Demand Metric	Expected Impact/Tech Shift
Intra-DC (Back-end)	GPU-to-GPU synchronization for LLM training.	10x fiber density increase.	Shift to 1.6T/3.2T Ethernet; Co-packaged optics (CPO).
DCI (Data Center Interconnect)	Distributed training & "Check-pointing" across regions.	145% CAGR in pluggable optics.	800G ZR/ZR+ becomes the standard for long-haul.
DC to Edge/User	Real-time Video GenAI & "Agentic" workflows.	30% - 50% annual traffic surge.	Growth in Edge AI data centers to lower latency.
User to DC (Uplink)	AI Wearables (Video glasses) & sensor-rich IoT.	Shift in Asymmetry.	Uplink demand starts to rival Downlink in specific sectors.
Network Management	AI-driven traffic orchestration (NaaS).	38% CAGR in software.	Shift toward Network-as-a-Service (NaaS) for dynamic scaling.

It is reasonable to expect many other app and behavior-related changes to happen as well. 

If AI is rapidly increasing network traffic volume and unpredictability, enterprises and global Internet infrastructure providers will have to redesign their systems for resilience. 

So new enterprise bandwidth, latency, and congestion controls will be needed to handle these loads, many will argue, including:

• Treating AI workloads as having distinct traffic patterns

• Add traffic shaping, rate limiting, intelligent filtering, and workload isolation features  

• Reducing cross-region data movement, placing data closer to models

• Building redundancy across regions and providers

• Using real-time monitoring and predictive analytics to detect anomalies

• Assuming double to five times “typical” traffic spikes.

Compute platforms supporting “AI compute as a service” will likely have to consider:

• Expand bandwidth, backbone capacity, and route diversity

• Deploy distributed inference and GPU capacity at the edge

• Implement AI-aware routing and advanced congestion management

• Increase regional redundancy, maintain reserve capacity

• Enable dynamic scaling and improved tenant isolation.

The point is that the costs of creating new AI compute facilities has a number of costs in addition to shells (buildings), power and processors. 

Category	Examples	Why needed for higher AI compute	Very rough cost indications (order of magnitude)
High‑performance interconnect inside clusters	InfiniBand/Ethernet switches, NICs, NVLink bridges, optical cabling, spine‑leaf fabrics	Distributed training and large MoE models need low‑latency, high‑bandwidth links between thousands of GPUs; networking can be a large fraction of AI cluster capex.	Per large AI cluster, networking (switches, NICs, optics) can easily run into hundreds of millions of dollars; per GPU, interconnect can add roughly USD 3,000–10,000 over server cost depending on scale and topology
Storage systems	High‑performance NVMe in servers, parallel/distributed file systems, object storage, backup/archival storage	Training data lakes, checkpoints, model artifacts and logs require very high throughput and capacity; storage performance strongly affects GPU utilization.	Full AI stack hardware (servers + storage + networking) often implies base systems at roughly USD 5,000–45,000 per server before GPUs; petabyte‑scale storage systems add millions to tens of millions of dollars per region.slyd​
Advanced cooling infrastructure	Direct‑to‑chip liquid cooling, immersion tanks, rear‑door heat exchangers, upgraded chillers, pumps, heat‑rejection systems	Rack densities >30–100 kW for GPU servers make air cooling insufficient, forcing large investments in liquid cooling plants, distribution loops, and monitoring.	Liquid‑cooling deployments for AI halls can cost tens of millions of dollars per site; over life of the facility, cooling energy is a major part of 15–25% “power & cooling” share of AI TCO.
Power delivery beyond basic “power”	Substations, high‑voltage switchgear, UPS, PDUs, busways, redundant feeds (N+1/2N)	Dense AI clusters require huge, highly reliable power; providers must oversize and harden electrical systems to avoid outages and support higher rack densities .	Upgrading a site’s electrical plant for AI (substation, UPS, distribution) typically runs in the tens to hundreds of millions of dollars for hyperscale campuses, depending on MW added and redundancy.aegissoftte
Data‑center facility upgrades (non‑shell)	Containment systems, raised floors, structural reinforcement for heavy racks/tanks, fire suppression tuned for liquid cooling, white‑space re‑fit	Existing halls often must be rebuilt to handle heavier racks, new coolant loops and different airflow patterns; safety systems are upgraded for new thermal/chemical risks.	Retrofit of an existing hall to AI‑grade density can cost several thousand dollars per square meter; full hall conversions often run into the tens of millions of dollars per building.
WAN and inter‑DC networking	Metro and long‑haul fiber, DWDM equipment, edge routers, private backbone upgrades	AI workloads move large datasets and models between regions and availability zones; cross‑DC bandwidth demand grows sharply with multi‑region training and inference.	Large cloud backbones already represent multi‑billion‑dollar capex programs; incremental AI‑driven capacity (fiber pairs, optical gear) can be hundreds of millions of dollars over a few years for a major provider.
Orchestration , MLOps, and control‑plane software	Cluster schedulers, container platforms, model registries, CI/CD for ML, usage metering/billing	To sell “AI compute as a service,” providers need sophisticated software to allocate GPUs, manage jobs, track utilization, and integrate storage/networking; complexity grows with scale.	Many platforms are internally developed; external software licensing and support can be on the order of 10–15% of AI infrastructure TCO over five years in some deployments.
Observability, telemetry, and optimization tools	Monitoring for GPUs, fabric, cooling, DCIM/BMS integration, AI‑driven optimization (e.g., cooling control)	Keeping thousands of GPUs fully utilized and within thermal/power limits requires deep telemetry and automated tuning, which are non‑trivial engineering investments.aegissoftt	Enterprise‑grade observability stacks and DCIM/BMS integration typically cost millions of dollars per large site over their life (licenses plus engineering and integration work).
Security and compliance	Hardware security modules, key management, secure enclaves, data loss prevention, access controls, audits	Enterprise AI workloads often involve sensitive data and regulated industries; clouds must harden AI clusters against exfiltration and meet compliance standards.	Security tooling and compliance programs add ongoing opex in the millions per year for large environments, plus capex for dedicated hardware and secure facilities.
Personnel and specialized operations	Site reliability engineers, network engineers (InfiniBand/HPC fabrics), MLOps teams, facilities engineers for liquid cooling	AI data centers need more specialized skills than traditional IT: tuning fabrics, managing liquid cooling, optimizing training pipelines, and running large clusters efficiently.	Personnel can represent 20–30% of AI infrastructure TCO over time; for hyperscalers, this means tens to hundreds of millions of dollars annually across global AI regions.slyd​
Support, maintenance, and spares	Hardware support contracts, spare parts pools, planned refresh cycles, vendor field engineers	High‑availability AI services require rapid replacement of failed components and regular firmware/software updates, increasing support intensity per rack.	Maintenance and support are often modeled as about 10–15% of AI infrastructure TCO across a 3–7 year horizon.
Land, water, and sustainability programs	Additional sites for new AI regions, water treatment/recycling for cooling, heat‑reuse infrastructure, carbon procurement	AI data centers often face local constraints on water use and emissions; providers invest in water‑efficient cooling, heat reuse, and carbon/renewable projects.	Water‑optimized cooling, treatment and heat‑reuse can add millions to tens of millions per site; broader sustainability and renewable programs for AI loads are multi‑billion‑dollar commitments across portfolios.