Don't Expect Measurable AI Productivity Boost in the Short Term

Many have high expectations for the impact artificial intelligence could have on productivity. Longer term, that seems likely, even if it might be hard to measure, in all cases. But short term results will be harder to detect.

One way of evaluating the potential artificial intelligence represents as a potential driver of productivity advances is to note that, since the 1970s, sources of improvement not related to information technology seem to be waning.

In other words, information technology has become a crucial driver of productivity in the modern economy as other sources of improvement seem to have slowed. The caveat is that productivity of office and knowledge work  is very difficult to measure. 

One way of evaluating the potential amount of work in the U.S. economy in 2024, for example, is to note that the Bureau of Labor Statistics estimates 70 percent of U.S. jobs are in the services segments, while goods-producing jobs represent about 30 percent of total.

Pre-1970, IT contributions to productivity arguably were modest. IT boosts increased through the 1990s at a significant level, but have accelerated since 1990, by most estimates. 

Time Period	IT Contribution	Underlying Improvement	Total Productivity Growth
Pre-1970s	Low (5-10%)	High (90-95%)	Moderate (1-2% per year)
1970s-1990s	Moderate (20-35%)	Moderate (65-80%)	Moderate (2-3% per year)
1990s-Present	High (40-60%)	Low (40-60%)	Moderate (1-2% per year)

On the other hand, total productivity has not grown all that much, averaging one percent to two percent in the most recent decades, despite IT apparently representing a higher percentage of total improvement. 

So one way to position the potential AI role is to note that sustaining the present one-percent to two-percent annual productivity increases might hinge on wringing more value out of AI. There always is the possibility that some AI use cases could boost productivity growth measurably, which clearly is the hope. 

The other angle is that AI--and IT in general--seems to have more impact in some industries, compared to others. Finance is typically among the industries considered to benefit most from almost any IT innovation. Government, education and healthcare tend to rank lowest. 

Industry	Pre-IT Contribution	1970s-1990s	1990s-Present
Finance & Insurance	Moderate (20-30%)	High (30-45%)	Very High (45-60%)
Manufacturing	Low (10-20%)	Moderate (20-35%)	High (35-50%)
Retail Trade	Low (10-15%)	Moderate (15-25%)	High (25-40%)
Wholesale Trade	Low (5-10%)	Moderate (10-20%)	High (20-35%)
Healthcare	Low (5-10%)	Moderate (10-20%)	Moderate (20-30%)
Education	Low (5-10%)	Moderate (10-15%)	Moderate (15-25%)
Government	Low (5-10%)	Moderate (10-20%)	Moderate (20-30%)

.Some industries--including computing, telecommunications and software--are technology-based so might always show high value from applied IT, but financial returns might not be uniformly high. 

Also, note that the financial return from holding bonds arguably provides a benchmark, as bonds are a product in the financial services industry but arguably do not benefit much, if at all, from IT. 

Yes, retail friction is reduced as bonds can be purchased, held or sold electronically. But that might be among the few measurable operational or capital investment benefits. 

E-commerce seems to be the industry most helped by applied IT. 

Industry	5-Year Average Return	10-Year Average Return	15-Year Average
Telecommunications	5-8%	6-9%	7-10%
Computer Hardware	8-12%	10-14%	12-16%
Software	10-15%	12-17%	14-19%
Content (Media, Entertainment)	6-9%	8-11%	9-12%
Advertising & Marketing	7-10%	8-11%	9-12%
Retail (Traditional)	4-7%	5-8%	6-9%
Retail (E-commerce)	12-15%	15-18%	17-20%
Education	6-9%	7-10%	8-11%
Finance	8-11%	9-12%	10-13%
Government Bonds	3-5%	4-6%	5-7%

So don't be surprised when AI does not boost productivity figures near term.

Customers Not Too Fond of ISP, Linear Video, Social Media Providers, ACSI Shows

Some things do not seem to change all that much, among them consumer satisfaction with home broadband or linear video subscriptions.

Though some providers in each of those categories score higher than average, customer satisfaction with internet service providers and linear video service always tends to rank lowest to near the bottom of all industries tracked by the American Customer Satisfaction Index. 

Over three to four decades, those linear video rankings haven’t moved much. The home broadband business is younger, but satisfaction scores for ISPs have actually been lower than scores for linear video, which always have ranked at or near the bottom of the ACSI industry rankings. 

source: ACSI, cnet

Streaming video scores have ranged higher than linear video, roughly in the middle of the industry rankings, and have improved since tracking began about 2000. Satisfaction with social media has not changed all that much since tracking began slightly before 2000, remaining near the bottom of industry rankings. 

Mobile phone service consistently ranks higher than linear video, home broadband or social media. The ACSI index includes multiple factors beyond price, such as quality, value, service, and expectations, so low scores might reflect a combination of these factors, such as low perceived value for money, customer service impressions and so forth. 

 

source: ACSI

Where, and How Much, Might Generative AI Displace Search?

Some observers point out that generative artificial intelligence poses some risk for operators of search engines, as both search and GenAI support human searches for information. 

And some already predict a drop in usage of search, as AI chatbots become replacements. 

By 2026, traditional search engine volume will drop 25 percent, with search marketing losing market share to AI chatbots and other virtual agents, according to Gartner. 

It is a difficult transition to analyze as chatbots clearly can replace some traditional search queries. On the other hand, many customer service interactions also involve queries. So those making estimates have to make assumptions about qualifies as a “search query” and when an AI alternative displaces a specific query. 

Search engines might continue to excel at tasks including:

• Open-ended exploration and discovery: Finding new information, exploring different perspectives, and serendipitous encounters with unexpected content.

• Factual research and verification: Accessing reliable and verifiable information from various sources, with ability to cite links and sources. 

• Complex or multi-step queries: Breaking down complex tasks into smaller steps, providing comprehensive results for specific needs.

AI chatbots might be better suited for handling:

• Simple, factual questions: Providing quick answers to straightforward queries requiring readily available information.

• Personalized interactions and guidance: Tailoring responses based on user context, preferences, and past interactions.

• Streamlined information retrieval: Completing specific tasks or answering questions within a defined domain. 

At a high level, that might point to search relevance for generalized knowledge; AI chatbots for domain-specific queries (single company; single industry; single geography, app, company or person. 

Study	Year	Findings
ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience	2023	This study found that ChatGPT excelled at answering straightforward questions and providing general solutions but fell short in fact-checking tasks. Users perceived ChatGPT's responses as having higher information quality compared to Google Search, despite displaying similar trust in both tools.
Bing AI chatbot vs. Google Search: Who does it better, and what about ads?	2023	This analysis suggests that search engines remain superior for specific tasks like finding factual information, local businesses, or real-time updates. Chatbots might be better suited for open-ended questions, personalized recommendations, or complex tasks requiring multiple steps.
How AI Will Replace Search Engines	2023	Surveyed users indicated a preference for AI chatbots for finding information, with 42% expecting to use them in the future. - Highlights the potential for chatbots to offer personalized recommendations and faster access to information.
29 Top Chatbot Statistics For 2024	2024	Predicts that up to 90% of health and banking queries could be handled by chatbots by 2022. - Suggests increasing adoption of chatbots across various industries.
Xu et al	2023	Users found ChatGPT responses to be more engaging and enjoyable, but Google Search provided more accurate and verifiable information.
Microsoft	2023	Early testers reported using the chatbot for specific tasks and entertainment, suggesting potential for niche use cases alongside traditional search.

“Organic and paid search are vital channels for tech marketers seeking to reach awareness and demand generation goals,” said Alan Antin, Vice President Analyst at Gartner. “Generative AI (GenAI) solutions are becoming substitute answer engines, replacing user queries that previously may have been executed in traditional search engines. This will force companies to rethink their marketing channels strategy as GenAI becomes more embedded across all aspects of the enterprise.”

To a great extent, the possibility of replacement might turn on the types of queries being made. Customer service and smart agent use cases are not formally substitutes for traditional search, for example. Commerce-related search is arguably more at risk of disruption, as are informational inquiries that already have shifted to map applications or shopping platforms and websites.

Source	Year	Forecast
Gartner	2021	30% of all customer service interactions will be handled by AI chatbots by 2025.
Juniper Research	2022	The global market for conversational AI platforms will reach $16.4 billion by 2027.
McKinsey & Company	2023	AI-powered virtual assistants are expected to handle up to 80% of routine customer inquiries by 2030.
Forrester	2021	Projects that AI-powered virtual assistants will become a primary channel for customer interactions by 2024.