There are cover bands, and there are COVER BANDS!

It's a great song, performed wonderfully. Just for fun!

AI ROI Metrics are Coming, Even if They are Essentially "Soft" Measures of Impact

We might as well be honest and predict that enterprises are going to develop all sorts of metrics that purportedly show the positive impact of their artificial intelligence investments, but that the metrics will quite probably be proxies that measure all sorts of things other than direct AI impact.

Still, some common metrics are a starting point:

• Cost per unit of output — Does AI reduce the labor or compute cost to produce a document, resolve a ticket, process a claim, underwrite a loan?

• Throughput / cycle time — How many units processed per hour, or how much time shaved off a workflow (e.g., code review, contract drafting, customer onboarding)?

• Error rates and rework costs — Does AI reduce defect rates, compliance exceptions, or manual correction loops?

• Headcount avoidance — the ability to scale output without proportional headcount growth. Often measured as "FTE equivalents automated."

Revenue-side metrics are less common, but might include:

• Conversion lift — Does AI-personalized outreach or recommendation improve sales conversion rates?

• Revenue per sales rep — If AI handles pipeline qualification or proposal drafting, does rep productivity improve?

• Customer retention / churn reduction — Does AI-assisted support or proactive intervention improve net revenue retention?

• Time-to-market — Does AI-accelerated R&D or software development compress product cycles in ways that generate earlier revenue?

Other operational outcomes also sometimes are quantified:

• Accuracy or precision rates on specific tasks (e.g., document classification, anomaly detection in fraud)

• Audit findings or compliance exceptions reduced

• Model risk KPIs — false positive/negative rates in detection systems

• Employee time recaptured — hours per week freed from low-value tasks, redirected to higher-value work

• Employee satisfaction / retention — particularly in roles prone to burnout from repetitive work

• Decision quality — harder to measure, but some firms track downstream outcomes of AI-assisted decisions against historical baselines

As rational as all that sounds, the metrics are “soft.” The attribution problem is severe, as AI is almost never the sole variable changing in a deployment. 

AI might be deployed while other changes also are occurring:

• Process redesign — Most AI deployments force workflow reengineering. Efficiency gains may be 60% process change and 40% AI

• Training and change management — The same tool deployed with weak adoption programs vs. strong ones produces dramatically different outcomes

• Data quality improvements — Organizations often clean and structure data as a precondition to AI deployment; that alone drives gains

• Personnel changes — New hires, role restructuring, or management changes co-occur with AI rollouts

• Macroeconomic or market tailwinds — Revenue gains during an AI deployment may reflect market growth, not AI impact.

• Hawthorne effects — Measuring a team's performance changes behavior regardless of the tool.

The point is that it can be almost impossible to isolate the impact of AI cleanly. So most enterprise AI ROI figures are really "ROI of the initiative that included AI," not AI's marginal contribution.

The more interesting question might be "which specific processes have changed in ways we can measure, and do we understand why?"

Skeptics are correct to argue that attributing success purely to AI is often an oversimplification. But enterprises will have to try and do so, as investors will demand such “proof.”

So firms will supply such “proof” as best they can, even if the outcomes are not, strictly speaking, solely because of AI use. 

And that is not an unusual case. 

Research highlights that AI’s impact is heavily moderated by "complementary assets.” In other words, a firm’s  organizational structure, existing data quality and worker skill levels often do more to determine the outcome than the AI model itself.

Study Focus	Key Finding Regarding Attribution	Source
Productivity Paradox	AI adoption does not guarantee boosts; results are contingent on organizational structure and worker attributes.	Cho et al. (2026)
Social Penalty/Bias	Using AI for assistance causes observers to attribute success to the tool rather than the person, leading to negative competence assessments.	Reif (2025)
Supply Chain/Bias	In complex systems, responsibility is fragmented across vendors/platforms, making it nearly impossible to attribute specific outcomes to one source.	Sharma et al. (2026)
Task-based Impact	AI improves performance within its "capability frontier" but degrades it outside that range; attributing net gains requires granular task-level data.	Brynjolfsson et al. (2023)

The difficulty in quantifying the immediate return on investment for new technologies is a recurring theme in economic history.

During the 1970s and 1980s, despite massive corporate investment in information and communications technology, overall productivity growth in many industrialized nations remained stagnant. This led economists to question whether computers were truly providing the expected value.

Eventually, results were observed, but:

• Results lagged deployment: it took decades for firms to fully "reimagine" their organizational structures, business models, and workflows to leverage the new technology effective

• Value was indirect: better management, more efficient coordination or improved service quality, but correlation, not causation, remains a question. 

The measurable financial benefits of a transformative technology often became clear only after business processes were redesigned.

Technology	Scope of Impact	Key Findings	Source
ICT / General IT	U.S. Economy (1995–2000)	ICT accounted for 56% of labor productivity growth; added 1.18 percentage points to GDP growth.	Oliner & Sichel (2000)
Emerging Tech (AI/ML)	U.S. Public Firms (2009–2019)	Over a three-year period, “neither the mean nor the median abnormal ROE (expected performance) reaches statistical significance in the post-implementation period.” “The mean abnormal inventory turnover is −1.06, which is not significantly different from zero.” “Overall, our results…indicate no significant difference in performance between sample and control firms during the implementation period of emerging digital technologies.”	Li et al. (2024)
Internet / ICT	SME Growth (Global)	Web-savvy SMEs grew more than twice as fast as those with minimal web presence.	McKinsey (2011)

Still, in the meantime, we will see all sorts of metrics “demonstrating” AI impact. Enterprises making the investments have no choice but to try to do so, even if those metrics are “soft.”

We Used to "Google Ourselves," but Now We Will Want to Know Whether We are in the Weights

It was inevitable: perhaps we used to "Google ourselves." Now, with language models doing the heavy lifting, we want to know whether we are In the Weights. 

The “weights” are the numerical parameters that shape an AI model’s training and output, so the website tries to measure how well “a model includes a subject in its inference operations. 

Here's an example from website In The Weights. 

Management and Leadership are Two Different Things

There is a difference between "leadership" and "management." Most of us work for managers, most of the time. What we often want are leaders, even if some elements of both arguably are needed some of the time. 

Leadership might be said to be about influencing people, while management  might be said to be about control and creating predictable results.

source: Researchgate

Not all managers exercise leadership. Sometimes they don't have to do so. A manager possesses formal authority over resources, budgets, schedules, or people, but not every situation calls for exercise of leadership skills. 

Role	Why Management Matters More Than Leadership	Why Leadership Is Less Critical
Payroll manager	Accuracy, compliance, deadlines, controls	Processes are highly standardized
Accounts payable supervisor	Transaction processing and auditability	Little need to create organizational change
Air traffic control shift supervisor	Strict adherence to procedures	Innovation can be undesirable during operations
Nuclear power plant operations manager	Safety and process discipline dominate	Consistency outweighs vision
Warehouse scheduling manager	Resource allocation and throughput optimization	Employees typically follow established procedures
Regulatory compliance manager	Monitoring, reporting, and enforcement	Persuasion plays a smaller role than compliance
Manufacturing line supervisor	Quality, efficiency, staffing	Limited need for strategic transformation

Conversely, not all leaders manage. A leader possesses influence, whether or not formal authority exists. The examples include leadership in a combat situation.

Role	Leadership Characteristics	Management Authority
Scientific thought leader	Shapes research agenda through expertise	Often has no line authority
Distinguished engineer	Influences technical direction through credibility	May manage no employees
Open-source software creator	Mobilizes contributors around a vision	Usually lacks formal authority
Social movement organizer	Creates commitment and purpose	Few formal management responsibilities
University professor	Influences students and colleagues	Typically manages little organizational infrastructure
Industry analyst	Influences strategic decisions across firms	No direct authority over followers
Religious leader of a voluntary group	Influence depends largely on trust and shared values	Limited formal managerial control

One classic formulation is that managers do things right; leaders do the right things. 

The terms “leader” and “manager,” like the terms “leadership” and “management,” often are used interchangeably, and probably should not be, as they are very different things.

Dimension	Management	Leadership
Primary Purpose	Create order, consistency, and predictability	Create change, adaptation, and movement
Core Question	"How do we execute efficiently?"	"Where should we go next?"
Focus	Processes, systems, resources	People, purpose, direction
Time Horizon	Short- to medium-term	Long-term
Key Activities	Planning, budgeting, organizing, staffing, controlling	Vision-setting, aligning, motivating, inspiring
Relationship to Change	Minimizes unnecessary variation	Initiates and guides change
Source of Authority	Formal position and organizational role	Influence, credibility, and followership
Success Measure	Efficiency, reliability, consistency	Commitment, adaptation, transformation
View of Risk	Reduce and manage risk	Accept calculated risk for future gains
Communication Style	Instructions, coordination, monitoring	Inspiration, persuasion, meaning-making
Primary Resource Managed	Tasks, budgets, schedules, assets	Human energy, attention, commitment
Organizational Outcome	Stability and operational effectiveness	Renewal and strategic effectiveness

The classic example is combat leadership in a small team and bureaucratic management of the whole army, navy or air force. In combat, leadership is not so much exercised by the leader as assented to by the followers. In other words, you might say leaders are made by their followers. 

Managers and executives, on the other hand, never are really made by their followers. They hold positions or offices that confer authority. Bureaucratic authority, the holding of an office, is not the same thing as leadership. 

With the caveat that the balance could well be different in a fast-moving Internet business compared to a factory, a classic statement might be that “the manager’s job is to plan, organize and coordinate. The leader’s job is to inspire and motivate.”

The degree of predictability and time frames often dictate when management is key and when leadership is more important. Highly-predictable scenarios do not require leadership. 

On the other hand, any institution that expects to last over multiple human lifetimes is going to rely on management rather than leadership, for the most part, as it is necessary to create stable structures over long periods of time.