IT, Public Policy Initiatives Fail at a High Rate. IoT Will Not be Different

In either private information technology or public policy arenas, failure is a common outcome. That is worth remembering as we enter a period of experimentation around internet of things use cases, ranging from smart cities to industrial automation.

Half of enterprise software projects fail, it often is found. There is some evidence that enterprise project failure rates have improved. But maybe not in the internet of things area, where 75-percent failure rates are not unheard of.  

One study of World Bank projects found failure rates of 25 percent on operational grounds, but up to 50 percent on the ability to solve the stated problem.

“Historically, the majority of major projects in government have not delivered the anticipated benefits within original time and cost expectations,” says a report by the U.K. National Audit Office.

“We’ve been in business 11 years, we have 1,400 customers, and 80 percent of all the projects we’ve seen in 11 years are customers who come to us with an existing failed IoT project,” says Nick Earle , Eseye chairman and CEO.

Policy organizations like the World Bank judge success based on whether planned products are delivered through an efficient process; not whether policies solve the problems that warranted intervention in the first place, or whether the policies promoted development outcomes, says Matt Andrews of the Kennedy School at Harvard University.

In his paper Public Policy Failure: ‘How Often?’ and ‘What is Failure, Anyway’?, Andrews asks a hard question: “public policies absorb resources to address major social issues,” so “we should know if policies are proving bad social investments; routinely failing to solve focal problems at high costs.”

Public policy initiatives represented an estimated 16 percent of global gross domestic product in 2017,  about $13 trillion, he notes.So a high rate of policy failure would mean that we are wasting these resources.

Between 2016 and 2018, for example, the World Bank reported that, of 400 projects, failure rates ranged between 25 percent and 50 percent. The failure rate is about 25 percent  if the definition is simply processes being followed or output measures achieved.

The failure rate is about 50 percent if the definition asks whether the policy intervention solves the problem it was designed to solve, says Andrews. The former measure is what we might call “efficiency.” The latter measure is “effectiveness.”

Effectiveness is what we want: an outcome that actually solves the problem we were trying to solve. Efficiency only measures whether we did the right things, using the metrics we said we’d use.

Others might ask even more provocative questions: are public projects destined to fail? Among the chief problems of many possible influences, consultants at PwC see three big buckets of issues:

• Methods and processes

• Stakeholder and leadership

• Complexity and uncertainty

source: Building State Capability

source: PwC

The first bucket of issues related to efficiency and procedures. The other two are fuzzy, political and had to quantify, because they deal with stakeholder involvement, leadership and the complexity of the problems policies are intended to fix.