When Counting Anything, Methodology Matters

Methodology always matters, when conducting Internet speed tests or most any other phenomenon. So it is that Ookla apparently is taking some heat from small Internet service providers over a change of methodology by Ookla.

Specifically, Ookla now measures access speeds for the providers that serve most customers in any area. That is unfair, some small ISPs say, since, by definition, they do not serve many of the customers in any given city or state, and will not be included in speed indexes any longer.

The complaint is understandable. But Ookla’s methodology also makes sense. As somebody who routinely tracks the size and composition of markets, there are some realities. If you want to know what is happening in the U.S. mobile or fixed network market, you pretty much only need to know what is happening at the four biggest mobile providers, not “all” mobile providers, or only the few largest telco and cable TV companies.

The smaller providers do not have enough mass to affect the basic trends, one way or the other. It’s sort of the same reality as looking at “North American” trends in any area (Canada and the United States, for example).

Canada tends to represent about 10 percent of “U.S. plus Canada” activity in any area.

So to know the fundamental trend in “North America,” one basically must know what is happening in the U.S. market, as all of Canada only represents 10 percent.

A similar situation exists in other intra-U.S. markets, such as fixed network markets. In the past, all independent telcos collectively have represented up to 10 percent of total U.S. “telco” activity.

Today, with the emergence of cable TV providers as major providers, the rural and independent segment arguably represents an even smaller share of total local access activity.

To know a trend, one must know what is happening at the relatively small number of largest providers. The rest is nuance.