By: Khalid Raza

Since the public internet was first introduced in 1995, two metrics have grown exponentially: The importance of internet reliability and the complexity of the internet. Of course, complexity works against reliability. As a result, we’ve been engaged in an arms race since 1995. Every time the internet became more complex, we had to work harder to keep the internet reliable.

Here is some context. In 1995, there were 16 million users on the internet. (Remember, the public internet grew out of the private Arpanet, so there was an existing set of users from day one). Today, we have more than 5.6 billion users. That’s an annual growth rate of 23% per year for 28 years. Traffic has grown from 0.1 exabytes per month in 2000 to 403.3 exabytes per month today (annual 43% growth).

This exponential growth of users coincides with other complexity drivers, such as hybrid cloud, edge networks, IoT, and an explosion in remote workers to create a vastly more complex internet.

Internet reliability is falling. According to Uptime Institute’s 2022 Outage Analysis, the number of outages is slowly growing due to increasing network complexity.

The reality is that we’ve outgrown the way the internet works. We need to relook at how we build enterprise networks.

Where Outages Come From

Recent network outages that caused loss of connectivity to cloud providers have highlighted the need to change how we implement peering. What made sense in a simpler era now creates an increasingly fragile internet.

Figure 1: Tier 3 providers offer customers on-ramps through cable, fiber, and wireless line access.
_{click to enlarge}

To understand why we need to change peering, let’s explore how the internet routes traffic today. Most service providers connect with each other, and there is an established hierarchy:

Tier 1 Providers

These are usually the national or global carriers. They own extensive backbone and peer with prominent national or global carriers. These providers don’t buy transit from any regional carriers but provide them with transit. These carriers maintain full internet routing tables that provide transits to Tier 2 or regional providers.

Examples of Tier 1 service providers in the US are AT&T, Verizon, Sprint, Century Link, etc.

Tier 2 Providers

These are regional providers. They peer with Tier 1 providers, buy transit from them, and provide transit to downstream providers. Essentially, they connect the hierarchy of local providers to the global provider by purchasing and selling transit. Examples of Tier 2 service providers include Hurricane Electric, Comcast, and Vodaphone.

Tier 3 Providers

They are local loop connections, or are what are called “Stub” networks. They don’t sell any transit but buy transit from Tier 2 providers or sometimes directly from Tier 1 providers. Tier 3 providers offer customers on-ramps through cable, fiber, and wireless line access. Examples include Comcast, Deutsche Telekom, and Verizon Communications (see figure 1 above). Generally speaking, traffic flows up from Tier 3 to Tier 1, across Tier 1, and then down the tiers to the provider. But that’s changing. Most user-facing network providers like Google, Facebook, and Amazon have increased their relationships with Tier 3 or local providers, and direct peering with