QoE: A new era of test and measurement to determine the sweet spot of network provisioning

By: Chris Chapman

Monumental changes in how companies test and measure their networks and network devices do happen; but they tend to happen slowly, only once every ten years or so. Today, we’re at a tipping point. Companies need to move beyond legacy Quality of Service (QoS) tests and begin focusing on Quality of Experience (QoE).

What QoE uncovers is the actual experience of the people who use the networks and the application services riding on top. It calculates metrics based on historical trends and measurements against baselines that define minimum acceptable quality.

Initially, in the decade spanning 1995-2005, the focus was on packets, which provided a single metric analysis of bandwidth and packet loss. The idea back then was that if the network slowed or was dropping packets, the solution was to add more bandwidth. During that decade, network faults persisted even as network equipment advanced and performance improved. But those faults were mitigated by the fact that application performance—in relation to the network—wasn’t as critical. If the network went down, there were work-arounds. Many business processes could still be done manually, with paper and pen, and voice and data networks were still siloed.


In mid-2000, the network’s importance began to change. Voice over IP (VoIP) started making headway as a business tool, as did video. Because of this, the network pipes became increasingly crowded. QoS became the buzzword, but testing and analysis still focused on packets and loss. The prevailing mindset continued to be around bandwidth: the more bandwidth and less packet loss, the happier the end user. Over time, however, that correlation fell apart.

To compensate, the industry began testing networks in labs before deploying them. But as companies added services on top of their pre-tested networks, they still had problems. The reason? The services were simply outstripping the abilities of existing testing techniques. The techniques provided very little insight into the relationship between the lab-tested networks and real-world scenarios.

This all brings us to QoE and today’s paradigm shift in the evolution of network test and measurement. QoE goes above and beyond the traditional metrics such as bandwidth, connections per second, or open connections, which are narrowly-focused metrics that measure specific engineering attributes of a device. For example, open connections and connections per second provide insights into table scaling and bandwidth provides information about forwarding efficiency—but neither of these metrics put it all together and directly measure how users perceive quality over time. Instead, metrics need to be analyzed as a system that includes multiple layers of the network architecture and multiple protocols used in the delivery of an application. Finally, the metric should be economically meaningful. This is how test and measurement techniques can catch failure impact, which in turn leads to lost revenue.