Assessing Customer Experiences with an Advanced Mobile Broadband Benchmarking Methodology

MedUX’s mobile QoE benchmarking methodology continues to provide a customizable and reliable framework for assessing customer experience. This year we have redefined our unique scoring methodology built on technical and user-centric parameters and aligned with international standards. By doing so we empower Mobile Network Operators (MNOs) and National Regulatory Agencies (NRAs) to make data-driven decisions while promoting fair competition.  
 

Regulatory authorities and telecommunications operators face the ongoing challenge of ensuring superior Quality of Experience (QoE) while advancing investments in network deployments and new technologies.  
 

Our mobile QoE benchmarking remains highly relevant for multiple stakeholders, including Mobile and Fixed network operators, device manufacturers, equipment/service providers, content providers, and end-users. It allows them to evaluate service, device, and network performance as well as pinpointing areas for improvement. By offering objective and transparent data, our benchmarking promotes fair competition and enhances the overall quality of mobile and fixed digital services.  
 

Customer experience has solidified its role as a crucial performance metric for MNOs, integrated into operational reporting, customer satisfaction programs, and market positioning strategies.

Additionally, NRAs are increasingly prioritizing customer experience. Traditional Quality of Service (QoS) assessments are evolving to incorporate user perception and measure the real-world impact of network and service performance on customer satisfaction.

With our refined methodology, MedUX remains at the forefront of evaluating mobile end-user QoE—also referred to as customer-perceived network quality or network experience. This approach captures both technical QoS parameters, such as throughput, DNS resolution time, latency, jitter, and packet loss, as well as user-centric QoE indicators related to web browsing, media streaming, gaming, social networking, and cloud storage.

Our latest advancements combine multiple performance indicators into a synthesized score—the MedUX QoE Scorecard. This valuable tool supports competitive intelligence, network optimization, public visibility, and quality certifications.

Setting up the Benchmarking Framework

Our mobile QoE benchmarking framework remains flexible and customizable, adapting to the specific needs of each client.

To ensure reliable and repeatable benchmarking results, we adhere to a standardized methodology encompassing all key aspects of mobile QoE. This includes defining use cases, selecting appropriate devices and data collection methods, identifying relevant metrics, measuring technical parameters, and analyzing results to generate actionable insights.

The benchmarking process involves:

• Selecting target use cases: Identifying the needs and goals based on client type, such as marketing, network intelligence, technology/vendor analysis, or consumer advocacy.
• Choosing appropriate data collection methods: Utilizing an optimal mix of proprietary software and hardware tools to simulate real user behavior and capture authentic QoE insights. Our approach integrates drive and walk tests via proprietary network testing robots alongside complementary data sources, such as crowdsourcing, SDKs, and Web Agents.
• Defining key performance indicators (KPIs): Establishing metrics and test schedules aligned with the QoE assessment objectives. This includes configuring software and hardware tools to evaluate targeted technologies and services.
• Executing testing and analysis: Simulating user interactions and collecting KPIs through automated tools within a specific geographical and temporal scope. The results are then analyzed using our scoring methodology to deliver a fair, transparent, and independent evaluation. This is reflected in the MedUX QoE Scorecard and MedOS (MedUX Opinion Scores).

By following this structured approach, MNOs and NRAs can identify opportunities to enhance service quality and optimize end-user experiences.  
 

Combining International Standards and Best Practices into a Unique Scoring Methodology

The MedUX QoE Scorecard remains a trusted benchmarking tool for assessing network quality and performance. Our methodology continues to prioritize an end-user perspective by measuring real commercial equipment in real environments, with real services and endpoints.

The score integrates diverse KPIs across service categories, weighted according to international best practices and industry contributions. Our evaluations encompass all elements affecting user experience, from environmental conditions and device capabilities to network infrastructure and service configurations.

Our methodology includes active speed tests that measure true end-to-end throughput via multiple Content Delivery Networks (CDNs). Operators may deploy dedicated endpoint servers for specific projects but not for independent benchmarking initiatives.

By adhering to international standards, including ETSI TR 103 559 V1.2.1, we ensure a consistent, reliable, and comparable benchmarking process across operators and markets.

Beyond maintaining a standardized approach for global comparisons, we continuously refine our methodology and scoring models to reflect technological advancements, market dynamics, and evolving consumer behaviors. Our QoE Scorecard adapts annually, revisiting requirements and thresholds to ensure continued relevance.

Key principles of our scoring methodology include:

• International standards and recommendations
• Comparability
• End-user perspective
• End-to-end performance
• Real services and true QoE KPIs
• Technology and market relevance

Leveraging True QoE KPIs from Real-Field Data

Our QoE benchmarking campaigns focus on the KPIs that best reflect customer experiences with real services. Through ongoing research and development, we continuously enhance our testing methodologies and metrics.

Our evolving service experience categories, now featuring separate MedOS scores for 5G-only and blended (3G/4G/5G) results, include:

• Overall QoE Score: An aggregate of availability, accessibility, speed, streaming, and OTT experience results.
• Accessibility (responsiveness): Evaluates network access timespeed, latency, and responsiveness, emphasizing 5G network entry times.
• Value for Speed (throughput): Assesses network capacity under stress, utilizing our patented Cloud Speed Test.
  • Downlink Speed
  • Uplink Speed
• Video Streaming Experience: Considers video resolution, loading time, and stall rate.
• Data & OTT Experience:
  • Gaming performance (latency, packet loss, jitter) for popular online games
  • Web browsing performance (loading times, DNS resolution, time to first-byte)
  • Social media (considering session duration and time to content) for popular social media platforms
  • HTTP Data Transfer (multi-thread download/upload performance)
  • Reliability: Assesses network consistency across all service categories, focusing on 5G stability by considering the success rate (%) of all tests and services described above. A test is considered as not successful if it fails or if does not meet minimum service thresholds.

Each category is assessed through numerous KPIs, generating vast datasets that are distilled into a synthetic performance indicator using our proprietary algorithm, grounded in international standards and recommendations.

Stay tuned for our upcoming Quality Seals and Awards, leveraging our QoE Scorecard and MedOS, highlighting top-performing operators and best-in-class networks.

Exploiting our unique synthetic QoE indicator for actionable insights

The MedUX QoE Scorecard covers a wide range of service categories, including web browsing, streaming video, online gaming, and cloud storage. This allows mobile network operators to understand the quality of experience that their subscribers are having, and to compare their performance to that of their competitors.

The MedUX Scorecard is designed to be accessible to different stakeholders within a mobile network operator, from executive management to more technical departments like engineering and optimization. This is because the service category scoring model and drill-down capabilities allow for granular analysis of the QoE of each service category, the so-called MedOS (MedUX Opinion Scores). This means that executives can get a high-level overview of the overall performance of the network, while engineers can dig deep into the data to identify and troubleshoot specific issues.

Our QoE Scorecard, and particularly the MedOS for each service category that we test, offer a simple way to translate several KPIs into a clear and easy-to-understand synthetic performance indicator.

With this approach, providers and operators can better understand their customers’ QoE and take proactive measures to improve it. The MedUX Scorecard provides a reliable benchmark for mobile broadband performance, allowing operators to compare their performance against industry averages and identify areas for improvement. By optimizing their networks and services, providers can offer a better experience to their customers and gain a competitive advantage in the marketplace.

Moreover, the QoE Scorecard can also provide insights into the impact of new technologies, such as 5G, on network performance and user experience. By evaluating the impact of new technologies and services on QoE, mobile operators can make informed decisions about network investment and service offerings.

Besides, the Scorecard can be used to assess the mobile experience in different geographical areas, allowing operators to identify areas for improvement and prioritize network investments. This is particularly important in ensuring equitable access to mobile services and bridging the digital divide.

In addition to providing valuable insights to mobile network operators, the MedUX QoE Scorecard is also used by industry regulators, governments, and consumer organizations to monitor the quality of mobile broadband services and to advocate for improvements in service quality. Overall, the MedUX QoE Scorecard is a powerful tool for promoting greater transparency and accountability in the mobile telecommunications industry, and for helping to ensure that consumers can enjoy the best possible mobile experience.

Illustrating scoring principles for a particular use case: Third 5G Quality of Experience Benchmark in Europe

MedUX is proud to launch its third 5G Quality of Experience Benchmark in Europe, expanding the scope to include more cities than in our first and second editions. The latest benchmarking campaign continues to build on our robust testing framework and methodology while incorporating refinements for even greater accuracy and insights. The results will be presented and discussed at MWC 2025, highlighting key trends and competitive insights.

Based on our standardized methodology:

• Select target use cases: Marketing and positioning, and network intelligence use cases.
• Select appropriate collection methods: MedUX Mobile robots with 5G and 5G-SA capabilities (in unforced mode).
• Define key performance indicators (KPIs): Data testing only (Both CDNs and independent dedicated servers), web browsing (WhatsApp, Amazon, Google, Kepler), social networks (Facebook, Twitter, Instagram and TikTok), gaming (Epic games, LOL, Valve, Blizzard), streaming (Youtube 4K), speed test, file download and upload, responsiveness and accessibility (ping), DNS resolution.
• Conduct testing and analysis: Expanded testing coverage across additional major European cities, with a continuous 3-4 day testing schedule in city centers.  
   

By assessing network availability, reliability, accessibility, and value for speed across a broad range of real-world user activities—such as web browsing, social networking, gaming, voice calls, and messaging—this benchmark provides a comprehensive and transparent view of the evolving 5G landscape in Europe. Our methodology ensures consistency and standardization, offering valuable insights to both mobile operators and regulatory authorities as they navigate the future of mobile connectivity.

Above indicators are also calculated across an operator’s 3G, 4G and 5G networks with a 5G device and subscription to obtain a blended result across technologies for 5G enabled devices. In this case, it does factor the results in 3G, 4G and 5G networks, and also the availability of each network technology.  
 

 CONTACT US 

Don’t forget to follow us on social media and subscribe to our newsletter to receive the latest updates and news.

Follow us on:

        

About MedUX

MedUX is the leading Quality of Experience (QoE) company, providing comprehensive and innovative solutions for measuring the performance of fixed, mobile, and TV telecommunications networks for telecom operators, governments, and digital enterprises. With a focus on delivering grand scale, end-to-end network, and service visibility, MedUX leverages real-time customer perspective data and advanced analytics to ensure quality and regulatory compliance, while also offering valuable insights for optimizing networks and improving customer experiences. With a presence in over 25 countries across Europe, America, Africa, and the Middle East, and monitoring over 60 operators worldwide, MedUX’s patented technology and expertise make it a trusted partner for improving the digital experiences of customers everywhere.