Why should you soak test your 5G CPE?

For any technology to become mature, a huge amount of tests have to happen behind the scenes. Given the typically tight deadlines and recent development of hardware and software components, rigorous testing is a reliable means of avoiding market failure 5G is no different in that respect. In fact, testing this technology becomes especially relevant because it is not a simple upgrade, it involves much complexity. Those offering 5G are looking at high-demand users who may depend on it for enabling further technological advancements. As such, the scope for flaws is practically nonexistent.

Much of the current 5G device testing has a focus on signaling and raw throughput tests using tools like iPerf. For example, Conformance Testing for 5G (3GPP 38.523) does not have any real-world data transfer scenarios. While the 5G signaling test is of much significance, there is also a critical need for real data transfer tests from an end-user angle. This allows a real, clear, on-ground understanding of how the device will perform. 

Wireless devices today are incredibly complex which, in spite of heavy testing, may leave possibilities for issues like memory leaks and race conditions. For mobile devices such issues are usually not observed because mobile phones usually get rebooted for one reason or the other – battery draining out, turning on the airplane mode (soft shutdown), etc. In cases where none of these happen the user might just reboot the phone when s/he thinks it is getting slow or unresponsive. Unlike mobile devices, the reboot option does not apply to 5G CPE devices.

Once installed, these devices are supposed to be out of sight and out of mind. They are meant to work for years, without any external support. Other devices such as  TVs, home IoT devices, security cameras, mobile phones, laptops, tablets, smart watches etc are connected to it constantly. Since 5G CPE devices are expected to run for years at a time, without any reboot, issues like minor memory leaks etc. might show up and affect their performance within a few months into usage.

These may be taken as symptoms for lower performance, difficulty in connecting new devices, etc. Certain parts of the system get tested only under heavy loads, e.g. buffer limits, queues. Many borderline events like race conditions get hit under heavy loads. Hence, the device behaviour must be tested in suboptimal conditions. A normal 5G end user (e.g.: a 5G connected home) will have multiple traffics that have different QoS requirements – video streaming, browsing, voice calls, security camera, IoT devices, etc. – running simultaneously. It is critical to simulate real world environments and multi QoS traffic loads in the test lab because the alternative to troubleshoot real field issues would be to send support staff onsite, a rather expensive option. 

It is possible to minimize this challenge by running multiple long duration soak tests on different setups under different client load and traffic conditions. Even a seemingly simple action like adding/removing multiple 5G cells might unearth some issues if done long enough. Another example is real time traffic like Skype for Business calls getting dropped during peak business hours when maximum number of employees are using the online resources. 

A two week soak test covering the different usage scenarios in a typical home or small office can help bringing out issues. For home use cases, video traffic at different resolutions might be of higher importance whereas for office use cases productivity tools like emails, ftp, Skype for Business calls might carry more weightage. VoIP call drops during peak business hours is a common issue reported from multiple real offices, so that is definitely one use case to cover in the soak test. 5G is still unproven in the real world. Any negative experience, even if it is caused by non-3GPP components in the device, may get blamed on 5G. So, the stakeholders in the ecosystem will have to leave no stone unturned to ensure that whatever they put in front of the customer is their best.

How do you test 5G CPE for long duration?

WiCheck emulates 100s or 1000s of WiFi clients and helps you to recreate real world scenarios very easily. It helps in long duration stress tests of 5G device for 1000s of hours with multiple users accessing the network simultaneously. It runs real applications(Facebook, YouTube, YouKu, Skype for business calls & more) and measures the end user experience.

Have you got any questions regarding 5G CPE end to end long duration stress tests?

Stay up-to-date on the latest developments in Alethea: aletheatech.com