Allion Labs / Romonz Kao

Mobile communication technologies have been flourishing incredibly in recent years, and 4G LTE has become one of the most compelling innovations. These innovations also give rise to products that support mobile network applications, such as portable routers and LTE routers. LTE routers enable users to share mobile connectivity over Wi-Fi or Ethernet. When LTE routers transmit mobile data via different interfaces, defects in product design or compatibility issues between chipsets and hardware devices might degrade throughput performance.

Here is a scenario: When a LTE router has a good throughput conversion, people can enjoy streaming 4K movies smoothly with the router at the same time. On the other hand, if the router has a lower throughput conversion, it may cause latency or take a longer time to load. The unstable network speed annoys users because they are likely stuck on a loading screen. These kinds of problems frustrate customers and lead to bad user experiences.

To put it simply, two major problems will degrade LTE router performance:

  • Low throughput conversion ratio
  • Throughput performance does not meet the specifications

This time, we choose three popular LTE routers available on the market. For each of them, we design conversion tests to verify the change of throughput performance in different LTE categories (Category 4 or Category 6) and transmission interfaces (Wi-Fi or Ethernet). This test review could be a reference for the future product development.

 

Device under Test (DUT):

The specifications of the three DUT are listed below:

  A B C
Wi-Fi 2.4GHz 802.11b/g/n 20/40MHz 802.11b/g/n 20/40MHz 802.11b/g/n 20/40MHz
Wi-Fi 5GHz 802.11ac 20/40/80MHz 802.11ac 20/40/80MHz Not support

LTE Category

Category 4 Category 4、6 Category 4

Table 1 LTE Router Specifications

 

Test Items:

LTE Category

  • LTE Category 4
  • LTE Category 6

Transmission Interface

-Wi-Fi Mode

  • Wi-Fi 2.4GHz: 802.11b/g/n 20/40MHz
  • Wi-Fi 5GHz: 802.11ac 20/40/80MHz

-Ethernet

 

Throughput Conversion Ratio Test Environment:

Test Configuration:

We run the test in an electromagnetic shielding environment. The radio communication tester is connected to the LTE router via a RF cable, and the test bed is connected to the LTE router via Wi-Fi.

Figure.1: Test Configuration

 

Test Equipment and Environment:

Radio Communication Tester (R&S CMW500) provides a stable mobile network for the test. The maximum transmission values are set up as below:

  • LTE Cat.4 : Downlink/Uplink : 127Mbps / 51Mbps
  • LTE Cat.6 : Downlink/Uplink : 254Mbps / 51Mbps

The RSSI value is set to maintain the maximum Wi-Fi speed (-30dBm to -35dBm).

Testing Methods:

The radio communication tester will send a signal to the tested LTE router. The LTE router will receive, convert and transmit that signal to the test bed. We define the following two values to evaluate the LTE performance:

  • Throughput Conversion Value: The data throughput collected by the test bed (LTE routers receive the signal from the radio communication tester, and later transmitt the signal to the test bed)
  • Conversion Ratio= Throughput Conversion Value / Transmission Value of Radio Communication Tester

According to Allion’s rich testing experience, we suggest that the conversion ratio should be above 90%.

 

Test Results:

For LTE Category 4, we verify whether the conversion performance of the three DUT in different interfaces meet the criterion—the throughput conversion ratio should be above 90%.

LTE Category 4 to Wi-Fi 2.4GHz: 802.11b/g/n (20 and 40MHz)

Based on the test results (Please refer to Figure 3 and Figure 5), for uploading, all three LTE routers meet the criterion. Plus, their throughput conversion values and ratios do not vary a great deal (The conversion values are around 49 Mbps).
However, for downloading (Please refer to Figure 2 and Figure 4), all three LTE routers do not meet the criterion because their conversion ratios stay below 90%. Moreover, for conversion values, Brand A has a lower one in either 20MHz or 40MHz. For example, in the case of 40MHz, Brand A’s conversion value is 97Mbps while two other values from Brand B and Brand C are around 107Mbps. Thus, compared with two other brands, Brand A performs poorly.

LTE Category 4 to Wi-Fi 5GHz: 802.11ac (20, 40 and 80MHz)

For 802.11 ac, we can only test Brand A and Brand B because Brand C does not support 5GHz band.

 

Based on the test results (Please refer to Figure 7, Figure 9 and Figure 11), for uploading, Brand A and Brand B meet the criterion. Their conversion values and ratios do not vary a great deal (both conversion values are around 49 Mbps). However, for downloading (Please refer to Figure 6, Figure 8, and Figure 10), only Brand B meets the criterion at 80MHz, and Brand B’s conversion value is also higher than that of Brand A. In conclusion, Brand B performs better at 802.11ac 5GHz..

LTE Category 4 to Ethernet Port

Based on the test results (Please refer to Figure 12 and Figure 13), for downloading, conversion ratios of three LTE routers on Ethernet meet the criterion. However, for uploading, Brand C has a much lower conversion ratio (only 71%).

 

Test Results Summary of LTE Category 4:

-Low Throughput Conversion Ratio

  • Wi-Fi Uploading: All three LTE routers’ conversion values are above 90%.
  • Wi-Fi Downloading: Only Brand B meets the criterion at 802.11ac 80MHz. Brand A and Brand C have significantly lower conversion ratios (60% to 85%).
  • LTE to Ethernet Uploading: Brand C’s throughput conversion ratio is only 71%, which is much lower than those of the two other brands.
  • LTE to Ethernet Downloading: The conversion ratios of all brands meet the criterion.

-Differences in LTE Throughput Performance

  • Wi-Fi Uploading: All brands meet the criterion and their values do not vary a lot, except Brand B, whose value is obviously lower on Ethernet.
  • Wi-Fi Downloading: Only Brand B meets the criterion at 802.11ac 80MHz.
  • Conversion Ratios while Downloading: Although Brand C performs better at 802.11bgn 20MHz, Brand B’s overall performance is the best.

In addition, Brand B supports the LTE Category 6. Does Brand B perform well enough like it does in LTE Category 4? Let’s take a look at the following test results.

LTE Category 6 to Wi-Fi 5GHz: 802.11ac (80MHz)

In Figure 15, we can see that Brand B’s uploading conversion values meet the criterion. However, for downloading, the conversion ratio in Category 6 is obviously lower (Please refer to Figure 14). Compared with the downloading condition of Category 4, the Category 6 conversion value is lower. Thus, Brand B does not achieve the throughput standard following Category 6 specifications which it claims to support.

LTE Category 6 to Ethernet Port

In Figure 17, we can see that Brand B’s uploading conversion value is above 90%. However, for downloading in Ethernet Port (Please refer to Figure 16), Brand B performs as poorly as what we saw in the previous Wi-Fi section—Brand B in Category 6 does not meet the criterion as it does in Category 4. Even though the value of Ethernet (128 Mbps) seems to be better than that of Wi-Fi (116 Mbps), the value (128 Mbps) fails to meet the throughput standard in Category 6.

 

Test Results Summary of LTE Category 6:

-Throughput performance does not meet the specifications:

Based on Allion’s rich, extensive experience in testing services, we discover that the maximum download transmission value for LTE Category 6 should be 300 Mbps. However, Brand B’s download conversion values in Wi-Fi and Ethernet are only 116Mbps and 128Mbps respectively. Those values are obviously lower than the standard. Thus, in LTE Category 6, Brand B does not meet the throughput conversion standard.

 

Conclusion

Based on the above test results, we pointed out the following problems:

  • Low throughput conversion ratio:

Conversion values of Wi-Fi downloading are generally low. Most Ethernet conversion values meet the criterion, but Brand C’s conversion ratio on Ethernet is significantly lower.

  • Throughput performance does not meet the specifications:

Among three LTE routers, Brand B performs the best. However, it is not up to par in Category 6.

  • Differences in LTE Throughput Performance:

Overall, Brand A has the least ideal throughput conversion performance.

 

In this test review, we have discussed certain issues of LTE routers, such as low throughput conversion ratio, low transmission speed and the performance failing to follow the product specifications. Those issues may frustrate customers and damage your brand image. To solve those issues that vendors may face, Allion can not only share expertise but customize the best testing services and solutions. We provide TR-398 standard test, following the first Wi-Fi in-premises performance testing standard in the world. This test can help ensure your router quality, build customer trust, and establish your brand reputation. If you are interested in testing your LTE routers, don’t hesitate to contact us.