Introduction
In recent blog posts we have been looking at the impact COVID-19 has been having on internet performance in the UK and USA. In this latest post we return to the UK to look more closely at the video streaming performance of Netflix, YouTube, and BBC iPlayer.
Netflix
Our Whiteboxes run regular tests to Netflix’s content servers, which are often hosted inside ISP networks. Netflix call these caches ‘Open Connect Appliances’ and it’s these same servers that you’ll be testing to if you use Netflix’s fast.com speed test.
Netflix announced in March that they would be reducing the streaming rate of videos for 30 days in Europe, and we can see the impact of that in the charts below.
The first chart, below, shows the Netflix download speed of two ISPs' 76Mbps VDSL products. Clearly visible each day is a dip during peak hours on ISP B. This is likely the result of some peak hour congestion between homes and the Netflix caches within ISP B’s network (we infer this because regular speed tests to our off-net test servers from ISP B see no such peak hour issues). ISP A sees dips during peak hours too, but they are far smaller. These peak hour drops predate the COVID-19 outbreak.
Most interestingly, you can see that on the 23rd March the peak hour dips are vastly reduced for both ISP A and ISP B. This is likely being driven by the change Netflix made to reduce streaming quality. Lower streaming quality means less bandwidth utilisation, which means more capacity for these tests to measure higher performance.
Netflix download speed for two UK ISPs' VDSL products, March 1st to March 25th. Enlarge chart
The next chart shows the change in Netflix download speeds as a percentage, from our reference date of March 1st for all major UK ISPs.
This chart is slightly harder to read, but it shows two key things. Firstly, both ISP B and ISP E have suffered Netflix congestion during weekends most noticeably. The reference date of March 1st was a Sunday, so the apparent improvement for the following days on ISP B and ISP E was simply due to them being weekdays, where Netflix usage is lighter. You can see that on the following weekend (8th and 9th), their performance fell to the same level again. Secondly, all ISPs saw the biggest dips in Netflix performance between the 20th and 22nd March (the first weekend of the lockdown), before suddenly seeing a big improvement around March 23rd.
Percentage change in Netflix download speed since 1st March 2020. Higher values are better. Results are derived from the Netflix video-streaming tests. Results from all hours of the day are used (not just peak). Enlarge chart
YouTube
There appears to be minimal impact to YouTube so far. All ISPs see between a 0.5% improvement and a 1.3% decline.
YouTube uses a slightly different streaming model to Netflix. Netflix’s servers will happily deliver you content as fast as you can receive it. However, YouTube’s servers will deliver content at just above the target bitrate of the video (they implement server-side rate limiting). This likely accounts for the significantly lower volatility of results than Netflix, as the broadband connections themselves are rarely ever reaching capacity, and it’s really YouTube’s rate limiting that we’re measuring here.
Percentage change in YouTube download speed since 1st March 2020. Higher values are better. Results are derived from the YouTube video streaming tests. Results from all hours of the day are used (not just peak). Enlarge chart
BBC iPlayer
BBC iPlayer is the video streaming service from the national broadcaster here in the UK. The chart below presents the percentage change in download speed from BBC iPlayer since 1st March.
There appears to be minimal impact to iPlayer so far, with one exception. Between March 11th and March 16th there was a significant drop in performance across all ISPs. Given that this drop was exhibited across all ISPs and is not seen in any other metrics, it suggests that the cause is the BBC or one of their CDN providers (the BBC use Akamai, Limelight, and their own “BIDI” CDN).
Aside from this decline in the middle of March, most ISPs see minimal change in iPlayer download speeds. This suggests that the BBC’s content delivery networks are holding up well under the increased load and the UK’s ISPs have sufficient capacity with these CDNs.
Percentage change in BBC iPlayer download speed since 1st March 2020. Higher values are better. Results are derived from the BBC iPlayer video streaming tests. Results from all hours of the day are used (not just peak). Enlarge chart
Conclusion
Netflix, YouTube, and iPlayer are all performing well across major UK ISPs, despite the significant increase in traffic that COVID-19 has brought about.
In fact, throughput from Netflix has noticeably improved since the crisis began. However, this is very likely due to Netflix decreasing the video quality offered to clients, which therefore frees up capacity on both the ISPs' networks and the Netflix content caches.
The strong performance from the video operators should not be a major surprise, as all three video streaming services tested here have deployments of caches deep inside the UK ISPs’ networks. This means that when you stream content from Netflix, for example, the traffic never even leaves your ISP’s network.
We will continue to monitor performance metrics related to COVID-19 and will be publishing further posts soon.
Methodology
The charts presented below are taken from the SamKnows UK panel of Whiteboxes, between 2020-03-01 and 2020-03-24. The Whitebox is a small hardware measurement device installed in a user's home, which carries out automated performance tests over their broadband connection many times per day. The results here are derived from a sample of approximately 4,000 Whiteboxes in the UK.
Crucially, all results are presented as the percentage from the beginning of March. In this report, we are not concerned with absolute speed or latency figures, we are instead concerned with whether the situation is degrading from a few weeks ago (before everyone was spending so much more time at home).
Results are split by major ISP only, and not by the individual broadband products they offer. We have also not split the results out by nation or county at this stage, but we may do so in the future.
A few notes on exclusions:
- All charts use a consistent set of Whiteboxes across the entire time range used in the chart. So a Whitebox must have generated measurements for every date in the period in order for it to be included. If a Whitebox did not generate measurements, even for a single day, then the Whitebox is excluded entirely.
- If a Whitebox changed ISP during the period then it is excluded entirely.
- If a Whitebox changed broadband product during the period, or its measured speed changed by more than 50% (either up or down), then it is excluded entirely.
All of these mechanisms help ensure that we have a consistent base for the comparison.