Benchmarking Dropbox, Google, and Microsoft
Online storage services have exploded in popularity since the launch of Dropbox eleven years ago. The proliferation of broadband services with high download and upload speeds has made online storage services a viable, and arguably more robust, alternative to using USB sticks.
What we tested and how
We uploaded a 500MB test file to publicly-accessible folders on Dropbox, Google Drive, and Office 365 Sharepoint. We then instructed approximately 180 SamKnows Whiteboxes, deployed on the fastest speed tiers of major UK broadband ISPs, to download these files and measure the transfer speed. The download duration was limited to 10 seconds. A single TCP connection was used for the download measurement (mimicking the behaviour of the real clients we tested here).
The IP address of the server that delivered the content was recorded as well, which allowed us to trace the location from where the content was delivered. We enabled the operating system (OpenWrt) on our Whiteboxes to choose to use either IPv4 or IPv6 for the transfer — we didn’t force the traffic to use one or the other. This meant that we’d connect to the storage providers over IPv6 when both the user’s broadband connection and online storage provider supported it.
In all cases, the online storage services use TLS to encrypt the data in transit. This presented a challenge for even our latest generation Whitebox, which employs a dual core 880Mhz CPU. On the fastest connections, this would mean that we’d need to decrypt content at hundreds of megabits per second. This would drive the Whiteboxes’ CPU to 100%, meaning that we wouldn’t be measuring the performance of the broadband connection or the online storage service, we’d be measuring our CPU performance (which is clearly not what we want). Fortunately, we were able to devise a solution to this problem: once we had the final URL of the content, often there would be many redirects before we got to the final URL, we would obtain a handle to the TCP socket (not the TLS-wrapped socket) and read data from this socket directly. Of course, the data would be unreadable to us as we’d be getting raw encrypted bytes, but that’s fine — we only care about the volume and speed of data, not the content itself. With this change applied, we were able to measure ~940Mbps reliably.
In summary, the metrics we captured for Dropbox, Google Drive, and Office 365 Sharepoint were:
- Download speed
- The volume of bytes transferred
- The IP address of the server that delivered the test content
The findings below were obtained from a sample of 9,335 tests that were run across 165 Whiteboxes in the UK over a two-day period in September 2018. As always, we used SamKnows One, our web-based internet measurement platform, to analyse the collected measurement data. This allows us to generate high-level summaries, identify patterns and then perform a deep-dive into the results to understand the cause of these patterns.
Initial comparisons of the online storage providers
Figure 1 shows the average download speed achieved from each of the online storage providers. On average, Google Drive delivers significantly higher throughput than Dropbox and Office 365 Sharepoint. The 36% difference between Google Drive and Office 365 Sharepoint is significant, particularly when dealing with broadband connections in hundreds of megabits per second. Of course, these are just averages and could be skewed by a few outliers, so we will need to explore a few more charts to see if a pattern forms.
Comparing online storage providers by ISP and product
Figure 2 below compares the average download speed for each of the major ISPs and products in the UK to each online storage service. As might be expected, Virgin’s 200Mbps and 350Mbps products deliver the highest absolute throughput. However, there are significant differences in the performance of Virgin’s fastest 350Mbps product — on Office 365 an average throughput of only 158Mbps was achieved, whilst on Google Drive the very same broadband connections averaged 338Mbps. This level of difference might be expected when dealing wiht a very high bandwidth-delay-product, but all of the online storage services tested here had hosting locations in the UK. In short, if you’re a Virgin user and looking for an online storage service, Google is probably worth looking at!
On the VDSL-based products from BT, Sky, Vodafone, and TalkTalk, the same pattern could be observed. Google Drive tended to always deliver the highest throughput, with Dropbox typically following second, and Microsoft in third.
IPv6 usage amongst online storage providers
There have been some eyebrows raised internally at SamKnows that so few of our recent blog posts have reported any usage of IPv6 for the real applications we’re measuring. Things are a bit different in this instance, with our testing showing that multiple online storage providers and ISPs are using IPv6 to exchange traffic. Figures 3, 4, and 5 below shows that more than 50% of tests run on BT and Sky broadband connections were carried out over IPv6, when testing to Google and Dropbox. Only Google, Dropbox, BT, and Sky appeared to employ IPv6. In other words, Microsoft, Virgin, and TalkTalk did not make use of IPv6 at all. There was no discernible performance difference in the IPv4 vs IPv6 measurements.
Figure 3: Dropbox
Figure 4: Google Drive
Figure 5: Office 365 Sharepoint
We carried out traceroutes to all of the IP addresses that the online storage providers served content from. In almost all cases traffic was served from London-based IP addresses. The few exceptions were seen with Google Drive, which occasionally served content from Amsterdam, most often for TalkTalk, suggesting that there might be some address ranges that TalkTalk own which are incorrectly geolocated by Google.
Our measurements have shown that Google Drive consistently appears to be the fastest online storage service, whilst Microsoft Office 365 consistently appears to be the slowest. This difference between Google and Microsoft is significant (over two-fold) on very fast connections, but is still noticeable even on lower speed connections. There are no discernable ISP-specific issues with any of the online storage services we’ve tested.
It is interesting to note that BT and Sky frequently use IPv6 for transfers with Google and Dropbox, whilst other ISPs and online storage services still rely wholly on IPv4. This shows that IPv6 can be used in production in the UK and deliver high performance — take note BBC and ITV!
The online storage service measurements are available in our Whiteboxes and Router SDK right now. Additional online storage providers will be introduced in the future so we’ll be keeping our beady eyes on their performance too.