SamKnows is now part of Cisco | Learn more

Spotlight: Measuring Broadband America Retrospective

This is the story of Measuring Broadband America. How it started, how it has helped improve the nation’s broadband, and how it’s continuing to drive progress in nationwide connectivity.

Download this issue

Fired up - ready to go

It’s a freezing February morning in Springfield, Illinois, and a rookie senator has boldly announced his intention to run for the White House. There’s a lot that’s different about Barack Obama. He’s young, only 45, and he’s vying to become the first black president of the United States, of course. And he’s talking about technology.

“Let us transform this nation,” he says in the 2007 speech announcing his candidacy for the presidency. “Let us be the generation that reshapes our economy to compete in the digital age.”

“Let’s make college more affordable, and let’s invest in scientific research, and let’s lay down broadband lines through the heart of inner cities and rural towns all across America. We can do that.”

It wasn’t mere rhetoric. As part of Obama’s presidential campaign, he launches a nine-page technology and innovation plan in which he pledges to “redefine broadband”, promising the “speeds demanded by 21st century and business communications”. To do that, however, America first needs to know how fast its broadband really is.

Four years later, with Obama now midway through his first term of office, the Federal Communications Commission (FCC) publishes the inaugural Measuring Broadband America Report, “the first broad-scale study of directly measured consumer broadband performance throughout the United States”. It’s a ground-breaking moment of clarity for the US broadband industry, and a ground-breaking moment for SamKnows, whose technology is being used to provide the first true picture of the country’s internet performance.

This is the story of Measuring Broadband America – how it started, how it has helped improve the nation’s broadband, and how it’s continuing to drive progress in nationwide connectivity.

Let us transform this nation. Let us be the generation that reshapes our economy to compete in the digital age.

– Barack Obama, 2007

The Obama effect

“If it wasn’t for President Obama, we wouldn’t have been working in America,” says Alex Salter, CEO of SamKnows, looking back on ten years of Measuring Broadband America. 

That change of administration put fresh impetus into the US broadband rollout, according to Walter Johnson, who was then part of the FCC’s Office of Engineering & Technology and a chief architect of Measuring Broadband America. “The powers that be, not only in government but in industry, had been late to realize the impact that broadband was going to have,” he says.

“It was clear that things were going to change. There was a change of administration, and the new chairman, Julius Genachowski, came into the Federal Communications Commission and he started developing what became known as the National Broadband Plan.”

However, it wasn’t only President Obama’s election victory that paved the way for Measuring Broadband America, but SamKnows’ earlier work with the UK telecoms regulator Ofcom, which proved the company’s technology could be used to successfully measure broadband performance on a national scale.

The first UK broadband speed report was published in January 2009, just 11 days before President Obama was inaugurated, and it provided a template for how the research could be conducted on a much larger scale in the US.

SamKnows won the FCC tender to perform similar tests in the US, but Measuring Broadband America was initially met with resistance from the broadband providers when the FCC first outlined its plans. “They were concerned that a government regulatory agency would do something that they weren’t aware of and it might have a negative effect on their business,” says Johnson. “We had a very heated meeting on this topic.”

If it wasn’t for President Obama, SamKnows wouldn’t have been working in America.

– Alex Salter

It was soon decided a more collegiate approach was necessary, bringing in consumer rights groups, academics, thought leaders, journalists and experts from the broadband providers themselves to help work through the testing methodology. Every aspect of the program was discussed in painstaking detail, the shared goal to ensure that testing was accurate and representative.

Alex Salter and his co-founder Sam Crawford would spend a good chunk of 2010 in Washington DC, holding regular meetings with the FCC and representatives of the broadband providers in an attempt to earn their trust. “Sam and Alex had regular project meetings with the FCC, to which they invited all of the other stakeholders to take part,” says Roxanne Robinson, Director of Government Projects at SamKnows.

“We set up a collaborative process where we presented our methodology, the tests, the platform and how it works, and we had these discussions every three weeks. It’s always been an open forum, and it’s still like that; we value transparency and communication very highly. We still have meetings every month and we openly communicate any changes, such as adding new measurements to the program and that kind of thing.”

Although the majority of big US broadband providers participated from the very beginning of Measuring Broadband America, there was one major provider that decided after initial reports that it didn’t want to take part.

However, a decade on, and with long-running broadband measurement programs being run across the world, Robinson says broadband providers are “now asking us to take part, because the data we collect simply isn’t available through any other program.”

It’s always been an open forum, and it’s still like that; we value transparency and communication very highly.

– Roxanne Robinson

Broadband improvement since the program began

Measuring the gains

What kind of difference has Measuring Broadband America made to the overall performance of US broadband? The graphs on this page will show you how some of the basic performance metrics have improved since the program first reported in 2011. Many of the gains would be the result of advancements in broadband technology and available bandwidth over the past decade, of course. But there is evidence that the Measuring Broadband America program has made a direct – and lasting – difference to performance.

Even in that very first report, published a decade ago, there were signs of broadband providers immediately upping their game. “For most participating broadband providers, actual download speeds are substantially closer to advertised speeds than was found in data from early 2009 and discussed in a subsequent FCC white paper – although performance can vary significantly by technology and specific provider,” the executive summary of that 2011 report reads.

Walter Johnson believes that the great strength of Measuring Broadband America is that it gave the underperforming ISPs a benchmark to aim for. “What occurred to us, and it’s only a theory, is that for the first time ISPs could look at the performance of other ISPs and judge how well, from an engineering perspective, they were doing,” he says. “The really low performers realized that they were just doing lousy engineering. And some of them obviously cleaned up their act and they improved their performance.”

For the first time, ISPs could look at the performance of other ISPs and judge how well they were doing.

– Walter Johnson

Lasting improvements are also borne out in the data. One of the criticisms leveled at Measuring Broadband America is that the data used for the reports is only collected for two months of the year, and that the broadband providers are informed when these measuring periods will be. Surely, critics argue, the broadband providers will simply pull out all the stops during that measuring period to put maximum gloss on their results, returning to ‘normal service’ when monitoring stops?

The data does indeed show that broadband providers up their performance ahead of the measuring period, but those improvements aren’t temporary – and the SamKnows test equipment is collecting data all year round, so it would be very easy to spot if providers were simply turning on the taps for the September/October measurements. “We haven’t seen it where people just turbocharge performance for September,” says Robinson.

“What you do see,” adds Alex Salter, “is ISPs set their investment schedule to get things completed by the measurement period.

So, you’ll see a lot of improvement occur before the measurement period and this continues afterwards. This is a benefit of continuous measurement, that we can track improvement over time. In the case of MBA, we have a dataset going back more than ten years.”

Moving to mobile

Broadband isn’t confined to fixed-line services, of course. Indeed, by the time President Obama was running for election for the second time, the focus had switched to mobile broadband, exposing the cellular networks to the same level of scrutiny as their fixed-line counterparts.

Once again, the FCC and SamKnows worked collaboratively with the major carriers to painstakingly refine the measuring process and develop apps that could accurately measure both speed and coverage. The mobile measuring apps also presented a particular challenge.

“We were talking about what to do with the data, how to release it, how to map it, and there was obviously a real concern about privacy, making sure that we don’t collect data that can be re-identified,” remembers Roxanne Robinson. “If you’re living in rural Montana and you’re the only house in that whole [mapping] hexagon, when we release results is it very obvious, for example, that we know Jane Bloggs lives in that house? So we did lots of work on privacy, and the app itself anonymizes all of the results, so you can’t tie any test results or handset to a particular person.”

The Android measuring app was released in November 2013, with the iOS version following shortly after, launched at Mobile World Congress in March 2014. In addition to the measurement app, SamKnows subsequently developed a second cell audit app that allowed the FCC’s enforcement bureau to start auditing the cellular networks’ coverage maps by taking continuous readings of mobile signal strength and other environmental factors.

“The FCC has a bunch of engineers that they’ll send out to various places to gather data in the field, to make sure the ISP is doing what it said it was going to do,” says Roxanne Robinson.

In one instance, this helped bring coverage to an area that wasn’t being served at all, because the coverage maps at the time were inaccurate. “There was a big uproar from the people on the ground who knew that they weren’t getting the coverage that was being stated,” says Robinson. “The FCC collected data in these areas to corroborate what was said and that ended up being written into a big report that was published on the FCC website.”

The project made such a real-world impact that “the guy who managed the project received an award for all of the work that he did on it,” Robinson adds.

There was a big uproar from people on the ground who knew that they weren’t getting the stated coverage.

– Roxanne Robinson

March 2020 download speeds

Added benefits

As mentioned earlier, the measurement period for Measuring Broadband America is only a couple of months per year – but that doesn’t mean all those Whiteboxes and other test equipment are sitting idle for the rest of the time. Data is collected all year round and it has been put to good use in various academic studies over the past few years.

These MBA-Assisted Research Studies (MARS) have included projects trying to identify broadband bottlenecks over the ‘last mile’ of the connection, as well as projects that aim to improve the granularity of the data collected by Measuring Broadband America.

One of the most recent MARS projects has been attempting to identify whether there’s any performance overhead of moving to encrypted DNS services. With browser makers such as Google and Firefox increasingly encouraging users to switch on encrypted DNS, broadband providers have argued that performance could suffer when DNS is no longer routed through their own servers. However, a two-year study conducted by academics at Princeton and elsewhere found little evidence of any performance hit from encrypted DNS, although there were quite large differences in performance between the DNS providers! Such studies could make a huge difference to the security and privacy afforded by internet connections in the future.

The data collected by Measuring Broadband America also provided unique insights into how well broadband networks held up during the Covid-19 pandemic. The sudden shift to home working as lockdowns were instigated across America put unprecedented pressure on domestic broadband and the applications that were used to keep businesses ticking over.

For instance, there was a big spike in the use of videoconferencing services, as employees strained to keep in touch with one another and their clients.

SamKnows called on the vast swathes of application performance data that is collected by its Whiteboxes to deliver the MBA Critical Services report in the height of the pandemic, allowing government and businesses to see the impact of homeworking.

The report showed that most American ISPs suffered only a very small decrease in download speeds, with the biggest hit to download performance coming as a result of the launch of a new video game (Call of Duty: Warzone), not homeworking.

The good news for the millions of people having to connect with colleagues via Zoom, Microsoft Teams and the like was that latency, too, only suffered a minor increase. The SamKnows measurements showed that the eight biggest US broadband providers only saw around a 3% increase in latency in the fortnight immediately following the first lockdowns, which shows just how resilient the broadband networks proved to be.

Again, it was only the millions of gamers that were rushing to download a massive Call of Duty update that caused any significant latency issues for the majority of providers in that initial period following the first lockdown.

Latency change since 1st March 2020

(Lower is better)

Mapping out the future

A slight bump in latency figures would feel like a ‘first-world problem’ to people who had no broadband at all during a pandemic. There are millions of Americans who are still struggling to get any type of broadband connection, and the Covid-19 lockdowns has pushed their plight higher up the political agenda.

There are billions of dollars of rural subsidies waiting to be spent to expand 4G coverage in underserved areas over the next decade, but to ensure the money is invested in the right places, the FCC has been working to ensure that the broadband coverage maps are as accurate as possible.

At the beginning of August, the FCC released a new set of mobile coverage maps that provided much greater detail of the coverage offered by the four biggest mobile carriers: AT&T, T-Mobile, UScellular and Verizon.

“This is the first public map released by the FCC to show updated mobile coverage and represents a significant improvement over data previously published by the agency,” the agency said, adding that it “will improve the uniformity and consistency of broadband availability data collected by the FCC.”

The new maps rely on data provided voluntarily by the networks themselves, but with the Biden administration putting a renewed focus on US broadband infrastructure, the aim now is to create even more accurate maps. The FCC plans to do this by gathering data directly from millions of users on the ground, using an app on their smartphone to measure the actual coverage in their location. “There’s a real appetite for it in the US at the moment and everyone knows it’s a big issue,” says Robinson.

A decade on, the role of Measuring Broadband America is bigger than ever.

Listen now

Part 1 is an interview with Walter Johnson, Chief Architect of the Broadband America Program. Part 2 is an interview with Roxanne Robinson, Director of Government & Academia at SamKnows.

MBA part 1: Interview with Walter

MBA part 2: Interview with Roxanne