Communications systems are integral to emerging and developed democracies; the capability to rapidly transmit information from one point to another can help fuel revolutions and launch information campaigns about unpopular decisions to ‘meter’ the Internet. In foreign nations and at home in Canada we regularly see ISPs interfere with transmissions of data content. Both abroad and at home, researchers and advocates often have difficulties decoding what telecom and cableco providers are up to: What systems are examining data traffic? How is Internet access distributed through the nation? Are contractually similar data plans that are sold in different geographic regions providing customers with similar levels of service?

To date, Canadian advocates and researchers have been limited in their ability to draw on empirical data during major hearings at the CRTC. This makes research and advocacy challenging. Over the past several years, researchers, advocates, counsel, and members of industry that I’ve spoken to have complained that they need hard data. (It’s a gripe that I’ve stated personally, as well). With your help, numbers will be on the way.

Most regulatory proceedings see corporate data filed in confidence, and what is made available to the public typically lacks the methodology by which data is collected, or the steps taken to generate final percentages and conclusions that are sometimes made public. Effectively, this means that advocates and researchers alike have to turn to foreign nations’ traffic information, data taken from independent Internet observatories, or perform theoretical extrapolations from the data that is provided to identify whether the numbers industry provides match up with the networks’ actual conditions. I want to change that and work with interested people to create an accessible and public dataset that maps Canada’s broadband conditions.

At a recent event in Victoria, I passionately pitched that I should win the available $800 to help pay for a developer to create a broadband analysis tool. Once developed and deployed, the end-user would just see an online form that let them choose their ISP, their service package, insert their postal code, and then press ‘go’. After initiating the analysis, data would be pumped to and from a series of servers around Canada/the world and the user’s computer. As a result, we could identify latency, jitter, broadband service levels, open ports, and whether traffic was being throttled or not.

Anyone who’s looked at broadband analysis immediately come upon the same question: why use this tool over the other, already existing, tools that are available? First, we can customize the code-based to capture exactly the information that we need. Second, it should be harder to ‘game’ than the existing public tools. Third, many of the existing options make it challenging to easily access the data provided the end-user. With the data we plan on collecting, we should be able to actually map Canada’s broadband services.

The talk was a hit (proposal available in .pdf). Unfortunately I didn’t win the money I was competing for. Fortunately, I got a lot of people at the event interested. Even better, several of those people were willing to lend their time and services to roll out the tool. Since then, the broadband analysis tool has been in development. Hopefully the open-sourced code will be ready for a production environment in a month or two. The tool is composed of three elements:

1. A front-end Flash-based object that let’s individuals select their ISP, their broadband plan, and where they receive Internet service;
2. Bandwidth servers that are hosted by people with public IP addresses and have bandwidth to spare for testing;
3. A central server that has a master list of the bandwidth servers and aggregate statistics from the bandwidth servers.

While running code is (clearly) a key part of what we need to start testing broadband service in Canada, it’s not enough. We need people to actually host the bandwidth servers (code will be open sourced, so you can check that nothing funky is going into your infrastructure) and someone to help design the Flash object (it’s not particularly pretty right now). While the initial work is aimed at Canadian Internet transparency, the code will be publicly available so there isn’t any reason why it couldn’t be used to similarly map broadband services in other nations and jurisdictions around the world.

We’ll be working with the folks at Open Data BC to figure out how best to disseminate the data that we collect: we’re not looking to create a new silo! We’re committed to making our findings and raw data accessible to anyone who’s interested. Since Canadians are amongst the most prolific users of broadband in the world it’s time that light be shed on how broadband services are actually being provided.

Ultimately, this is a call: the code is coming, but infrastructure is needed. Can you, or someone you know, help in making some infrastructure available to bring transparency to the contemporary Canadian broadband landscape? If you can help please get in touch!