Over the past few months I’ve been watching news that is emerging from think tanks, independent researchers, and news analysts about the ‘dramatic’ increases in bandwidth usage in North America. In this post I’d like to pull together a host of sources on the recent use of the UDP protocol for transferring files, and how that relates to bandwidth scarcity. Over the next month or so, I’m hoping to put together some additional pieces on packet inspection, Enhanced Drives Licenses (EDLs), and more topical IT and privacy issues. But first, to UDP data traffic…

Peer-to-Peer and Link Points

This summer Bell Canada argued that they needed to use Deep Packet Inspection (DPI) devices to stem the use of peer-to-peer (P2P) applications during peak usage time, because P2P applications were causing congestion at major link points along Bell’s network. Bell’s practices became an issue when the Canadian Association of Internet Providers (CAIP) filed a complaint with the CRTC; Bell’s traffic shaping was being applied to all traffic that ran along Bell’s ADSL lines, rather than being localized to Bell’s customer. CAIP lost their complaint, with the CRTC noting that Bell was not discriminating against CAIP customers. The CRTC decision did not, however, condone or authorize the legality of Bell’s use of DPI technologies to filter data traffic.

The equipment that major telecommunications companies are using to filter data traffic, DPI technologies, is presently aimed at TCP/IP data traffic. Most P2P applications use TCP/IP (as do most computer programs that send and receive traffic from the ‘net), but a P2P program called uTorrent has begun to shift its traffic streams to another data traffic protocol, UDP. The goal hasn’t been to evade present data filtering processes (as Simon Morris, Product Manager at BitTorrent has stated), but rather to more efficiently move traffic without running into congestion at major link points.

The Shift to UDP and the End of the Internet

Richard Morris, over at the Register, took uTorrent’s shift to the UDP protocol as an opportunity to cry that the sky was falling. Drawing a parallel between the use of UDP and “Congestion Collapse” or “Internet meltdown”, he argues that shifting to UDP, and away from TCP/IP’s native congestion management features, demands that ISPs be permitted to take a more active role in managing their networks. His gem paragraph, on the second page, reads:

The internet evolved as a gentleman’s system in the comfortable confines of the ivory towers of academe, but now that it’s an essential part of daily life for more than a billion people, the time has come to get realistic about its management. Some of the people who use this system are spoiled children with no more concern for the greater good than junkies looking for their next fix. They can’t be allowed to spoil it for the rest of us, and the only practical means to prevent their doing so is to unleash effective management upon them. (Source)

TCP/IP’s way of alleviating congestion requires the computer that is sending packets to realize that its packets are being dropped as they course along the ‘net; once enough packets are lost the computer slows the rate of packet transmission until it reaches an equilibrium with congestion and packet transmission success. uTorrent is using a micro-transmission protocol, uTP, on top of the UDP protocol to realize in real time when congestion occurs, letting the P2P program slow data rates in real time as it encounters congestion. This is intended to make uTorrent traffic, and BitTorrent traffic generally, more network friendly. Iljitsch van Beijnum at Ars Technica notes that there are good reasons for this, stating that avoiding congestion on ISPs’ networks is important for torrent companies,

because using UDP makes BitTorrent traffic light up like a Christmas tree in even the simplest traffic shaping equipment—little or no deep packet inspection required. So if ISPs can legitimately claim this traffic causes trouble, they’ll start filtering it faster than the EFF can say “net neutrality.” (Source)

The concern, of course, is that in finding another space to shuttle packets along there will be an increased amount of bandwidth congestion. This is precisely what Richard Bennett was trying to convince people of in his article at the Register, and exactly what ISPs in North America publicly state that they are afraid of.

The Coming Exaflood

Generally, those who want to approach bandwidth as a precious resource state that we should be very afraid of the coming exaflood. Probably one of the ‘best’ pieces that talks to our deep-seated fears of using up our bandwidth (largely because of upstarts like YouTube, and other streaming services) is Bret Swanson (from the Discovery Institute). In his piece “The Coming Exaflood,” he warns us that if we do not enable the FCC to allow for discriminatory bandwidth usage by ISPs, or at least let ISPs charge content providers to transmit their content, that the Internet as we know it will collapse. If you want a particularly good scare, spend a moment to hear what the Internet Innovation Alliance (a group backed by AT&T and who has been preaching about the exaflood for a while now) has to say:

The issue? We’ve been hearing about the ‘exaflood’ for a long, long time. In fact, since 1995 we’ve been warned that some kind of data-triggered collapse of the ‘net was imminent, and it has yet to occur. While it is certainly true that there is a growth in broadband use, and the amount of data that ISPs are expected to transmit, the growth has remained relatively steady. The University of Minnesota MINTS project on Internet traffic rates has recently found that average growth at major peering points hovers between 50-60% per year, and that based on today’s growth, with zero investment on the part of telcos, that in four to five years ISPs will hit their genuine bandwidth caps at link points (Source). While UDP traffic does contribute to the increases in bandwidth use, the shift to UDP for P2P is intended to reduce the congestion at link points. What’s more, if uTP is implemented in a way that is more ISP-friendly than sending data of TCP/IP, P2P will be less of a headache for ISPs.

Investing in Infrastructure

What really, really needs to happen, regardless of the viability of uTP, is that ISPs need to get more active in investing in their infrastructures (I’m not suggesting that there isn’t investment, but that it has to increase). This is, in the short-term, incredibly unpopular – until you lay out vast swathes of new infrastructure, it’s hard to realize any substantial benefit for infrastructure costs. It has to be noted that it is really expensive to update an ISP’s network; Verizon is spending about $23 billion dollars in upgrades, rolling out fiber to each home so that home users have more bandwidth to use for their various online activities. This said, once the fiber has been laid, Verizon isn’t sure that it will be able to monetize (i.e. fully capitalize on) the vast bandwidth that they will be providing, but at the same time recognizes that they will attract customers from other carriers who are looking for fast, unlimited data transfers.

While Verizon is leading the charge in upgrading their network, other telcos in the U.S. may be close behind, depending on how receptive the American government is to putting some cash in the top hats that Verizon and their competitors have outstretched. Suggesting that a ‘mere’ five billion dollar stimulus package would create almost 100,000 new jobs, one has to wonder if telcos aren’t just looking to enhance their current networks, on the public’s dime, for private gain. Given that Verizon has already noted that it will cost in the vicinity of five times the suggested ‘stimulus’ money to upgrade their network, I have to wonder if by the logic in the stimulus proposal that Verizon alone won’t be creating (roughly) 500,000 jobs in their investments. If so, I think that all the money should go to them on the basis that they have been the telco taking the lion’s share of the risks, and that they should have to pay back all of the stimulus money if they don’t manage to create, say, 200,000 jobs.