Late last week The Register reported that Virgin Media is going to be trialling Detica’s Deep Packet Inspection (DPI) appliances to measure the levels of copyright-infringing file sharing that is occurring along Virgin Media’s networks. It’s important to note a few things right up front:
- I have a request in to the company manufacturing these appliances, Detica, and have been promised responses to my questions. In light of this, I’m not accusing Detica or Virgin Media of engaging in any ‘privacy invasive’ uses of DPI, at least not at the moment.
- The information that I’ll drawing on is, largely, from a consultation paper that Detica presented in late September of 2009.
- This post is largely meant as a ‘let’s calm down, and wait to hear about the technology’s details’ before suggesting that a massive campaign be mounted against what might be a relatively innocuous surveillance technology.
With that stated…
Detica describes themselves as a “business and technology consultancy specialising in helping clients collect, manage and exploit information to reveal actionable intelligence. As the digital revolution causes massive amounts of data to converge with a new generation of threats, many of our clients see this as one of their greatest challenges.” Their CView DPI system is meant to let ISPs better identify the amount of copyright infringing work that is coursing across their networks, in an effort to give ISPs better metrics as well as to determine whether arrangements between ISPs and content providers has a significant, measurable effect on the transfer of copyright infringing files.
The consultancy piece that is provided by Detica maintains that their DPI system is meant to preserve customer privacy, though the lack of technical insight in the paper itself means that many individuals and groups are deeply concerned about the actual instantiation of privacy-protective measures (e.g. Twitter’s #virginmedia hashtag is filled with concerns and complaints). What I’ll do is outline the most relevant parts of the consultancy piece, and follow each part with the questions that were provided to Detica. I’ll also include one question set that I forgot to ask, and will be sending along once I get my response from Detica. First, what’s exactly is this DPI appliance, anyways?
CViewTM is a pre-built, secure service providing a statistically-significant sample of all illegal file sharing activity across an ISP network. CViewTM resides within the secure environment of an ISP network, operating in a “lights out” environment (i.e. without human intervention). It aggregates all of the individual “in-network” file sharing activity and performs analysis on this dataset with pre- defined statistical models to present ISPs and CPs with detailed reports of the volume and nature of the P2P activity by subscriber groups (achieved by clustering similar behaviour types).
That the technology operates without human intervention is a no-brainer; the company is selling a device that is meant to massively aggregate and analyze data traffic. If an individual human, or team of network specialists, had to watch the logs and then run their own calculations based on what the log revealed then the product would be a dud for the purposes that it’s being sold for. Something that I expect is unsettling for many, especially those concerned with the impact of imposing statistical behaviour sets on users, is that there are pre-defined statistical models that will report on users by clustering similar behaviour types. What I failed to ask in relation to this includes:
- What are these kinds of behavioural types?
- Can the ISP with a CView product in their network infrastructure alter how behaviour types are collated?
- When Detica suggests (earlier in the consultancy piece) that they will be creating a ‘piracy index’, is this index meant to be a standard that is controlled by Detica, or is it fungible so that ISPs can configure it to suitably engage with their own customer base and customer habits? If the latter is true, then doesn’t this suggest that a industry standard index is immediately jeopardized?
There are a set of four principles that are ingrained with the development of the CView devices themselves. The first:
anonymous data collection — all records collected from the network have their IP addresses strongly anonymised such that no reference to an individual can be made, even in conjunction with other ISP systems. No content data is recorded (e.g. URLs).
As pertains to records being ‘strongly anonymised’, I want to know what, exactly, this entails. While Google claims to have ‘strong’ anonymization for the IP address information that they collect, they only remove the last 8 bits of the IP address in their logs. Given that this comprises the last octet only, and each octet can contain the values from 1-255, Google’s technique lets a computer user hide amongst 254 computers at most. Google’s approach is juxtaposed against, say Microsoft’s, which deletes cookies and full IP addresses along with other identifiable information after 18 months. What, exactly, is entailed in Detica’s ‘strong anonymisation’ process?
The second:
proportional to right to privacy — traffic is inspected to establish what the content is and the application being used, with no persistence of traffic data or identity information.
I presume that this means that there is simply an inspection of the content, a record or log kept concerning what is (and what isn’t?) identified, and then no efforts to store content streams offline. Is traffic inspected inline with the Virgin network, or is content being offloaded and subsequently analyzed ‘offline’? I fully expect that CView examines known protocols (which DPI appliances are generally capable of doing) but wonder what method is used to identify content. Is Detica using a file hash-based identification process or fingerprinting system? I ask because broadly identifying protocol alone would render any analysis of P2P data traffic as inherently infringing somewhat problematic, given that P2P is also used for legitimate file transfers (in Canada, our national news station, film board, and other government bodies are using P2P for the dissemination of public content, as an example), and there are substantial differences between the application of fingerprinting or hash-based systems (fingerprints might catch mash-ups, whereas hash-based targets full files). Further, when it is suspected that encrypted P2P traffic is crossing a network, does this constitute infringing traffic, non-infringing, or place a user in an entirely separate behavioural category-type?
The third and fourth:
closed system — no traffic data or identity information is ever made available to a person. Traffic application data is produced by an entirely closed and automated “lights out” system. Appropriate hardware, software and process controls prevent intentional or accidental breaches of privacy (e.g. preventing access to the live system when data is being processed).
no feedback loop — none of the behavioural data collected can ever be attributed back to a person or drive action against an individual.
This appears to be a positive maneuver, though I would wonder how much access ISPs actually have to these devices: are they prevented from reconfiguring these devices, or offloading information to a SAN for their own analysis? Should an ISP demand it, is is even possible for these devices to disclose the traffic data or identity information of ISP subscribers and their related data traffic? How are updates performed to the device, and what would such updates comprise (e.g. would they update the protocols/files that are detected, or go so far as to modify the ‘piracy index’ as well and extend the ability to discretely associate infringing content with particular IP addresses)? Finally, given that different categories of users are being established, while the device cannot use behavioural data to target individuals, can it be used to target the groups that the device identifies?
As stated at the head of this post, I haven’t heard back from Detica, and until I do I’m refraining from decrying (or praising) this technology, in part because I’ve been aware of this kind of technology for some time: despite Detica’s suggestions, DPI manufacturers such as iPoque have included this in some of their devices for some time (also, and contrary to their consultancy piece, Canadian ISPs have been tracking P2P use for some time). Identifying and preventing the distribution of copyright infringing files, while certainly a problem for the P2P movement, would likely be read (in the UK) as complying with the recent ‘Digital Britain’ initiatives. If the technology genuinely provides some significant level of anonymity, and presuming that the ‘piracy index’ isn’t rigged in some manner (perhaps have it open-sourced?), then this could just be a manifestation of a company selling a very particular product to address a particular need for network intelligence in compliance with British Law. This isn’t something that is strongly desired by some parties – the ‘dumb pipe’ position is commonly adhered to by network neutrality advocates – but perhaps speaks to the real need to address the misconceptions about information services, and how they legally (at least in Canada and the US) differ from telephone services.
I’ll end this by noting that I’m less familiar with UK law and regulations; I don’t know RIPA in and out, and I’m not trying to justify the Detica appliance. Instead, I’m just suggesting that until more data is released that privacy advocates and network neutrality advocates alike should take a step back, take a deep breath, and wait for a little more information before letting loose the dogs of war.
Technology, Thoughts & Trinkets 
i take note that you say your less familiar with UK law and regulations, or i take it, the ‘EU derectives’ they are directly related to, indeed have to be translated from.
your from canada i take it?, and so some of your legal perspective is from that angle.
now its unclear if your coming from the technology POV, the Logical POV and/or the general legal POV so rather than reiterate the whole long term UK (self) Informed (by necessity) user perspective regarding mission creep of all the CCTV, RIPA, and many other UK laws etc.. so il just recap some of the many UK peoples stated points of fact….
from the tech POV, ALL Deep Packet Interception/Inspection hardware Must by it very design and
as its very first act, ‘Intercept’ all of a given dataflow Before any other later action.
second, it must ‘store’ this data weather temporarily in ram, or permanently on some other storage, to then perform some work on this given data.
thirdly, it must perform the act of ‘processing’ (or as the DPI vendors PR like to rename it, inspecting/inspected/inspection, as its better PR OC) to produce the intended outcome of a ‘derivative work’ from the input of this core unique personal dataflow.
to justify its installation into any core network it Must perform at least these 3 key stages, and perhaps an optional 4th stage of permanant storage of the outcome data to be used elsewere.
you agree?, yes.
from a logical POVm that is also the case.
you agree, yes?
now from both a UK, And especially current EU directives legal POV as stated many time before elswere, in this case, myself, alex, Portly_Giraffe,RobertJ etc
Under EU and UK law it doesn’t matter what they do with the data after they have intercepted and inspected it, the very act of intercepting it in the first place without a warrant or prior informed consent from all parties.
the law don’t care nor provide for any provision exceptions if they discard IP addresses and other personally identifiable data after the fact, it still does not over ride the fact that they need a warrant or consent of all the partys in the first place.
“Named Day Written question to: Home Office for answer on 30 Nov 2009 12:00 AM
Annette Brooke: To ask the Secretary of State for the Home Department what recent representations he has received on the protection of internet users in the UK, with particular reference to (a) trials of deep packet inspection hardware and (b) the consent of internet users to the interception of their communications; and if he will make a statement. [302777]
Answer:
Mr. Hanson: Deep packet inspection can be used by internet service providers for a variety of uses, including the blocking of unwanted e-mails and “spam”. The circumstances under which interception can be carried out with the consent of the users are set out in section 3(1) & (2) of the Regulation of Investigatory Powers Act 2000. The Home Office, together with other Government Departments, has received a number of representations relating to the use of targeted on line advertising systems.
David Hanson – Minister of State (Crime and Policing), Home Office (since 10 Jun 2009)
MP for Delyn N Wales
http://www.theyworkforyou.com/mp/david_hanson/delyn
http://www.davidhanson.org.uk/
http://www.guardian.co.uk/politics/person/2192/david-hanson
Mr Hanson previously worked at the MoJ – Minister of state, Ministry of Justice (Jun 2007 – Jun 2009) so should have some familiarity with ICO/DPA/PECR issues.
”
“highlighting that even under the Digital Economy Bill, the onus will be on the Copyright Holder to identify the traffic they believe to be unlawful, and not on the ISP. This is set out in Sections 4 and 5 of the Bill. And the reporting of the success or failure of the proposed measures is the responsibility of OFCOM and not the ISP. This is set out in Section 9 of the Bill.
RIPA Chapter 1 Para 3 (3) b allows an interception if:
Quote
it takes place for purposes connected with the provision or operation of that service or with the enforcement, in relation to that service, of any enactment relating to the use of postal services or telecommunications services.
In this context, CView fulfils no purpose “connected with the provision or operation of that service or with the enforcement, in relation to that service, of any enactment relating to the use of postal services or telecommunications services” either now or in the future.
It therefore appears that there is no logical argument that the proposed CView interceptions are exempt under RIPA, though of course there may be a legal argument.”
the stanford RIPA case law URL
http://www.lawdit.co.uk/reading_room/room/view_art
icle.asp?name=../articles/Cliff%20Stanford.htm
“”Legal search
A spokesman for Virgin Media insists the company’s use of DPI isn’t illegal. “There are exceptions [in the RIPA regulations] for network management purposes and this falls into that category,” he said.
”
Ohh No it Doesnt as Virgin Media legal Know all to well IF they actually did a simple search of current statute
again for ease ill refer to the stanfard RIPA case law as PCPro should find it Very easy to research this and provide Accurate reporting…
“At trial, Stanford had sought to rely on a section of RIPA that gives a defence to a person who intercepts “a communication in the course of its transmission by means of a private telecommunication system” if either: (a) he is a person with a right to control the operation or the use of the system; or (b) he has the express or implied consent of such a person to make the interception.Stanford relied on the position that he had gained access to the emails through a company
employee. The employee apparently was given access to usernames and passwords on the email server.
Therefore, Stanford argued, he was entitled to access the emails as “a person with a right to
control the operation or the use of the system”.
Geoffrey Rivlin QC, the trial judge had a different view. He pointed out that “right to control”
did not mean that someone had a right to access or operate the system, but that the Act required
that person to of had a right to authorise or to forbid the operation.
Stanford appealed the judge’s decision. However, the Court of Appeal upheld Rivlin’s view. It
pointed out that the purpose of the law was to protect privacy. Therefore Stanford’s sentence of 6
months imprisonment (suspended for two years) and a fine of £20,000 with £7000 prosecution costs
were upheld.
Daniel Doherty
”
”
“the use of Deep Packet Inspection to make derivative works of copyrighted material (that automatic copyrighted material being the piracy of your unique personal data streams)
for commercial gain, thats a criminal offence BTW..
http://www.patent.gov.uk/copy/legislation/legislat
ion.pdf
when its copyright piracy for commercial gain, it clearly falls under criminal law.
not mere civil/tort law, as might for instance downloading for personal use, as any first year law student would know.
and the actual use and or supply of the DPI kit without to perform a criminal act is covered there too.
see:
S.107 of the Copyright Designs and Patents Act 1988 (“CDPA”) established the following categories of offences: making or dealing in infringing copies of copyright works;
making or possessing an article specifically designed or adapted for making copies of copyright works; and
causing a work to be performed, played or shown in public.
Making or Dealing in Infringing Articles
It is an offence under s.107 (1) of the CDPA to
(a) make for sale or hire,
(b) import into the United Kingdom otherwise than for private and domestic use,
(c) possess in the course of a business with a view to committing any act infringing copyright,
(d) in the course of a business
(i) sell or lets for hire,
(ii) offer or expose for sale or hire,
(iii) exhibit in public, or
(iv) distribute, or
(e) distribute otherwise than in the course of a business to such an extent as to affect prejudicially the owner of the copyright,
an article which is, and which is known to be or where there is reason to believe it to be, an infringing copy of a copyright work. Anyone convicted of such making, importing or distribution may be fined or sentenced to up to 2 years in prison upon conviction on indictment or 6 months imprisonment and a fine up to the statutory minimum on summary conviction, or both (s. 107 (4) CDPA). The maximum penalty for any other offence under s.107 (1) is 6 months imprisonment or a fine up to level 5 on the standard scale on summary conviction, or both (s. 107 (5)).
Making or Possessing Specially designed or adapted Articles for Making Infringing Copies
It is an offence under s.107 (2) to make an article specifically designed or adapted for making copies of a particular copyright work, or possess such an article, knowing or having reason to believe that it is to be used to make infringing copies for sale or hire or for use in the course of a business. The maximum penalty for an offence under this sub-section is 6 months imprisonment or a fine up to level 5 on the standard scale on summary conviction, or both (s.107 (5)).
Communicating the Work to the Public
The new offence of communicating a copyright work to the public is provided by a new s.107 (2A). The penalty for that offence is a imprisonment not exceeding 3 months, a fine up to the statutory maximum or both on summary conviction, or 2 years imprisonment, a fine or both under a new s.107 (4A).
”
these are just a main points restated, Uk and EU law is very protective of the consumer, the main problem right now seems to be actually getting the UK lagal forces to actually act on the existing criminal case file that been in the system for over a year and wny the EU are now nearly in the 3rd stege of taking the UK in the EU courts around or just before cristmas eve i think it is, so just a few more weeks, although i forget the exact timeframe…..
LikeLike
“the main problem right now seems to be actually getting the UK legal enforcers The CPS to actually act on the existing criminal DPI case file thats been in the UK system for over a year now.
and why the EU are now nearly in the 3rd and final stage of taking the UK to the EU high courts, around or just before cristmas eve i beleave it is, so just a few more weeks to go, although i forget the exact timeframe…..
LikeLike
you should also perhaps read the the ‘All Party Parliamentary Group On Privacy’ paper too and take full note of this:
If something isnʼt “communications data” it is almost overwhelmingly “content” and so requires a warrant.
and if your still not clear on the why the use of DPI is wrong outside these normal business duties for the purpose of fulfilling the contract and using Anything But the provided ‘Header packets’ is deemed illegal and very Bad for this or indeed any countrys long term future, I.E Your Jobs and Your Wages etc
read the ‘All Party Parliamentary Group On Privacy’ paper
http://privacyappg.org.uk/Documents/appg_IMP_brief
ing.pdf
All Party Parliamentary Group On Privacy
…..
” P14
What is “communications data”?
In their consultation paper regarding the Interception Modernisation Programme,
Protecting the Public in a Changing Communications Environment the Home Office says:
“Communications data is information about a communication.
It does not include the content of a communication.
It can show when a communication happened, where it came
from and where it was going, but it cannot show what was said or written […]
For a given telephone call, communications data can include the telephone numbers involved, and the
time and place the call was made, but not what was said. For an e-mail it might include the
e-mail address from which the message was sent, and where it was sent to, but not the
content…….
….
” P21
the current separation of “communications data” from “content” looks unworkable: interpretations in
individual cases are difficult; even when an interpretation is forthcoming, the practical
problems of separating the one from the other are considerable.
If something isnʼt “communications data” it is almost overwhelmingly “content” and so requires a warrant
from the Secretary of State and is inadmissible in evidence.16
This is surely not an outcome a law enforcement investigator wants…..”
”
“P63
A Chilling Effect?
As the general public becomes aware of the practice of collecting and collating all this
personal information, the risk is that it will generate a chilling effect on the individual’s right
to free expression, association and might dissuade people from participating in
communications transactions.
Already, following from the media coverage of
the Government ‘wanting to get access to social networking profiles’ there has been a rising
concern about what people do or say on social networking sites. As we try to build ‘Digital
Britain’ we may in fact be creating a barrier to people accessing online services and
applications out of fear of surveillance.
This chilling effect could, in turn, have serious ramifications for industry.
If developments like ‘cloud computing’ and increasing virtual communications and modes of work are
placed under similar scrutiny then the policy of modernising policing powers could restrict
innovation, or drive infrastructure out of the UK.
Every time an individual shares a
document with a colleague, this process generates communications traffic data. Every
online video conference or sharing of knowledge through discussion boards across
organisations will generate communications traffic data over public networks. This will
result in a level of surveillance never seen before, with ever weakening safeguards.”
LikeLike
quote
“Nicholas Bohm | December 8th, 2009 at 15:31 UTC
Does using CView involve interception under RIPA?
Yes. The question of whether a human needs to see something before it counts as interception is answered in my paper on Phorm at http://www.fipr.org/080423phormlegal.pdf in paragraphs 14 to 17. No human access is necessary – machine examination of content is still interception, and unlawful unless justified.
It remains to be seen whether a convincing case can be made for an ISP’s need to know how much of its traffic infringes copyright – is this really required for purposes connected with the provision or operation of its service?
“
LikeLike
Sorry for not responding earlier – I’ve been away on a research trip, which has limited my time for responses.
I’ll be putting up something on this soon, based on what I’ve heard back from CView. The question that I would have is whether or not the inspection of elements of data flow, in a (seemingly) anonymized fashion that cannot be tracked back to an individual, and that doesn’t constitute the full capture of a piece of content (i.e. not all of a music file apparently is captured, only the first few packets) then does this actually constitute an ‘interception’ any more than scanning text on a post card (NOT a letter) would constitute ‘interception’. With what you’ve posted, that still doesn’t come off clearly to me.
LikeLike
given Deep Packet Inspection (DPI) refers to examining both packet header and payload, hence even one single packet intercepted and then processed for payload content is stated to be criminal offence without warrant or prior consent of both the sender (VM user)and receaver (website etc) in UK legislation already mentioned , then clearly “NOT a post card” type argument, but a clear opening the royal mail letter and looking at it payload/content.
were as Shallow Packet Inspection refers to examining both the generic packet header for routing operations, and ONLY the packet header, can be claimed to be your postcard and covered by a explicit network managment in this instance….
as already made clear, they are and must by design and desciption of of the Deep packet Interception/inspection be intercepting every packet, even one full packet contains both header and content data, its that content data that makes this or breaks this, Not the ammount of data intercepted and processed.
we have not even considered Copyright legislation and how the amount of the consumers data/Intellectual Property use might contribute to this as a civil violation or a criminal vialation, one things cristal clear, ‘piracy for commercial profit’ IS a Criminal offence in the UK at least.
LikeLike