By ANDREW KEEN
Illustrated by Stuart Bradford
It’s the Rorschach test for the digital age. Mention the idea of “deep packet inspection” (DPI) to media reform groups like the Electronic Frontier Foundation (EFF) or Free Press and they’ll warn of an imminent takeover of the Internet by dark corporate or authoritarian forces. Yet utter the same three letters to a seasoned technologist and you’ll generally get a nuanced answer about both the challenges and opportunities for managing a network, confronting online spam, intellectual property theft and Internet viruses.
So what is the truth about DPI? Is it a generic type of digital technology which enables the reassembly of complex data by intermediaries such as Internet service providers (ISP) or website operators like YouTube? Is DPI the 21st-century realization of an always-on electronic big brother or, more prosaically, is it simply a useful tool for efficient network management and for fighting online crime?
The truth is that our schizophrenic attitude toward DPI might actually be a symptom of a more general schizophrenia about today’s security technologies. We are paranoid about the threat of airline terrorism, for example, but also increasingly obsessed by threats to individual liberty at airports. And our schizophrenia about security technology is particularly marked on the Internet where many of us are both obsessed about our own right to privacy, yet also fearful of the manifold online criminal threats to intellectual property and individual privacy, not to mention computer viruses.
It’s hard not to be nostalgic for the 20th century when the technology of surveillance was so much simpler. Think of those iconic 20th-century motion pictures that chronicled our fear of being watched. In Rear Window, for example, Hitchcock’s classic 20th- century surveillance movie, Jimmy Stewart and Grace Kelly spied on both their criminal and law-abiding neighbors, mostly with just the help of their own eyes. And even in Coppola’s The Conversation or von Donnersmarck’s The Lives of Others, 20th-century spying— whether on American or East German citizens—was realized by steaming open envelopes, taking secret photographs, planting physical recording devices or by paying informers.
But in the 21st century, the surveillance picture has dramatically changed. The digital networking revolution—with its massive eruption of electronic communications—changes everything. The Internet has quickly become the dominant platform on which we do our business, get our information and have our fun. Messaging, commerce, commentary, love, advertising, entertainment and, most troubling, criminality have all migrated online. Watching people on the Internet, therefore, is like watching them in real life. Our online secrets—legal and otherwise—reveal who we really are, what we are doing and how we think.
Therein lies both the essential need and the dangerous threat of digital surveillance. Evolving technologies like DPI, which enable third parties to look “deeper” at Internet traffic and even potentially reassemble electronic messages through the reconstruction of IP packets, represent both a challenge to our 20th-century assumptions about privacy as well as a potential solution to the general lawlessness on the Internet.
Even though DPI technology, utilizing machine-to-machine communication, reassembles the structure rather than the full and exact content of the online message, there’s no doubt that this technology has an Orwellian dimension. Indeed, this technology might, in theory, represent the holy grail for marketers, salesmen, secret policemen, blackmailers and even nosy parents. A technology that “sniffs” information by inspecting online packets represents our deepest nightmare. It could be the technology deployed in the Room 101 of the digital age.
That said, however, there are some myths about DPI which need to be put to rest. Firstly, DPI technology isn’t alone in seeking to know us better than we know ourselves. There are already a number of non-DPI online technologies available that enable marketers, salesmen and secret policemen to determine our most intimate interests, desires and secrets. One of these is Google, a generally beloved search engine algorithm that most of us use many, many times a day. Google is in the business of interpreting our interests and desires so it can serve up appropriate advertising. That’s why, for example, the advertisements that appear on our Gmail accounts often are somehow related to what we’ve written in our e-mails.
A couple of years ago, when a journalist at the Financial Times asked Google Chairman and CEO Eric Schmidt where he wanted the company to be in five years, he said that he hoped the search engine would be so knowledgeable about each of us that it could tell us not only what we wanted to do that day, but also what job we aspired to. In a sense, then, we are already living in a DPI-free Room 101. And it’s called Google.
Secondly, the phrase “deep packet inspection” is, as the think tank Digital Society’s policy director George Ou told me, a “marketing term” to describe a type of technology. So, like Web 2.0 or the real-time Internet or other digital buzz words, DPI is a term that covers a huge swathe of technologies from companies like Audible Magic, which can conceivably be used to sniff the packets of hackers, criminals, and peer-to-peer pirates, as well as ordinary, law-abiding Internet users like you and me. Thus, while DPI does raise our suspicions about digital surveillance, it is such a broad technological term as to be relatively meaningless to a non-technical audience.
Thirdly, the anxiety of groups like Free Press, EFF and other Internet consumer groups about DPI does reflect a misunderstanding about how this technology is deployed. Yes, DPI does enable intermediaries like ISPs or websites to look more “deeply” at data trafficked over the Internet. But, no, the core purpose of DPI is not to read our private e-mails or forensically sniff our online emotions.
Instead, DPI is currently used for many critical applications that enable the Internet to operate effectively—from ensuring security against malicious viruses, spam and identity theft, to distributed denial of service attacks. DPI is also used to speed up online applications, such as Content Delivery Networks, as a way of making sense of content expiry controls. Last but not least, current applications include what is known as “DPI data flow analysis,” which enables the enforcement of Internet access policies—from workplace protocol to empowering parents to manage and control their home routers.
So what’s the truth about DPI? The truth, of course, is that there really isn’t a single truth about this technology. Just as it’s people rather than guns that kill people, so the efficacy and morality of DPI technology depends on its user. There’s no doubt that DPI technology can—and is—deployed by authoritarian governments in Iran or China to eavesdrop on and wiretap the online behavior of their citizens in order to censor content. Yet DPI can also be used legally and responsibly in democratic countries to manage the security of the network; fight against illegal spam; police viruses, botnets and server hijacking; and aggressively counter malicious barrages such as Denial of Service (DoS) attacks designed to disable the Internet.
This criminal threat cannot be underestimated. Web security firm Symantec published a 2008 report saying that “the release rate of malicious code and other unwanted programs may be exceeding that of legitimate software applications.” And the Finnish antivirus firm F-Secure has estimated that an equal amount of Internet malware was created in 2007 as was created between 1987 to 2007. As Richard Bennett, fellow of the Information Technology and Innovation Foundation (ITIF) and a noted authority of network architecture, told me: DPI is an essential management tool for maintaining the security of the Internet.
“Should we be terrified of it?” I asked Bennett, when we met at the ITIF offices in Washington, D.C.
“Look, we have cybercriminals trying to steal people’s identities and bring down the Internet,” Bennett explained in defense of DPI. For him, as for many network experts, DPI is just a technology. In the hands of accountable network managers, Bennett insisted, DPI is “absolutely indispensable.”
“So why are we so paranoid about it?” I asked.
“Sounds like Big Brother, doesn’t it?” he responded dryly.
Bennett is, of course, correct about the roots of our mistrust. But if 99 percent of people were engaging in illegal activity in any arena, they would be subject to repercussions. So wouldn’t users who are breaking the law on the Internet justify the use of DPI technology?
This isn’t as absurdly theoretical as it sounds. Some recent research from Princeton University, which randomly studied 1,021 files from the open source file-sharing application BitTorrent, found that 99 percent of the sampled files were copyrighted content. Therefore, 99 percent of BitTorrent traffic is involved in the pirating of content. BitTorrent clients are stealing movies or books or music from their legal owners. These users are common thieves. Through their collective online kleptomania, they are unintentionally killing our culture by making it increasingly difficult for creative people to be financially rewarded for their work.
But what would happen if DPI technology could expose and block these criminals? What would be the value of DPI then?
Herein, of course, lies one of the most controversial and complex aspects of this technology. In addition to load balancing, Web caching and all those other highly technical yet indispensable applications that enable the functionality of the Internet, DPI can indeed enable ISPs to filter content so that they can identify copyrighted material that has been illegally transmitted.
In other words, DPI can intentionally help save our culture by blocking and tackling its unintentional killers. You see, the truth about most online thieves is that they aren’t hardened criminals obsessed with breaking the law. Instead, they are ordinary people like you and me who are stealing simply because they can do it with only the tiniest risk of detection. Thus technologies like DPI, which can automatically spot illegal Internet activity, could become a valuable deterrent against theft because it radically raises the chances of detection and punishment.
In principle, DPI technology can be effectively deployed by ISPs in the short term to both block access to specific sites as well as to specific URLs. It can also be used to block the sharing of specific files on peer-to-peer networks and to help enforce user policies and sanctions on Internet pirates. Unfortunately, however, in the long term, DPI technology will be stymied by the use of encryption by pirates to hide the identity of the transmitted content.
That’s the biggest limitation on a full rollout of DPI technology as a frontline deterrent against piracy. Whatever law enforcement officials can do on the Internet, criminals can do just as easily. So while DPI could be used as an effective deterrent against the many relatively law-abiding consumers who are stealing because it’s socially acceptable and low risk, it isn’t technology that can, in the long run, effectively fight organized copyright theft on the Internet.
The future, therefore, of DPI technology is an increasingly sophisticated cat-and-mouse game between Internet authorities, governments and organized online crime. The smarter the packet inspection, the smarter the encryption used by pirates to disguise their criminality.
So let’s be clear here. DPI isn’t a magic bullet that will end piracy overnight. In addition to the encryption issue, DPI technology is not a long-term solution for blocking access to specific URLs, nor will it stop peer-to-peer sharing of files with known hashtags.
While DPI technology is perhaps less critical for fighting piracy than it is for maintaining the security of the network, it is certainly one highly legitimate means to help fight online theft of media content. At the present time, it’s one useful tool in the multifaceted attack on Internet piracy. So given the billions of dollars lost each year to the movie, music and book industries through piracy, it is simply good business to want to utilize any technology that can legally help save the creative economy.
As part of the Guild’s ongoing effort to inform members about complex and multifaceted Internet issues, the DGA Quarterly continues its ongoing series of stories. In this edition, author and essayist Andrew Keen (
The Cult of the Amateur) dispels some of the misconceptions about deep packet inspection and explains its value in network management and combating piracy. Keen’s opinion does not reflect the opinion of the DGA on this issue.