Recently I took part in a Technology Liberation Front podcast about the Comcast controversy, with Adam Thierer, Jerry Brito, Richard Bennett, and James L. Gattuso. There’s now a (slightly edited) transcript online.
Economics of Eavesdropping For Pay
November 2, 2007 by
Following up on Andrew’s post about eavesdropping as a profit center for telecom companies, let’s take a quick look at the economics of eavesdropping for money. We’ll assume for the sake of argument that (1) telecom (i.e. transporting bits) is a commodity so competition forces providers to sell it essentially at cost, (2) the government wants to engage in certain eavesdropping and/or data mining that requires cooperation from telecom providers, (3) cooperation is optional for each provider, and (4) the government is willing to pay providers to cooperate.
A few caveats are in order. First, we’re not talking about situations, such as traditional law enforcement eavesdropping pursuant to a warrant, where the provider is compelled to cooperate. Providers will cooperate in those situations, as they should. We’re only talking about additional eavesdropping where the providers can choose whether to cooperate. Second, we don’t care whether the government pays for cooperation or threatens retaliation for non-cooperation – either way the provider ends up with more money if it cooperates. Finally, we’re assuming that the hypothetical surveillance or data mining program, and the providers’ participation in it, is lawful; otherwise the law will (eventually) stop it. With those caveats out of the way, let the analysis begin.
Suppose a provider charges each customer an amount P for telecom service. The provider makes minimal profit at price P, because by assumption telecom is a commodity. The government offers to pay the provider an amount E per customer if the provider allows surveillance. The provider has two choices: accept the payment and offer service with surveillance at a price of P-E, or refuse the payment and offer reduced-surveillance service at price P. A rational provider will do whatever it thinks its customers prefer: Would typical customers rather save E, or would they rather avoid surveillance?
In this scenario, surveillance isn’t actually a profit center for the provider – the payment, if accepted, gets passed on to customers as a price discount. The provider is just an intermediary; the customers are actually deciding.
But of course the government won’t allow each customer to make an individual decision whether to allow surveillance – then the bad guys could pay extra to avoid being watched. If enough customers prefer for whatever reason to avoid surveillance (at a cost of E), then some provider will emerge to serve them. So the government will have to set E large enough that the number of customers who would refuse the payment is not large enough to support even one provider. This implies a decent-sized value for E.
But there’s another possibility. Suppose a provider claims to be refusing the payment, but secretly accepts the payment and allows surveillance of its customers. If customers fall for the lie, then the provider can change P while pocketing the government payment E. Now surveillance is a profit center for the provider, as long as customers don’t catch on.
If customers know that producers might be lying, savvy customers will discount a producer’s claim to be refusing the payments. So the premium customers are willing to pay for (claims of) avoiding surveillance will be smaller, and government can buy more surveillance more cheaply.
The incentives here get pretty interesting. Government benefits by undermining providers’ credibility, as that lowers the price government has to pay for surveillance. Providers who are cooperating with the government want to undermine their fellow providers’ credibility, thereby making customers less likely to buy from surveillance-resisting providers. Providers who claim, truthfully or not, to be be refusing surveillance want to pick fights with the government, making it look less likely that they’re cooperating with the government on surveillance.
If government wants to use surveillance, why doesn’t it require providers to cooperate? That’s a political question that deserves a post of its own.
Comcast and Net Neutrality
October 24, 2007 by
The revelation that Comcast is degrading BitTorrent traffic has spawned many blog posts on how the Comcast incident bolsters the blogger’s position on net neutrality – whatever that position happens to be. Here is my contribution to the genre. Mine is different from all the others because … um … well … because my position on net neutrality is correct, that’s why.
Let’s start by looking at Comcast’s incentives. Besides being an ISP, Comcast is in the cable TV business. BitTorrent is an efficient way to deliver video content to large numbers of consumers – which makes BitTorrent a natural competitor to cable TV. BitTorrent isn’t a major rival yet, but it might plausibly develop into one. Which means that Comcast has an incentive to degrade BitTorrent’s performance and reliability, even when BitTorrent isn’t in any way straining Comcast’s network.
So why is Comcast degrading BitTorrent? Comcast won’t say. They won’t even admit what they’re doing, let alone offer a rationale for it, so we’re left to speculate. The technical details of Comcast’s blocking are only partially understood, but what we do know seems hard to square with claims that Comcast is using the most effective means to optimize some resource in their network.
Now pretend that you’re the net neutrality czar, with authority to punish ISPs for harmful interference with neutrality, and you have to decide whether to punish Comcast. You’re suspicious of Comcast, because you can see their incentive to bolster their cable-TV monopoly power, and because their actions don’t look like a good match for the legitimate network management goals that they claim motivate their behavior. But networks are complicated, and there are many things you don’t know about what’s happening inside Comcast’s network, so you can’t be sure they’re just trying to undermine BitTorrent. And of course it’s possible that they have mixed motives, needing to manage their network but choosing a method that had the extra bonus feature of hurting BitTorrent. You can ask them to justify their actions, but you can expect to get a lawyerly, self-serving answer, and to expend great effort separating truth from spin in that answer.
Are you confident that you, as net neutrality czar, would make the right decision? Are you confident that your successor as net neutrality czar, who would be chosen by the usual political process, would also make the right decision?
Even without a regulatory czar, wheels are turning to punish Comcast for what they’ve done. Customers are unhappy and are putting pressure on Comcast. If they deceived their customers, they’ll face lawsuits. We don’t know yet how things will come out, but it seems likely Comcast will regret their actions, and especially their lack of transparency.
All of which – surprise surprise – confirms my position on net neutrality: there is a risk of harmful behavior by ISPs, but writing and enforcing neutrality regulation is harder than it looks, and non-regulatory forces may constrain ISPs enough.
Comcast Blocks Some Traffic, Won't Explain Itself
October 23, 2007 by
Comcast’s apparent policy of blocking some BitTorrent traffic, which has been discussed on tech sites [example] for months, has now broken out into the mainstream press. Comcast is making things worse by refusing to talk plainly about what they are doing and why. (This is an improvement over Comcast’s previously reported denials, which now appear to be inconsistent with the facts.)
To the extent that Comcast has explained itself, its story seems to be that it is slowing traffic from heavy users in order to keep the network moving smoothly. This would be a reasonable thing for Comcast to do (if they were open about it) – but it’s not quite what they’re actually doing.
For starters, Comcast’s measures are not aimed at heavy users but rather at users of certain protocols such as BitTorrent. And not even all users of BitTorrent are targeted, but only those who use BitTorrent in a particular way: uploading a file to non-Comcast users while not simultaneously downloading parts of the same file. (In BitTorrent jargon, this is called “seeding”.) To get an idea of how odd this is, consider that an uploader who is experiencing blocking can apparently avoid the blocking by adding some download traffic.
It would likely be easier for Comcast to simply measure how much traffic each user is generating and drop the heaviest users’ packets, or just to discard packets at random (a tactic that falls most heavily on those who send and receive the most packets).
Beyond its choice of what to block, Comcast is using an unusual and nonstandard form of blocking.
There are well-established mechanisms for dealing with traffic congestion on the Internet. Networks are supposed to respond to congestion by dropping packets; endpoint computers notice that their packets are being dropped and respond by slowing their transmissions, thus relieving the congestion. The idea sounds simple, but getting the details right, so that the endpoints slow down just enough but not too much, and the network responds quickly to changes in traffic level but doesn’t overreact, required some very clever, subtle engineering.
What Comcast is doing instead is to cut off connections by sending forged TCP Reset packets to the endpoints. Reset packets are supposed to be used by one endpoint to tell the other endpoint that an unexplained, unrecoverable error has occurred and therefore communication cannot continue. Comcast’s equipment (apparently made by a company called Sandvine) seems to send both endpoints a Reset packet, purporting to come from the other endpoint, which causes both endpoints to break the connection. Doing this is a violation of the TCP protocol, which has at least two ill effects: it bypasses TCP’s well-engineered mechanisms for handling congestion, and it erodes the usefulness of Reset packets as true indicators of error.
People have apparently figured out already how to defeat this blocking, and presumably it won’t be long before BitTorrent clients incorporate anti-blocking measures.
It looks like Comcast is paying the price for trying to outsmart their customers.
Radiohead Album Available for Free, But Fileshared Anyway
October 18, 2007 by
The band Radiohead is trying an interesting experiment, offering its new album In Rainbows for download and letting each customer decide how much to pay. You can name a price of zero and download the album for free, if you want, or you can pay whatever price you think is fair.
Now Andy Greenberg at Forbes is reporting that despite Radiohead’s free-if-you-choose offer, many users are downloading the album from P2P systems rather than getting it from the band’s site. Some commentators find this surprising, but in fact it should have been predictable.
Why are some people getting In Rainbows from P2P rather than the band’s site? Probably because they find P2P easier to use.
Radiohead’s site makes you click and click to get the music. First you have to click through a nearly content-free splash screen. Then you click through another splash screen telling you things you probably already knew. Then you click an “ORDER” button, and click away a dialog box telling you something you already knew. Then after some headscratching, you realize you need to click the “VIEW BASKET” button, which takes you to a form asking you to name your price, in U.K. currency. (They link you to a third-party site, offering a large collection of currency-conversion tools – several more clicks to find the one you want.) After choosing your price, you click “PAY NOW”, at which point you get to stare at a “You are currently in a queue” screen for a while, after which you set up an daccount enter some personal information (including your email address and mobile phone number) and agree to some terms of service (which are benign, but it’s more time and more clicks to verify that). Finally, you get to download the music.
It’s easy to see why somebody might prefer a P2P download. Leaving aside legal issues – and let’s face it, many people do – the moral argument against unauthorized P2P downloading seems pretty weak in this case, where downloaders aren’t depriving the band (or anyone else) of revenue.
This is an interesting natural experiment that tells us something about why people use P2P. If people normally choose P2P over authorized channels because P2P is cheaper, we would expect customers to shift toward the authorized channel when it offers a zero price. But if people choose P2P for convenience, then we’d expect a shift toward more P2P use for this album, because people have fewer moral qualms about P2P downloading this album than they would for a normal album. The clunkiness of Radiohead’s site improves the experiment by sharpening the ease-of-use factor.
It’s too early to tell how the experiment will come out, but news reports so far indicate that the ease-of-use factor is probably more important than some pundits think. This is yet more evidence that had the record industry embraced easy-to-use Internet music technologies early on, things would be very different now.
[UPDATE (Oct 21, 2007): Bill Zeller documents how technical issues completely prevent a large number of users from legally downloading In Rainbows from Radiohead’s site.]
