In my last post I told a story about the Level 3/Comcast dispute that portrays Comcast in a favorable light. Now here’s another story that casts Comcast as the villain.
Story 2: Comcast Abuses Its Market Power
As Steve explained, Level 3 is an “Internet Backbone Provider.” Level 3 has traditionally been considered a tier 1 provider, which means that it exchanges traffic with other tier 1 providers without money changing hands, and bills everyone else for connectivity. Comcast, as a non-tier 1 provider, has traditionally paid Level 3 to carry its traffic to places Comcast’s own network doesn’t reach directly.
Steve is right that the backbone market is highly competitive. I think it’s worth unpacking why this is in a bit more detail. Let’s suppose that a Comcast user wants to download a webpage from Yahoo!, and that both are customers of Level 3. So Yahoo! sends its bits to Level 3, who passes it along to Comcast. And traditionally, Level 3 would bill both Yahoo! and Comcast for the service of moving data between them.
It might seem like Level 3 has a lot of leverage in a situation like this, so it’s worth considering what would happen if Level 3 tried to jack up its prices. There are reportedly around a dozen other tier 1 providers that exchange traffic with Level 3 on a settlement-free basis. This means that if Level 3 over-charges Comcast for transit, Comcast can go to one of Level 3’s competitors, such as Global Crossing, and pay it to carry its traffic to Level 3’s network. And since Global Crossing and Level 3 are peers, Level 3 gets nothing for delivering traffic to Global Crossing that’s ultimately bound for Comcast’s network.
A decade ago, when Internet Service Retailers (to use Steve’s terminology) were much smaller than backbone providers, that was the whole story. The retailers didn’t have the resources to build their own global networks, and their small size meant they had relatively little bargaining power against the backbone providers. So the rule was that Internet Service Retailers charged their customers for Internet access, and then passed some of that revenue along to the backbone providers that offered global connectivity. There may have been relatively little competition in the retailer market, but this didn’t have much effect on the overall structure of the Internet because no single retailer had enough market power to go toe-to-toe with the backbone providers.
A decade of consolidation and technological progress has radically changed the structure of the market. About 16 million households now subscribe to Comcast’s broadband service, accounting for at least 20 percent of the US market. This means that a backbone provider that doesn’t offer fast connectivity to Comcast’s customers will be putting themselves at a significant disadvantage compared with companies that do. Comcast still needs access to Level 3’s many customers, of course, but Level 3 needs Comcast much more than it needed any single Internet retailer a decade ago.
Precedent matters in any negotiated relationship. You might suspect that you’re worth a lot more to your boss than what he’s currently paying you, but by accepting your current salary when you started the job you’ve demonstrated you’re willing to work for that amount. So until something changes the equilibrium (like an competing job offer), your boss has no particular incentive to give you a raise. One strategy for getting a raise is to wait until the boss asks you to put in extra hours to finish a crucial project, and then ask for the raise. In that situation, not only does the boss know he can’t lose you, but he knows you know he can’t lose you, and therefore aren’t likely to back down.
Comcast seems to have pursued a similar strategy. If Comcast had simply approached Level 3 and demanded that Level 3 start paying Comcast, Level 3 would have assumed Comcast was bluffing and said no. But when Level 3 won the Netflix contract, Level 3 suddenly needed a rapid and unexpected increase in connectivity to Comcast. And Comcast bet, correctly as it turned out, that Level 3 was so desperate for that additional capacity that it would have no choice but to pay Comcast for the privilege.
If Comcast’s gambit becomes a template for future negotiations between backbone providers and broadband retailers, it could represent a dramatic change in the economics of the Internet. This is because it’s much harder for a backbone provider to route around a retailer than vice versa. As we’ve seen Comcast can get to Level 3’s customers by purchasing transit from some other backbone provider. But traffic bound for Comcast’s residential customers have to go through Comcast’s network. And Level 3’s major customers—online content providers like Netflix—aren’t going to pay for transit services that don’t reach 20 percent of American households. So Level 3 is in a weak bargaining position.
In the long run, this could be very bad news for online businesses like Netflix, because its bandwidth costs would no longer be constrained by the robust competition in the backbone market. Netflix apparently got a good deal from Level 3 in the short run. But if a general practice emerges of backbone providers paying retailers for interconnection, those costs are going to get passed along to the backbone providers’ own customers, e.g. Netflix. And once the precedent is established that retailers get to charge backbone providers for connectivity, their ability to raise prices may be much less constrained by competition.
Conclusions
So which story is right? If I knew the answer to that I wouldn’t have wasted your time with two stories. And it’s worth noting that these stories are not mutually exclusive. It’s possible that Comcast has been looking for an opportunity to shift the balance of power with its transit providers, and the clumsiness of Level 3’s CDN strategy gave them an opportunity to do so in a way that minimizes the fallout.
One sign that story #2 might be wrong is that content providers haven’t raised much of a stink. If the Comcast/Level 3 dispute represented a fundamental shift in power toward broadband providers, you’d expect the major content providers to try to form a united front against them. Yet there’s nothing about the dispute on (for example) the Google Public Policy blog, and I haven’t seen any statements on the subject from other content providers. Presumably they’re following this dispute more closely than I am, and understand the business issues better than I do, so if they’re not concerned that suggests maybe I shouldn’t be either.
A final thought: one place where I’m pretty sure Level 3 is wrong is in labeling this a network neutrality dispute. Although the dispute was precipitated by Netflix’s decision to switch CDN providers, there’s little reason to think Comcast is singling out Netflix traffic for special treatment. In story #1, Comcast is be happy to deliver Netflix (or any other) content via a well-designed CDN; they just object to having their bandwidth wasted. In story #2, Comcast’s goal is to collect payments for all inbound traffic, not just traffic from Netflix. Either way, Comcast hasn’t done anything that violates leading network neutrality proposals. Comcast is not, and hasn’t threatened to, discriminate against any particular type of traffic. And no, declining to upgrade a peering link doesn’t undermine network neutrality.
I addressed the technical and policy issues behind this debate in my blog post at Innovation Policy, http://www.innovationpolicy.org/now-playing-video-over-the-internet
One important fact is that neither Level 3 nor Comcast fits the cubbyholes to which Steve assigns them; Level 3 is in this mess because of the CDN business, not their transit business; Comcast peers with a dozen large networks in the US, and sells transit and business services as well as eyeball network services. They actually send more traffic off net to their peers than they receive.
ISPs have operating costs above and beyond transit. Comcast has a network of 750,000 route miles to maintain, and Level 3 has only 100,000 mile to worry about. While CDNs reduce transit costs, the increase operational expenses on the 600,000 miles of internal route miles that Comcast runs.
When you drill down in the operational details of the networks, it’s clear that Level 3 (and Netflix) wants something for nothing and is playing the net neut card for strategic purposes.
Many commentators are spending a great deal of effort trying to detail things like traffic ratios, distance of respective transport, and cost of upgrades and maintenance. They are also trying to detail exactly which businesses the players are in, and the shape of their networks.
The most fundamental issues are instead:
1. Comcast has market power over a significant portion of last mile consumers, giving it the ability and incentive to make demands on its interconnection providers that it would not be able to make in a competitive market
2. Level 3 operates in competitive markets for backbone and CDN services, and seeks to deliver traffic to customers in Comcast’s last mile
These roles are especially clear in the case of Netflix traffic. Whether or not Comcast also sells transit and business services is irrelevant. Whether or not you classify Level 3’s Netflix traffic as “on-net” peering or plain-old transit is irrelevant.
You’re right, the market as a whole is more complicated than the simple categories I laid out (and I even noted this in my post and in the comments). Nevertheless, with respect to the dispute at hand, they are helpful and accurate descriptors.
I don’t see how Comcast has market power. Level 3 is free to tell Comcast to pound sand and rely on transit. Unless it wants control over the amount of capacity between it and Comcast. In which case it looks a lot more like a customer than a peer.
Comcast may have market power with respect to it’s retail service. But it has no market power over Level 3, which is entirely free to ignore Comcast.
Let’s imagine that Level 3 refused to pay for additional peering, so the Netflix traffic flows over the existing Level 3-Comcast peering links and saturates them. Netflix customers who use Comcast notice a lot of dreaded “buffering”. If the Netflix-Level 3 CDN contract includes an SLA, that SLA has been violated, potentially invoking some kind of penalty for Level 3. There is no way for Level 3 to provide adequate CDN service without paying Comcast, so they pay.
That’s absolutely right. But if Level 3 needs Comcast to help it deliver on an SLA to Netflix, it better be prepared to pay Comcast. Otherwise Level 3 can make whatever commitments it wants to it’s customers and in so doing can also commit Comcast without Comcast’s consent.
Yeah, what Wes said.
Also:
http://www.voxel.net/blog/2010/12/peering-disputes-comcast-level-3-and-you
Not only can they arbitrarily degrade the traffic over the Level 3 link, they can do it regardless of how it enters their network… and from the description on that blog post they have frequently used this power in the past as leverage in peering/transit negotiations.
If Comcast is willing to degrade the quality of its own Internet service, there’s not much anyone can do about that. But a real third party transit provider would laugh pretty hard if Comcast informed them that they had to pay for the privilege of providing Comcast with service.
I’m not sure I understand your point about Level 3 being unable to route around Comcast. There are many ISPs to which Level 3 may not connect directly. Like everyone else, Level 3 reaches them via one or more third party networks. If Level 3 didn’t like Comcast’s proposal, it should have shown some “backbone” here, and told Comcast to go ahead and access this content via third party transit relationships. The fact that Level 3 did not do so would tend to indicate that Level 3 would rather pay Comcast so that it has control over interconnection, then let Comcast decide unilaterally (as a transit customer) how much interconnection capacity it needs.
I imagine that while that might work for a short period of time, Comcast could just as easily make the same demands of whoever ends up delivering the traffic to them, regardless of whether both parties call it “transit” or “peering” traffic. What’s more, Comcast has already demonstrated the willingness to do things like send traffic over highly congested transit links when upstream providers refuse to peer with them on their own terms.
If Level 3 decides not to pay Comcast and Comcast decideds to pay someone else to get Level 3 CDN traffice, other tier 1 providers may see this increase of traffic over their networks from Level 3 and decide that Level 3 is no longer a peer, and they begin to charge Level 3 for access.