April 18, 2024

Comcast and Net Neutrality

The revelation that Comcast is degrading BitTorrent traffic has spawned many blog posts on how the Comcast incident bolsters the blogger’s position on net neutrality – whatever that position happens to be. Here is my contribution to the genre. Mine is different from all the others because … um … well … because my position on net neutrality is correct, that’s why.

Let’s start by looking at Comcast’s incentives. Besides being an ISP, Comcast is in the cable TV business. BitTorrent is an efficient way to deliver video content to large numbers of consumers – which makes BitTorrent a natural competitor to cable TV. BitTorrent isn’t a major rival yet, but it might plausibly develop into one. Which means that Comcast has an incentive to degrade BitTorrent’s performance and reliability, even when BitTorrent isn’t in any way straining Comcast’s network.

So why is Comcast degrading BitTorrent? Comcast won’t say. They won’t even admit what they’re doing, let alone offer a rationale for it, so we’re left to speculate. The technical details of Comcast’s blocking are only partially understood, but what we do know seems hard to square with claims that Comcast is using the most effective means to optimize some resource in their network.

Now pretend that you’re the net neutrality czar, with authority to punish ISPs for harmful interference with neutrality, and you have to decide whether to punish Comcast. You’re suspicious of Comcast, because you can see their incentive to bolster their cable-TV monopoly power, and because their actions don’t look like a good match for the legitimate network management goals that they claim motivate their behavior. But networks are complicated, and there are many things you don’t know about what’s happening inside Comcast’s network, so you can’t be sure they’re just trying to undermine BitTorrent. And of course it’s possible that they have mixed motives, needing to manage their network but choosing a method that had the extra bonus feature of hurting BitTorrent. You can ask them to justify their actions, but you can expect to get a lawyerly, self-serving answer, and to expend great effort separating truth from spin in that answer.

Are you confident that you, as net neutrality czar, would make the right decision? Are you confident that your successor as net neutrality czar, who would be chosen by the usual political process, would also make the right decision?

Even without a regulatory czar, wheels are turning to punish Comcast for what they’ve done. Customers are unhappy and are putting pressure on Comcast. If they deceived their customers, they’ll face lawsuits. We don’t know yet how things will come out, but it seems likely Comcast will regret their actions, and especially their lack of transparency.

All of which – surprise surprise – confirms my position on net neutrality: there is a risk of harmful behavior by ISPs, but writing and enforcing neutrality regulation is harder than it looks, and non-regulatory forces may constrain ISPs enough.

Comments

  1. Just wondering how long Seth has been on Comcast’s payroll.

    I will NEVER trust any big business that lies and secretly degrades a service its customers PAY for.

    They won’t even tell you what their cap is.

    They pretty much thumb their nose at the FCC.

    They keep changing their story. If I did something similar while being interrogated by the police, they would immediately suspect LIES! Can’t keep his story straight.

    ANY corporation that gets so big for its britches that it blatantly defies all comers needs to be summarily dismantled.

    I WILL NOT STAND BY and watch one of the last bastions of TRUE freedom be relegated to some corporate entity who’s only goal is to separate as much money as it can from the average person and quality be damned.

    Seth you seem like a pretty bright guy, I find it hard to believe you spend soooo much time defending Comcast, when it’s pretty obvious they have been secretive, underhanded and have lied about this whole affair. Unless of course, you completely ignore all that and just believe that Comcast, with it’s huge compassionate heart, ONLY has it’s customers best interests in mind. If you really believe the stuff I’ve seen you post here, I have a bridge you might be interested in.

    Sort of reminds me of Neville Chamberlain, there’s no problem, Germany and Italy are our friends, no, really they are.

    Those who fail to learn from history are doomed to repeat it.

  2. I’m getting the impression that running the file sharing program Shareaza is causing Comcast to do something that completely stops my internet connection until i reset the cable modem.

    This is happening only when a file sharing program is running, and it started around the time the stories about Comcast messing with file sharing programs started.

    Of course it could have some other explanation.

  3. Ned Ulbricht says

    Anyhow, the cable companies were the ones who designed the DOCSIS standard…

    Tel,

    James Martin and James Westall, who co-authored, “<a href=”http://people.clemson.edu/~jmarty/papers/bittorrentBroadnets.pdf”Assessing the Impact of BitTorrent on DOCSIS Networks (to appear in Proceedings of the 2007 IEEE Broadnets), also make an observation in one of their earlier papers that seems relevant here.

    In “Validating an ‘ns’ Simulation Model of the DOCSIS Protocol” (July 2006), the co-authors note on p.1 (in PDF):

    DOCSIS systems are extremely complex. The DOCSIS HFC cable specification is a 500 page document. Due to the complexity and cost, there are no open source DOCSIS platforms that are available to researchers. In contrast to the Internet community, where academic researchers can introduce new protocols or protocol enhancements through the IETF’s RFC process, in the industry-centric HFC cable and WiMAX communities, standards are developed in members-only industry consortiums. As a result, the evolution of DOCSIS is being
    directed by industry with little involvement of academia. […]

  4. Oversubscription rates of 10x are not uncommon in the ISP industry but the honest ISP will at least be polite enough to advertise what the “contention ratio” of the service really is (they will never say “oversubscribe”).

    So what I propose is a dynamic cap on uploads that works like sliding windows in reverse: the more you upload, the lower your cap and the lower your priority. If you’re an interactive user, you’ll always have good response time, and if you’re a file sharing demon, you’ll always be able to do your thing, but gracefully.

    That’s kind of tricky to implement. It should be achievable but you will go a long way looking for a device that handles your design.

    Unfortunately, the current generation of DOCSIS modems can’t support this, because they only support a static cap that they take on at boot time. But when you get into the distributed bandwidth reservation protocols that we designed for the MBOA UWB protocol, it’s possible.

    Surely the modems can at least support a selection of channel on the upstream carrier, that would be enough. Let’s suppose you have 200 users on a given upstream. If there are 5 upstream channels then that’s 40 users per channel (on average). Each channel has enough bandwidth for 4 users to max out at the rate they thought they were buying (oversubscription is 10x).

    All you need to do run a bandwidth usage report once per month and sort the users by their usage. Put the 40 biggest users into the same channel to battle it out together, then the 40 next biggest into their channel and so on. It’s consistent oversubscription in all channels and it’s fair because eash user gets to share channel with similar users to themselves. Each month you run another usage report and sort the users again.

    I’m sure that the cable modem upstream is already divided into a number of frequency bands, not sure if the modems randomly hop between them or get allocated positions.

    Anyhow, the cable companies were the ones who designed the DOCSIS standard… if they can’t get it right then they just have to spend more money on infrastructure rather than blame their customers.

  5. I wonder why the attention of this blog has shifted to the slow-burning, US-only wiretapping/FISA issue so thoroughly when P2P is under acutely crushing attack from, I guess, the RIAA. Let’s see, we’ve had the Jammie Thomas judgment, and swift on its heels Comcast’s BT disruption.

    Unnoticed here but mentioned at other blogs is that large OiNK torrent site in the UK going dark.

    Unnoticed seemingly at all the usual blogs are:
    * Limewire seems to be slowly creeping into bed with the recording industry; a lot of people have shifted to Frostwire because of this.
    * Frostwire is now bundling some kind of browser toolbar. They let you opt out of installing it, and they *claim* it lacks spyware/adware, but they also imply that they get paid somehow as a result of people installing it. If it generates revenue for someone, it almost has to be spyware/adware despite their denial.
    * Shareaza is apparently kaput. Limewire, Frostwire, and Shareaza are THE three big Gnutella clients, and Shareaza also does BT and eD2K, with both of the others experimenting with BT capabilities (still at beta quality). About a month ago the Shareaza web site went down and never came back up. The software of course still works, but it looks like the beginning of the end, unless someone forks it (it’s open source, so a fork analogous to Frostwire forking Limewire is possible and would be legal, as with a further fork of Frostwire). Most interesting is that the Shareaza domain hasn’t expired yet, and in the meantime you can see what happens when you try to view the site. Not a timeout, interestingly…

    The connection was reset
    ——————————————————–
    The connection to the server was reset while the
    page was loading.

    Bogus RST packets are, of course, exactly how Comcast is meddling with BT. This appears at the former Shareaza web site’s address with any ISP, however. Only a backbone provider or someone with control of the Shareaza site’s own hardware would be able to do this. Lots of file sharers registered with forums at that site, so if someone bad has control of their hardware, they have obtained data on a lot of filesharers, a concern also raised in connection with OiNK and other downed torrent sites. The interesting thing is that they didn’t merely pull the plug but configured the firewall to deny HTTP access instead (as evidenced by the RST). The RST also does not appear instantly. I can think of only one reason to do this after capturing a site: to log all the attempts to reach it and perform tracing. Scary. Scarier is the alternative, which is that a backbone provider is forging RSTs. Great Firewall of China behavior there. A pilot project testing the feasibility of an American version of broad-based net censorship, conducted on the quiet? (The site, being hosted in America last I checked, would be affected for worldwide users by a hypothetical American Great Firewall.) If so it’s interesting that they test it on a p2p vendor web site. While China censors political and human rights sites primarily, the shape of things to come may include America censoring net access to filesharing and other consumer-empowering sites that Big Business doesn’t like. It seems communists and fascists sometimes differ in their censorship priorities.
    * eD2K seems to have been crippled somehow in the past week or so. Global searches perform poorly and usually only find what’s on the one server you’re connected to, even if you have highID. A majority of the servers in the server lists won’t even connect these days; I haven’t connected lately to anything myself except Razorback 3.1 and once in a blue moon one single other server whose name I forget. If a majority of the servers are simultaneously down and the remaining ones are isolated due to the network’s severe fragmentation resulting from the outages, it would explain these symptoms. Only a massive attack, either a legal crackdown or a virus/worm/similar network intrusion attack, seems able to explain such a large number of simultaneous downtimes, and if the state of affairs persists, it basically means eD2K users will effectively have lowID forevermore.
    * The local DSL provider in my area provides DSL modems with router functionality that have been firmware crippled to deny the user the right to forward ports. Fortunately the modem works just fine in bridge mode, but this will potentially cripple P2P users. I’ve seen Limewire working decently from behind one of their modems, so they may actually support UPnP, but not manual port forward, which puts eD2K users out in the cold if they don’t know enough about these things to figure out how to put the modem in bridge mode, which requires also configuring a PPPoE connection manually in Windows and protecting their computer with firewall software like ZoneAlarm (or a router they have more control over, e.g. if they have a home network and need one anyway). Of course, manual port forwarding is not for the faint of heart even when it “works normally”; I wonder how many users of these DSL modems even know that they have web interfaces?

    Regardless, BT, Gnutella, and eD2K all seem to be under simultaneous crippling attacks, in all cases NOT limited to specific ISPs’ persecution.

  6. Andrew Crystall says

    The problem is they’re not blocking “P2P”. They’re blocking any traffic which acts in a certain way. This is not limited to traditional P2P apps – at least one games series has a matchmaking protocol which works this way, and which is likely to remain forever broken because this ISP’s actions.

    That’s the real issue here, as far as I’m concerned. This is just one in a string of incidents, breaking apps which are in themselves in no way offensive because someone later comes up with an idea which uses a similar protocol, which is then blocked/shaped/whatever.

    PS; Feel free to suggest that you should have to have a ISP business account to set up a matchmaking server for four people to play a game. I’m afraid at that point I’ll laugh at you.

  7. @Richard Bennett

    * Don’t you know that the Internet is built on the concept of sharing, not hogging all the bandwidth you can get?

    Which is why its ironic that Comcast is trying to prevent people from SHARING by blocking seeding, no?

  8. @Richard Bennett:

    “The idea that Comcast should lower its upload cap on all of its users because a small number are consuming most of the network’s bandwidth illustrates just how ludicrous this discussion has become.”

    But, as those readers who have not been ignoring every mention of PowerBoost are aware, that cap only applies to sustained transfers anyway! The users who upload small amounts of data infrequently are subject to the burst cap, not the sustained cap. Lowering the sustained cap will only affect users who upload for long periods at the maximum possible rate, i.e. the ones you *want* to affect.

    “Where do BitTorrent users get their sense of entitlement?”

    From Comcast’s promotional material and service agreements, of course. When you sign up for a service that has a 384 kbps upload rate, and there’s no mention of any limit on the amount of data you can upload or the amount of time you can spend uploading at that rate, it’s reasonable to conclude that you’ll actually be allowed to use it.

    “Don’t you know that the Internet is built on the concept of sharing, not hogging all the bandwidth you can get?”

    Don’t you know that commerce is built on the concept of sellers providing the goods or services they claim to be providing? If you tell me I can upload at 384 kbps, but then you start whining when I actually do it, you’re undermining commerce.

    It’s simple: if you don’t want your customers using so much bandwidth, tell them how much they *can* use. Don’t tell them they can use more than you’re really prepared to give them, and don’t leave them in limbo with only the vague notion that their “unlimited” service isn’t really unlimited, but no idea of what the limit actually is. (Remember the “pieces of flair” argument from Office Space? If wearing only 3 pieces is unacceptable, don’t pretend 3 is the minimum.)

  9. In theory, any uploading application is disruptive to any assymmetrical network, but in practice the application is always BitTorrent of one of its close relatives among the “swarm” variety of P2P apps. But to be precise, modern packet-switched networks are not designed to handle constant load. We have another network that performs quite well with that traffic profile, it’s called The Telephone Network.

    The idea that Comcast should lower its upload cap on all of its users because a small number are consuming most of the network’s bandwidth illustrates just how ludicrous this discussion has become.

    * Where do BitTorrent users get their sense of entitlement?

    * Don’t you know that the Internet is built on the concept of sharing, not hogging all the bandwidth you can get?

    * Didn’t your mama teach you any manners?

    Comcast simply needs to apply Bennett’s First Law of Traffic: the more bandwidth you consume, the lower your priority becomes. That’s not a problem, it’s network engineering in action.

    Anyhow, this discussion has ceased to yield any benefit, so sayonara.

  10. @Richard Bennett:

    “I take it the problem is that BT is capable of disrupting this system if left unchecked, hence the need to throttle it.”

    Again, BT is no more capable of disrupting it than any other protocol; uploading at 384 kbps is uploading at 384 kbps no matter what program is doing it. The problem, to the extent that there actually is a problem, is that Comcast has oversold their upstream capacity: upstream cap * number of users * average utilization per user > total capacity. They advertised the ability to upload at 384 kbps, expecting that very few users would actually do it, and it turns out they guessed wrong.

    Comcast is trying to solve it by reducing average utilization per user — that is, preventing people from using the bandwidth they’ve paid for, while still maintaining the marketing fiction that their customers are allowed to make sustained transfers at 384 kbps.

    But if they manage to succeed at making BitTorrent unusable on their network, people will just switch to something else, maybe an application that opens fewer connections but still puts the same load on the network, and then Comcast will have to come up with some way to break *that* application. All because they don’t want to invest in their network to reduce the number of users per segment, or lower the cap to a more sustainable (but less marketable) rate.

  11. I continue to object to characterizing the problem in terms of BT. The problem is a mismatch between customers’ desires to share files and the available upstream bandwidth in the installed plant. The same problem would exist if the customers were sharing files using FTP, coordinating their activities by email, etc…

    Comcast has received bad publicity because their vendor, Sandvine, failed to understand the root cause of the problem, or if they did understand, failed to apply this understanding correctly. If Sandvine had characterized the problem correctly perhaps they would have come up with a variable bandwidth cap and we wouldn’t be having this discussion.

  12. In fact, what you described — “If you’re an interactive user, you’ll always have good response time, and if you’re a file sharing demon, you’ll always be able to do your thing, but gracefully” — is already a reality for Comcast customers in areas where BT isn’t being disrupted.

    Nobody has a problem with it, do they? I take it the problem is that BT is capable of disrupting this system if left unchecked, hence the need to throttle it.

  13. Ned Ulbricht says

    Users would thus be acting in their own interest, as well of those of other users, by using properly-designed TCP stacks.

    supercat,

    Although the immediate Comcast discussion here has moved down to the MAC layer, I think this is worth posting anyhow…

    Last June, Sally Floyd queried the “TCP Maintenance and Minor Extensions Working Group” about redesignating RFC 2861 “TCP Congestion Window Validation” (Experimental) as a “Proposed Standard”.

    Also see Dr. Floyd’s slides.

    According to the IETF-69 tcpm minutes (26 July 2007), one member of the working group was opposed.

    In a technically related matter, last month’s draft-ietf-tcpm-rfc2581bis-03.txt maintains the SHOULD recommendation in the revised section 4.1. The draft section concludes:

    [A] TCP SHOULD set cwnd to no more than RW before beginning transmission if the TCP has not sent data in an interval exceeding the retransmission timeout.

    My understanding of the BitTorrent spec is that a peer MAY* terminate a connection if it has received no messages for a configurable amount of time. Keep-alives messages SHOULD* be used.

    (*Keywords use in this paragraph are my interpretation.)

  14. Tony Lauck says

    Comcast will solve their problems by technical or economic means, or they will be the victim of creative destruction.

  15. @Richard Bennett:

    “Unfortunately, the current generation of DOCSIS modems can’t support this, because they only support a static cap that they take on at boot time.”

    Did you miss all the times I mentioned PowerBoost? The cap changes dynamically. Comcast already does lower the cap for sustained transfers, from a burst rate to the advertised rate.

    In fact, what you described — “If you’re an interactive user, you’ll always have good response time, and if you’re a file sharing demon, you’ll always be able to do your thing, but gracefully” — is already a reality for Comcast customers in areas where BT isn’t being disrupted.

  16. The question of user expectations vs. the engineering of packet networks is the key issue, for sure. Customers seem to expect that networks engineered for bursty traffic are somehow ripping them off if they can’t use constant bit rates right at the network’s peak rate, but clearly we can’t all do that. In fact, the fundamental deal that packet switching is based on is the we won’t use peak rate all the time, and if anyone needs to do that the packet switching model of bandwidth allocation is not for him.

    Comcast, for example, caps uploads at 384 Kb/s on a network that can deliver 10 Mb/s and provisions 100-300 users per headend. The network is “oversubscribed” by 4-12 times, and that never was a problem until people started using P2P software to run constant downloads and uploads. And of course people never would have turned to P2P software if there weren’t excess bandwidth sitting around for it to exploit.

    And now that 5% of Comcast’s customers use 80% of the upload bandwidth (and the rest get crappy response time), what are they supposed to do? I don’t think the idea of lowering the cap is correct, because that punishes everybody. And I don’t think banning P2P is correct, because sometimes it’s useful.

    So what I propose is a dynamic cap on uploads that works like sliding windows in reverse: the more you upload, the lower your cap and the lower your priority. If you’re an interactive user, you’ll always have good response time, and if you’re a file sharing demon, you’ll always be able to do your thing, but gracefully.

    Unfortunately, the current generation of DOCSIS modems can’t support this, because they only support a static cap that they take on at boot time. But when you get into the distributed bandwidth reservation protocols that we designed for the MBOA UWB protocol, it’s possible.

    In the meantime, P2P freaks will complain and Comcast will police to effectively create a dynamic cap. It’s the unhappy medium, and network neutrality regulations won’t alleviate the misery.

  17. Tony Lauck says

    I think that Richard may have quoted Bram Cohen out of context. Bram identified a problem and found a solution. When my Azureus Bit Torrent client is busy downloading or seeding there may be 20 to 50 open TCP connections, but there are seldom more than 2 active upload connections.

    In my opinion, Bit Torrent is getting a bad rap in this discussion. It is just another network application. If it overloads some networks, it is because it is a popular application and because those networks don’t measure up to their user’s expectations.

  18. @Richard Bennett: “So even when caps are in place, the network math still depends on load coming in bursts rather than at steady state.”

    At that point, it becomes an economic issue — how much are they prepared to invest in their network, and how much can they lower their rate caps without scaring off customers — not a technical one.

    Comcast sells a connection with a certain sustained upstream rate cap (say, 384 kbps), and it’s their responsibility to do the “network math” needed to make sure the cable can handle that. If constant uploading at 384 kbps presents a problem, then they can lower the cap to the point where it stops being a problem.

    Note that this doesn’t mean they can’t oversubscribe. But what you’re describing is not the typical oversubscription model, where they advertise 500 kbps per customer but only provide a fraction of that, knowing that some customers will use the whole 500 kbps while others use next to nothing, and on average they’ll have just enough capacity for their actual network load.

    Instead, if what you say is true, *none* of their customers can use the whole advertised rate without disrupting the network, which means the advertised rate is pure fiction. It’s not a burst rate: they already advertise a separate burst rate, and it’s much higher than the sustained rate.

  19. @Jesse: Yes, there is a capping mechanism in the cable modem (don’t try to defeat it, you can’t), but it doesn’t prevent collisions on large networks with large numbers of users, because it doesn’t want to. There is only one collision window, and it’s very small in relation to the network’s propagation delay. So even when caps are in place, the network math still depends on load coming in bursts rather than at steady state.

    @supercat: BitTorrent operates with 20-50 neighbors at a time, a scenario that TCP congestion control can’t handle. See: http://www.bittorrent.org/protocol.html :”TCP congestion control behaves very poorly when sending over many connections at once.” That’s not my opinion, it’s the opinion of the guy who created BitTorrent, Bram Cohen.

    And see remarks on collisions to Jesse.

  20. //The capping mechanism doesn’t prevent users from overloading the cable, as it doesn’t take effect until after an upstream packet has been transmitted ON THE CABLE. The filters above the headend simply cause the user station to retransmit ON THE CABLE.//

    Properly-functioning TCP implementations will reduce the rate at which they send packets in congested conditions. While it is true that the TCP stack may have to generate retries if some of its packets get dropped on the floor, thus resulting in a slight temporary increase in traffic, a properly-functioning stack will adjust its window parameters to minimize the number of packets that will end up getting dropped (and thus the number of packets that are transmitted more than once on the local segment).

    While it would be possible for someone to break their TCP implementation in such a way as send and retry packets overly aggressively, there would be no reason to do so, since it would not improve performance. If a gateway will deliver a maximum of 200 packets/second, randomly dropping packets beyond that rate, optimal throughput will be achieved when 200 packets/second are sent once each (net 200 packets delivered). Sending each packet twice each (400 total packets/second) would drop throughput by 25% (since 50% of packets would be delivered once, 25% twice, and 25% not at all). Sending more retries would drop throughput even further.

    While throttling at the modem would probably be a good idea (among other things, to protect the network in case somebody’s TCP stack goes berzerk) throttling anywhere in the data path will work.

  21. Chuck McManis says

    @cm: No, I wasn’t suggesting that you change your email address if you change ISPs. Only that you forward your email through their SMTP server (which you probably already do). They could even spoof this if they wanted to, responding to any DNS MX packet request giving their server as the best MX host, that would give them an opportunity to forward legitimate mail, make a copy for the Feds who are probably listening in, and keep spammers off their network. By a number of estimates, eliminating Zombie PC spam can free up nearly 25% of an ISP’s upstream bandwidth.

    Comcast doesn’t mind spammers because they aren’t criminally liable as they might be under the DMCA and the facilitating of copying copyrighted works. Since there is no RIAA going after spammers, low risk, so they let their customers suffer.

    –Chuck

  22. @Richard Bennett:

    Is that true? My understanding is that the capping is actually done in the cable modem. Wikipedia at http://en.wikipedia.org/wiki/DOCSIS backs that up:

    “Most DOCSIS cable modems have caps (restrictions) on upload and download rates. These are set by transferring a configuration file to the modem, via TFTP (Trivial File Transfer Protocol), when the modem first establishes a connection to the provider’s equipment.”

    “Comcast, the largest cable provider in the United States, caps downstream bandwidth at 4, 6, or 8 Mbit/s and upstream bandwidth at 384 kbit/s (48 kB/s), or 768 kbit/s (96 kB/s) for the 8 Mbit/s downstream package, for standard home connections. In some areas, they are offering 16 Mbit/s downstream and 1 Mbit/s (125 kB/s) or 2 Mbit/s (250 kB/s) upstream as a more expensive, yet speedier alternative; or to keep customers from switching to Verizon’s FiOS. These differing speed options are made possible by loading a particular configuration file, for the respective pricing tier or region, into the modem.”

  23. man-in-the-middle says

    This is just a classic man-in-the-middle attack at the packet level. Comcast needs to be smacked by a huge cluestick, whether it is by everyone going to Fios, or by lawsuits, i don’t care.

    Aren’t they risking common carrier status by doing this? They are forging packets that customers have paid to have sent across their network? How is this any more legitimate than me doing an ol’ Smurf attack? I mean i’m just sending forged packets. Why would that be against the law? Why is it my fault that machines respond the way they are programmed to respond.

    It’s not really my problem that Comcast is trying to do whatever it is they are really trying to do when they are defrauding me. Sorry, just because you have some problem doesn’t mean you can break the law.

  24. @Jesse, Jesse, Jesse; no, no, no. The capping mechanism doesn’t prevent users from overloading the cable, as it doesn’t take effect until after an upstream packet has been transmitted ON THE CABLE. The filters above the headend simply cause the user station to retransmit ON THE CABLE.

    This is critical because of the nature of the DOCSIS protocol, where a request for bandwidth is sent out in contention space before each packet can be delivered on the cable. The key parameter that affects the stability of the network is the collision probability of the requests for bandwidth, so Comcast needs to throttle them. The TCP packet drop mechanism offers no relief to this problem, especially on stations with 20 active TCP streams.

  25. @Richard Bennett:

    You have consistently failed to address the facts of the situation…

    Comcast can and does set an upstream bandwidth cap for cable modem users, and BitTorrent can’t and doesn’t get around it. This fact is easily observed by starting a BT session and noticing that your upstream speed never exceeds your cap, except for brief periods in the beginning as provided by PowerBoost.

    Comcast can and does identify users whose bandwidth use they deem to be excessive. This fact is easily observed with a quick Google search to turn up reports from users who have been warned by Comcast for their use.

    Since those are *already* sufficient for capping or charging excessive users, why exactly is “traffic shaping at the TCP level” necessary?

  26. @Brianary, who says: “Shockingly false. All that’s required is practical examination of the end result, or an unwillingness to accept the inevitable bugs and overhead of maintenance and complexity of this network filter.”

    In the first place, DOCSIS isn’t a “network filter”, it’s a medium access control protocol with some serious defects. An understanding of those defects in combination with an understanding of how BitTorrent works leads me to believe that traffic shaping at the TCP level is absolutely necessary on the Comcast network.

    To use your analogy, it’s a good idea to understand who uses firearms to shoot people before passing too many firearms regulations, otherwise you’ll simply annoy a lot of law-abiding citizens and leave guns in the hands of criminals. That’s an analogy, don’t get too excited about it.

  27. “…an understanding of this process is vital to forming a judgment about the legitimacy (or lack thereof) of Comcast’s traffic shaping.”

    Shockingly false. All that’s required is practical examination of the end result, or an unwillingness to accept the inevitable bugs and overhead of maintenance and complexity of this network filter.

    Do you need to know the complete history and science of firearms to know you don’t want to be shot?

  28. Tony Lauck says

    To Mitch Golden,
    …and sometimes the government has to step in to ensure that the funders of the politicians make money off of the public.

    To Jesse,
    Sounds like an opportunity for Sandvine to increase its revenues. 🙂

  29. @Tony Lauck:

    “For the same as the typical Comcast residential customer I get DSL service that delivers 600 KBPS downloads consistently and 90 KBPS uploads consistently.”

    Comcast gives me a consistent 750 KB/s down and 50+ KB/s up, with bursts up to 1500 and 100+ respectively (the aforementioned PowerBoost). They don’t even interfere with BitTorrent around here.

  30. Mitch Golden says

    To cm: There are lots of analogous situations to the situation the internet finds itself in now. The old analog telephone company, the local gas company, railroads, etc. In all these cases, there is a well-established tradeoff: if the monopoly is “natural”, it should be permitted to exist, but with substantial regulation. I have had a gas utility as a client, and there is a wall between the “monopoly” side of the business and the other parts.

    And thus, in the old days, Ma Bell would never have said (as AT&T has recently) that they would use their market power to prevent illegal traffic on their network. Even drug dealers and prostitutes can get phones.

    My concern, as Ed has indicated in his piece, is that the companies making these decisions are far from disinterested parties simply trying to optimize their networks for all their users. They are aggressively using their market power to further their other economic interests.

    Therefore, I believe that even if Comcast were to spell out exactly what it is doing (thus eliminating the claim of fraud) it shouldn’t be allowed *both* to be a monopoly (or a part of a small oligopoly) and to do whatever it wants.

    So, to Tony Lauck: No one wants “the government calling the shots”. What we want is the market calling the shots. But sometimes the government has to step in to make sure that the market will actually function.

  31. Tony Lauck says

    complaints from the ISP or any downtime. Each year the service has gotten faster. (I would also say more reliable, but it has been years since there has been an outage). There are no restrictions on running servers.

    Perhaps the reason why my local monopoly has been doing such a good job of providing networking service is because they are much smaller than Comcast. But Comcast is much smaller than the Government. Do we really want the biggest elephant in the room to be calling the shots?

  32. Tony Lauck says

    I must thank Richard Bennett for his kind personal comments.

    Network Neutrality is a very complex issue, involving technical, economic, and political factors. And the technical factors are among the most difficult parts of networking, involving resource management, congestion control, traffic shaping, or whatever you want to call the particular slice of the problem you are currently focusing on.

    I will say this however. I consider my self very fortunate to have a progressive internet provider. This company owns the local internet service, cable service and telephone service. For the same as the typical Comcast residential customer I get DSL service that delivers 600 KBPS downloads consistently and 90 KBPS uploads consistently. I have been moving a GB of data a day for a number of months without any co

  33. Sorry, tag closed this time:

    There’s some level of reasonableness that can be achieved in any regulatory system, and, of course, room for abuse. That’s not unique to net neutrality, yet some regulation seems to help some markets. When I first wrote about Comcast’s conflict of interest in throttling heavy bandwidth users as video competition, I linked to an FCC ruling about DSL providers being forbidden to interfere with VOIP traffic. To me, there are some no-brainers that regulators can do like that without hosing the Internet forever.

    As far as folks here complaining about the bandwidth hogs – imagine if electricity usage were flat-rate. That’s what we have with bandwidth packages now. In the case of the natural monopoly, such as an ISP with xTTH, history has shown that metered usage with public oversight of rates is the model that is least-bad given the forces involved.

    All these issues of TCP resets, port filtering, SPAM zombies, bloated websites, torrent seeding, illegal p2p usage, etc. would work themselves out with a reasonable per-GB fee (20 cents, perhaps) and a minimal connection fee ($10 per mo, maybe). I suspect that those on the $15/mo package now would still pay a similar rate, and those on the $80/mo package would also still pay a similar rate. But there’d be none of this hassle of offering and ‘managing’ free, because ‘free’ doesn’t exist.

  34. There’s some level of reasonableness that can be achieved in any regulatory system, and, of course, room for abuse. That’s not unique to net neutrality, yet some regulation seems to help some markets. When I first wrote

  35. Ned Ulbricht says

    On October 28th, 2007 at 12:34 pm, Spudz <http://www.nowhere.bogus/&gt; wrote:

    Ned Ulbricht said: “[U] <snip>

    Your attribution seems misleading. Unless you have a comment about my use of square brackets to insert capitalization, you probably should complain about Peter Svensson’s bylined story to his editors at the AP.

  36. //Perhaps the three Sandvine marketing white papers can shed some light on this matter://

    I looked through them briefly and saw no mention of the implications of violating RFC’s, nor do I fully understand the rationale.

    If one assumes that people put packets on the network for the purpose of having them delivered, then having the network gateway delay or in extreme cases drop packets from people who are using excessive bandwidth should be a perfectly fine way of controlling congestion. If the gateway is configured to transmit one 1K per user every 4ms, with a maximum queue of 500Kbytes (two seconds’ worth) packets, then a user will achieve optimal throughput when sending 1K/second, all of which will go through. Unless the user is trying to use many dozens of simultaneous TCP connections, a properly-functioning TCP stack will automatically adjust itself to match that level of bandwidth availability.

    While it is true that a broken TCP stack may generate excessive retries that would clog the local network, such excessive retries would degrade the performance of the machine sending them. Users would thus be acting in their own interest, as well of those of other users, by using properly-designed TCP stacks.

    In what way would such a method of traffic management not be optimal?

  37. Mitch: One practical problem is that you can lay only so much network infrastructure (cables, wireless stations, …); it’s prone to a “natural monopoly” issue much like rail, water, or power.

    Of course that can be worked around by legislating equipment access/sharing schemes. There are precedents in the telco sphere suggesting it can work.

  38. P.S. I’d agree that the only net neutrality regulation we need is of the antitrust variety. Perhaps also enforced line sharing, with infrastructure costs also being equally shared.

  39. BT wrote “Spuds your analogy does not work” followed by an assortment of insulting BS. I’m sorry, but I do not take kindly to this type of rude behavior. When I say something you are to accept it, not argue with me. Capisce?

    What would you say if your landlord sent nasty mail, unsigned, with a return address of your apartment number and later that day you got a visit from the police about it?

    Brianary said: “Comments that do not publish without any indication of why, or even any message that the comment has not published? Classy.”

    This has been a recurring problem around here. It’s sad to see indications that it has still not been corrected.

    Brianary also said deceptive marketing has led to Verizon being slapped with a $1 million fine. Unfortunately, to any kind of large business in America, $1 million is chump change. This slap on the wrist probably won’t change Verizon’s behavior one iota.

    Ned Ulbricht said: “[U]sers also reported Comcast blocking some transfers of e-mails with large attachments through an application that is fully in the legal sphere: Lotus Notes, an IBM Corp. program used in corporate settings.”

    BitTorrent is “fully in the legal sphere” or hadn’t you noticed? You may need to re-read your Universal v. Sony. Pay particular attention to the bits containing the phrase “substantial noninfringing uses”.

  40. Ed,
    I agree, you’re right.
    The first commentator to this entry said:
    Want to lose this fight? Keep calling it “Net Neutrality”.
    What we’re fighting against is extortion, plain and simple.

    We have laws against extortion. If he’s right, we don’t need more laws to deal with Comcast.

  41. BTW kids, follow up on Tony Lauck’s suggestion and go read about what Sandvine does and why, it’s enlightening. And in case y’all don’t know, Tony is one of the great luminaries of networking. When I met him in 1984 he was already one of the guiding lights of the standards world.

  42. Mitch Golden says

    I hesitate to wade into this back-and-forth, but I’ll point out that recent ISP shenanigans has persuaded Stephen H. Wildstrom of Business Week magazine to change his mind and he now supports greater regulation of the net.

    http://www.businessweek.com/technology/content/oct2007/tc20071024_623695.htm

    What I find very odd about this discussion (including the original article) is that Net Neutrality disappears as an issue if there is genuine competition and real customer choice among ISPs. None of the anti-regulation voices on this blog (and Ed’s original post) seem to address this obvious point: most users have limited to no choice in broadband ISPs. If there are no choices, the market can’t address anything.

    My preferred approach would be to reduce concentration in ISP space. Then, if you didn’t like what Comcast was doing, you could leave.

  43. The fundamental bottleneck of cable modem systems is the cable itself which is a shared medium. On the uplink you have multiple transmitters and single receiver and approx 40MHz of shared bandwidth to work in. It all comes down to how smart the modems are and what systems are in place to manage the bandwidth (and it comes down to how many customers they hang off each leg of cable).

    I did a bit of searching and it seems that Comcast is using DOCSIS 2.0 which provides TDMA (i.e. slotted upstream with centrally allocated talk-time for each modem and a request-for-bandwidth / acknowledge handshake) and supports a range of QOS options in the modem itself. That should be all the tools any network manager could ask for; not sure what features they actually bother using. In principle, the network management device can simply decide not to allocate slots to the customer who is over-using their upstream bandwidth… their modem will be blocked at the MAC layer.

    Also, people can bring their own modems to the party so there’s no doubt a mix of devices hanging off the cable which would complicate things (but the DOCSIS standard should cover that).

  44. I recently moved house and had to take Comcast as there is no low-cost DSL in my area. As a non-power user (specifically not downloading large files that benefit from sustained data rate) I’m profoundly unimpressed with the bandwidth, and my perception, though uncorroborated by numbers, is that it’s slower than my previous DSL for browsing and VPN sessions.

  45. Chuck: Many people use mail services outside the Comcast network, or whatever local network they happen to be roaming in. Do you think I will switch my email addresses everytime I switch ISPs?

  46. Perhaps the three Sandvine marketing white papers can shed some light on this matter:

    http://www.sandvine.com/solutions/p2p_policy_mngmt.asp

  47. Richard Bennett: Ideally, any throttling would be done between the home user’s LAN and the common segment of cable. I don’t know whether it’s possible to request that the cable modem throttle its rate of packet transmission, or whether there is any cable company equipment between the cable modem and the common segment that would allow for flow control.

    Provided that the cable company controls equipment between the user and the Internet at large, though, I don’t think it really matters. If the gateway between the cable segment and the Internet has a separate queue for each subscriber, and limits the rate at which packets received from each queue will be sent outward, that will limit the total TCP bandwidth usage of any subscriber with an even remotely RFC-compliant TCP implementation. To discourage people from breaking their protocol implementations by implementing excessive retries, the capacity for a user’s queue could be reduced if excessive retires were detected. In case of more severe problems, if flow control couldn’t be implemented at the modem, Comcast could offer software to implement PC-side throttling. Since a user would get maximal bandwidth utilization by having the PC send out the maximum allowable amount of traffic and having it get through (instead of the PC sending out 2x the allowable amount of traffic but having half the packets get randomly dropped), such flow-control software would enhance user performance at the same time as it reduced network congestion.

  48. Question: How many of the people who are whacking Comcast here can tell me, in general terms, what a DOCSIS cable modem has to do in order to send a packet upstream? My impression is that none of you can, yet an understanding of this process is vital to forming a judgment about the legitimacy (or lack thereof) of Comcast’s traffic shaping.

    @Ned, thanks for the homework assignment, I’m touched by your concern for my continued education, but I have to decline for the moment. I’d like to point you to this memo by Bram Cohen explaining the enhancements to the “choking” algorithm in one particular version of BT. Have a ball.

  49. No business can afford to sell server-level bandwidth at home-use-level cost.

    It’s that simple. BitTorrent is a huge bandwidth-hog. It’s designed to suck-up as much bandwidth as it can, for file-transfers.

    Ha, ha… what rubbish. Check out Exetel prices — the excess charge on bandwidth is $3 per Gig. At that price, the heavy users pay for their usage, but both servers and slurpers find it affordable. A packet is a packet after all. Most ISPs actually find their upload traffic costs them LESS than their download traffic because they are forced to buy symmetric links and the market in hosting has fewer buyers.

    The problem is that geek morality recognizes the negatives of bandwidth-hogs – people who use too much of a shared service are in the wrong.

    No, it just means that the billing model is wrong and everyone should be charged for what they use. Please don’t tell me that usage is difficult to measure or that I should feel sorry for someone setting up a business and not thinking carefully about their billing model.

    However, the contract is pretty clear, no servers on home service. That’s a big problem, since it means the geek is violating the contract, a major sin, and puts Comcast in the right.

    The distinction between a client and a server is very vague, even to experts and certainly does not exist at the packet level.

    Is a peer-to-peer network made of clients or servers? What about Skype? Are SIP clients really clients when they can both initiate and answer connections? What about real-time network gamers? What about ftp where connections are initiated from server to client? What about uploading large image files to flickr? What if I use an ssh client to connect to a remote ssh server which tunnels a remote client application back to the X11 server I’m using at home?

    DSL is just too slow

    Hmmm, there’s ADSL2 technology to put a 10 megabit link over a mile of very ordinary grade UTP, admitedly it might be a bit variable but I would have thought that even 5 megabit would be enough for most people.

    Yup, that precious bandwidth that Comcast is trying to get back by shutting down BitTorrent users could easily be gotten back if they would shut down the zombies running on their network. This is the real story Ed, why don’t they do this?

    Comcast could do a better than just reclaim the bandwidth, they could provide real service by a bit of careful statistical traffic analysis to politely inform the owners of machines that have been invaded by spammers that they have been done over (or are about to be done over). We know that zombie machines are used for credit-card theft, identity theft, illegal bank transfers, staging points for break-ins and you name it… the owners of those machines have a lot to lose.

    Even if technological abuse is uncovered, proving it can be a very difficult and time consuming process. On an individual basis, we have virtually no chance.

    Systematic packet tampering can relatively easily be detected by a small group of cooperating individuals. There are plenty of network performance measurement tools out there and writing your own is not that difficult. Those individuals have various options when it comes to notifying other customers of the results of their tests (or distribute their performance measurement tools under GPL).

    Yet when reading about these settlements, including my own personal experience, I fail to see how corporations are even being punished.

    Very few problems are solved by more lawsuits. What we want is to avoid lawsuits used as means to shut down the individuals who speak up about dishonest traders and shonky products.

    “If they were worried about too much internet traffic, they should drop packets, not forge TCP RST packets.”

    This is wrong. It just adds to the congestion, since the server will retry.

    TCP is designed to backoff on lost packets, lots of work has been done on TCP and congestion control — search on “Reno”, “Vegas”, “Westwood”, “Hybla”. Certainly, all common operating systems have TCP congestion control backoff in some form or other. The best thing a network node can do is slowly increase the end-to-end delay then start to drop packets (which happens naturally as a queue fills up, then overflows). If the endpoints are well behaved, they will backoff, if not well behaved then RST packets won’t fix the problem either. Sending RST packets is what firewalls do when they want to prevent any traffic on blocked ports. Randomly sending RST packets into the middle of live data streams is a way of telling your customers that your service is unreliable.

    Cable modem networks have limited upstream bandwidth, and there is often a crunch between interactive users (web surfers) and non-interactive users like BitTorrent. Why isn’t it appropriate to delay the non-interactive user so that the interactive user sees good performance?

    This is Network Engineering 101, folks.

    It is perfectly appropriate to delay non-interactive traffic and give preference to interactive traffic — if you can figure out which is which, and if you have a modem that knows how to apply QOS to the upstream. Sadly, many of these systems were badly designed to begin with and don’t provide useful features like QOS (you think I’m joking, no one would be that stupid, well they are).

    You can search the RFCs high and low and not find any mention of how to throttle applications that circumvent backoff and slow start by queueing multiple connections, which is what BitTorrent does.

    A non-standard endpoint, uninterested in cooperating with the network can circumvent anything except packet loss. That’s a simple fact, regardless of RFCs. The RFCs describe situations where both the network and the endpoints are designed to be mostly cooperative. Overloading RST packets merely encourages the endpoints to be uncooperative.

    Comcast not the customer owns the IP address that is on the packet. If would be different If I were to program one of my systems to send a reset with my next door neighbors IP address, but in the case of Comcast they own the IP.

    So let’s say someone at Comcast were to “borrow” a customer’s IP address and post a semi-pornographic (but not illegal) picture to Wikipedia then someone at Wikipedia found a clever way to track back that IP address to “name and shame” the Comcast customer. Comcast could claim that since they “own” the IP, they are completely in the clear? Somehow I don’t think it works that way…

    No matter how many packets the user generates, only a limited number will go onto the bandwidth-constrained network segment.

    Fully agree, this is ideal behaviour. However, I strongly suspect that the cable modems are too clunky to properly implement media-level upstream bandwidth management (e.g. they use simple ALOHA with no awareness of how much the other modems are using).

  50. Ned Ulbricht says

    Ned, you’re engaging in sophistry. The TCP RST that Comcast uses to throttle BitTorrent is quite effect in its solution space.

    Richard,

    According to Peter Svensson’s AP report, several days ago, Mitch Bowling, senior vice president of Comcast Online Services, acknowledged responsibility for problems with Lotus notes clients:

    [U]sers also reported Comcast blocking some transfers of e-mails with large attachments through an application that is fully in the legal sphere: Lotus Notes, an IBM Corp. program used in corporate settings.

    Kevin Kanarski, a network engineer for a major law firm, noticed the disruption in August and eventually traced the problem to Comcast. But he got the cold shoulder from the company’s customer support department.

    On Tuesday, Bowling acknowledged the problem, saying it was unintentional and due to a software bug that has been fixed. Kanarski said transfers started working again last week.

    Jesse,

    As far as news and mail clients go, I’d direct your attention to Van Jacobson and Mike Karels’ 1988 paper, Congestion Avoidance and Control, specifically p.19. The paper is cited in RFC 2581, in section 4.1 “Re-starting Idle Connections”.

    As Richard correctly points out, though, robust networking requires experimentation and observation. So, I’m sure he will tell us which versions of which BitTorrent client he’s using, on which version of which TCP stack. And, I expect he’ll post some sample packet traces to illustrate how BitTorrent “circumvent[s] backoff and slow start”.

  51. Precedent: Looks like deceptive use of the word “unlimited” is costing Verizon $1 million.

  52. @Richard Bennett:

    “BT opens multiple socket pairs and queues them for use when its upload rate is degraded. It’s not the same as the normal process of opening a socket pair per object and using all ones you have open.”

    Web browsers and Usenet clients do the same thing – keep connections open in case they might be reused. It’s nothing that can’t be controlled by the same bandwidth limiting techniques that are used for other traffic.

    “So the only entity that can reliably limit the flow of packets in this direction is the cable modem itself. And it’s not able to detect how busy the upstream path is because it’s an RF device that receives at one frequency and transmits at another.”

    Comcast’s PowerBoost feature temporarily raises the cap in certain situations. Surely, if it were impossible to ration usage according to availability, everyone else in the area would lose connectivity when one person’s PowerBoost kicked in and his upstream rate jumped from 384 kbps to 2 Mbps.

    “It’s not ideal, but given all the engineering constraints it’s about the best they can do.”

    Nonsense. We already know they can adjust the caps, accomodate even-faster-than-normal uploads without disturbing other users, and warn/terminate users who use too much bandwidth. The last one alone would be sufficient to deal with those “bandwidth hogs” – that is, if Comcast were interested in a general, simple, and fair solution instead of just scapegoating.

  53. Richard Bennet: I’m not familiar with how cable networking is set up, but are you saying that all the cable users in an area are on a shared network segment without so much as a buffering switch between them? If that is the case, how does the cable company restrict access to subscribers, and prevent non-subscribers from jabbering on the network?

  54. Ed, while Comcast clearly has the motive and the opportunity to commit the “crime” of violating the rules of net neutrality, I believe the only incentive they really need to inhibit BitTorrent uploads during congested times is to preserve bandwidth for the vast majority of their subscribers. Comcast’s network is asymmetrical, with a lot less upstream capacity than downstream, and the BitTorrent application can use a ton of upstream capacity, creating congestion for many of their high speed internet subscribers. Why shouldn’t Comcast act to preserve quality of service for the bulk of their subscribers? I’ve posted more on this topic at http://ikeelliott.typepad.com/telecosm/2007/10/comcast-as-traf.html

  55. I’m confused. The way i understood it, seeding torrents was the OPPOSITE of hogging – you’re actually sacrificing part of your bandwidth to help other people who want to get a file you ALREADY HAVE.

    @Richard. Wouldn’t building out more network infrastructure to remove the bottleneck also be a clean solution?

  56. @supercat: “The proper approach to limiting traffic is to limit the flow of packets onto the bandwidth-limited part of the network;”

    OK, that’s true enough. On the Comcast network, the bandwidth bottleneck is the path upstream from the cable modem to the cable modem controller. So the only entity that can reliably limit the flow of packets in this direction is the cable modem itself. And it’s not able to detect how busy the upstream path is because it’s an RF device that receives at one frequency and transmits at another. The upstream path, you see, is write-only.

    The only clean solution to this dilemma is to re-work the upstream protocol in the cable modem to adapt to information about network conditions coming down from the controller. And that means migrating to DOCSIS 2.0 or 3.0, and obsoleting a lot of customer-owned equipment.

    In the meantime, Comcast employs a stop-gap solution to prevent overload, namely limiting the requests for upstream data that are allowed inside the cable modem network. It’s not ideal, but given all the engineering constraints it’s about the best they can do.

    Now if you want to argue that dropping SYNs is a better way to do that than injecting RSTs, fine, that’s a nice academic discussion but it doesn’t about to much.

  57. @enigma_f: Yes, that section is dishonest. Comcast certainly does delay BitTorrent seeding, and in some cases appears to block it outright. They may have a good reason for doing so, but the FAQ is clearly not accurate.

    I would prefer it if this discussion had focused on that issue rather than the question of whether TCP RSTs are legitimate according some whitebread reading of the RFCs, frankly. The real issue is how the traffic shaping affects the customer, not the means by which it’s done.

    So I would say they deserve criticism for not communicating clearly with their customers, but not for traffic shaping. As you know, I believe traffic shaping is legitimate if done for the good of the majority of customers and the stability of the network. I don’t expect carriers to do magic, but I do expect them to be straight about the goals they’re trying to achieve and the general parameters of the service.

  58. Richard Bennett:

    Is this dishonest or not?

    From Comcast FAQS:

    Do you block access to peer-to-peer applications like BitTorrent?
    No. We do not block access to any Web site or applications, including BitTorrent. Our customers use the Internet for downloading and uploading files, watching movies and videos, streaming music, sharing digital photos, accessing numerous peer-to-peer sites, VOIP applications like Vonage, and thousands of other applications online.

  59. @Richard Bennett:

    Comcast has clearly lied about their policies, and deserve all the criticism they get.

    I certainly hope there is a lawyer getting a class action suit going against them.

    @Brianary:

    Yes, I have noticed that too. Very annoying-I think it is connected with anti-spam…

  60. Seth Finkelstein: The proper approach to limiting traffic is to limit the flow of packets onto the bandwidth-limited part of the network; a system should buffer a reasonable number of packets, but drop packets when the buffer is full.

    If a user has a reasonable number of TCP sockets open, each trying to send data as fast as possible, then hitting the bandwidth limit will cause packets to start piling up in the buffer. Since a packet won’t be acknowledged until it actually gets sent, the sender will stop sending once the amount of unacknowledged data reaches its window size (it will resume sending once some of the data gets acknowledged).

    If a user tries to open so many sockets that the total window size exceeds the buffer size, then packets will start getting dropped. If the user’s system has a good TCP implementation, it will reduce the window size to minimize packet loss. A poor implementation will keep sending packets that are going to get dropped, resulting in poor performance, but that’s a problem for the user, not the system. No matter how many packets the user generates, only a limited number will go onto the bandwidth-constrained network segment.

  61. @Brianary: Most people don’t consider Comcast a “telco” as their main business is cable TV. Their network was designed for analog streaming in one direction, and was adapated for the brave new world of interactive communications. Or I should say, is in the process of being adapted. Obviously, it’s not there yet.

    Verizon is a telco, and their FiOS service is as good as anything in the world. It operates at a true upload/download rate of 20 Mb/s, same as the Japanese system that’s advertised as 100 Mb/s. Verizon is no slouch, so I think your complaint is actually with AT&T, and on that score I agree with you: AT&T sucks.

  62. @Jesse, who said “It circumvents the conventional rules of internet traffic by opening multiple connections at once? You must be kidding! Web browsers, Usenet clients, and download managers all do that. BitTorrent doesn’t do anything special that can’t be handled by the same mundane bandwidth restrictions used for other traffic.”

    Close, but no cigar. BT opens multiple socket pairs and queues them for use when its upload rate is degraded. It’s not the same as the normal process of opening a socket pair per object and using all ones you have open. Read the BT protocol description of choking.

  63. Well, nevermind the reference links. Too much trouble in this lousy comment system. I’ll put them at Brianary.Blogspot.com.

    (Comments that do not publish without any indication of why, or even any message that the comment has not published? Classy.)

  64. (1) “Top 30 Countries for Broadband Internet Access”

  65. (cont.)

    All of the US telcos have really let us down with lousy bandwidth (1) and terrible upload speeds (2), while taking $200 billion dollars from taxpayers (3). Complaints about bandwidth saturation just further underscore this point for me, and anything that shakes the complacency of the telcos seems good to me.

  66. (cont.)

    Sandvine is likely to have bugs, and I suspect I’ve already seen the effect of that when I tried to get to Google on two separate days and got a “Connection was reset by server” for several hours. It’s the same reason I don’t run Vista: I don’t need the DRM layer there adding more hassle to my life.

  67. @Seth:
    I’m sorry about this, but it sure felt like I was being accused of something for occasionally downloading linux distro ISOs via BitTorrent, and that I should just accept interruption or sabotage because I’m a home user.

  68. …testing…

  69. Spuds your analogy does not work. Your landlord can use the address of the building all he wants. But he can’t use your signature. You own the signature he owns the building.

    In fact your landlord can change your apartment number if he chooses. He owns the building, not you.

    The IP address is not your signature. It is the building address. The address you own is the MAC (Media Access Control) address that is burned into your equipment. That is the address you own, assuming that you own and not lease your cable modem.

    Bill Tedeski

  70. “Figure out how much bandwidth a customer can use without disturbing the network, and if they want to use more than that, charge them for it.”

    This is exactly what my current ISP does. There’s even a usage-tracking page I can check. If I go over a certain amount of transfer, they charge an extra buck fifty for every additional gig, or 100MB, or some such of transfer.

    “Comcast not the customer owns the IP address that is on the packet. If would be different If I were to program one of my systems to send a reset with my next door neighbors IP address, but in the case of Comcast they own the IP.”

    This “justification” doesn’t wash IMHO. Your argument justifies the landlord in my apartment building forging my signature on snail-mail because “he owns the address that is on my mail; I just lease it”. Somehow I suspect the courts would look dimly upon my landlord if he actually did forge my signature on a snail-mail, and even more dimly on his defense attorney if said attorney tried to argue justification with an argument like you just made.

  71. How can your forge your own name?

    >>They are *forging* packets, leading one customer to
    >>believe that another said something that he didn’t.

    Comcast not the customer owns the IP address that is on the packet. If would be different If I were to program one of my systems to send a reset with my next door neighbors IP address, but in the case of Comcast they own the IP.

    Bill Tedeski
    Network Engineer

  72. Seth

    you have it backwards,

    >> “If they were worried about too much internet traffic,
    >> they should drop packets, not forge TCP RST packets.”

    >>”This is wrong. It just adds to the congestion,
    >> since the server will retry.

    If you drop packets the server will retry, a TCP RST will shut down the connection creating a complete disconnect with no retry.

    Forged TCP Resets are a standard practice for firewalls, IDS and IPS systems. Every firewall product that I use has the ability to send a Forged TCP Reset. Most IDS or IPS (Intrusion detection systems, Intrusion prevention systems) can send a reset to break a session it does not like.

    Bill Tedeski
    Network Engineer

  73. @Richard Bennett:

    “They’ve apparently found a way to discover BitTorrent from its traffic profile, and stifle excessive seeding connections when the Comcast network is busy. We know this mechanism works because would-be bandwidth hogs complain about it.”

    What makes you think it applies only to “excessive” connections, only when the network is busy, and only “would-be bandwidth hogs” are affected? Do you mistakenly believe that all seeding is excessive and only bandwidth hogs do it, or do you know something about Comcast’s BitTorrent interference policy that the rest of us don’t?

    “You can search the RFCs high and low and not find any mention of how to throttle applications that circumvent backoff and slow start by queueing multiple connections, which is what BitTorrent does. This is an app that doesn’t want to play by the conventional rules of Internet traffic, so other measures are required.”

    It circumvents the conventional rules of internet traffic by opening multiple connections at once? You must be kidding! Web browsers, Usenet clients, and download managers all do that. BitTorrent doesn’t do anything special that can’t be handled by the same mundane bandwidth restrictions used for other traffic.

  74. As a sometime network engineer, “net neutrality” regulations seem to me very, very difficult.

    Here in Japan, you buy your fibre connection from one of three providers (NTT, KDDI or Usen), and then in the case of NTT, at least, you pick one of twenty or thirty ISPs. It’s the same with DSL. This works well; if I don’t like one ISP due to poor connectivity, tech. support, or whatever, I can switch my connection to another. This is a solution to the problem that can’t be gamed.

    Strange that in a socialist country like Japan they can manage to create a competitive market for this, yet in a “capitalist” country like the U.S., they grant Internet access monopolies to local loop carriers.

  75. Ned, you’re engaging in sophistry. The TCP RST that Comcast uses to throttle BitTorrent is quite effect in its solution space. They’ve apparently found a way to discover BitTorrent from its traffic profile, and stifle excessive seeding connections when the Comcast network is busy. We know this mechanism works because would-be bandwidth hogs complain about it.

    You can search the RFCs high and low and not find any mention of how to throttle applications that circumvent backoff and slow start by queueing multiple connections, which is what BitTorrent does. This is an app that doesn’t want to play by the conventional rules of Internet traffic, so other measures are required.

    And we do know from experience that BitTorrent will retry later. You don’t have to be a network engineer to see it happen, just run a test of your own, as I have.

  76. Ned Ulbricht says

    This is Network Engineering 101, folks.

    Richard,

    First, the vast majority of Comcast’s residential broadband customers are not network engineers. I know it, Comcast knows it, Comcast’s customer’s know it, and I suspect you know it, too. Comcast’s customer’s are surely should be expected to rely on Comcast’s customer support FAQs—whether or not the contract is unclear or ambiguous enough to require extrinsic evidence for its interpretation.

    Second, as Steve Bellovin recently pointed out, Comcast’s characterization of RST as just a delay is wrong:

    That won’t fly. Stating that the software will retry assumes a certain model of software. Perhaps some particular clients will retry. Others may not. The semantics of a TCP Reset are quite well-defined; there’s even an Internet Best Current Practice that warns against other inappropriate TCP Resets.

    Mr. Bellovin’s blog entry points to BCP 60, and I’d call your attention specifically to section 2.3 which “recommend[s] that the TCP reset not be used as a congestion control mechanism, because this overloads the semantics of the reset message, and inevitably leads to more aggressive behavior from TCP implementations in response to a reset.”

    I’d also call your attention to STD 7, on p.82,

    RST A control bit (reset), occupying no sequence space, indicating that the receiver should delete the connection without further interaction. […]

    Of course, you know all that networking 101 stuff, I merely point it out for the benefit of those who might be able to understand the difference between “delay the connection” (not what the standard says) and “delete the connection” (what the standard says.)

  77. Would it also be fair to ask for all the billions in dollars in telecoms subsidies back?

  78. Apparently, yes. So the rule is you can violate your contract as long as you don’t make a nuisance of yourself, but if you do, you’re history.

    That’s fair.

  79. @Seth:

    “If a certain application is a big problem, in practice it makes sense to focus some attention on that protocol.”

    Not when the general problem of excessive bandwidth use can be tackled *directly* with less controversy and less effort.

    “People are violating their service plan.”

    That’s an amusing claim, since Comcast’s customer support (as quoted earlier) says they allow BitTorrent, and they explicitly support other “server” activities like Xbox Live matches.

    “Do you know this [that Comcast can target bandwidth usage directly] for a fact?”

    Yes, I do. It’s basic networking, and they already do something similar – look into their PowerBoost feature, which dynamically changes the upstream and downstream caps. They also keep track of users who are using excessive bandwidth – it’s easy to find reports from people who’ve been warned for violating the unwritten limits of their supposedly unlimited service.

    @Richard Bennett:

    “No servers on the residential account, boys and girls”

    So, all those residential users hosting Halo games are violating their service contracts? And Comcast, by providing tech support for Xbox connections, is encouraging users to violate their contracts?

  80. Oversubscribing bandwidth is a reasonable business model, and I for one am glad to have capacity available for when I need it. If the only connection available to my home were based on the expectation that I’d use 100% of it all the time, I’m sure it would be painfully slow compared to what’s currently available.

    People who DO use 100% of their bandwidth all the time are the tragedy in this commons. Comcast must do something or everyone’s connection will be dog-slow. I suspect that killing bittorrent upload connections is simply the most expedient way they currently have. It also allows them (for a time) to continue to offer very simple Internet connection services (e.g., flat monthly fee, one size fits all, no metering).

    Sooner or later, the next big thing in peer-to-peer filesharing will come out and its transport layer will be encrypted. Comcast won’t be able to masquerade as anybody and their current countermeasures will cease to be effective. It’s my hope that Comcast is taking the time available to set up a reasonable solution to the bandwidth hogs. Perhaps something involving metering, which I doubt they can just switch on right now. I hope they don’t just start throttling encrypted traffic – things will get *very* ugly in that case.

  81. Please.

    BitTorrent, operating as a seed after a download has completed, is clearly a server. It sets there and takes requests for files from elsewhere on the Internet and serves up files to them. This is exactly what an ftp server does.

    Cable modem networks have limited upstream bandwidth, and there is often a crunch between interactive users (web surfers) and non-interactive users like BitTorrent. Why isn’t it appropriate to delay the non-interactive user so that the interactive user sees good performance?

    This is Network Engineering 101, folks.

  82. I disagree with the contention that running BitTorrent automatically classifies a computer as a server. If Comcast believes BitTorrent is server software, it should classify BitTorrent as such and clearly communicate that to its customers. Otherwise, customers should continue to treat it as a client application for downloading large files efficiently.

    I think most coverage glosses over the legit uses for BitTorrent. For example, Silkroad Online http://www.silkroadonline.net is a free massively-multiplayer online role-playing game (MMORPG) distributed using BitTorrent. Silkroad competes in the same game genre as World of Warcraft. To get the game, you download an 850 MB file. Without BitTorrent you would download the game from a single server limited by the number of simultaneous connections it has available. Because it takes a long time (one to three hours) for each person to completely download such a large file, subsequent downloaders will wait for quite a long time before they can begin their own downloads. Making the game available as a BitTorrent distributes the game more efficiently, separates the large file into bits and pieces, and allows users to connect to each other to download portions that another user has. Instead of setting up a server room to handle traffic for those few people, the company can set up one or two seed computers and distribute its game to many in a shorter time period (theoretically, at least).

    Comcast plays a monopoly role in almost every single instance it operates in. It controls prices at its whim and fancy. When will its misbehavior begin conversations about breaking it up like AT&T once was? Because if it continues to mistreat its customers and slows innovation in the telecommunications market, it will one day end up on the chopping room floor.

    I think “net neutrality” should be respun as “consumer choice.” As of today, there is no real choice. I’m hoping that municipalities and municipal utilities start laying thick fiber like they do sewers, water lines, and roads. Then any company can trip over themselves to provide real and good service to me.

  83. @Seth:
    “The answer to your question is that the 1% running filesharing servers can saturate the connection for the other 99%.”

    Is this a problem in countries that have actually invested some money in network infrastructure? Can you even confirm that this has ever actually happened? Is this even physically possible, given their topology?

    Besides, I doubt that most BitTorrent users have the program completely unthrottled, or else they wouldn’t be able to do anything else with their connection.

    “This is exactly why server-level traffic can’t be supported on home-use-level plans.”

    If I have set up a web server to remotely control my lights, or sprinklers, or DVR, is that morally wrong in your view? This is a very old-school, broadcast-media, top-down, authoritarian view of the Internet. We’re moving beyond that. Everyone is a potential “server” now, whether Comcast has prepared for it or not.

    ““Unlimited” doesn’t mean “feel free to set-up a server for all the bandwidth”. They aren’t unclear about it, and the people construing it as such are plainly in the wrong.”

    Can you provide a link to a Comcast definition that clarifies “unlimited”? Are you making your own assumption about it? Does it justify misrepresenting my traffic by sending reset packets? Couldn’t they simply throttle heavy traffic, or are they just hoping that less-technical users will give up after things stop working, and assume there was something wrong with their software?

    And yes, construing “unlimited” as actually meaning “unlimited” is unforgivably wrong. Clearly these customers need to be punished for abusing whatever bandwidth Comcast feels like giving them. And Comcast should be able to call that level of service anything they want, because we’re just home users.

    “They do try to tamp [BitTorrent] down, to keep it from saturating the network. This is good.”

    I’m guessing you don’t use much BitTorrent, Seth. You seem to have significant contempt for those that do. You also seem to think they are the cause of all network congestion; not companies that take tax dollars and refuse to increase capacity as usage increases, not code to eavesdrop and log communications for the government, not botnets, not YouTube or Skype or high-def video podcasts or podcasts in general. BitTorrent isn’t evil, it just does the same thing that CacheFly and others do, just more dynamically, and at a price that we plebs can afford. BitTorrent isn’t evil, and singling it out to tamp down isn’t “good”.

    @enigma_foundry:
    “Yes, net neutrality is slippery to define, and would be difficult to enact into law.”

    Why can’t we just call it extortion, and make it illegal to charge specific websites to ensure their traffic is not “lost”?

  84. Seth, I congratulate you on your stamina. The horde is whacking on you harder than the Red Sox whacked the Rockies, and you stood up to them.

    No servers on the residential account, boys and girls; there’s a whole other service plan if that’s what you want to do. If you wants to play, you gots to pay.

  85. @Tito Villalobos

    “If they were worried about too much internet traffic, they should drop packets, not forge TCP RST packets.”

    This is wrong. It just adds to the congestion, since the server will retry.

    “They should do “traffic management” of heavy users NOT specific protocols.”

    If a certain application is a big problem, in practice it makes sense to focus some attention on that protocol.

    [Pre-emptive: “Gotcha! You said above they were targeting traffic, but now you said specific protocol, so which is it, huh huh huh?” – the two aren’t exclusive, the idea is that focusing _some attention_ on a protocol can be done as part of a overall server management]

    “then comcast needs to change its service plans and/or reprice them”

    People are violating their service plan.

    “That agreement means I can fully utilize those transfer rates, 24/7 for the whole month.”

    I GIVE UP! If the very clear NO SERVERS contract provision does not convince you, then it’s futile for me to say anything further.

    @Jesse

    “if they want to reduce bandwidth usage, they can target it directly”

    Do you know this for a fact? Are you absolutely and completely certain that you know the best way of handling the specific situation they face? So confident that if they aren’t doing what you think they should do, it must be a sham reason rather than you’re mistaken?

    I addressed the “targeting” point above.

    Y’know, this is really tedious, and I’m not cut out for it 🙁

  86. Seth, you wrote, “Jesse, how many people send connection-saturating email for hours at a time?”

    Not many, I imagine, but that’s not the point. The point is, if they want to reduce bandwidth usage, they can target it directly. That would be simpler, more fair to the BT users who *don’t* use excessive bandwidth (e.g. stop seeding once the share ratio reaches 1:1), and it would catch excessive bandwidth use from other protocols (email, FTP or game servers, etc.).

    You wrote, “And why punish people – and what are you going to do to them?”

    Cap their bandwidth usage or ask them to upgrade to a higher service level. Pretty simple. Figure out how much bandwidth a customer can use without disturbing the network, and if they want to use more than that, charge them for it.

    You wrote, “Comcast in fact isn’t after BitTorrent per se, they appear to be after file-sharing servers.”

    I think you’re mistaken. They are specifically targeting BitTorrent seeds, and seeding is a normal part of the BT protocol. You don’t have to be a “file-sharing server” in order to seed a file; downloaders are expected to stick around seeding after their file is done, at least until they’ve uploaded as much as they downloaded.

    In a technical sense, every BT user is a “server” because they accept incoming connections and send data upon request, but I must point out that every Xbox gamer is also a server in that sense. Millions of people use cable modems to play video games online, and the matchmaking system of popular games like Halo is designed so that the games are hosted by players. Just like BitTorrent, acting as a “server” is an inherent part of the protocol – but of course Comcast doesn’t want to upset all those gamers.

  87. Ed, you are correct that “There is a risk of harmful behavior by ISPs, but writing and enforcing neutrality regulation is harder than it looks …”. All one has to do is look at our tax code to see how “neutral” regulation has become a morass.

    Nevertheless, the mantra of let the market solve these problems has become quite tiresome for the same reason: “It is much harder than it looks”.
    1. Few people have the technical knowledge to uncover the misuse of technology by a company. If I recall correctly, it took over one year before the Sony Rootkit debacle was accidental discovered. Of course once it became public knowledge, a firestorm of protest arose.
    2. Even if technological abuse is uncovered, proving it can be a very difficult and time consuming process. On an individual basis, we have virtually no chance. One can only hope that activist organizations such as the Electronic Frontier Foundation would have the power to stand-up to large corporations.
    3. The apologies of corporations appear meaningless. When caught – “Oops Sorry about that.”, and they move onto a new scam. What continues to bother me about the response of the do-not-regulate crowd is the continued insistence that market forces will somehow “solve” this malfeasance. The fact that corporate misconduct continues is proof that the market is imperfect. So why don’t the do-not-regulate people demand that corporations modify there behavior to be ethical? If individuals steal from corporation, they go to jail. If a corporation steals from an individual shouldn’t there be a demand by the do-not-regulate crowd that it be similarly punished?
    4. Speaking of corporate punishment. There have been numerous articles on class action lawsuits related to corporate misconduct. Yet when reading about these settlements, including my own personal experience, I fail to see how corporations are even being punished. The class action lawsuits seem to end with no admission of guilt and the defrauded consumer only gets a coupon off his/her next purchase from the company that you should not even being doing business with.
    5. Finally, if companies don’t want to be burdened with onerous regulation, simple logic dictates that they would not do undisclosed underhanded activities to steal from their customers. Logically it seems we need regulation, however imperfect, based on continued and long term corporate misconduct. The market has not solved this problem and there is no expectation it would do so in the future.

  88. Chuck McManis says

    I run my own mail domain on a server. I’m perhaps more technically literate than some but I am certainly not particularly unusual in Silicon Valley. So riddle me this…

    I keep a log of all the SPAM I reject. Its a ton. And as everyone knows, and you can prove easily to yourself, most of that spam comes from one source. ZOMBIE PCs ON BROADBAND NETWORKS. Yup, that precious bandwidth that Comcast is trying to get back by shutting down BitTorrent users could easily be gotten back if they would shut down the zombies running on their network. This is the real story Ed, why don’t they do this? This would be both useful (people would cheer) and no one (except the spammers) would be pissed off at them. How could they do this?

    Sure tracking a bunch of dynamic IP hopping encrypted BotNets might be tough but to talk to my mail server they have to connect it on good old port 25, defined back in the stoneage of networking. So as a first approximation Comcast could block *ANY* originating packet which is headed for port 25 outside their network. If you’re a legitimate Comcast user you’ll have set your PC to use their mail host (convieniently something like smtp.comcast.net) if your a spammer you don’t want your filth going through a choke point where bayesian filters can weed it out. Next if your a legitmate customer you probably don’t tell the mail server your connecting to that your return address is somewhere not on Comcast’s network. Another EASY EASY way to filter out needless traffic.

    And once you can’t send spam from Comcast’s users, you’ve just made botnets based on Comcast connected PCs less valuable. Taking money right out of the botnet operator’s pockets.

    Instead, Comcast goes after BitTorrent users ? Isn’t that like arresting the security guard at a superfund site because he tossed his cigarette butt on the ground?

    –Chuck

  89. Tito Villalobos says

    @Seth:
    To repeat much of what was said above, but hopefully more concisely (and politely):
    The issue here is that
    a) If they were worried about too much internet traffic, they should drop packets, not forge TCP RST packets.
    b) They should do “traffic management” of heavy users NOT specific protocols.
    c) If the Comcast claim that “heavy users are using too much bandwidth” is true then comcast needs to change its service plans and/or reprice them. They state (from the link above) that Comcast offers un-capped transfers at the listed bandwith. That agreement means I can fully utilize those transfer rates, 24/7 for the whole month. If that is unrealistic, then the fault is Comcast’s for offering and advertising a service it cannot or will not deliver, not the customer’s for actually using the service to it’s full extent.

    In fact, Slashdot had this similar (in my opinion at least) case earlier this week:
    http://yro.slashdot.org/article.pl?sid=07/10/25/1237202

  90. Ned: That’s right. They don’t prevent you from using BitTorrent in an absolute sense. They do try to tamp it down, to keep it from saturating the network. This is good.

    The TOS provision against servers is not enforced fanatically and rigidly. Only when something threatens to become a big problem, and only to an extent to keep it from doing lots of damage. That’s also good.

    It really tells you something – or should – that this minor bit of traffic management is being flamed as, literally, a Federal crime.

  91. Ed:

    Yes, net neutrality is slippery to define, and would be difficult to enact into law.

    But what about a more modest approach, simply legislating transparency in service bandwidth and protocol carrying by the ISP, making it hard for them to legally hide what they do?

    That avenue seems to have some promise, especially in markets where folks have some choice.

    EF

  92. Ned Ulbricht says

    However, the contract is pretty clear, no servers on home service.

    Seth,

    From Comcast High-Speed Internet FAQ:

    Do you block access to peer-to-peer applications like BitTorrent?

    No. We do not block access to any Web site or applications, including BitTorrent. Our customers use the Internet for downloading and uploading files, watching movies and videos, streaming music, sharing digital photos, accessing numerous peer-to-peer sites, VOIP applications like Vonage, and thousands of other applications online.

    So, while the contract may be very clear to you, your answer is contradicted by Comcast’s customer support.

    They say that BitTorrent is an acceptable use. And they say that uploading files is an acceptable use.

  93. Without regard to whether Comcast’s goals are legitimate, it seems that their tactics may be illegal under wire fraud statutes. They are *forging* packets, leading one customer to believe that another said something that he didn’t.

    It’s as if my mom and I were sending streams of post cards to each other, and someone at the post office decided we were sending too many. So he creates a card saying “Go away, I hate you,” forges my name to it, and sends it to my mom. He simultaneously sends me one with my mom’s forged signature.

    Is anyone aware of any criminal investigations here?

  94. If Comcast keeps forging reset packets (which is reportedly the way they throttle traffic) for people who upload without downloading, doesn’t that ultimately increase bandwidth consumption?

    I’m more than a little wary of the idea that lawsuits or customer choice will have any significant impact on Comcast’s behavior. What are the damages for having some small (and arguably TOS-violating) part of your traffic disrupted?

  95. Brianary: The answer to your question is that the 1% running filesharing servers can saturate the connection for the other 99%. This is exactly why server-level traffic can’t be supported on home-use-level plans.

    “Unlimited” doesn’t mean “feel free to set-up a server for all the bandwidth”. They aren’t unclear about it, and the people construing it as such are plainly in the wrong.

    Same way “Staying out” doesn’t mean never writing comment in a favorite blog (though sadly, I have come to regret even that).

  96. So which is it, Seth?

    “Note 99% of individual Internet users download and don’t upload except in a trivial sense.”

    – or –

    “No business can afford to sell server-level bandwidth at home-use-level cost.”

    No business can afford to sell what you (rather subjectively) call “server-level” bandwidth for 1% of their customers? Check out the amount of subsidies the telcos have gotten from the taxpayers, and what we’ve gotten in return. Compare that to other countries. The US is near the bottom for broadband speed.

    Comcast needs to throttle their customers? Great, but they have to own up to it. They don’t get to say “unlimited” in every mailing and on the side of every bus if they don’t really mean it. They set up that expectation, and they need to come clean if they can’t keep up any more because they blew all of our tax subsidies and overpriced service charges on CEO yachts and bubblegum instead of infrastructure.

    Don’t insult us all with the “geek morality” condescension, Seth. Extortion is wrong. Bribery is wrong. False advertising is wrong. Your thinly-stretched rationalizations notwithstanding.

    And what happened to STAYING OUT of it, Seth?

  97. Jesse, how many people send connection-saturating email for hours at a time? And why punish people – and what are you going to do to them? Comcast in fact isn’t after BitTorrent per se, they appear to be after file-sharing servers.

    What would it take to get people to consider that this makes sense? A foot-stamping of “They must do exactly like I would do it, from my armchair!”, is not reasonable.

  98. Seth, if Comcast wants to limit bandwidth usage, they have much more direct means at their disposal. It makes no sense to specifically target BitTorrent when they could target bandwidth usage in general.

    That is, if they don’t want people maxing out their upstream 24 hours a day, they could simply look at overall bandwidth usage, and punish the users whose upstream usage they deem to be excessive. It doesn’t matter what protocol they’re using – bits are bits, and someone who’s constantly sending huge email attachments is just as much of a bandwidth hog as someone who’s constantly seeding torrents.

    The fact that they ignored this *simpler* means of achieving the alleged goal of bandwidth conservation, in favor of a complicated and secretive method that targets one specific protocol, suggests that their goal is, in fact, something else entirely.

  99. I think my argument about net neutrality isn’t whether or not a czar would investigate Comcast now, but rather what would happen when Comcast starts to offer “Home Service Plus” which is exactly the same as the current service, but doesn’t “delay” BitTorrent traffic. On the one hand, Comcast would be admitting exactly what they’re doing, but on the other hand they’re charging extra for one protocol versus another. The whole “Pay Per Protocol” is what I want to avoid by encouraging this nebulous idea of Net Neutrality.

    I agree that if Comcast starts dropping speeds of certain classes of traffic that the market would help punish them, but that assumes that they have competition. The trouble comes when your only broadband provider doesn’t offer you sufficient service through the only cable line coming to your home. I’m not riddled with choices here, (DSL is just too slow) but if I have a choice I will buy the fastest connection possible.

    I really am not able to dig up my neighbor’s back yards and put new lines in, so I have to try to squeeze everything I can through the one that Comcast happens to have put in my back yard. If there is anything I can do to improve the market competition for the radio frequences going through that hunk of copper in my back yard, I’ll do it.

  100. Ned, I think there’s an attempt to rationalize the ranting against Comcast within “geek morality”. The problem is that geek morality recognizes the negatives of bandwidth-hogs – people who use too much of a shared service are in the wrong. And Comcast’s High-Speed Internet is a shared service. The way people try to argue out of that, is to focus obsessively on construing Comcast as offering a guaranteed individual service, which then gives the geek moral permission to rant at Comcast for not living up to the contract. However, the contract is pretty clear, no servers on home service. That’s a big problem, since it means the geek is violating the contract, a major sin, and puts Comcast in the right. The geek reaction is to try to say BitTorrent isn’t a server (laughable), or that since overall defining a server is difficult, the contract is not meaningful. But again, this is not a hard case, since BitTorrent’s mission in life is file-serving. Hence trying to bring in everything else.

    So, my answer to you is that whatever dubious representations Comcast may have in terms of selling its service, this case, right here, this particular situation, is not one of them.

  101. I disagree with your fundamental presence:

    BitTorrent is an efficient way to deliver video content to large numbers of consumers — which makes BitTorrent a natural competitor to cable TV. BitTorrent isn’t a major rival yet, but it might plausibly develop into one. Which means that Comcast has an incentive to degrade BitTorrent’s performance and reliability, even when BitTorrent isn’t in any way straining Comcast’s network.

    Comcast is in the content delivery business. Historically, the content they delivered has been fixed bandwidth channels of 6 MHz wide video/audio. In the recent past they used some of those 6MHz channels for packet data delivery. In the age of digital set-top boxes, can there be much of a distinction between the broadcast video/audio channels and the narrowcast video/audio files? Isn’t there a trend towards just-in-time delivery of video entertainment (where the users selects what they want to see) versus just-in-case delivery (broadcasting all of those channels whether I watch them or not)? If Comcast were more with it, I’d think they’d be looking for a way to improve their capability of being a common carrier of all types of content, not just the pre-programmed video channels.

  102. Ned Ulbricht says

    No business can afford to sell server-level bandwidth at home-use-level cost.

    Seth,

    Whether that’s true or not, take a look at Comcast’s High-Speed Internet Fact Sheet, available from the “press room” at their site.

    Do you think that Comcast’s “Fact Sheet” is sufficiently informative in New York?

  103. There is a reason to apply traffic shaping policies to traffic that is not responsive to network congestion, as described in RFC896. But somehow I doubt that this is the primary motivation here.

  104. “what we do know seems hard to square with claims that Comcast is using the most effective means to optimize some resource in their network.”

    Actually, I’d say the opposite – what we know seems to indicate they are making very reasonable choices.

    Look, there are hard cases, but this isn’t one of those hard cases.

    No business can afford to sell server-level bandwidth at home-use-level cost.

    It’s that simple. BitTorrent is a huge bandwidth-hog. It’s designed to suck-up as much bandwidth as it can, for file-transfers.

    And anyone who would appoint themsevles as czar is by definition confident they’d make the right decision. After all, aren’t you confident in what you wrote in the quote above?

  105. Mitch Golden says

    Ed, I disagree that this confirms your position that there isn’t something to worry about. My belief is that this confirms the position of the skeptics that the ISPs are bound and determined to play games, and that in the long run, just having hackers and activists monitor what they doi. It remains to be seen whether Comcast actually stops doing whatever it is they’re doing, and I think it’s considerably optimistic to assume that in the future people will be able to figure out what is going on and stop the ISPs from doing it.

    Frankly, I think it’s likely that the major ISPs start to collude to enforce these sorts of restrictions. AT&T has already indicated that they are going to play games with their backbones.

  106. Ned,

    I didn’t mean to imply that the market alone, without the threat of legal enforcement, can cope with all forms of deception.

    What I meant to say is that serious deception can be deterred (or punished) by legal means; and in the absence of such deception the market can operate.

  107. Ned Ulbricht says

    Arggh. Messed up blockquote above. Just read Brad Stone’s post that I linked to. And do consider the possibility that the reporter messed up. But then contrast the unamed executive’s statements with other Comcast press statements, elsewhere.

    P.S. Ed, when will your blog here get a preview on comments? If there’s one thing that might pursuade me to stop commenting here, the lack of preview would be it. Of course, maybe you think that’s a feature….

  108. Ned Ulbricht says

    So why is Comcast degrading BitTorrent? Comcast won’t say. They won’t even admit what they’re doing, let alone offer a rationale for it, so we’re left to speculate.

    Ed,

    According to Brad Stone, writing in The New York Times BITS blog, “Comcast: We’re Delaying, Not Blocking, BitTorrent Traffic” (22 Oct 2007):

    The executive declined to talk in detail about the technology, citing spammers or other miscreants who might exploit that knowledge. But he insisted the company was not stopping So why is Comcast degrading BitTorrent? Comcast won’t say. They won’t even admit what they’re doing, let alone offer a rationale for it, so we’re left to speculate.file transfers from happening, only postponing them in certain cases. He compared it to making a phone call and getting a busy signal, then trying again and getting through. In cases where peer to peer file transfers are interrupted, the software automatically tries again, so the user may not even know Comcast is interfering.

    When I compare this report to the AP’s report of their testing, then I’m left to speculate that Comcast is being intentionally deceptive.

    You wrote, “If they deceived their customers, they’ll face lawsuits.”

    Whatever your stance on network neutrality, there’s another issue concerning the appropriate role of the FTC, and other public agencies, in regulating unfair practices in the marketplace. It’s a pretty fringe position to advocate that market mechanisms alone are sufficient to deter deceptive practices.

  109. Want to lose this fight? Keep calling it “Net Neutrality”.

    What we’re fighting against is extortion, plain and simple.

  110. Michael Kohne says

    I can’t get off of Comcast right now (I’m under contract till next spring, and Verizon doesn’t yet have FIOS to my house), but the minute I can go, I will. And this incident (along with the fact that my comcast service is more than bit spotty) will be one of the stated reasons.

    I’d have a lot less trouble with them doing stuff if they’d just admit what they are doing. They have spent DAYS trying to pretend nothing is wrong when clearly, something is wrong. All they had to do to avoid this storm of criticism was to admit what was going on from day one.

    When will they learn?