Last week, Comcast offered a defense in the FCC proceeding challenging the technical limitations it had placed on BitTorrent traffic in its network. (Back in October, I wrote twice about Comcast's actions.)
The key battle line is whether Comcast is just managing its network reasonably in the face of routine network congestion, as it claims, or whether it is singling out certain kinds of traffic for unnecessary discrimination, as its critics claim. The FCC process has generated lots of verbiage, which I can't hope to discuss, or even summarize, in this post.
I do want to call out one aspect of Comcast's filing: the flimsiness of its technical argument.
Here's one example (p. 14-15).
As Congresswoman Mary Bono Mack recently explained:
The service providers are watching more and more of their network monopolized by P2P bandwidth hogs who command a disproportionate amount of their network resources. . . . You might be asking yourself, why don’t the broadband service providers invest more into their networks and add more capacity? For the record, broadband service providers are investing in their networks, but simply adding more bandwidth does not solve [the P2P problem]. The reason for this is P2P applications are designed to consume as much bandwidth as is available, thus more capacity only results in more consumption.
(emphasis in original). The flaws in this argument start with the fact that the italicized segment is wrong. P2P protocols don't aim to use more bandwidth rather than less. They're not sparing with bandwidth, but they don't use it for no reason, and there does come a point where they don't want any more.
But even leaving aside the merits of the argument, what's most remarkable here is that Comcast's technical description of BitTorrent cites as evidence not a textbook, nor a standards document, nor a paper from the research literature, nor a paper by the designer of BitTorrent, nor a document from the BitTorrent company, nor the statement of any expert, but a speech by a member of Congress. Congressmembers know many things, but they're not exactly the first group you would turn to for information about how network protocols work.
This is not the only odd source that Comcast cites. Later (p. 28) they claim that the forged TCP Reset packets that they send shouldn't be called "forged". For this proposition they cite some guy named George Ou who blogs at ZDNet. They give no reason why we should believe Mr. Ou on this point. My point isn't to attack Mr. Ou, who for all I know might actually have some relevant expertise. My point is that if this is the most authoritative citation Comcast can find, then their argument doesn't look very solid. (And, indeed, it seems pretty uncontroversial to call these particular packets "forged", given that they mislead the recipient about (1) which IP address sent the packet, and (2) why the packet was sent.)
Comcast is a big company with plenty of resources. It's a bit depressing that they would file arguments like this with the FCC, an agency smart enough to tell the difference. Is this really the standard of technical argumentation in FCC proceedings?
Front page posts
Sadly, there's truth, and then there's political truth. The difference is in the amount of money each will put into one's campaign coffers over the next two, four, or six years. Which one do you think Mary Bono Mack is interested in?
One way of avoiding wasted bandwidth would be to abolish copyright.
Then the instantaneous diffusion of the Internet could be better harnessed for distribution of new art instead of compensating for a ridiculous prohibition against unauthorised copies of old art.
I would not go so far as to say that they can tell the difference, or that they will even check. Look at some of the more ridiculous patents -- and it is the patent office's *job* to check these things..
Sigh ... I don't know if this is really a technical issue, and I'm again getting the feeling that ...
Oh, why bother, what am I thinking, of course it's not a technical issue, and I'm only going to get in trouble by not going along with the ISP-ARE-CENSORS!!!!!!! party line :-( :-( :-(
Anything more authoritative than necessary to convince the (current) FCC is wasted effort. Comcast has a duty to their stockholders not to waste excess money on this.
And then it will have to explain to its shareholders why it lost so many customers.
Oh, wait, they already have to do that.
"If you build it, they will come" only applies up to a point. To use a traffic analogy (as Comcast does in their testimony), building a bigger highway might attract more people to use it, and thus not permanently reduce congestion, but that's no reason not to build bigger highways. Without investing in greater capacity, how does Comcast plan to handle the massive bandwidth consumption of movie streaming services such as Netflix's "Watch Instantly" or movie rental services as in iTunes? These things are only going to get more popular.
Secondly, regardless of whether Comcast wants to call these RST packets "forged" or not, people are reporting being unable to seed torrents to non-Comcast users. If the reason you use BitTorrent is to distribute a file efficiently, Comcast's practice makes it nigh impossible to do so. This is not simply traffic shaping, this is breaking the functionality of one particular program.
As much as I disagree with what Comcast was doing, I tend to agree with dr2chase. Get this out of the FCC!
I first started working on congestion control in datagram networks in 1978 when I took over managing the DECnet architecture. It didn't take me long to recognize that the problem was more a research problem than an engineering problem and one of my first steps was to engage some of Digital Equipment's researchers, including Raj Jain and K.K. Ramakrishnan.
Thirty years have gone by. Networks are more than 1000 times faster. Demand still exceeds capacity on occasion. With the benefit of progress in theory and practice, I would characterize the problem today as primarily an engineering problem, but there are still aspects that are far from being completely understood.
What disturbs me is that greedy special interests of all flavors are converting this into a political problem. As one who has watched decades of technological progress and decades of political decline, this doesn't strike me as a favorable development.
George Ou explaining that Mac OS X users have their computers frequently compromised but just don't know it:
http://talkback.zdnet.com/5208-10533-0.html?forumID=1&threadID=18366&mes...
This is not a rational person. For him to be used as a reliable source is sad.
James
[...] the P2P blog picked up via Ed Felten’s Freedom to Tinker blog, the definition of a BitTorrent as presented by Comcast is in fact one that was used in a speech by [...]
Actually, the P2P systems, working at their best, use less bandwidth than a central delivery system.
Consider a 1gb linux distro hosted at ubuntu.com and two neighbours on the same ISP who both want it. They could download it the old way, by connecting to ubuntu.com. In this case it sends out 2gb, 2gb comes into the ISP and 1gb goes down each pipe to each user.
Now consider a bittorent download. In this case 1gb goes out of ubuntu.com (very good for them) and 1gb comes into the ISP (good for them.) As before, 1gb goes down each pipe to each user.
The other difference is that 500mb goes up each pipe from each user, and the 1gb that didn't come in the main pipe to the ISP still does transit the ISP's internal backbone. The 500mb that came up from each user is upstream bandwidth that otherwise largely sits unused.
Well, almost. On a DSL ISP this is pretty true. Under Docsis, the upstream is badly managed so they don't handle it as well as DSL. On a wireless ISP, the upstream and downstream come from the same pool, so it does have an effect.
It gets worse -- the quote again, Alex ... "P2P applications are designed to consume as much bandwidth as is available" ... and now the evil truth. Replace the P2P with any internet technology, and that statement remains true. That's right -- gaming applications, instant messaging, web applications, video on demand services, streaming radio -- they're all designed to use as much bandwidth as is available. There is nothing special that P2P applications do -- they all open sockets using their various TCP/IP stacks. Give any one of those applications 1 KB/s of additional pipe, and it will consume it. --funchords
[...] the P2P blog picked up via Ed Felten’s Freedom to Tinker blog, the definition of a BitTorrent as presented by Comcast is in fact one that was used in a speech by [...]
“P2P applications are designed to consume as much bandwidth as is availableâ€
I wonder what the "bandwidth throttle" in Bittorrent, Azureus, etc. does.
I know that downloads, streaming and P2P take a big bite out of a provider's bandwidth. I support an ISP that throttles excess traffic the proper way, by dropping packets. Sending RSTs stinks, as the lotus notes users have found out. Comcast severely messed things up and their denial didn't help the issue.
"Never ascribe to malice that which is adequately explained by incompetence." I suggest adding "Comcast competence" to the English language.
Robb Topolski wrote:
"It gets worse — the quote again, Alex … “P2P applications are designed to consume as much bandwidth as is available†… and now the evil truth. Replace the P2P with any internet technology, and that statement remains true. That’s right — gaming applications, instant messaging, web applications, video on demand services, streaming radio — they’re all designed to use as much bandwidth as is available. There is nothing special that P2P applications do — they all open sockets using their various TCP/IP stacks. Give any one of those applications 1 KB/s of additional pipe, and it will consume it."
I don't think that's at all accurate. It would be more accurate to say that programs use as much bandwidth as is necessary. Excluding BitTorrent for the moment, all of the applications you cite - gaming, IM, video and audio streaming - all have limited bandwidth needs. Playing Counter-Strike requires a finite amount of bandwidth. If you provide twice the necessary amount, it will not magically expand to use all that capacity. This is even more obvious with IM, an extremely low-bandwidth application, comparatively. Streaming needs the amount of bandwidth to handle the stream - it will not expand to use more unless the stream is higher quality.
BitTorrent is bandwidth-intensive. If you provide it more bandwidth, AND there are more peers available, AND it is allowed to initiate further connections, AND it is not bandwidth throttled in the actual client, then it will use more bandwidth. But even here, it will not use an infinite amount, or even indefinitely expand. Obviously, it will only receive about as much as the file size being downloaded. As for uploading, it will continue to upload until a ratio is met, or until all connected "leechers" have the file as well, or until you tell it to stop uploading.
Furthermore, BitTorrent's bandwidth usage is limited by the bandwidth of all the connected peers - even if you have a 10 Mbps connection, unless there are hundreds or thousands of peers, it is unlikely you will be able to saturate it, simply because there are not enough fast peers to connect to.
"There is nothing special that P2P applications do — they all open sockets using their various TCP/IP stacks."
That doesn't make any sense. I'm no computer scientist, but I'm under the impression that there is typically one IP stack implemented per operating system, and that various programs use various protocols in the protocol suite to do what they need.
Perhaps Comcast's argument is disingenuous rather than just false...
If you look at this from a purely technical perspective, their argument is clearly wrong. They say they can't increase bandwidth because the p2p _protocols_ will consume it all. But as discussed above, p2p is inherently more efficient and doesn't work like this.
But if you look at this from a political perspective, it probably is fair to say that if they increase bandwidth, p2p _users_ will consume more bandwidth. It's not the protocol, but the users.
This is inherently a political problem. Comcast are hiding behind a false technical argument, but their actual argument is political - they're just not prepared to admit it.
I know this is a technical blog, but there's no denying this is one area where technology and politics collide.
Anyone who has worked at an ISP knows that torrents and most P2P software does indeed use as much bandwidth as they can. I don't know how anyone can claim otherwise. When you are seeding a file and there are enough leechers, you will max out your upload and continue to do so until every leecher is running at maximum download. With the ratio of seeds to leechers 100:1 or worse, its a joke to say that p2p does not eat up any bandwidth you give it.
Having said that comcasts still doesn't have a defense. So what if p2p eats up bandwidth. If you are going to sell internet with unlimited download tough. In Australia we have to pay for how many gigs we download, so we never have problems like this, people pay for what they use, which is fair.
"Anyone who has worked at an ISP knows that torrents and most P2P software does indeed use as much bandwidth as they can."
There is a difference between a program using as much bandwidth as it _can_ versus using as much bandwidth as is _available_, which is what the original claim was ("P2P applications are designed to consume as much bandwidth as is available").
The latter claim really doesn't make any sense when you think about it. Reductio ad absurdum: it implies that if you gave BitTorrent infinite bandwidth, it would use it all. In order for that to be true, there would have to be infinite demand for whatever is being distributed, which is not true.
Is this the same congressional Bono who wants perpetual copyright?
I'll jump in on the side that P2P does use all the bandwidth available to it. I'll admit that I've not tested the limiting case of infinite bandwidth, I doubt anyone else has or will in the near future either. In the more commonplace situation, upgrading a finite link with a larger finite link results in:
* more peers discovering this node is fast
* the users discovering that files move through faster, so they request more files
* requests for larger files, since human patience is the constant variable
... and now the evil truth. Replace the P2P with any internet technology, and that statement remains true. That’s right — gaming applications, instant messaging, web applications, video on demand services, streaming radio — they’re all designed to use as much bandwidth as is available.
All told, yes I'd agree that as bandwidth becomes more widely available, people find stuff to fill it up faster. With things like games and web, they are limited by game designers and web designers who usually have some approximate knowledge of efficiency, plus they are aiming for a wide market and don't want to make their application too much of a hog. With streaming radio, you still only can listen to one channel at a time.
P2P is worse because people just search out big lists of "want to have" files and fill up their queue and walk away leaving it chugging.
At the end of the day, if you sell someone a fat pipe and you advertise it as "unlimited" then you would expect people to use it as advertised. Suppose you offer a fast car and you keep paying for gas no matter how far they drive... suddenly they long tours across the country where before they only drove to work and back. Kind of obvious what is going to happen.
I've wondered about this forgery claim. Isn't it the case that the only way to cleanly interrupt a TCP stream is to send a packet claiming to be from the other side? AFAIK there is nothing in the protocol that lets a packet say, this is from someone other than a given peer, but we are interrupting the connection to that peer, so don't bother trying to send any more packets there. Rather, the IP address and port number serve to the TCP client software as identifiers for a particular logical channel. The only way to send channel-specific information to the TCP software is by putting the IP address and port number into the "from" fields of the packet, isn't that the case? So what else can someone do who interrupts the channel than engage in this so-called "forgery"?
it's going to be even more depressing when the FCC accepts this crappy argument to rule "no harm, no foul"
There's a glaring flaw in Comcast's argument that no one seems to have noticed so far: the amount of bandwidth that's "available" to an individual P2P program is *not* what people have asked Comcast to increase.
Yes, it may be true that if you increase a customer's upstream cap from 768 kbps to 1.5 Mbps, he'll just start using twice as much upstream bandwidth. That's what the previous commenters are arguing about, but it's irrelevant.
Comcast sells a service with a certain amount of upstream bandwidth -- let's say 1 Mbps -- and they count on the fact that most people won't want to use the whole 1 Mbps that they're paying for, which means they can put 20 customers on a shared line that has less than 20 Mbps of capacity. But now as P2P becomes more popular, more people want to use their whole 1 Mbps, and the capacity of that shared line has to increase.
Mary Bono Mack responds as if people were demanding that Comcast increase the 1 Mbps figure, which is flat-out wrong. The problem isn't that individual caps are too low, it's that Comcast doesn't have enough real bandwidth to provide the advertised 1 Mbps to everyone who wants it. If they build up their network *without* increasing caps, that problem will be solved.
@Hal: "So what else can someone do who interrupts the channel than engage in this so-called 'forgery'?"
They can drop packets, which in fact is what happens naturally when a connection is overloaded (and Comcast claims overloaded connections are what inspired this mess in the first place). TCP is designed to back off when the packets start dropping.
We've already gone through this with the electrical power companies. For years, they argued against co-generation and alternative power sources because system stability required a central power generating core and a power using periphery. The power companies lost, and the entire business has been restructured with generation and transmission entities being separate and distinct. The internet has implicitly done this restructuring for the cable companies, but the corporations still have their active fantasy lives.
It comes down to a consumer issue. What is the definition of a 1Mb/s download, 0.5Mb/s upload line? Any naive interpretation recognizes that I can download 86,400,000,000 bits per day, assuming there is something out there that can pump me those bits. If I have a 150amp 110V power system hookup, I can run it flat out, and it is the power companies job to find me the power I am demanding. When the aggregate load gets too high, the power company may brown me out, cutting the power I can draw. If I am a big power user, like an aluminum smelter, I might have a contract to cut my demand when asked, in exchange for a better overall rate.
If Comcast and the other cable companies cannot provide the bit rates they advertise for technical reasons, they need to refine their terms of service and explain exactly what they are delivering. Forging packets to drop connections is the same as randomly imposing black outs. A lot of work was done on computer utility metrics back in the 1960s and 1970s. My satellite provider states explicit maximums and explains its throttling policy quite clearly. It isn't great, but they only have so many satellites and I have to share. Maybe someone should drag out some of that work, apply for a few patents, now that the ideas have become novel and non-obvious, and we can develop a forward looking model for ISPs.
Isn’t it the case that the only way to cleanly interrupt a TCP stream is to send a packet claiming to be from the other side?
Hal,
No, see RFC 792 "Internet Control Message Protocol" (STD 5: ICMP). And see RFC 1122 "Requirements for Internet Hosts -- Communication Layers" (STD 3).
Note, though, that despite the requirement that some ICMP errors MUST be passed up to the transport layer, actual deployed code may work differently. Some stacks may silently discard ICMP for a variety of reasons, both semi-justifiable and clueless. Middleboxes may also have an effect.
Fwiw, I'd call everyone's attention to the well-known case of ICMP Source Quench.
It seems to me the solutions we see being proposed basically break down into four categories:
1. Implement the network infrastructure necessary to provide the level of service that is being offered: i.e., upgrade so that everyone, or at least more people, can use their connection fully.
2. Reduce the level of service being offered to a point where the current infrastructure can handle it: i.e., stop offering 16 Mbps down / 1 Mbps up connections (Comcast actually advertises this) if you can't deliver it.
3. Redefine the pricing model: e.g., switch to a $x / GB model, or offer tiered services (web/e-mail/IM only is $x / mo.; "full" internet access is $3x / mo.), etc.
4. Use questionable methods, such as these RST packets or completely blocking particular protocols, to avoid having to implement any of the above solutions.
And it would seem that Comcast has gone this latter route.
If I have a 150amp 110V power system hookup, I can run it flat out, and it is the power companies job to find me the power I am demanding. When the aggregate load gets too high, the power company may brown me out, cutting the power I can draw.
If you are running a data center, they can't brown you out. The computer supplies will pull constant power until the input voltage dips and the UPS + backup generator kicks in. In effect, the power grid can either supply the full demand of the data center or cut it off an supply nothing (which is why any decent size data center does need a backup generator). When the data center drops off the grid, everyone else on the grid will see a massive surge as that power redistributes itself.
Every power company I've ever heard of charges for usage. It would be unthinkable to hand out a fixed-size power connection and tell the consumer to "just go nuts on this".
@ Gabriel J. Michael
http://www.freedom-to-tinker.com/?p=1218#comment-379421
This has already been run through once already and I believe my proposed solution is implementable on their existing platform (or would require only minor changes), it doesn't require a change of business model, it doesn't involve packet forging and within that business model it is about as "fair" as you can get.
A COUPLE THOUGHTS:
(1) P2P like bittorrent only uses bandwidth equal to whatever is being transferred. Thus, if everyone was sharing Textfiles (example books), P2P would not even be a blip on the radar, because textfiles are teeny-tiny things (just a few kilobytes per book).
(2) Therefore if the pipes are getting clogged, it is not the fault of the protocol, but the CONTENT being transferred. Most P2P content is video and said video takes-up huge amounts of space (1 gig for a SD movie; 5-10 gigs for a HD movie).
(3) Which means even if P2P was outlawed, comcast and other providers would still have a problem with other video-related programs such as:
- AppleTV (downloadable video)
- Itunes (ditto)
- NBC.com, FOX.com, CW.com and other streaming tv sites
- youtube.com, googlevideo.com, et cetera
- and on and on
Banning P2P protocols is NOT going to solve the problem.
There are other sources that also provide video,
and eat a lot of bandwidth, thus the problem still exists.
Comcast needs to deal with the ACTUAL problem (insufficient pipe to transfer on-demand videos). Comcast needs to deal with the congestion, else risk their users not being able to access Itunes video, NBC.com video, youtube.com video, and on and on.
(4) Forgery
- I have an envelope here stamped with Federal Aviation Administration. If I sent-out that envelope, pretending to be the government's FAA representative, I'd find myself in a lot of trouble.
It's forgery to pretend to be somebody else.
Which is what Comcast is doing when they send-out packets pretending to be a remote person that I'm trying to talk to. They are forging somebody's else's address/letterhead (just as if they used an FAA-stamped envelope), and pretending to be somebody else.
It's identity theft.
"Die moderne Frau
kocht ohne Sau."
I disagree with Seth and Jacob ;-)
[...] the P2P journal picked up via Ed Felten’s Freedom to Tinker blog, the definition of a BitTorrent as presented by Comcast is in fact digit that was utilised in a [...]
Ed, it's rather silly to claim Comcast's entire defense is predicated on the few things you paraphrase and attribute to me which weren't actual quotations. Here is a proper framing of my position on Comcast.
http://blogs.zdnet.com/Ou/?p=1001
This comment you made below is flat out deceptive.
"P2P protocols don’t aim to use more bandwidth rather than less."
P2P uses less bandwidth for the content distributor and offloads that bandwidth by multiples to others. It uses less bandwidth for Vuze such that Vuze no longer needs to spend the millions of dollars that Youtube spends on server bandwidth. Vuze would rather offload that to the broadband companies and let them pay the entire freight of not only the downstream, but it uses other people's infrastructure to deliver content to the rest of the world.
P2P is so inefficient in terms of excessive network load that the P4P working group is working to make it more network-aware and avoid unnecessary and overly long paths. This fantasy that P2P is "efficient" is getting out of hand. Sure it's "efficient" for the freeloader like Vuze who wants to avoid paying distribution costs, but it's not efficient for the entities Vuze freeloads from.
Brad Templeton's description of BitTorrent is misleading and inaccurate. It places more load on an ISP's internal network than conventional downloading methods because it uses lots of short packets and generates a great deal of protocol chatter. And many BT sessions are abandoned midway because the seeder is shut down. P4P is a much better approach.
Interestingly, Mr. Templeton is the chairman of the Electronic Frontier Foundation, the non-profit that's been among the harshest critics of Comcast's efforts to control BitTorrent's bandwidth appetite, and he's also a board member of the for-profit BitTorrent, Inc.
I think that's a problem.
Very good point Richard,
Ed Felton also makes the same mistake as many others like Brad Templeton when he describes how "efficient" BitTorrent is. They look "efficiency" purely from the selfish standpoint of the content distributor and not the havoc it wreaks on the Internet in terms of excessive and unnecessary load. P2P is often so stupid that it will download files all the way from China or Europe rather than something in the same region. If it didn’t do that, we would not have a need for the P4P working group that wants to implement network-aware P2P.
It's financially "efficient" to BitTorrent and Vuze in the sense that they can offload their server bandwidth costs on to the broadband companies and freeload off their infrastructure to serve the Internet. When Comcast enforces their terms of service that prohibit servers or 24x7 upstream saturation and shuts down this freeloading, Vuze will simply complain to the FCC and DEMAND that Comcast should spend their own money to upgrade the network to something like FiOS. Now of course it’s easy make demands like that when you’re not flipping the bill.
What I find ironic is that Templeton's EFF says metered Internet service is better when such a system would completely price BitTorrent or P2P usage out of existence.
Ingo: who is Jacob? I see a Seth but no Jacobs among the commenters to this one.
George and Richard: nobody is asking ISPs to "foot the bill" for distributing something. The ISPs' customers will foot the bill, in one way or another, since ISPs pass their costs on to their customers like most businesses. My ISP provides flat rate net access up to $60GB a month; any usage above that in any given month costs extra that month, $1.50 extra for 60-61GB, $3.00 for 61-62, and so forth. A higher flat rate is another possibility, and you yourself mentioned metering.
Ed writes:
"But even leaving aside the merits of the argument, what’s most remarkable here is that Comcast’s technical description of BitTorrent cites as evidence not a textbook, nor a standards document, nor a paper from the research literature, nor a paper by the designer of BitTorrent, nor a document from the BitTorrent company, nor the statement of any expert, but a speech by a member of Congress."
Since the FCC is a creature of Congress, I'm not surprised to see a Member cited as an expert in Comcast's pleading. They know full well this is a political contest, not a technical one.
To Spudz:
"George and Richard: nobody is asking ISPs to “foot the bill†for distributing something. The ISPs’ customers will foot the bill, in one way or another, since ISPs pass their costs on to their customers like most businesses."
The customer has paid for RESIDENTIAL broadband service. Residential broadband service is built upon the fact that not everyone uses their bandwidth at the same time. A Cable DOCSIS 1.1 network has a total of 10 mbps upstream per second shared among 450 users and it is actually closer described as a wireless ISP with very limited shared resources than a DSL let alone a FiOS network. Just 26 users saturating their upstream at 384 kbps will hit the theoretical peak capacity of an entire neighborhood and the network fundamentally does not support 24x7 BitTorrent seeders or other servers.
That's why the terms of service say you can't host servers and you can't saturate the pipe 24x7. BitTorrent seeders and servers operate 24x7 at or near peak capacity and it fundamentally breaks the network. This is not as if Comcast created some rules stating that VoIP can’t be used. These terms of service were NOT implemented for anti-competitive reasons; they were implemented to keep the network from melting down which clearly falls under the category of REASONABLE network management. They were implemented to prevent 5% of the users from saturating the network and preventing the 95% of users from accessing the broadband service they paid for.
If you want a pipe that you can host servers and saturate the pipe, buy a commercial-grade connection such as a T1 for $400/month where you can saturate up and down at 1.554 mbps.
(I'm assuming that Comcast are selling connectivity based on link speed
rather than transfer volume. I don't live in the US)
The alleged bandwidth supply costs problem that Comcast is suffering
from is going to be a natural consequence of a fixed supply of bandwidth
and having no caps on the demand of the end user.
Instead of pissing off their users they really should see it as supply vs
demand problem rather than a p2p problem.
The solution is then going to be:
a) Invest more resources to keep up with bandwidth demand:
i) Be less profitable;
ii) Charge the big bandwidth users more money;
iii) Be more cost efficient with bandwidth supply.
b) Be honest in their limited bandwidth supply capacity and set about
to limit the demand through either:
i) Data transfer quotas
ii) Traffic shaping (based on quota, time of day, etc)
What am I missing here ?
Emmanuel,
Comcast does not sell "unlimited" data service for their consumer residential product. Their terms of service says no servers and no 24x7 pipe saturation. Why are you saying Comcast is not being honest when you don't even know the terms of service?
As for metered Internet service, that would certainly solve the supply problem but it would effectively price BitTorrent usage out of existence since very few users would use it and they'd use it sparingly. Metered pricing is hostile towards consumers and innovation and I am happy that the US market has rejected it. Even the cell phone market is moving to an “all you can eat within reason†model. That all you can eat model doesn’t work if users violate the terms of service. You can’t go to an all you can eat buffet and feed 30 of your friends out the side door just like Comcast can’t afford to let residential users turn offer commercial-grade bandwidth to the rest of the Internet.
As for "Traffic shaping (based on quota, time of day, etc)", they are using the only mechanism available to them on a shared medium network. Normal Layer 3 network traffic shaping mechanisms do not work as effectively since the damage is already done on the first mile before you even get to the router where traffic shaping is done. Comcast is using TCP RSTs to shut down excessive 24x7 upstream connections that violate the terms of service and they're only using the TCP RSTs when the network is slammed.
@Richard Bennett:
It places more load on an ISP’s internal network than conventional downloading methods because it uses lots of short packets and generates a great deal of protocol chatter.
You have made this claim several times but you have never provided convincing supporting evidence: you only speculate that this is the case. In particular, the two papers that you have cited in the past by Jim Martin do not support this specific claim. One paper shows that Docsis is vulnerable to a syn flood attack by a malicious attacker that chooses packet timing, a poor model for a Bittorrent client. The other merely shows that highly utilizing a link with Bittorrent will slow down other users: This is an unsurprising claim and is also true of other protocols. There is no comparison to heavy HTTP traffic to demonstrate that it has a comparatively more burdensome effect on the network.
Again, you have given no evidence of the claim that Bittorrent, as a protocol, has some kind of extraordinary adverse effect on networks compared to other protocols.
Finally, I fail to understand why you spend so much time picking on Bittorrent. Don't you support traffic management of any heavy network usage regardless of protocol? Isn't it enough to say it uses a lot of bandwidth and therefore needs no more justification for de-prioritization?
@jon: you say: "Isn’t it enough to say it uses a lot of bandwidth and therefore needs no more justification for de-prioritization?"
No, it's not. Compare BitTorrent traffic to FTP traffic. A client inside the Comcast network doing an FTP download only creates traffic on the DOCSIS downstream side, where there is plenty of capacity. The same client doing the same transfer using BitTorrent creates traffic on both the downstream and the upstream side, and the upstream side is severely constrained by the cable topology. And in addition to the raw data, the BT client also has to field (typically) thousands of connection requests per minute from peers testing his speed and get peer list updates from the tracker over UDP.
FTP uploads over symmetrical links, BitTorrent uploads over asymmetrical links. That alone is a huge difference, so it annoys me to read people like Templeton claiming "efficiency" for BitTorrent.
That's simply a lie.
It was implied that the direction of the traffic was the same as whatever the ISP was concerned with.
So: "Isn’t it enough to say it uses a lot of bandwidth in a given direction and therefore needs no more justification for de-prioritization?" Why are you trying to carve out a special case for Bittorrent when, at the end of the day, you want to justify managing any heavy traffic in a given direction, regardless of protocol? If all Bittorrent users became FTP uploaders (with the same upstream traffic patterns) next week would you go and try to make a special case for why FTP should be throttled? Or when it changes the week after to protocol X? Why not just argue the root issue?
Regardless, I'm still interested in some empirical evidence of the claimed pathological behaviour of Bittorrent.
There are many conflicting issues at work here. I will comment on a couple that seem to come up over and over again, year after year over the past decade.
First of all, we shouldn't feel any more sorry for the ISP or the Carrier contending with bandwidth demand by customers than we feel sorry for the phone company contending with usage by teenagers who use the landline phones 24 hours a day talking to each other or swamping the local radio station call in number. As anyone who works for a phone company knows, teenagers will tie up all the phone lines they can come into contact with.
Nevertheless, it is the phone company's job to increase capacity as required. It's the law. They of course are common carrier, and so should be the Internet provider (in my opinion both the ISP/Carrier as well as the backbone providers, but in any case at a minimum the backbone providers). That would subject them to the kind of performance requirements that have made landline phone service reliable for more than 100 years.
The distinction between phone service and "information services" is now without question a distinction without a difference. Proof: you can have voice over ip over a voice line and it is indistinguishable from voice over ip over a data line. You can have voice over data over voice or you can have data over voice over data. (The same is true for wirless and cable.) The presumed technical difference that was the original and that remains the most highly vaunted justification for exempting ip traffic from common carrier status is now clearly bogus and it needs to be relegated to the heap of history. Ergo, net traffic providers need to be common carrier, either de jure or de facto. If not for rates then at least for carriage and capacity. You can regulate the technology without regulating rates. See FCC Part 15.
As to the technical issues of traffic management, routing, shaping, allocation, balancing, throttling, call it what you will, to the extent that the carriers feel the need to create new methods and protocols to accomplish this, then when these methods are being applied to Internet traffic, and as long as the carriers are not common carrier, then they should participate in an RFC standards process and gain acceptance for their methods before implementing. Congress-FCC-Industry triangulation is not a replacement for the long established Internet procedures. Carriers need to participate and agree to comply.
At the same time, any regulatory framework must enable the carriers to seek new ways to achieve higher capacity and higher efficiency while also increasing consumer choice. As a consumer maybe I want an account with throttled, bit-limited, shaped, and balanced traffic and maybe I want an account without it. I should have a choice, so should you, and carriers need to be prepared to provide it and not just unilaterally shove things onto the market.
With a variety of service types and levels available to customers, the issues raised by Comcast's actions, its defense and its critics all become moot.
To Bob Schmidt:
" As a consumer maybe I want an account with throttled, bit-limited, shaped, and balanced traffic and maybe I want an account without it. I should have a choice, so should you, and carriers need to be prepared to provide it and not just unilaterally shove things onto the market. "
You DO have that choice. It's called a commercial grade account from Comcast or some other provider where the terms of service permit you to have servers and saturate the uplink 24x7. What you don't have the right to do is buy the cheaper consumer-grade service meant for data consumption not data distribution to the whole Internet, and then violate the terms and harm other customers. It isn't about "feeling sorry" for Comcast, it's about protecting the rights of 95% of Comcast's customers to have steady surfing, VoIPing, and gaming.
Ultimately the issue is for all providers, who controls the data stream. Do the providers have the right to limit useage to certain types of service and deny it to others or do they not.
I doubt that anyone would argue against the providers being able to deny their datastreams to spammers or malware so there are grounds for limiting use.
Everyday some new use of the internet is created. Telephony, pirating video from home tvs to office, video and most recently renting movies over the internet. Renting movies over the internet should really bog down the net when a million or so people decide to rent a video at the same instant.
Perhaps the real decision will be to create separate networks for video, phone, data, and so on rather than trying to squeeze it all through one pipe?
Or maybe a tiered system where data streams become more expensive with more use. Why should anyone pay the same for basic email and browsing as the gamer who burns up the bits 16 hours a day or the guy piping his home TV to his office over the internet. One may use a gigabyte a month and the other 20 Gigs a day, or whatever the number is.
Many people here are missing the point (e.g. with "residential" vs. "commercial" service based arguments).
Others have hit the nail on the head: it's a supply and demand problem.
The solution to supply and demand problems has been known for centuries: a thriving, competitive marketplace and the Invisible Hand.
It's high time we had such a marketplace for broadband in North America.
Fix the lack of broadband competition and watch all the other problems disappear in the space of a few months or years.
One sign that the current lack of competition is causing market failure is the observable lack of a tier in between $20/month consumer broadband with major limits on upstream speed, total transfer per month, and server running and $400/month full-blown commercial-grade service. Where's the $30/month or $50/month or whatever "consumer premium" that allows everything the commercial-grade does but has lower up and down caps and lacks the things like business-grade support, serious hosting stuff, a static IP, and suchlike? Why are there packages for surfers and e-mailers, packages for businesses running heavily-loaded Web servers with PHP and CGI doing complex e-commerce and the like, but no packages for the heavy home user that games, YouTubes, VOIPs, file-shares, and does other high-usage things with their net connection? The answer can only be a lack of competition, because there isn't a lack of demand, and there really isn't even a lack of supply. (Cable network topologies have problems supplying uplink, because the network was originally designed for broadcast, but this doesn't apply to DSL, which uses a phone network designed for two-way traffic from the get-go. This may be why we see DSL providers pulling fewer traffic-shaping shenanigans and having more generous upload caps. Room here for some company to lease capacity here and there and offer a hybrid service that uses cable to supply a fat down-pipe and the phone line to supply individual up-pipes, hm? If only the market were more competitive.)
@spudz:
This wordpress installation reacted with "too much load" (or somethin), and put my comment here instead of http://www.freedom-to-tinker.com/?p=1257 :-(
I related to the research page http://citp.princeton.edu/memory/media/ last picture, wallpaper above the microwave ;)
Internet bandwidth is just as important to the US economy as oil, go ahead and start placing limitations on "certain" things....I'd like to kick comcast in the balls #1 for its crappy service #2 for its never even close bandwidth #3 for being a bunch of pompous bigots that want to eliminate or hinder the sharing of free information. Nazis suck.
To all you people citing games and/or VoIP as a high-bandwidth culprit, you people do not know what you are talking about. Online games (I play them and have measured them) use about 40 kbps up/down and even a resource-challenged network like Comcast DOCSIS 1.1 network can handle that kind of load 24x7 for nearly every user on the network. VoIP uses about 80 kbps up/down and it's mostly sporadic usage which means you add about 4 kbps up/down to the base load on the network on average. It takes less than 26 24x7 BitTorrent seeders who saturate the upstream at 384 kbps to kill a DOCSIS 1.1 network.
Now Vuze says too bad, build a new network like Verizon FiOS to support our freeloading of server bandwidth so we don’t have to pay any money unlike YouTube paying millions of dollars for server bandwidth. If we do this, are we not saying that any network that can’t support 24x7 upload deserves to be fined/regulated out of existence? What do we do about Wireless ISPs who have very scarce spectrum? You all realize that Cable plays under the same limitations of Wireless ISPs right? Cable providers will be forced to carry analog TV well beyond February 2009 when TV broadcasters no longer have to carry analog TV. Do we tell the Wireless ISPs with scare resources that they can’t prevent BitTorrent seeders from killing their network? Do you all realize that without Cable broadband, there’d be less competition/reason for Verizon to put out FiOS or less reason for them to do reasonable prices for FiOS?
I hear a lot of people being ignorant in this forum saying that we need to go to some sort of tiered or metered Internet service. You all do realize that this effectively prices BitTorrent usage out of the mainstream because it becomes cost prohibitive to operate right? Maybe you all need to be careful what you wish for because you might just get it.