Mike Godwin notes a serious error in the recent NYT future-of-TV article:
A more important problem with the article is that it gives a false impression of the normal user experience of BitTorrent:
Created by Bram Cohen, a 29-year-old programmer in Bellevue, Wash., BitTorrent breaks files hundreds or thousands of times bigger than a song file into small pieces to speed its path to the Internet and then to your computer. On the kind of peer-to-peer site that gave the music industry night sweats, an episode of “Desperate Housewives” that some fan copied and posted on the Internet can take hours to download; on BitTorrent, it arrives in minutes.
That hasn’t been my experience of BitTorrent, and I doubt many other ordinary users routinely experience the downloading of TV programs in “minutes.” On the off chance that BitTorrent speeds had suddenly improved since I had last used the application, I conducted an experiment – I downloaded the latest episode of Showtime’s program “Huff,” which stars Hank Azaria, within 24 hours of its having aired. (Downloading a program shortly after it has aired, when interest in the episode is at its peak, is the way to maximize download speed on BitTorrent.) The result? Even with the premium broadband service I have at my office, downloading Episode 13 of “Huff” – the final episode of the season – took six hours, with download speeds rarely exceeding 30KB/sec.
The NYT article seems to make a common error in thinking about BitTorrent. BitTorrent’s main effect is not to make downloads faster as the number of users increases, but to keep downloads from getting much slower. A simple model can explain why this is so. (As with all good models, this one gets the important things right but ignores some details.)
Let’s assume that a file is being offered by a server, and the server’s net connection is fast enough to transmit the entire file in S seconds. We’ll assume that N users want to download the file simultaneously, and that each user has a net connection that would take U seconds to transmit the entire file . (Generally, the user is willing to pay less for Net service than the server, so S < U.)
In an old-fashioned (pre-BitTorrent) system, all N copies of the file must go through the server's connection. That takes SN seconds. One copy goes through each user's connection, which takes U seconds. The two steps, taking times SN and U, can happen simultaneously, so the time to do both is equal to the larger of SN and U.
T_old(N) = max(SN, U)
If the file is popular (i.e., N is large), the SN term will dominate and the download time will be proportional to N. For popular files, adding users makes downloads slower.
In BitTorrent, the file only needs to go through the server’s connection once, which takes N seconds. On average, each block of the file must traverse a typical user’s link twice, since each block must be downloaded once, and BitTorrent expects each user to upload as many blocks as it downloads. So with BitTorrent, the total time to serve the N users is max(S, 2U). Recalling that S < U, we can see that
T_bt(N) = 2U
Now we can see the big win offered by BitTorrent: the download time is independent of the number of users (N), and of the speed of the server’s connection (S). Adding more users doesn’t make the download faster, but it doesn’t make it slower either. (It’s also worth noting that if N, the number of users, is small, BitTorrent is worse than old-fashioned systems, by a factor of two.)
With BitTorrent, the bottleneck is the end user’s net connection, only half of which can be used for BitTorrent downloads. (The other half must be used for uploads.) Most users’ connections, even the broadband ones, will take an awfully long time to download high-quality video content, BitTorrent or not.
Mike didn’t see much improvement, because he was a second class participant. There are actually two classes of users on bittorrent: those that can accept inbound connections, and those that don’t. Those that setup the NAT box (or firewall) to allow others to connect to them see full bandwidth transfers in ten to fifteen minutes of connecting (typically). Those that don’t may never see that happen.
Now with more stable material, such as a BSD distribution, the ratio of seeders to feeders is much higher (since no one is worried about receiving a cease and desist letter), so even a second class participant can realize fairly good transfer rates.
Small nit: “In BitTorrent, the file only needs to go through the server’s connection once, which takes N seconds” — shouldn’t that read “which takes S seconds”?
In my experience the ratelimiting factor isn’t either upload or download speed, but the ability of my computer to process the BitTorrent, the media player, the screen visualizer, Seti@Home, &c…
–Bob.
I see that the NYT is up to their usual standards of accuracy and clear writing. Ah well.
Your BitTorrent experience will vary, of course, due to lots of complicated factors. There are a couple of simple things you can do to max out your download speed, though.
The most important thing is to make sure that you can accept incoming connections. If you’re on the real Internet, this happens automatically. If, like many, you’re on the Interweb instead, you need to make sure the BitTorrent ports are opened on your firewall or NAT server. BitTorrent connections don’t work between two firewalled systems, only when at least one is on the real Internet.
Second, many Internet connections, especially DSL, have poor traffic shaping, which often leads to very poor download bandwidth when the upload is saturated (as often is the case with BitTorrent, but is rare when browsing the Interweb. Many routers and firewalls have traffic shaping features, but these are not trivial to configure (I haven’t yet on my network, and I consider myself moderately studly).
You can, however, implement poor man’s traffic shaping just by setting your BT upload cap to just under the flat-out maximum your connection will allow. If doing this dramatically improves your downloads, then you know that traffic shaping was the problem.
With these two tricks, I routinely get 150kB/s downloads, maxing out my wimpy DSL. I’m sure that if I had a good Internet connection (in Korea and Japan, 30Mb/s is standard), I’d see even better download throughput.
Ah, the ever-appropriate Google ad: “BitTorrent Accelerator — Speed up BitTorrent downloads by up to 100% – Download BT Engine free!”
Let me guess — it disables the quid-pro-quo thingie and makes your BT client a pure leech? Won’t even do anything on typical home broadband network connections, which separately throttle upload and download speeds anyway. (Mine is 3 megabits up and down, so I’d get 3 megabits down no matter how much I was uploading, rather than up to six with no uploading. Yes, that is like having a pair of T1 lines into my box. :))
I think this diagram illustrates this very clearly.
This article also ignores the release scene which is composed of more than ‘some fan’ of shows, in fact rippers are often automatic processes done remotely on a computer in a dorm room or university server room by a few people in the us and europe who don’t even watch all the shows they are ripping.
Bittorrent’s huge strength is that it leverages the release scene, the people who run the trackers know about the release scene and leverage the nuke process to provide high quality releases.
In terms of peer-peer speeds of BitTorrent clients, the resulting download bandwidth depends
a) on the aggregate bandwidth and connection availability of other peers, and
b) on the upstream bandwidth of the computer you are downloading to.
The intrinsic design (see the “BitTorrent economy” paper on bittorrent.com) lets the client “choke” other peers, when certain thresholds are exceeded. Metrics are up/download ratio, time and so on.
Simplified version: Other peers are only willing to send you bits, if you feed them other bits, or time passes by.
So, effectively, your upstream bandwidth affects your overall download speed and that’s the main point. I downloaded a Linux install CD image onto a server which is hosted with plenty of (symmetric up-/downstream) bandwidth in a data center with excellent connectivity. With a total of approx. 120 available peers, the download maxed out the 100MBit/s ethernet interface. The consumed upload bandwidth was about half of the download bandwidth.
Regards,
/k
The other important thing that BitTorrent does for the server is greatly ease bandwidth on the server. This allows small time websites to host a large, massively popular file and they only have to upload the entire file one time, after that many people will be nice enough to seed the file for some time after they have it fully downloaded. After that initial one upload the server only has to worry about tracking the file which, in essence, is simply small chucks of text generally consisting of ip addresses so the bandwidth that is required per file is minimal.
Some Gnutella clients *cough*limewire*cough* support “partial file uploading”, meaning that you can be a source for the chunks of a file that you already have long before you have the entire file.
BitTorrent always works in the context of at least one “seeder” which has the whole file and which can act as a conventional server even if no one is uploading. So BT should never be slower than the conventional model.
One problem with BT is that there is no enforcement of the rule that people should upload as much as they download. The protocol does have a “tit for tat” element that encourages people to upload, but there is not necessarily going to be numerical equivalency. Generally, users on high speed connections will be uploading proportionately more than users on slower systems.
If your download finishes before you have uploaded your full quota, you can be polite and let it continue to upload. This doesn’t prevent you from watching the show, but it might interfere with you downloading the next thing you want to watch.
Home users often have their upload bandwidth capped at a fraction of download bandwidth, perhaps by a factor of 1/3. If users run BT until they have a 1:1 up/down ratio, and if this is the bottleneck in their TV viewing, then it could increase the total time to 3U (or 4U for half duplex), if U is the download time.
The reported 30 KB/sec for a popular show on a business premium broadband connection is surprisingly low. I’d suggest checking to see if a firewall was preventing uploads from going out, causing BT’s tit for tat rule to kick in and slow the download speed to a relative trickle.
I think comparing Bittorrent to a client/server environment doesn’t really make sense here. Obviously by stating “an episode of ‘Desperate Housewives” that some fan copied and posted on the Internet’ the Times meant other P2P systems, and not a server.
If you compare Bittorrent to other P2P systems, you have to take into account that it is one of the few protocols that allows instant sharing of data before you have received the full file. Gnutella, Kazaa & Co. wait for the complete file to be transmitted – which makes sense in a big, distributed network with lots of files.
Bittorrent on the other hands forms lots of small, file-based instantaneous networks and shares data right away – which allows much higher download speeds.
I frequently max out my (cable) network connection with BitTorrent downloads. A MPEG 2 version of a TV recording runs about 1 GB/hr which takes less than an hour to download on a 3 Mbs network. MPEG 4 encoding (DivX, Xvid) can result in better compression and even faster download times. So, it really does take minutes, not hours, to download TV content.
The key is to find torrents where the number of intact copies (seeds in the jargon) is >> than the number of clients.
Then there’s the question if any of the TV recordings are worth downloading, but that’s for another day…
Florian,
Good point. For connections that are full-duplex and run at the same speed in both directions, the BitTorrent time should be U rather than 2U. This means that the whole incoming side of the connection can be used by BitTorrent. But still the speed of that connection will be the bottleneck.
Small nit: Typically, DSL is full duplex, so uploading does not substantially slow down your downloads. I believe that cable modems are capped much below the actual wire speed, and I would be a bit surprised if Internet over cable exhibits true half-duplex properties. So uploading should not slow down your downloads substantially.