A few years ago, I lived in Cambridge, Massachusetts. I subscribed to Comcast cable. With my trusty Hauppauge WinTV-PVR-150 I enjoyed the ability to watch TV on my desktop computer — even to record it for later viewing or to occasionally edit and re-upload it to YouTube (with critical commentary and within the bounds of fair use of course, with nary a DMCA notice). Then I moved across the river, to Boston. When I had Comcast cable installed, they also installed a set-top box connected to my television, for tuning all of my channels. I plugged my PVR-150 into another cable connection and got almost no channels at all (including many of the channels I had gotten just across the river in Cambridge). Despite being a geek, it took me awhile to figure out what had happened. As it turns out, the Boston Comcast system technology was entirely independent of the Cambridge Comcast system, and Boston was ahead of the technology curve in adopting digital cable. My tuner card worked only with analog cable signals, so I set out to learn about digital tuners. As it turns out, the solution was not as simple as purchasing a digital equivalent of my PVR-150. Although such products exist, they can tune only un-encrypted digital TV signals… which are increasingly rare. The cable companies have implemented encryption to fight “theft” of channels that subscribers have not paid for. In the process, I lost the ability to view channels I had paid for on a device of my choosing. The cable set-top-box was the only device in my house that could tune to those channels.

The FCC saw this problem on the horizon all the way back in 1996, when it included requirements for compatibility of consumer-provided “navigation devices” in the 1996 Telecommunications Act (Sec. 304, adding Sec. 629 to the Communications Act, subsequently codified as 47 USC 549). The FCC ultimately settled on a solution called CableCard, which specified a universal standard for a small device that would perform the decryption and tuning functions of a full set-top-box, but in a form factor that could be inserted into third-party “host” devices. The problems with CableCard have been many-fold. To begin with, the long process of implementation (both technically and politically) failed to facilitate a competitive market for devices that are an alternative to set-top-boxes provided by the cable companies. Second, the functionality of CableCard was outdated almost as soon as it was designed. For instance, it did not anticipate two-way interactive services, program guides, or video-on-demand. Third, the standard mandated particular hardware, which contributed to the implementation delays and difficulties. This technology was also cable-specific, and even there it was bound to be obsoleted as more of the logic moved into software. For instance, the cable industry has increasingly shifted to new standards for delivering digital video which are not compatible even with CableCard-equipped devices. Finally, and perhaps most damning to the project, is the fact that all devices that wish to be CableCard “host” devices must first pass certification by the cableco-surrogate organization CableLabs. This means that cable companies still hold de facto control over the market.

Much of the concern over certification was related to “theft” of content, and as a result it was exceedingly difficult to become certified unless all components of your product (and the products it shared cable content with) were certified to prevent the user from many actions — including copying, editing, and redistribution. The result is that, in 2010, there are still very few CableCard-compatible PC devices, and all of them require running the locked-down Windows 7 operating system. What’s more, the push for control was extended even further into the home — encompassing the outputs of all CableCard-certified devices via a technology called HDCP. HDCP encrypts the digital video signal coming out of CableCard-certified devices so that only HDCP-certified devices can decrypt it. Ed pointed out in 2006 that HDCP is trivially easy to crack, so although it won’t stop a dedicated pirate (who could also just do this or this), it could nevertheless prevent a legitimate third-party video device market. Your flatscreen television must be HDCP certified, as must your flatscreen computer monitor, as must your third-party tuner or DVR. Hauppauge has not to date shipped a CableCard or HDCP certified device.

The FCC has struggled with how to address these shortcomings. In 2007, it realized that the industry players were not going to update CableCard to support two-way interactions, and issued rules intended to force it to do so. This ushered in the “true2way” standard at the heart of “CableCard 2.0.” It also ushered in yet another certification regime. This time, CableLabs imposed an even greater degree of restriction on the design of third-party devices, insisting that the manufacturers surrender control of major portions of the user interface to their own restrictive “middleware”. This undermines one of the competitive advantages of these devices (better user interface), and mires their developers in the process of revamping their products to meet these arbitrary “compatibility” requirements. Tivo has not to date shipped a CableCard 2.0 certified device.

In the absence of a better digital solution, the only way third-parties have been able to reclaim their ability to exercise independent in-home viewing, recording, and fair use rights has been to rely on the “analog hole” and some truly creative hacks to tune their devices. The “analog hole” refers to the fact that set-top-boxes are required to output analog “component” video in order to maintain compatibility with older devices. In order to perform navigation functions, people will use an “IR Blaster” which is literally taped to the front of the set-top-box and simulates the remote control. This is not workable for most consumers, but the FCC has started to compromise on even the analog hole option by permitting “selectable output control” in some cases, which turns off analog outputs during certain programming.

The next stage of this battle began when several public interest groups filed a petition asking for several fixes to the system, including a new standards-based video gateway. Consumer electronics manufacturers supported this effort, and TiVo sent a letter explaining the various ways in which CableCard through its various instantiations had failed to facilitate a competitive market (reversing decades of a vibrant market for VCRs and other “cable ready” devices). Recently, Google announced Google TV, a platform for integration of internet and cable content in set-top-boxes and televisions. They did not mention if Google TV devices would aim to use CableCard or some other method to get access to the cable content. Ironically, Google may be one of the few big players in this space that can deal with the overhead of the certification process, although the requirements of being certified may also work against the company’s general ethos of content openness.

In any case, the Commission released a Notice of Inquiry (see the docket here) asking several questions about a possible new approach to the CableCard issue, which would hopefully resolve some of the regime’s existing shortcomings. We should see much more activity on the docket as comments come due. At the center of the FCC’s NOI is the notion of an “AllVid” standard that defines a single set of protocols that a video provider’s equipment must support so that other devices can both control tuning/interaction/etc and receive a video stream for display. The “all” in AllVid refers to the fact that this standard is designed to be agnostic to the delivery technology (ie: Cable, Fiber, Satellite, etc.) This far in the proposal I am sold. However, the NOI assumes that this functionality must come from a new type of physical device:

25. AllVid Equipment. The AllVid equipment would be designed to operate specifically with one MVPD and offered through the MVPD’s preferred mechanism, whether leased or sold at retail, manufactured by one company or competitively. We foresee two possible physical configurations for the AllVid equipment. In the first configuration, the AllVid equipment would be a small “set-back” device, capable of communicating with one navigation device or TV set and providing at least two simultaneous video streams to allow for picture-in-picture and to allow subscribers to watch a program on one channel while recording a program on another channel. In the second configuration, the AllVid equipment would act as a whole-home gateway, capable of simultaneously communicating with multiple navigation devices within the home, and providing at least six simultaneous video streams within the home (which would allow picture-in-picture in three different rooms), possibly through a modular system that could accommodate more streams as necessary.

The NOI then goes on to note that the FCC may suggest Ethernet as the universal port, IP as the base protocol, and DTCP-IP (“Digital Transmission Content Protection over IP”) for encryption (it, too, is cryptographically weak). At this point, I start to have issues with the direction things are going. First of all, it is unclear to me why there needs to be a new mandated technical standard that is rooted in a new hardware device. There are already hardware devices that intermediate between the proprietary backend network and the home — set top boxes and (increasingly) centralized media boxes that redistribute content to the rest of the home via simpler adapters. I don’t see the benefit of mandating something different if the functionality can simply be built into the existing devices that every consumer receives. It is unlikely to be any cheaper to invent a new hardware standard to do the same thing via a new protocol standard. It is better, it seems, to let the market innovate when it comes to devices that connect to the backend and simply mandate that any such device (including today’s set-top-boxes) provide a standard hardware interface (such as the USB, firewire, or ethernet already on such boxes) that exposes a yet-to-be-defined software interface for sending and receiving navigation information. Essentially, if you can input queries via the remote or view information via the screen, you should be able to do the same thing using something like a standard and extensible XML query-response protocol. In this case, the FCC’s proposal could be implemented as a software update. This is indeed what seems to be hinted at in paragraph 21 of the CableCard Notice of Proposed Rulemaking that the FCC released on the same day as the AllVid NOI:

21. We also tentatively conclude that we should require cable operators to enable bi-directional communication over these interfaces. We propose that, at a minimum, these interfaces should be able to receive remote-control commands from a connected device. We also propose to require that these outputs deliver video in any industry standard format to ensure that video made available over these interfaces can be received and displayed by devices manufactured by unaffiliated manufacturers. We believe that these proposals will improve the functionality of retail consumer electronics devices significantly. We seek comment on this proposed rule and tentative conclusions. We also seek specific comment on whether cable operators could implement these changes inexpensively with firmware upgrades, and if so, whether January 1, 2011 would be a reasonable effective date for such a rule change. If not, we encourage commenters to propose an effective date for this proposed rule change based on how complex it would be to execute.

Of course, this depends on “smart video devices” being able to receive the digital video signal coming from one of these devices. I’m not sure we need them to output the video itself via these interfaces. Currently, the HDMI port typically outputs a signal that could be plugged straight into a standard DVI port on any device, but for the fact that it has been encrypted with HDCP. The problem is not that the output is non-standard, but that it is encrypted, and the keys for decryption are held by a commercial party that is understandably hostile to competition. In that sense, the Commission’s suggestion in the AllVid NOI that the new adapters use DTCP-IP represents more of the same, but worse. Less is known about the proprietary standard because the specification is closely held by the five corporations that developed it, and their restrictions on licenses for implementers are likely to be even more restrictive than CableCard or HDCP. In any event, it is hard to imagine that under this mindset any device that allows fair use copying, editing, and redistribution will be certified by even a more lenient administrator.

At the end of the day, the market failure for smart video devices is not due to technology. It is due to the policy decision to mandate failed content protection measures in every stage of the home entertainment chain. Given that this policy decision is unlikely to change, my only hope for ever re-crossing the river seems to rest on the eventual death of the entrenched video distribution path at the hand of the Internet. Of course, some of the same interests are present there, and this battle will likely be perpetuated for some time to come.