I wrote Friday about weaknesses in the HDCP handshake protocol that is being used to set up encryption of very high-def TV content that is in transit from devices like next-gen DVD players to television monitors. This was not news to those who follow the area. The ideas in my post came from a 2001 paper by Crosby, Goldberg, Johnson, Song, and Wagner – all I did is abstract away some of the mathematical detail to simplify the explanation.

As far as I can tell, the publication of the Crosby paper, and the spread of the news about HDCP’s flaws, did not trigger any change in Hollywood’s plans for next-gen TV. They’re still relying upon HDCP, in the same way they seemed to be planning five years ago. As far as I can tell, HDCP’s vulnerability hasn’t changed anything – yet.

This is quite interesting, when we consider that the system as designed is almost certain to be completely broken. The security of the design relies entirely on the secrecy of 1600 special numbers (which form a 40-by-40 matrix), whose disclosure to the public would release the knowledge to build a device that could do absolutely everything that HDCP is supposed to prevent. It is well known how to extract these numbers – though it requires some technical effort – but once the numbers are published, HDCP’s security is shot forever. This is virtually certain to happen in the next few years.

There’s a good chance that the HDCP people knew beforehand that this would be the case. If they used a good due diligence process on the HDCP algorithm, they would almost certainly have discovered these weaknesses. But even if they didn’t know from the very beginning, they found out in 2001.

There’s no doubt that they could have done better. Instead of using the flawed homebrew algorithm they chose, the HDCP designers could have avoided this security problem by using a well-known algorithm known as offline Diffie-Hellman.

Recall from Friday that each HDCP device has a private value and a public value. When two devices – call them A and B – handshake, they send each other their public values. Then A combines its private value with B’s public value, and vice versa. The result of that computation is a secret key, which is guaranteed to be the same for both participants.

In HDCP as designed, the recipe for combining public and private values involves only addition. And because is uses only addition, a would-be attacker (like the conspiracy I discussed on Friday) can set up a series of N equations in N unknowns, and those equations involve only addition – not multiplication, division, or other mathematical operators. Because the equations are this simple (“linear” in math-speak), you can solve them by standard methods, assuming you have enough equations.

The offline Diffie-Hellman algorithm, which I mentioned above, combines the public and private values in a particularly nasty non-linear fashion, to prevent the equation-solving attack. (For more detail on how Diffie-Hellman works, see the entry in Wikipedia.) An adversary can set up equations, but as far as we know there is no practical way to solve them.

The Crosby paper suggests an explanation for the use of a homebrew algorithm: the HDCP design requirements said that the entire design had to implementable in hardware with no more than 10,000 logic gates. (Logic gates are one of the simplest building blocks used to design digital circuitry. In 2001, a state-of-the-art microprocessor chip would have contained tens of millions of gates.)

Indeed, the structure of the homebrew algorithm they used is so similar to that of offline Diffie-Hellman (with simpler mathematical structures replacing the more secure ones used by offline D-H) that it seems likely that the designers knew what they really wanted but couldn’t figure out how to fit it within the 10,000-gate budget they were given – so they settled for a less secure design.

The alternative, of course, would have been to increase the gate budget in order to allow a more secure design. Why didn’t that happen? I’ll discuss some possible reasons next time.

Tim Lee at Tech Liberation Front points out an interesting aspect of the new MovieBeam device – it offers its highest-resolution output only to video displays that use the HDMI format.

(MovieBeam is a $200 box you buy that lets you buy 24-hour access to recent movies. There is a rotating menu of movies. Currently video content is trickled out to MovieBeam boxes via unused broadcast bandwidth rented from PBS stations. Eventually they’ll use the Internet to distribute movies to the devices.)

This is a common tactic these days – transmitting the highest-res content only via HDMI. And it seems like a mistake for Hollywood to insist on this. The biggest problem is that some HDTVs have HDMI inputs and some don’t, and most consumers don’t know the difference. Do you know whether your TV has an HDMI input? If you do, you either (a) don’t have a high-def TV, or (b) are a serious video geek.

Consider a (hypothetical) consumer, Fred, who bought an early high-def set because he wanted to watch movies. Fred buys MovieBeam, or a next-gen DVD player, only to discover that his TV can’t display the movies he wants in full definition, because his TV doesn’t do HDMI.

Fred will be especially angry to learn that his MovieBeam box or high-def DVD player is perfectly capable of sending content at higher definition to the inputs that his TV does have, but because of a bunch of legal mumbo-jumbo that Hollywood insists upon, his set-top box deliberately down-rezzes the video before sending it to his TV. Just imagine what Fred will think when he sees news stories about how pirated content is available in portable, high-def formats that will work with his TV.

The official story is that HDMI is a security measure, designed to stop infringers. It’s been known for years that HDMI has serious security flaws; even Wikipedia discusses them. HDMI’s security woes make a pretty interesting story, which I’ll explore over several posts. First I’ll talk about what HDMI is trying to do. Then I’ll go under the hood and talk about how the critical part of HDMI works and its well-known security flaws. (This part is already in the academic literature; I’ll give a more accessible description.) Finally, I’ll get to what is probably the most interesting part: what the history of HDMI security tells us about the industry’s goals and practices.

Officially, the security portion of HDMI is known as High-bandwidth Digital Content Protection, or HDCP. The core of this security design is the HDCP handshake, which takes place whenever two devices communicate over an HDMI cable. The handshake has two goals. First, it lets each device confirm that the other device is an authorized HDCP device. Second, it lets the two devices agree on a secret encryption key which only those two devices know. Subsequent communication over the cable is encrypted using that key, so that eavesdroppers can’t get their hands on any content that is distributed.

In theory, this is supposed to stop would-be infringers. If an infringer tries to plug an authorized video source (like a MovieBeam box) into a device that can capture and redistribute video content, this won’t work, because the capture device won’t be able to do the handshake – the authorized video source will recognize that it is unauthorized and so will refuse to sent it content. Alternatively, if an infringer tries to capture content off the wire, between an authorized source and an authorized TV set, this will be foiled by encryption. That’s the theory at least. The practice is quite different, as I’ll describe next time.

Recently I’ve been trying to figure out the politics of technology policy. There seem to be regularly drawn battle lines in Congress, but for the most part tech policy doesn’t play out as a Republican vs. Democratic or liberal vs. conservative conflict.

Henry Farrell, in a recent post at Crooked Timber, put his finger on one important factor. This was part of a larger online seminar on Chris Mooney’s book “The Republican War on Science” (which I won’t discuss here). Here’s the core of Henry Farrell’s observation:

There’s a strand of Republican thinking – represented most prominently by Newt Gingrich and by various Republican-affiliated techno-libertarians – that has a much more complicated attitude to science. Chris [Mooney] more or less admits in the book that he doesn’t get Newt, who on the one hand helped gut OTA [the Office of Technology Assessment] (or at the very least stood passively to one side as it was gutted) but on the other hand has been a proponent of more funding for many areas of the sciences. I want to argue that getting Newt is important.

What drives Newt and people like him? Why are they so vigorously in favour of some kinds of science, and so opposed to others? The answer lies, I think, in an almost blindly optimistic set of beliefs about technology and its likely consequences when combined with individual freedom. Technology doesn’t equal science of course; this viewpoint is sometimes pro-science, sometimes anti- and sometimes orthogonal to science as it’s usually practiced. Combining some half-baked sociology with some half arsed intellectual history, I want to argue that there is a pervasive strain of libertarian thought (strongly influenced by a certain kind of science fiction) that sees future technological development as likely to empower individuals, and thus as being highly attractive. When science suggests a future of limitless possibilities for individuals, people with this orientation tend to be vigorously in its favour. When, instead, science suggests that there are limits to how technology can be developed, or problems that aren’t readily solved by technological means, people with this orientation tend either to discount it or to be actively hostile to it.

This mindset is especially dicey when applied to technology policy. It’s one thing to believe, as Farrell implies here, that technology can always subdue nature. That view at least reflects a consistent faith in the power of technology. But in tech policy issues, we’re not thinking so much about technology vs. nature, as about technology vs. other technology. And in a technology vs. technology battle, an unshakable faith in technology isn’t much of a guide to action.

Consider Farrell’s example of the Strategic Defense Initiative, the original Reagan-era plan to develop strong defenses against ballistic missile attacks. At the time, belief that SDI would succeed was a pretty good litmus test for this kind of techno-utopianism. Most reputable scientists said at the time that SDI wasn’t feasible, and they turned out to be right. But the killer argument against SDI was that enemies would adapt to SDI technologies by deploying decoys, or countermeasures, or alternatives to ballistic missiles such as suitcase bombs. SDI was an attempt to defeat technology with technology.

The same is true in the copyright wars. Some techno-utopians see technology – especially DRM – as the solution. The MPAA’s rhetoric about DRM often hits this note – Jack Valenti is a master at professing faith in technology as solving the industry’s problems. But DRM tries to defeat technology with technology, so faith in technology doesn’t get you very far. To make good policy, what you really need is to understand the technologies on both sides of the battle, as well as the surrounding technical landscape that lets you predict the future of the technical battle.

The political challenge here is how to defuse the dangerous instincts of the less-informed techno-utopians. How can we preserve their general faith in technology while helping them see why it won’t solve all human problems?