March 19, 2024

Archives for August 2007

Debate: Will Spam Get Worse?

This week I participated in Business Week Online’s Debate Room feature, where two people write short essays on opposite sides of a proposition.

The proposition: “Regardless of how hard IT experts work to intercept the trillions of junk e-mails that bombard hapless in-boxes, the spammers will find ways to defeat them.” I argued against, concluding that “We’ll never be totally free of spam, but in the long run it’s a nuisance—not a fundamental threat—to the flourishing of the Internet.”

Does Apple Object to iPhone Unlocking?

I wrote Monday about efforts to “unlock” the iPhone so it worked on non-AT&T cell networks, and the associated legal and policy issues. AT&T lawyers have aggressively tried to stop unlocking; but Apple has been pretty silent. What position will Apple take?

It might seem that Apple has nothing to lose from unlocking, but that’s not true. AT&T can exploit customer lock-in by charging higher prices, so it has an obvious incentive to stop unlocking. But AT&T also (reportedly) give Apple a cut of iPhone users’ fees, reportedly $3/month for existing AT&T users and $11/month for new users. This isn’t surprising – in exchange for creating the lock-in, Apple gets to keep a (presumably) hefty share of the resulting revenue.

Apple’s incentive is much like AT&T’s. Apple makes more money from iPhone customers who use AT&T than from those who use other cell providers, so Apple gains by driving customers to AT&T. And it’s not pocket change – Apple gets roughly $150 per user – so even though Apple gets money for selling iPhones to non-AT&T users, they get considerably more if they can drive those users to AT&T.

Thus far, Apple seems happy to let AT&T take the blame for intimidating the unlockers. This mirrors Apple’s game plan regarding music copy-protection, where it gestures toward openness and blames the record companies for requiring restrictive technology. If this works, Apple gets the benefit of lock-in but AT&T gets the blame.

From Apple’s standpoint, an even better result might be to have iPhone unlocking be fairly painful and expensive, but not impossible. Then customers who are allergic to AT&T would still buy iPhones, but almost everybody else would stick with AT&T. So Apple would win both ways, selling iPhones to everybody while preserving its AT&T payments.

What a clever Jobsian trick – using a business model based on restriction, while planting the blame on somebody else.

iPhone Unlocked; Legal Battle Looming?

In the past few days several groups declared victory in the battle to unlock the iPhone – to make the iPhone work on cellular networks other than AT&T’s. New Jersey teenager George Hotz published instructions (starting here) for a geeks-only unlock procedure involving hardware and software tweaks. An anonymous group called iPhoneSimFree reportedly has an easy all-software unlock procedure which they plan to sell. And a company called UniquePhones was set to sell a remote unlocking service.

(Technical background: The iPhone as initially sold worked only on the AT&T cell network – the device was pretty much useless until you activated AT&T wireless service on it. People figured out quickly that you could immediately cancel the wireless service to get an iPhone that worked only via WiFi; but you couldn’t use it on any other mobile phone/data network. This was not a fundamental technical limitation of the device, but was instead a technological tie designed by Apple to drive business to AT&T.)

Unlocking the iPhone helps everybody, except AT&T, which would prefer not to face competition in selling wireless services to iPhone users. So AT&T, predictably, seem to be sending its lawyers after the unlockers. UniquePhone, via their iphoneunlocking.com site, reports incoming lawyergrams from AT&T regarding “issues such as copyright infringement and illegal software dissemination”; UniquePhones has delayed its product release to consider its options. The iPhoneSimFree members are reportedly keeping anonymous because of legal concerns.

Can AT&T cook up a legal theory justifying a ban on iPhone unlocking? I’ll leave that question to the lawyers. It seems to me, though, that regardless of what the law does say, it ought to say that iPhone unlocking is fine. For starters, the law should hesitate to micromanage what people do with the devices they own. If you want to run different software on your phone, or if you want to use one cell provider rather than another, why should the government interfere?

I’ll grant that AT&T would prefer that you buy their service. Exxon would prefer that you be required to buy gasoline from them, but the government (rightly) doesn’t try to stop you from filling up elsewhere. The question is not what benefits AT&T or Exxon, but what benefits society as a whole. And the strong presumption is that letting the free market operate – letting customers decide which product to buy – is the best and most efficient policy. Absent some compelling argument that iPhone lock-in is actually necessary for the market to operate efficiently, government should let customers choose their cell operator. Indeed, government policy already tries to foster choice of carriers, for example by requiring phone number portability.

Regardless of what AT&T does, its effort to stop iPhone unlocking is likely doomed. Unlocking software is small and easily transmitted. AT&T’s lawyers can stick a few fingers in the dike, but they won’t be able to stop the unlocking software from getting to people who want it. This is yet another illustration that you can’t lock people out of their own digital devices.

Why Was Skype Offline?

Last week Skype, the popular, free Net telephony service, was unavailable for a day or two due to technical problems. Failures of big systems are always interesting and this is no exception.

We have only limited information about what went wrong. Skype said very little at first but is now opening up a little. Based on their description, it appears that the self-organization mechanism in Skype’s peer-to-peer network became unstable. Let’s unpack that to understand what it means, and what it can tell us about systems like this.

One of the surprising facts about big information systems is that the sheer scale of a system changes the engineering problems you face. When a system grows from small to large, the existing problems naturally get harder. But you also see entirely new problems that didn’t even exist at small scale – and, worse yet, this will happen again and again as your system keeps growing.

Skype uses a peer-to-peer organization, in which the traffic flows through ordinary users’ computers rather than being routed through a set of central servers managed by Skype itself. The advantage of exploiting users’ computers is that they’re available at no cost and, conveniently, there are more of them to exploit when there are more users requesting service. The disadvantage is that users’ computers tend to reboot or go offline more than dedicated servers would.

To deal with the ever-changing population of user computers, Skype has to use a clever self-organization algorithm that allows the machines to organize themselves without relying (more than a tiny bit) on a central authority. Self-organization has two goals: (1) the system must respond quickly to changed conditions to get back into a good configuration soon, and (2) the system must maintain stability as conditions change. These two goals aren’t entirely contradictory, but they are at least in tension. Responding quickly to changes makes it difficult to maintain stability, and the system must be engineered to make this tradeoff wisely in a wide range of conditions. Getting this right in a huge P2P system like Skype is tricky.

Which brings us to the story of last week’s failure, as described by Skype. On Tuesday August 14, Microsoft released a new set of patches to Windows, according to their normal monthly cycle. Many Windows machines downloaded the patch, installed it, and then rebooted. Each such machine would leave the Skype network when it shut down, then rejoin after booting. So the effect of Microsoft’s patch release was to increase the turnover in Skype’s network.

The result, Skype says, is that the network became unstable as the respond-quickly mechanism outran the maintain-stability mechanism; and the problem snowballed as the growing instability caused ever stronger (but poorly aimed) responses. The Skype service was essentially unavailable for a day or two starting on Thursday August 16, until the company could track down the problem and fix a code bug that it said contributed to the problem.

The biggest remaining mystery is why the problem took so long to develop. Microsoft issued the patch on Tuesday, and Skype didn’t get into deep trouble until Thursday. We can explain away some of the delay by noting that Windows machines might take up to a day to download the patch and reboot, but this still means it took Skype’s network at least a day to melt down. I’d love to know more about how this happened.

I would hesitate to draw too many broad conclusions from a single failure like this. Large systems of all kinds, whether centralized or P2P, must fight difficult stability problems. When a problem like this does occur, it’s a useful natural experiment in how large systems behave. I only hope Skype has more to say about what went wrong.

E-Voting Ballots Not Secret; Vendors Don't See Problem

Two Ohio researchers have discovered that some of the state’s e-voting machines put a timestamp on each ballot, which severely erodes the secrecy of ballots. The researchers, James Moyer and Jim Cropcho, used the state’s open records law to get access to ballot records, according to Declan McCullagh’s story at news.com. The pair say they have reconstructed the individual ballots for a county tax referendum in Delaware County, Ohio.

Timestamped ballots are a problem because polling-place procedures often record the time or sequence of voter’s arrivals. For example, at my polling place in New Jersey, each voter is given a sequence number which is recorded next to the voter’s name in the poll book records and is recorded in notebooks by Republican and Democratic poll watchers. If I’m the 74th voter using the machine today, and the recorded ballots on that machine are timestamped or kept in order, then anyone with access to the records can figure out how I voted. That, of course, violates the secret ballot and opens the door to coercion and vote-buying.

Most e-voting systems that have been examined get this wrong. In the recent California top-to-bottom review, researchers found that the Diebold system stores the ballots in the order they were cast and with timestamps (report pp. 49-50), and the Hart (report pp. 59) and Sequoia (report p. 64) systems “randomize” stored ballots in an easily reversible fashion. Add in the newly discovered ES&S system, and the vendors are 0-for-4 in protecting ballot secrecy.

You’d expect the vendors to hurry up and fix these problems, but instead they’re just shrugging them off.

An ES&S spokeswoman at the Fleishman-Hillard public relations firm downplayed concerns about vote linking. “It’s very difficult to make a direct correlation between the order of the sign-in and the timestamp in the unit,” said Jill Friedman-Wilson.

This is baloney. If you know the order of sign-ins, and you can put the ballots in order by timestamp, you’ll be able to connect them most of the time. You might make occasional mistakes, but that won’t reassure voters who want secrecy.

You know things are bad when questions about a technical matter like security are answered by a public-relations firm. Companies that respond constructively to security problems are those that see them not merely as a PR (public relations) problem but as a technology problem with PR implications. The constructive response in these situations is to say, “We take all security issues seriously and we’re investigating this report.”

Diebold, amazingly, claims that they don’t timestamp ballots – even though they do:

Other suppliers of electronic voting machines say they do not include time stamps in their products that provide voter-verified paper audit trails…. A spokesman for Diebold Election Systems (now Premier Election Solutions), said they don’t for security and privacy reasons: “We’re very sensitive to the integrity of the process.”

You have to wonder why e-voting vendors are so much worse at responding to security flaw reports than makers of other products. Most software vendors will admit problems when they’re real, will work constructively with the problems’ discoverers, and will issue patches promptly. Companies might try PR bluster once or twice, but they learn that bluster doesn’t work and they’re just driving away customers. The e-voting companies seem to make the same mistakes over and over.