July 27, 2016

avatar

Diebold Quietly Recalled Voting Machine Motherboards

Diebold replaced the motherboard (i.e., the main electronic component) on about 4700 of Maryland’s AccuVote-TS voting machines in 2005, according to Cameron Barr’s story in Thursday’s Washington Post. The company and state officials kept the recall quiet – even some members of the state’s Board of Elections were unaware of it until contacted by the Post. (“If they had asked, we would have told them,” an official said.)

The original motherboards had a design error that caused the machines to become unresponsive, or “freeze”, sometimes during elections. In the 2004 general election, about four percent of Montgomery County’s machines had this problem, according to the county’s 2004 Presidential General Election Review: Lessons Learned report (page 11).

In March 2004, Diebold had sent the state a memo describing the problem in the original motherboards. The memo says that “stack-up of component tolerances” led to timing errors in accessing RAM memory.

Let’s decode that for non-engineer readers. A circuitboard uses many chips or components. The technical specifications for each chip give a set of tolerances, which might say something like this: “If the temperature is between 40 and 140 degrees, and the supply voltage is between 2.9 and 3.1 volts, and a stable signal is delivered on pin 13 for at least 30 nanoseconds, then the chip will respond by sending a signal on pin 19, between 30 and 70 nanoseconds after receiving the pin-13 signal.” This is a promise from the chip’s manufacturer to the system designer. Designers rely on promises like this to make sure their systems will work.

When the designer connects different chips together – when a signal produced by one chip is fed into another one – the designer has to make sure that the signal provided by the first chip falls within the tolerances accepted by the second chip. Otherwise the second chip might not work as advertised, and the overall system might be flaky or simply fail.

But sometimes design errors like these turn out not to cause trouble. If tolerances are just a little bit out of whack, you might just get lucky. Maybe a chip that is guarantted only for voltages over 2.9 volts will still work at 2.88 volts. Maybe a delay guaranteed between 30 and 70 nanoseconds tends to come out on the low end of that range in the batch of chips you got. Or maybe everything works fine, except when something unusual happens – a hot day, or a glitch in the building’s power supply, or an unusual sequence of button presses on the screen. A designer might choose to risk such problems to save money, in an application where reliability isn’t critical. But it shouldn’t happen in a voting machine.

Diebold’s March 2004 memo explains their design problem and says that they redesigned the motherboard to fix the problem. Newly manufactured machines were getting the redesigned motherboards, and any old machines that exhibited problems would have their motherboards replaced. But at that time old machines that hadn’t been seen malfunctioning were left in the field. Diebold estimated that fewer than one percent of the old machines would have problems.

In the November 2004 election, about four percent of Montgomery County machines had screen freezes. Afterward, Diebold decided to recall the old motherboards, replacing them all with new redesigned boards. Today, every Maryland voting machine has one of the new motherboards. Will we see further problems with Diebold’s motherboard design? Only time will tell.

(You may be wondering how these design problems might have affected the accuracy of vote-counting in the 2004 election. I’ll consider that question in the next post.)

Comments

  1. >> You may be wondering how these design problems might have affected the accuracy of vote-counting in the 2004 election.

  2. “You may be wondering how these design problems might have affected the accuracy of vote-counting in the 2004 election. ”

    That’s easy, just compare the electronic results with a recount of the paper tra…nevermind.

  3. avatar Bryan Feir says:

    The ‘quietly’ part of this reminds me of a comment made by many people about the software industry in general (and Microsoft in particular) treating bugs as PR problems rather than technical problems…

  4. Bryan: There’s a reason software has “soft” in its name. The problem reported here is clearly a hardware issue, but then IMO a good deal of the cavalier attitude towards software quality has rubbed off on the whole computer industry, if not the larger tech industry.

  5. avatar Bryan Feir says:

    I’m aware that it’s a hardware issue. I was commenting on what you referred to as the ‘cavalier attitude’ in which security problems aren’t considered problems until they hit CNN and may affect the bottom line. Which is a bad attitude to take for a company that works in ATMs. (Granted, in ATMs any security problems will hit their bottom lines before it hits CNN, as the banks will be rather upset with them first and want it dealt with ASAP.)

    To a large extent, this gets back to Bruce Schneier’s comments about security being an economic externality for many companies. If it doesn’t directly affect their bottom line, they don’t care.

  6. The media is not doing its J O B. Democracy and freedom of the press my foot. What we need is a kick ass news paper or cable channel. when a girl dissappeared somewhere in the caribbean nancy grace and rita cosby were talking talking talking talking for months on end. But when s**t like this happens, nary a word or sound.

  7. avatar Stan Klein says:

    Even with all the failures, the motherboards got through the certification testing and were probably operating in the machines completely within the older Federal Election Commision and newly adopted Election Assistance Commission specifications. When you specify a Mean Time Between Failures (MTBF) of a paltry 163 hours, probably to cut the cost of testing for the vendors, almost anything goes. If business computers failed as often as voting machines, we would be replacing or repairing them monthly. How long would we stand for that? The MTBF for voting machines should be at least 15000 hours. That still allows failures (0.1% versus the current nearly 10% — actually 50% or more if early voting is considered), but a paper trail combined with proper fail safe design can mitigate them.