Recently the Election Science Institute released a fascinating report on real experience with e-voting technologies in a May 2006 primary election in Cuyahoga County, Ohio (which includes Cleveland). The report digs beneath the too-frequent platitudes of the e-voting debates, to see how , poll workers and officials actually use the technology, what really goes wrong in practice, and how well records are kept. The results are sobering.

Cuyahoga County deserves huge credit for allowing this study. Too often, voting officials try to avoid finding problems, rather than avoiding having problems. It takes courage to open one’s own processes to this kind of scrutiny, but it is the best way to improve. Cuyahoga County has done us all a service.

The election used Diebold electronic voting systems with Diebold’s add-on voter verified paper trail (VVPT) facility. One of the most widely discussed parts of the report describes ESI’s attempt to reconcile the VVPT with the electronic records kept by the voting machines. In about 10% of the machines, the paper record was spoiled: the paper roll was totally blank, or scrunched and smeared beyond reconstruction, or broken and taped back together, or otherwise obviously wrong. Had the election required a recount, this could have been a disaster – roughly 10% of the votes would not have been backed by a useful paper record, and Ohio election law says the paper record is the official ballot.

What does this teach us? First, the design of this particular VVPT mechanism needs work. It’s not that hard to make a printer that works more than 90% of the time. Printer malfunctions can never be eliminated completely, but they must be made very rare.

Second, we need to remember why we wanted to augment electronic records with a VVPT in the first place. It’s not that paper records are always more reliable than electronic records. The real reason we want to use them together is that paper and electronic recordkeeping systems have different failure modes, so that the two used together can be more secure than either used alone. In a well-designed system, an adversary who wants to create fraudulent ballots must launch two very different attacks, against the paper and electronic systems, and must synchronize them so that the fraudulent records end up consistent.

Third, this result illustrates why it’s important to audit some random subset of precincts or voting machines as a routine post-election procedure. Regular integrity-checking will help us detect problems, whether they’re caused by glitches or malicious attacks.

There’s much more in the ESI report, including a summary of voting machine problems (power failures, inability to boot, broken security seals, etc.) reported from polling places, and some pretty pointed criticism of the county’s procedural laxity. The best system is one that can tolerate these kinds of problems, learn from them, and do a better job next time.

[This entry was written by Avi Rubin and Ed Felten.]

A report by Harri Hursti, released today at BlackBoxVoting, describes some very serious security flaws in Diebold voting machines. These are easily the most serious voting machine flaws we have seen to date – so serious that Hursti and BlackBoxVoting decided to redact some of the details in the reports. (We know most or all of the redacted information.) Now that the report has been released, we want to help people understand its implications.

Replicating the report’s findings would require access to a Diebold voting machine, and some time, so we are not in a position to replicate the findings at this time. However, the report is consistent with everything we know about how these voting machines work, and we find it very plausible. Assuming the report is accurate, we want to summarize its lessons for voters and election administrators.

Implications of the Report’s Findings

The attacks described in Hursti’s report would allow anyone who had physical access to a voting machine for a few minutes to install malicious software code on that machine, using simple, widely available tools. The malicious code, once installed, would control all of the functions of the voting machine, including the counting of votes.

Hursti’s findings suggest the possibililty of other attacks, not described in his report, that are even more worrisome.

In addition, compromised machines would be very difficult to detect or to repair. The normal procedure for installing software updates on the machines could not be trusted, because malicious code could cause that procedure to report success, without actually installing any updates. A technician who tried to update the machine’s software would be misled into thinking the update had been installed, when it actually had not.

On election day, malicious software could refuse to function, or it could silently miscount votes.

What can we do now?

Election officials are in a very tough spot with this latest vulnerability. Since exploiting the weakness requires physical access to a machine, physical security is of the utmost importance. All Diebold Accuvote machines should be sequestered and kept under vigilant watch. This measure is not perfect because it is possible that the machines are already compromised, and if it was done by a clever attacker, there may be no way to determine whether or not this is the case. Worse yet, the usual method of patching software problems cannot be trusted in this case.

Where possible, precincts planning on using these machines should consider making paper backup systems available to prepare for the possibility of widespread failures on election day. The nature of this technology is that there is really no remedy from a denial of service attack, except to have a backup system in place. While voter verified paper trails and proper audit can be used to protect against incorrect results from corrupt machines, they cannot prevent an attack that renders the machines non-functional on election day.

Using general purpose computers as voting machines has long been criticized by computer scientists. This latest vulnerability highlights the reasoning behind this position. This attack is possible due to the very nature of the hardware on which the systems are running. Several high profile studies failed to uncover this. With the current technology, there is no way to account for all the ways that a system might be vulnerable, and the discovery of a problem of this magnitude in the midst of primary season is the kind of scenario we have feared all along.

Timeline and Perspective

This is not the first time Diebold has faced serious security issues – though this problem appears to be the worst of them all. Here is a capsule history of Diebold security studies:

2001: Doug Jones produces a report highlighting design flaws in the machines that became the Diebold touchscreen voting machines.
July 24, 2003: Hopkins/Rice study finds many security flaws in Diebold machines, including ones that were pointed out by Doug Jones.
September 24, 2003: SAIC study finds serious flaws in Diebold voting machines. 2/3 of the report is redacted by the state of Maryland.
November 21, 2003: Ohio’s Compuware and InfoSentry reports find critical flaws in Diebold touchscreen voting machines
January 20, 2004: RABA study finds serious security vulnerabilities in Diebold touchscreen voting machines.
November, 2004: 37 states use Diebold touchscreen voting machines in general election.
March, 2006: Harri Hursti reports the most serious vulnerabilities to date discovered.

None of the previously published studies uncovered this flaw. Did SAIC? It might exist in the unredacted report, but to date, nobody outside of Maryland officials and SAIC has been able to see that report.

We believe that the question of whether DREs based on commodity hardware and operating systems should ever be used in elections needs serious consideration by government and election officials. As computer security experts, we believe that the known dangers and potentially unknown vulnerabilities are too great. We should not put ourselves in a position where, in the middle of primary season, the security of our voting systems comes into credible and legitimate question.

David Card and Enrico Moretti, two economists from UC Berkeley, have an interesting new paper that crunches data on the 2004 election, to shed light on the effect of touchscreen voting. The paper looks reasonable to me, but my background is not in social science so others are better placed than me to critique it. Here, I’ll summarize the paper’s findings.

The researchers start with datasets on county-by-county vote results in the 2004 U.S. presidential election, and county-by-county demographics, along with a list of counties that used DREs (i.e., touchscreen voting machines). It turns out that counties that used DREs tended to vote more strongly for Bush than counties that didn’t. This effect, by itself, isn’t very interesting, since there are many possible causes. For example, DREs were more popular in the South, and Bush was more popular there too.

To get a more interesting result, they redid the same calculation, while controlling for many of the factors that might have affected Bush’s vote share. To be specific, they controlled for past voting patterns (Republican and third-party voting shares in the 1992, 1996, and 2000 presidential elections), for county demographics (percent black, percent Hispanic, percent religious, percent college-educated, percent in the military, percent employed in agriculture), for average income, and for county population. They also included a per-state dummy variable that would capture any effects that were the same across all counties in a particular state. After controlling for all of these things, they still found that DRE counties tended to tilt toward Bush, compared to non-DRE counties. This discrepancy, or “DRE effect” amounted to 0.21% of the vote.

So did Republicans steal the election? The researchers turn to that question next. They observe that if the DRE effect was caused by Republican cheating, then we would expect the DRE effect to be larger in places where Republicans had a motive to cheat (because the election was close), and where Republicans had an opportunity to cheat (because they controlled the election bureaucracy). Yet further analysis shows that the DRE effect was not larger in states where the election was close, and was not larger in states with Republican governors or Republicans secretaries of state. Therefore it seems unlikely that outright vote-stealing can account for the DRE effect.

The researchers next looked at how DRE use correlated with voter turnout. They found that voter turnout was roughly 1% lower in counties that used DREs, after controlling for all of the factors listed above. Interestingly, the drop in turnout tended to be larger in counties with larger Hispanic populations. (The same effect does not seem to exist for black voters.) This suggests a possible cause of the DRE effect: DREs may suppress turnout among Hispanic voters, who tend to vote for Democrats overall (although not in Florida).

Why might DREs suppress the Hispanic vote? Perhaps Hispanics are more likely to be intimidated by the high-tech DREs. Perhaps DREs are harder to use for voters who aren’t native English speakers. Perhaps DREs made people wait longer to vote, and Hispanic voters were less able or less willing to wait. Or perhaps there is some other cultural issue that made Hispanic voters wary of DREs.

It’s worth noting, though, that when the researchers estimated the magnitude of the Hispanic-vote-suppression mechanism, they found that it accounted for only about 15% of the overall DRE effect. Most of the DRE effect is still unexplained.

This is an interesting paper, but is far from the last word on the subject.

UPDATE (Thur. May 19): Steve Purpura, who knows this stuff much better than I do, has doubts about this study. See the comments for his take.

Yesterday, a team of social scientists from UC Berkeley released a study of the effect of e-voting on county-by-county vote totals in Florida and Ohio in the recent election. It’s the first study to use proper social-science modeling methods to evaluate the effect of e-voting.

The study found counties with e-voting tended to tilt toward Bush, even after controlling for differences between counties including past voting history, income, percentage of Hispanic voters, voter turnout, and county size. The researchers estimate that e-voting caused a swing in favor of Bush of up to 260,000 votes in Florida. (A change of that many votes would not be enough to change the election’s result; Bush won Florida by about 350,000 votes.)

No e-voting effect was found in Ohio.

The study looks plausible, but I don’t have the expertise to do a really careful critique. Readers who do are invited to critique the study in the comments section.

Regardless of whether it is ultimately found credible, this study is an important step forward in the discourse about this topic. Previous analyses had shown differences, but had not controlled for the past political preferences of individual counties. Skeptics had claimed that “Dixiecrat” counties, in which many voters were registered as Democrats but habitually voted Republican, could explain the discrepancies. This study shows, at least, that the simple Dixiecrat theory is not enough to refute the claim that e-voting changed the results.

Assuming that the study’s authors did their arithmetic right, there are two possibilities. It could be that some other factor, beyond the ones that the study controlled for, can explain the discrepancies. If this is the case, we can assume somebody will show up with another study demonstrating that.

Or it could be that e-voting really did affect the result. If so, there are several ways this could have happened. One possibility is that the machines were maliciously programmed or otherwise compromised; I think this is unlikely but unfortunately the machines are designed in a way that makes this very hard to check. Or perhaps the machines made errors that tended to flip some votes from one candidate to the other. Even random errors of this sort would tend to affect the overall results, if e-voting counties different demographically from other counties (which is apparently the case in Florida). Another possibility is that e-voting affects voter behavior somehow, perhaps affecting different groups of voters differently. Maybe e-voting scares away some voters, or makes people wait longer to vote. Maybe the different user interface on e-voting systems makes straight party-line voting more likely or less likely.

This looks like the beginning of a long debate.