Recently the Election Science Institute released a fascinating report on real experience with e-voting technologies in a May 2006 primary election in Cuyahoga County, Ohio (which includes Cleveland). The report digs beneath the too-frequent platitudes of the e-voting debates, to see how , poll workers and officials actually use the technology, what really goes wrong in practice, and how well records are kept. The results are sobering.
Cuyahoga County deserves huge credit for allowing this study. Too often, voting officials try to avoid finding problems, rather than avoiding having problems. It takes courage to open one’s own processes to this kind of scrutiny, but it is the best way to improve. Cuyahoga County has done us all a service.
The election used Diebold electronic voting systems with Diebold’s add-on voter verified paper trail (VVPT) facility. One of the most widely discussed parts of the report describes ESI’s attempt to reconcile the VVPT with the electronic records kept by the voting machines. In about 10% of the machines, the paper record was spoiled: the paper roll was totally blank, or scrunched and smeared beyond reconstruction, or broken and taped back together, or otherwise obviously wrong. Had the election required a recount, this could have been a disaster – roughly 10% of the votes would not have been backed by a useful paper record, and Ohio election law says the paper record is the official ballot.
What does this teach us? First, the design of this particular VVPT mechanism needs work. It’s not that hard to make a printer that works more than 90% of the time. Printer malfunctions can never be eliminated completely, but they must be made very rare.
Second, we need to remember why we wanted to augment electronic records with a VVPT in the first place. It’s not that paper records are always more reliable than electronic records. The real reason we want to use them together is that paper and electronic recordkeeping systems have different failure modes, so that the two used together can be more secure than either used alone. In a well-designed system, an adversary who wants to create fraudulent ballots must launch two very different attacks, against the paper and electronic systems, and must synchronize them so that the fraudulent records end up consistent.
Third, this result illustrates why it’s important to audit some random subset of precincts or voting machines as a routine post-election procedure. Regular integrity-checking will help us detect problems, whether they’re caused by glitches or malicious attacks.
There’s much more in the ESI report, including a summary of voting machine problems (power failures, inability to boot, broken security seals, etc.) reported from polling places, and some pretty pointed criticism of the county’s procedural laxity. The best system is one that can tolerate these kinds of problems, learn from them, and do a better job next time.
E-voting E-OK or not
Responding to the insinuations made from my earlier comment, I must ask where exactly I ever suggested malicious intentions of any of the people conducting the evaluation? It was my obvious assertion that anyone who may be inclined to use the information obtained for unscrupulous reasons would probably be a dishonest politician.
“My point is this: These men were testing the weaknesses in their system to see how they can better exploit it.”
If their goal was exploiting the weaknesses, why did they publicly release their findings in a report?
I’m surprised this comment has passed unchecked, considering that many of us in the security field make a career out of testing the weaknesses in systems. And, without fail, someone accuses us of malicious intent.
To me, this is like accusing the National Academy of Engineering of planning to blow up skyscrapers, because of their careful analysis of how the Trade Center towers collapsed.
There is no way that Internet voting will be used in statewide or national elections for a long while. Google the SERVE security report which is still the state of the art in evaluations of Internet voting. As Dan Wallach has said, “Internet voting is ok when you don’t care what the outcome is.”
I’ll mention another trend which is often overlooked in discussions of electronic voting. That is the move towards absentee voting. Here’s a chart showing the California experience:
http://www.ss.ca.gov/elections/hist_absentee.htm
Absentee votes have increased from 2% back in the 1960s up to 40% in the most recent election, and the trend is continuing upward.
The state of Oregon runs all their elections absentee now (vote by mail).
This is another example of a reality which Internet voting discussions prefer to overlook. Voting from home has even more possibilities for abuse than paper voting at the polling place – vote selling, voting for others, votes being “lost in the mail”, etc. Yet look at the trends. Vote by mail is becoming common and may soon become universal.
IMO it is inevitable that we will move to internet voting before long. I know, I know, computer people are going even more ballistic than they were already. But that’s the reality of the situation. The only thing that can compete with vote by mail is vote by net. People aren’t going to be willing to come all the way down to polling stations.
Too much discussion of electronic voting is occuring in an idealized and abstract world separated from reality. People need to understand that the world is imperfect and make judgements on that basis. All too often I still see analysists comparing a corruptable computerized voting system with an idealized, perfect, paper voting system. That’s not how things work in the real world.
I can say with a degree of certainty that the reason for Cuyahoga County’s participation are not altruistic. I commend the author for recognizing what would normally be a helpful deed by a concerned local government. Ah, but as we should well know by this point in time, a politician in Ohio with altruistic concerns is hunted down and lynched for the dangerous heathen he is….
As a person who actively keeps my evil impulses in check, I am keenly aware of the darker impulses of those around me, and Ohio is on my personal radar. My point is this: These men were testing the weaknesses in their system to see how they can better exploit it. And exploit it they will, wait and see. Actually, I find it terribly foreboding to have learned of their participation, especially if that little pixie of a tyrant J. Kenneth Blackwell was involved. Or should I say Governor- Elect Blackwell?
Having both electronic and paper ballots sounds like a good idea because sometimes paper can be annoying, but we can’t completely depend on technology. However, if this is going to work they need to seriously fix the problems with the system. A faulty electronic system will just make it worse than using paper.
The solution to this problem is to use machines to address the existing problems of error and fraud without introducing new opportunities.
Until those new machines are available, I would prefer to stick with the existing problems of error, and the occasional, but necessarily difficult to achieve fraud.
One other issue though that needs to be adressed by e-voting machines, and other than a realible (and legally conclusive) voter audited paper trail, no one has proposed, is that of transparency. The recount, if required, needs to be able to be performed by lay people, not only those ordained into the cadres of those knowledgable about the inner workings of the e-voting machines…
Hal,
By paper voting, I am guessing you are referring to things like scan forms and punch cards. Problems with those are: random marks invalidating votes (mostly for scan cards), confusing “butterfly ballots”, and the extreme difficulty of the voter to verify the votes were cast as desired. I have voted using scan forms and punch cards. I dislike the punch cards because once I remove the card, I have no way of knowing what little hole in the paper is for my candidate and which is not. Additionally, these are not *really* paper ballots, they are electronic ballots using paper as the input.
If you want a true balanced analysis, then we must include completely manual paper voting.
The reason so many people are opposed to completely electronic voting without a paper trail is because there are very real, very possible undetectable abuses that can result in fraud. Add to that the almost blind faith our elected officials place on the technology as the only way to go. And let’s not forget the equipment manufacturers balking about even providing details about how the machine operates or providing the paper trail in the first place.
Any new system being built to replace and old system should fix the problems inherent in the old system AND not create new issues NOT possible in the old system without finding ways to prevent or mitigate them first.
Actually, Dan Wallach, the stopwatch attack would breach ballot secrecy. (It won’t help you breach voter anonymity if the voter shows up in a cloak and ski mask, will it? :))
The previous commentary, on voter confusion, shows where the weakest link in any voting system is: not the paper trail, nor the computers, but the voters, who are after all only human.
I agree that IDEALLY we would like to do as you say, use machines to address existing problems of error and fraud without introducing new opportunities. But again, that is an IDEAL standard. In practice, what we should look at is how electronic voting compares on balance with paper voting.
You list several kinds of attacks that are possible with electronic voting but are arguably more difficult with paper voting, and that’s fine. Now please list some attacks that are possible with paper voting, but which electronic voting would make more difficult. This would be a step towards a balanced analysis. It is the failure to take this step which I object to.
Hal,
I think part of the misunderstanding comes from the crucial difference between error and overt malfeasance. Computers are better at reducing natural errors, but are far more useful for malfeasance.
Computers reduce most physical constraints on tampering with data, allowing individuals to do the kind of damage in seconds that used to require a team working for hours or days. After all, this is what computers are good at: they concentrate the work of many people, and make mountains of data far easier to manage.
Second, computers give entirely new opportunities for malfeasance. The actual counting of votes can be altered by many people besides the counters. Anyone with access to the machine over its lifetime can potentially alter the counting behavior—because the voting apparatus itself has become a sophisticated component in the counting process.
Finally, computers allow for arbitrarily complex malicious behavior. In the old days an election official could try to tamper with a voting apparatus, but couldn’t make it register a 1% bias that only happens on November 2, 2008, automatically reverting to normal behavior at midnight.
The solution to this problem is to use machines to address the existing problems of error and fraud without introducing new opportunities.
To Hal: I think its safe to say that computer science types have decidely not advocated allowing the paper record to govern in the case of the dispute. In fact, Doug Jones piece from the Oct 2004 issue of Comm. of the ACM on auditing specifically states that the usual case with audit trails is not to default trust one or the other… but to do an investigation to see if any evidence would lead you to believe that one or the other is suspect.
Unlike Hal, I don’t see the ESI report as an exposé on the problems with paper ballots. I see it as an exposé on the problems with one vendor’s implementation of machine-printed paper records. The main point of the report isn’t that VVPAT is a problem. The point is that VVPAT records need to have headers to help you disambiguate them and that voters and poll workers need to be cognizant of the different failure modes of the printer (or, if you prefer, that a different paper/printer device is necessary).
This study illustrates the enormous disconnect between theory and reality in online discussions of electronic voting.
People hold electronic voting to an ideal of perfection which has never been realized in any large-scale voting system. The truth is that paper voting systems have significant problems. Paper votes have been effectively manipulated for so many years that election fraud is an ever-present reality. Electronic voting should be compared to an ideal, it should be compared to the reality of highly imperfect paper voting systems.
One thing I’ve noticed is that computer scientists, very knowledgeable about all the problems with computers and electronics, think that electronic voting is a bad idea and that paper voting is far better. But vote registrars and election officials, who have struggled for years with all of the problems with paper votes and are well aware of their imperfections, think that electronic voting is far better. Each side knows all of the problems with the technology they are familiar with, and just assumes that the other technology is superior.
This disconnect is pointed out by a problem which I have never seen frankly discussed. Suppose we did have an electronic voting system with a paper trail; suppose there were a recount, and suppose the two systems disagreed. Which one should we believe?
Computer scientists would say to believe the paper version, but they have no grounds for this conclusion. This study illustrates how bad paper can be. And the problem can’t be fixed by just saying, get better printers. Have you all forgotten the lesson of Florida in 2000? That was not an exception, it is the rule with paper ballots! Every election based on paper has substantial uncertainty in the voting results. And that’s ignoring what happens when people try to cheat. Paper is a weak technology; it is damaged easily, it must be handled just right, it has a low degree of robustness. One of the amusing aspects of paper is this: every recount you do gets a different answer! How can such a technology provide confidence in a voting situation?
It’s time for computer scientists and electronic voting experts to climb out of their ivory towers and take a look at the real world. If you’re going to pass yourself off as an expert on voting systems, start by spending a few years studying the problems with paper voting. We need experts who have this kind of cross domain experience in order to get an objective and unbiased analysis of how we can best combine the two technologies in order to benefit from the strengths of both. Computer scientists smugly sitting back and saying that all you need is a paper backup do not help the situation. That is bad advice and will only lead to more problems in the future.
Just to confuse things further, the Texas governor (himself running for reelection) announced a special election, occurring on the same day as the general election. The special election will select somebody to fill Tom DeLay’s shoes for basically November and December. Shelley Sekula-Gibbs will be listed on the “special” ballot, but will be write-in-only for the “regular” ballot. To confuse things further, Democratic candidate Nick Lampson will be on the “regular” ballot but chose not to appear on the “special” ballot. And, to make things even more confusing, if no candidate on the special election ballot gains a majority, there will need to be a run-off election in December. Regardless, the winner of the regular ballot will take office in January.
Confused yet?
One can imagine that the whole situation is the result of a variety of politically-motivated maneuvers, but the net result will be a serious amount of voter confusion. In particular, Sekula-Gibbs, when she appears on local TV news, is generally quoted citing the need to vote for her twice. At least her voters will have her name nearby to help them with the spelling.
There’s some debate about how close you need to be to her name in order for your write-in to be considered a vote for her. One possible interpretation: “Any write-in votes for the CD-22 race that do not match a pre-programmed alias will be looked at by a resolution board, whose function is to determine who the voter intended to vote for… The elections chief also said anyone voting a straight GOP ticket will not cast a vote for Sekula-Gibbs, because the Republican Party has no official candidate in the race. A straight Republican ticket would not ‘trigger’ a write-in vote.”
Having voted on a Hart InterCivic machine for several years now, I can say that these machines are reasonably efficient for “normal” votes, but I’ve never tried doing a write-in before. Scrolling in the name will be slow. An interesting security risk occurs: if you’ve got a stop-watch, you can use completion time as a predictor for how somebody voted. That’s an important violation of voter anonymity.
I’ve thought about the problem caused by electronic voting machines. We already have a system of electronic machines that are designed to handle vasts amount of data that leave a very secure and complete paper audit trail. I’ve written about it at http://tinyurl.com/rpdm4 .
I haven’t seen the machines in question or tried to use them, but if either ten per cent of them had spoilt ballots, and none of the voters reported that problem, then there’s either a serious problem with the interface (in that it doesn’t actually allow for voter verification of the audit trail, even though it claims to), with the election procedures (does a poll worker have a way to recover if a voter does complain?), or with voter education (did any voter actually validate the audit trail?).
Such a large fraction of bad ballots must indicate a systemic problem.
—
The parallels between 1876 and our time are eerie.
http://elections.harpweek.com/09Ver2Controversy/Overview-1.htm
Joe: “To Scott Craver: I would argue that ease of use isn’t the only real argument for having DREs… I can think of many (accessibility, minority language accomodation, ballot style complexity, efficiency, the lack of paper is a cost-saving measure, etc.).”
Except for the last one, all of those are usability issues.
They all involve a user interface problem that keeps people from voting at all, or from voting correctly. If ballots are in a different language, or too complex, or you must wait an hour for your turn because voting takes so dang long, these are failures of the interface.
This was the main argument for DRE machines, making it easier for people to vote and reducing the rate of miscast ballots.
The problem is that we had this idea of using touch-screen computers to address all the interface problems, but then people figured that, as long as there was a computer in the booth, we might as well use the machine for the actual ballot counting, storage and transmission.
Sounds like the Diebold system isn’t a VVPT. If it were really “voter-verified”, there would have been no problems with the paper ballots.
Of course, this may be a problem with voter education on the new systems, as much as anything else. Even if the screen says, “Check the printed ballot behind the glass {insert giant arrow here}. Is the ballot legible, and does it show the votes you made today?” some people are still going to hit yes even if the ballot looks like a monkey went wild with a purple crayon.
To Nikita: There have been a few informal studies that show that there is, in fact, very little voter verification going on. At first, I was surprised by this result. Then I realized that what we’re asking of users is quite complex compared to all other kiosk-like embedded systems out there. No where do you compare information on screen with printed information displayed to you under glass as a “check” on the two sets of information. This means we need to educate voters (and poll workers) about why it is important to check the paper record so that we can shoehorn this process into the user experience.
To Scott Craver: I would argue that ease of use isn’t the only real argument for having DREs… I can think of many (accessibility, minority language accomodation, ballot style complexity, efficiency, the lack of paper is a cost-saving measure, etc.). I think most usability people would consider all the DRE-based user interfaces to be particularly poor when compared with other sectors of information technology (just look at ATMs… they’re quite good). It’s clear that the vendors don’t spend much if any time doing user evaluation methods and are largely locked into their current designs (it’s not trivial to bring a new voting system design to market). Hopefully, Texas districts will construe anything close to Shelley Sekula-Gibbs as being a vote for her… best, Joe
It’s important to note that this is not the final word on the ESI report. DESI has responded, in their usual corporate manner, calling the ESI report inaccurate and that the report might even constitute defamation. ESI responded to DESI saying that reconciling the discrepancies they noted will take very tight cooperation with both the county and DESI.
This is an important point. Without a working knowledge of these voting systems, which very few people come close to outside the walls of the vendor, it is very difficult to impossible to due effective audits of modern voting technology. I have high hopes for the collaboration between ESI and DESI to reconcile the discrepancies that were found, and I further hope that a similar audit conducted after November’s election will show that such scrutiny can only be good for everyone involved.
Just in the news, (scroll to the bottom) e-voting machines deliver a surprising blow to republicans in Tom Delay’s district.
As you probably know, Tom Delay is out of the race, forcing Republicans to run a write-in campaign for Shelley Sekula-Gibbs—but the e-voting machine is really bad at inputting write-in names. From the article:
“The vast majority of voters in the district for the first time will use eSlate voting machines that will require voters, for a write-in, to dial up one letter at a time and press ”enter” after each letter. It will take around two minutes for a voter to dial in the name of the Republican candidate, Houston City Councilwoman Shelley Sekula-Gibbs (with no option for a hyphen).”
This is a bafflingly bad UI, and a tremendous irony given that ease of use is the only real argument for having a DRE machine in the first place.
On a related note, Avi Rubin was the guest on the Diane Rehm Show this morning, speaking about his new book “Brave New Ballot: The Battle to Safeguard Democracy in the Age of Electronic Voting.”
A stream should be available by noon today from the program site.
I think this study points out a problem not only with the machines but with the VVPT process. If people are willing to use a machine that does not print out any paper receipts at all — despite the fact that by law this means that they effectively did not vote — you have to wonder how much voter verification there really is going on.