May 20, 2018

Archives for September 2009

Finnish Court Orders Re-Vote After E-Voting Snafu

The Supreme Administrative Court of Finland has ruled that three municipal elections, the first in Finland to use electronic voting, must be redone because of voting machine problems. (English summary; ruling in Finnish)

The troubles started with a usability problem, which caused 232 voters (about 2% of voters) to leave the voting booth without fully casting their ballots. Electronic Frontiers Finland explains what went wrong:

It seems that the system required the voter to insert a smart card to identify the voter, type in their selected candidate number, then press “ok”, check the candidate details on the screen, and then press “ok” again. Some voters did not press “ok” for the second time, but instead removed their smart card from the voting terminal prematurely, causing their ballots not to be cast.

This usability issue was exacerbated by Ministry of Justice instructions, which specifically said that in order to cancel the voting process, the user should click on “cancel” and after that, remove the smart card. Thus, some voters did not realise that their vote had not been registered.

If you want to see what this looks like for a voter, check out the online demo of the voting process, from the Finnish Ministry of Justice (in English).

Well designed voting systems tend to have a prominent, clearly labeled control or action that the voter uses to officially cast his or her vote. This might be a big red “CAST VOTE” button. The Finnish system mistakenly used the same “OK” button used previously in the process, making voter mistakes more likely. Adding to the problem, the voter’s smart card was protruding from the front of the machine, making it all too easy for a voter to grab the card and walk away.

No voting machine can stop a “fleeing voter” scenario, where a voter simply walks away during the voting process (we conventionally say “fleeing” even if the voter leaves by mistake), but some systems are much better than others in this respect. Diebold’s touchscreen voting machines, for all their faults, got this design element right, pulling the voter’s smart card all of the way into the machine and ejecting it only when the voter was supposed to leave — thus turning the voter’s desire to return the smart card into a countermeasure against premature voter departure, rather than a cause of it. (ATM machines often use this same trick of holding the card inside the machine to stop the user from grabbing the card and walking away at the wrong time.) Some older lever machines use an even simpler method against fleeing voters: the same big red handle that casts the ballot also opens the curtains so the voter can leave.

I’d be curious to know what the rules are about fleeing voters in Finland. I know that New Jersey procedures say that if a voter leaves without performing the final step of pushing the “Cast Vote” button, poll workers are supposed to push the button on the voter’s behalf (without looking at the voter’s choices). Crucially, the design of the New Jersey voting machine (for all its faults) makes it almost certain that such a non-cast ballot will be discovered promptly — the machine makes a noise when the ballot is cast, and the machine will complain if the poll worker tries to enable the next voter’s ballot before the previous voter’s ballot has been cast.

It seems likely that the Finnish machine, in addition to its usability problems that led to fleeing voters, had other design/process problems that made a non-completed ballot less noticeable to poll workers. (I don’t know this for sure; the answer isn’t in any English-language document I have seen.)

Fortunately, the damage was not as bad as it might have been, because the e-voting system was used in only three municipalities, as a pilot program, rather than nationwide. Presumably, nationwide use of the flawed system is now unlikely.

Consolidation in E-Voting Market: ES&S Buys Premier

Yesterday Diebold sold its e-voting division, known as Premier Election Systems, to ES&S, one of Premier’s competitors. The price was low: about $5 million.

ES&S is reportedly the largest e-voting company, and Premier was the second-largest, so the deal represents a substantial consolidation in the market. The odds of one major e-voting company breaking from the pack and embracing up-to-date security engineering are now even slimmer than before. Premier had seemed like the company most likely to change its ways.

The sale represents the end of an embarrassing era for Diebold. The company must have had high hopes when it first bought a small e-voting company, but the new Diebold e-voting division never approached the parent companies standards for security and product quality. Over time the small e-voting division became an embarrassment, and the parent company distanced itself by renaming the division from Diebold to Premier and publicizing the division’s independence. Now Diebold is finally rid of its e-voting division and can return to doing what it does relatively well.

Finding and Fixing Errors in Google's Book Catalog

There was a fascinating exchange about errors in Google’s book catalog over at the Language Log recently. We rarely see such an open and constructive discussion of errors in large data sets, so this is an unusual opportunity to learn about how errors arise and what can be done about them.

The exchange started with Geoffrey Nunberg pointing to many errors in the metadata associated with Google’s book search project. (Here “metadata” refers to the kind of information that would have been on a card in the card catalog of an traditional library: a book’s date of publication, subject classification, an so on.) Some of the errors are pretty amusing, including Dickens writing books before he was born, a Bob Dylan biography published in the nineteenth century, Moby Dick classified under “computers”. Nunberg called this a “train wreck” and blamed Google’s overaggressive use of computer analysis to extract bibliographic information from scanned images.

Things really got interesting when Google’s Jon Orwant replied (note that the red text starting “GN” is Nunberg’s response to Orwant), with an extraordinarily open and constructive discussion of how the errors described by Nunberg arose, and the problems Google faces in trying to ensure accuracy of a huge dataset drawn from diverse sources.

Orwant starts, for example, by acknowledging that Google’s metadata probably contains millions of errors. But he asserts that that is to be expected, at least at first: “we’ve learned the hard way that when you’re dealing with a trillion metadata fields, one-in-a-million errors happen a million times over.” If you take catalogs from many sources and aggregate them into a single meta-catalog — more or less what Google is doing — you’ll inherit all the errors of your sources, unless you’re extraordinarily clever and diligent in comparing different sources to sniff out likely errors.

To make things worse, the very power and flexibility of a digital index can raise the visibility of the errors that do exist, by making them easy to find. Want to find all of the books, anywhere in the world, written by Charles Dickens and (wrongly thought to be) published before 1850? Just type a simple query. Google’s search technology did a lot to help Nunberg find errors. But it’s not just error-hunters who will find more errors — if a more powerful metadata search facility is more useful, researchers will rely on it more, and will therefore be tripped up by more errors.

What’s most interesting to me is a seeming difference in mindset between critics like Nunberg on the one hand, and Google on the other. Nunberg thinks of Google’s metadata catalog as a fixed product that has some (unfortunately large) number of errors, whereas Google sees the catalog as a work in progress, subject to continual improvement. Even calling Google’s metadata a “catalog” seems to connote a level of completion and immutability that Google might not assert. An electronic “card catalog” can change every day — a good thing if the changes are strict improvements such as error fixes — in a way that a traditional card catalog wouldn’t.

Over time, the errors Nunberg reported will be fixed, and as a side effect some errors with similar causes will be fixed too. Whether that is good enough remains to be seen.