Lawsuits! Everybody’s filing lawsuits. Premier Election Systems (formerly Diebold) is suing SysTest, one of the EAC’s testing authorities (or, more properly, former testing authorities, now that the EAC is planning to suspend their accreditation). There’s also a lawsuit between the State of Ohio and Premier over whether or not Premier’s voting systems satisfy Ohio’s requirements. Likewise, ES&S is being sued by San Francisco, the State of California, and the state of Oregon. A Pennsylvania county won a judgment against Advanced Voting Systems, after AVS’s systems were decertified (and AVS never even bothered showing up in court to defend themselves). And that’s just scratching the surface.

What’s the real problem here? Electronic voting systems were “certified”, sold, deployed, and then turned out to have a variety of defects, ranging from “simple” bugs to a variety of significant security flaws. Needless to say, it takes time, effort, and money to build better voting machines, much less to push them through the certification process. And nobody really understands what the certification process even is anymore. In the bad old days, a “federally certified voting system” was tested by one of a handful of “independent testing authorities” (ITAs), accredited by the National Association of State Election Directors, against the government’s “voluntary voting system guidelines” (the 2002 edition, for the most part). This original process demonstrably failed to yield well-engineered, secure, or even particularly usable voting systems. So how have things improved?

Now, NASED has been pushed aside by the EAC, and the process has been glacial. So far as I can tell, no electronic voting system used in the November 2008 election had code that was in any way different from what was used in the November 2006 election.

Regardless of whether we jettison the DREs and move to optical scan, plenty of places will continue using DREs. And there will be demand for new features in both DREs and optical scanners. And bug fixes. The certification pipeline must be vigilant, yet it needs to get rolling again. In a hurry, but with great caution and care. (Doesn’t sound very feasible, I know.)

Okay, then let’s coerce vendors to build better products! Require the latest standards! While brilliant, in theory, such a process is doomed to continue the practical failures (and lawsuits) that we’re seeing today. The present standards are voluminous. They are also quite vague where it matters because there is no way to write a standard that’s both general-enough to apply to every possible voting system and specific-enough to adequately require good development practices. The present standards err, arguably correctly, on the vague side, which then requires the testing authorities to do some interpretation. Doing that properly requires competent testing labs and competent developers, working together.

Unfortunately, they don’t work together at all (never mind issues of competence). The current business model is that developers toil away, perhaps talking to their customers, but not interacting with the certification process at all until they’re “done,” after which they pitch the system over the wall, write a big check, and cross their fingers that everything goes smoothly. If the testing authority shoots it down, they need to sort out why and try again. Meanwhile, you’ve got the Great States of California and Ohio doing their own studies, with testers like yours truly who don’t particularly care what the standards say and are instead focused on whether the machines are robust in the face of a reasonable threat model. Were the problems we found outside of the standards’ requirements? We don’t care because they’re serious problems! Unfortunately, from the vendor’s perspective, they now need to address everything we found, and they have no idea whether or not they’ll get it right before they may or may not face another team of crack security ninjas.

What I want to see is a grand bargain. The voting system vendors open up their development processes to external scrutiny and regulation. In return, they get feedback from the certifying authorities that their designs are sound before they begin prototyping. Then they get feedback that their prototypes are sound before they flesh out all the details. This necessarily entails the vendors letting the analysts in on their bugs lists (one of the California Secretary of State’s recommendations to the EAC), further increasing transparency. Trusted auditors could even look at the long-term development roadmap and make judgments that incremental changes, available in the short term, are part of a coherent long-term plan to engineer a better system. Alternately, the auditors could declare the future plans to be a shambles and refuse to endorse even incremental improvements. Invasive auditing would give election authorities the ability to see each vendor’s future, and thus reach informed decisions about whether to support incremental updates or to dump a vendor entirely.

Where can we look for a a role model for this process? I initially thought I’d write something here about how the military procures weapon systems, but there are too many counter-examples where that process has gone wrong. Instead, let’s look at how houses are built (or, at least, how they should be built). You don’t just go out, buy the lumber and nails, hire people off the street, and get banging. Oh no! You start with blueprints. Those are checked off by the city zoning authorities, the neighborhood beauty and integrity committee, and so forth. Then you start getting permits. Demolition permits. Building permits. Electrical permits. At each stage of construction, city inspectors, the prospective owners, and even the holders of the construction loan, may want to come out and check it out. If, for example, there’s an electrical problem, it’s an order of magnitude easier to address it before you put up the interior walls.

For voting systems, then, who should do the scrutiny? Who should scrutinize the scrutineers? Where’s the money going to come from to pay for all this scrutiny? It’s unclear that any of the testing authorities have the deep skills necessary to do the job. It’s similarly unclear that you can continually recruit “dream teams” of the best security ninjas. Nonetheless, this is absolutely the right way to go. There are only a handful of major vendors in the e-voting space, so recruiting good talent to audit them, on a recurring part-time basis, is eminently feasible. Meta-scrutiny comes from public disclosure of the audit reports. To save some money, there are economies of scale to be gained from doing this at the Federal level, although it only takes a few large states to band together to achieve similar economies of scale.

At the end of the day, we want our voting systems to be the best they can be, regardless of what technology they happen to be using. I will argue that this ultimately means that we need vendors working more closely with auditors, whether we’re considering primitive optical scanners or sophisticated end-to-end cryptographic voting schemes. By pushing the adversarial review process deeper into the development pipeline, and increasing our transparency into how the development is proceeding, we can ensure that future products will be genuine improvements over present ones, and hopefully avoid all these messy lawsuits.

[Sidebar: what about protecting the vendors’ intellectual property? As I’ve argued before, this is what copyrights and patents are about. I offer no objection to vendors owning copyright on their code. Patents are a bit trickier. If the auditors decide that some particular feature should be mandatory and one vendor patents it, then every other vendor could potentially infringe the patent. This problem conceivably happens today, even without the presence of invasive auditors. Short of forbidding voting machine patents as a prerequisite for voting system certification, this issue will never go away entirely. The main thing that I want to do away with, in their entirety, are trade secrets. If you want to sell a voting machine, then you should completely waive any trade secret protection, ultimately yielding a radical improvement in election transparency.]

California Secretary of State Debra Bowen has sent a letter to Chair Gineen Beach of the US Election Assistance Commission (EAC) outlining three proposals that she thinks will markedly improve the integrity of voting systems in the country.

I’ve put a copy of Bowen’s letter here (87kB PDF).

Bowen’s three proposals are:

• Vulnerability Reporting — The EAC should require that vendors disclose vulnerabilities, flaws, problems, etc. to the EAC as the system certification authority and to all the state election directors that use the affected equipment.
• Uniform Incident Reporting — The EAC should create and adopt procedures that jurisdictions can follow to collect and report data about incidents they experience with their voting systems.
• Voting System Performance Measurement — As part of the Election Day Survey, the EAC should systematically collect data from election officials about how voting systems perform during general elections.

In my opinion, each of these would be a welcome move for the EAC.

These proposals would put into place a number of essential missing elements of administering computerized elections equipment. First, for the users of these systems, election officials, it can be extremely frustrating and debilitating if they suspect that some voting system flaw is responsible for problems they’re experiencing. Often, when errors arise, contingency planning requires detailed knowledge about specific details of a voting system flaw. Without knowing as much as possible about the problem they’re facing, election officials can exacerbate the problem. At best, not knowing about a potential flaw can do what Bowen describes: doom the election official, and others with the same equipment, to repeatedly encounter the flaw in subsequent elections. Of course, vendors are the most likely to have useful information on a given flaw, and they should be required to report this information to both the EAC and election officials.

Often the most information we have about voting system incidents come from reports from local journalists. These reporters don’t tend to cover high technology too often; their reports are often incomplete and in many cases simply and obviously incorrect. Having a standardized set of elements that an election official can collect and report about voting system incidents will help to ensure that the data comes directly from those experiencing a given problem. The EAC should design such procedures and then a system for collecting and reporting these issues to other election officials and the public.

Finally, many of us were disappointed to learn that the 2008 Election Day survey would not include questions about voting system performance. Election Day is a unique and hard-to-replicate event where very little systematic data is collected about voting machine performance. The OurVoteLive and MyVote1 efforts go a long way towards actionable, qualitative data that can help to increase enfranchisement. However, self-reported data from the operators of the machinery of our democracy would be a gold mine in terms of identifying and examining trends in how this machinery performs, both good and bad.

I know a number of people, including Susannah Goodman at Common Cause as well as John Gideon and Ellen Theisen of VotersUnite!, who have been championing one or another of these proposals in their advocacy. The fact that Debra Bowen has penned this letter is a testament to the reason behind their efforts.

The recount of the 2008 Minnesota Senate race gives us an opportunity to evaluate the accuracy of precinct-count optical-scan voting. Though there have been contentious disputes over which absentee ballot envelopes to open, the core technology for scanning ballots has proved to be extremely accurate.

The votes were counted by machine (except for part of one county that counts votes by hand), then every single ballot was examined by hand in the recount.

The “net” accuracy of optical-scan voting was 99.99% (see below).
The “gross” accuracy was 99.91% (see below).
The rate of ambiguous ballots was low, 99.99% unambiguous (see below).

My analysis is based on the official spreadsheet from the Minnesota Secretary of State. I commend the Secretary of State for his commitment to transparency in the form of making the data available in such an easy-to-analyze format. The vast majority of the counties use the ES&S M100 precinct-count optical-scanners; a few use other in-precinct scanners.

I exclude from this analysis all disputes over which absentee ballots to open. Approximately 10% of the ballots included in this analysis are optically scanned absentee ballots that were not subject to dispute over eligibility.

There were 2,423,851 votes counted for Coleman and Franken. The “net” error rate is the net change in the vote margin from the machine-scan to the hand recount (not including change related to qualification of absentee ballot envelopes). This was 264 votes, for an accuracy of 99.99% (error, one part in ten thousand).

The “gross” error rate is the total number of individual ballots either added to one candidate, or subtracted from one candidate, by the recount. A ballot that was changed from one candidate to the other will count twice, but such ballots are rare. In the precinct-by-precinct data, the vast majority of precincts have no change; many precincts have exactly one vote added to one candidate; few precincts have votes subtracted, or more than one vote added, or both.

The recount added a total of 1,528 votes to the candidates, and subtracted a total of 642 votes, for a gross change of 2170 (again, not including absentee ballot qualification). Thus, the “gross” error rate is about 1 in 1000, or a gross accuracy of 99.91%.

Ambiguous ballots: During the recount, the Coleman and Franken campaigns initially challenged a total of 6,655 ballot-interpretation decisions made by the human recounters. The State Canvassing Board asked the campaigns to voluntarily withdraw all but their most serious challenges, and in the end approximately 1,325 challenges remained. That is, approximately 5 ballots in 10,000 were ambiguous enough that one side or the other felt like arguing about it. The State Canvassing Board, in the end, classified all but 248 of these ballots as votes for one candidate or another. That is, approximately 1 ballot in 10,000 was ambiguous enough that the bipartisan recount board could not determine an intent to vote. (This analysis is based on the assumption that if the voter made an ambiguous mark, then this ballot was likely to be challenged either by one campaign or the other.)

Caveat: As with all voting systems, including optical-scan, DREs, and plain old paper ballots, there is also a source of error from voters incorrectly translating their intent into the marked ballot. Such error is likely to be greater than 0.1%, but the analysis I have done here does not measure this error.

Hand counting: Saint Louis County, which uses a mix of optical-scan and hand-counting, had a higher error rate: net accuracy 99.95%, gross accuracy 99.81%.

Yes, we know that there’s no such thing as a perfect voting system, but the Estonians are doing their best to get as far away from perfection as possible. According to the latest news reports, Estonia is working up a system to vote from mobile phones. This follows on their earlier web-based Internet voting. What on earth are they thinking?

Let’s review some basics. The Estonian Internet voting scheme builds on the Estonian national ID card, which is a smartcard. You get the appropriate PCMCIA adapter and you can stick it into your laptop. Then, through some kind of browser plug-in, it can authenticate you to the voting server. No card, no voter impersonation. The Estonian system “avoids” the problem of voter bribery / coercion by allowing the voter to cast as many votes as they want, but only the last one actually counts. As I understand it, a voter may also arrive, on election day, at some sort of official polling place and substitute a paper ballot for their prior electronic ballot.

The threats to this were and are obvious. What if some kind of malware/virus/worm contraption infects your web browser and/or host operating system, waits for you to connect to the election server, and then quietly substitutes its own choices for yours? You would never know that the attack occurred and thus would never think to do anything about it. High tech. Very effective. And, of course, somebody can still watch over your shoulder while you vote. At that point, they just need to keep you from voting again. They could accomplish this by simply having you vote at the last minute, under supervision, or they could “borrow” your ID card until it’s too late to vote again. Low tech. Still effective.

But wait, there’s more! The central database must necessarily have your vote recorded alongside your name in order to allow subsequent votes to invalidate earlier votes. That means they’ve almost certainly got the technical means to deanonymize your vote. Do you trust your government to have a database that says exactly for whom you voted? Even if the vote contents are somehow encrypted, the government has all the necessary key material to decrypt it. (And, an aforementioned compromised host platform could be leaking this data, regardless.)

Okay, what about voting by cellular telephone? A modern cell phone is really no different from a modern web browser. An iPhone is running more-or-less the same OS X and Safari browser that’s featured on Apple’s Mac products. Even non-smart-phones tend to have an environment that’s powerful and general-purpose. There’s every reason to believe that these platforms are every bit as vulnerable to software attacks as we see with Windows systems. Just because hackers aren’t necessarily targeting these systems doesn’t mean they couldn’t. Ultimately, that means that the vulnerabilities of the phone system are exactly the same as the web system. No better. No worse.

Of course, crypto can be done in a much more sophisticated fashion. One Internet voting system, Helios, is quite sophisticated in this fashion, doing end-to-end crypto in JavaScript in your browser. With its auditability, Helios gives you the chance to challenge the entire client/server process to prove that it maintained your vote’s integrity. There’s nothing, however, in Helios to prevent an evil browser from leaking how you voted, thus compromising your anonymity. An evil election server could possibly be prevented from compromising your anonymity, depending on how the decryption keys are managed, but all the above privacy concerns still apply.

Yes, of course, Internet and cell-phone voting have lots of appeal. Vote from anywhere! At any time! If Estonia did more sophisticated cryptography, they could at least have a hope at getting some integrity guarantees (which they appear to be lacking, at present). Estonians have absolutely no privacy guarantees and thus insufficient protection from bribery and coercion. And we haven’t even scratched the surface of denial-of-service attacks. In 2007, Estonia suffered a large, coordinated denial-or-service attack, allegedly at the hands of Russian attackers. I’m reasonably confident that they’re every bit as vulnerable to such attacks today, and cell-phone voting would be no less difficult for resourceful attackers to disrupt.

In short, if you care about voter privacy, to defeat bribery and coercion, then you want voters to vote in a traditional polling place. If you care about denial of service, then you want these polling places to be operable even if the power goes out. If you don’t care about any of that, then consider the alternative. Publish in the newspaper a list of every voter and how they voted, for all the world to see, and give those voters a week to submit any corrections they might desire. If you were absolutely trying to maximize election integrity, nothing would beat it. Of course, if you feel that publishing such data in the newspaper could cause people to be too scared to vote their true preferences, then maybe you should pay more attention to voter privacy.

(More on this from Eric Rescorla’s Educated Guesswork.)