Lawsuits! Everybody’s filing lawsuits. Premier Election Systems (formerly Diebold) is suing SysTest, one of the EAC’s testing authorities (or, more properly, former testing authorities, now that the EAC is planning to suspend their accreditation). There’s also a lawsuit between the State of Ohio and Premier over whether or not Premier’s voting systems satisfy Ohio’s requirements. Likewise, ES&S is being sued by San Francisco, the State of California, and the state of Oregon. A Pennsylvania county won a judgment against Advanced Voting Systems, after AVS’s systems were decertified (and AVS never even bothered showing up in court to defend themselves). And that’s just scratching the surface.
What’s the real problem here? Electronic voting systems were “certified”, sold, deployed, and then turned out to have a variety of defects, ranging from “simple” bugs to a variety of significant security flaws. Needless to say, it takes time, effort, and money to build better voting machines, much less to push them through the certification process. And nobody really understands what the certification process even is anymore. In the bad old days, a “federally certified voting system” was tested by one of a handful of “independent testing authorities” (ITAs), accredited by the National Association of State Election Directors, against the government’s “voluntary voting system guidelines” (the 2002 edition, for the most part). This original process demonstrably failed to yield well-engineered, secure, or even particularly usable voting systems. So how have things improved?
Now, NASED has been pushed aside by the EAC, and the process has been glacial. So far as I can tell, no electronic voting system used in the November 2008 election had code that was in any way different from what was used in the November 2006 election.
Regardless of whether we jettison the DREs and move to optical scan, plenty of places will continue using DREs. And there will be demand for new features in both DREs and optical scanners. And bug fixes. The certification pipeline must be vigilant, yet it needs to get rolling again. In a hurry, but with great caution and care. (Doesn’t sound very feasible, I know.)
Okay, then let’s coerce vendors to build better products! Require the latest standards! While brilliant, in theory, such a process is doomed to continue the practical failures (and lawsuits) that we’re seeing today. The present standards are voluminous. They are also quite vague where it matters because there is no way to write a standard that’s both general-enough to apply to every possible voting system and specific-enough to adequately require good development practices. The present standards err, arguably correctly, on the vague side, which then requires the testing authorities to do some interpretation. Doing that properly requires competent testing labs and competent developers, working together.
Unfortunately, they don’t work together at all (never mind issues of competence). The current business model is that developers toil away, perhaps talking to their customers, but not interacting with the certification process at all until they’re “done,” after which they pitch the system over the wall, write a big check, and cross their fingers that everything goes smoothly. If the testing authority shoots it down, they need to sort out why and try again. Meanwhile, you’ve got the Great States of California and Ohio doing their own studies, with testers like yours truly who don’t particularly care what the standards say and are instead focused on whether the machines are robust in the face of a reasonable threat model. Were the problems we found outside of the standards’ requirements? We don’t care because they’re serious problems! Unfortunately, from the vendor’s perspective, they now need to address everything we found, and they have no idea whether or not they’ll get it right before they may or may not face another team of crack security ninjas.
What I want to see is a grand bargain. The voting system vendors open up their development processes to external scrutiny and regulation. In return, they get feedback from the certifying authorities that their designs are sound before they begin prototyping. Then they get feedback that their prototypes are sound before they flesh out all the details. This necessarily entails the vendors letting the analysts in on their bugs lists (one of the California Secretary of State’s recommendations to the EAC), further increasing transparency. Trusted auditors could even look at the long-term development roadmap and make judgments that incremental changes, available in the short term, are part of a coherent long-term plan to engineer a better system. Alternately, the auditors could declare the future plans to be a shambles and refuse to endorse even incremental improvements. Invasive auditing would give election authorities the ability to see each vendor’s future, and thus reach informed decisions about whether to support incremental updates or to dump a vendor entirely.
Where can we look for a a role model for this process? I initially thought I’d write something here about how the military procures weapon systems, but there are too many counter-examples where that process has gone wrong. Instead, let’s look at how houses are built (or, at least, how they should be built). You don’t just go out, buy the lumber and nails, hire people off the street, and get banging. Oh no! You start with blueprints. Those are checked off by the city zoning authorities, the neighborhood beauty and integrity committee, and so forth. Then you start getting permits. Demolition permits. Building permits. Electrical permits. At each stage of construction, city inspectors, the prospective owners, and even the holders of the construction loan, may want to come out and check it out. If, for example, there’s an electrical problem, it’s an order of magnitude easier to address it before you put up the interior walls.
For voting systems, then, who should do the scrutiny? Who should scrutinize the scrutineers? Where’s the money going to come from to pay for all this scrutiny? It’s unclear that any of the testing authorities have the deep skills necessary to do the job. It’s similarly unclear that you can continually recruit “dream teams” of the best security ninjas. Nonetheless, this is absolutely the right way to go. There are only a handful of major vendors in the e-voting space, so recruiting good talent to audit them, on a recurring part-time basis, is eminently feasible. Meta-scrutiny comes from public disclosure of the audit reports. To save some money, there are economies of scale to be gained from doing this at the Federal level, although it only takes a few large states to band together to achieve similar economies of scale.
At the end of the day, we want our voting systems to be the best they can be, regardless of what technology they happen to be using. I will argue that this ultimately means that we need vendors working more closely with auditors, whether we’re considering primitive optical scanners or sophisticated end-to-end cryptographic voting schemes. By pushing the adversarial review process deeper into the development pipeline, and increasing our transparency into how the development is proceeding, we can ensure that future products will be genuine improvements over present ones, and hopefully avoid all these messy lawsuits.
[Sidebar: what about protecting the vendors’ intellectual property? As I’ve argued before, this is what copyrights and patents are about. I offer no objection to vendors owning copyright on their code. Patents are a bit trickier. If the auditors decide that some particular feature should be mandatory and one vendor patents it, then every other vendor could potentially infringe the patent. This problem conceivably happens today, even without the presence of invasive auditors. Short of forbidding voting machine patents as a prerequisite for voting system certification, this issue will never go away entirely. The main thing that I want to do away with, in their entirety, are trade secrets. If you want to sell a voting machine, then you should completely waive any trade secret protection, ultimately yielding a radical improvement in election transparency.]
