Voting technology has changed greatly in recent years, leading to problems with accuracy and auditability. These are important, but another trend has gotten less attention: the gradual erosion of the secret ballot.

It’s useful to distinguish two separate conceptions of the secret ballot. Let’s define weak secrecy to mean that the voter has the option of keeping his ballot secret, and strong secrecy to mean that the voter is forced to keep his ballot secret. To put it another way, weak secrecy means the ballot is secret if the voter cooperates in maintaining its secrecy; strong secrecy means the ballot is secret even if the voter wants to reveal it.

The difference is important. No system can stop a voter from telling somebody how he voted. But strong secrecy prevents the voter from proving how he voted, whereas weak secrecy does not rule out such a proof. Strong secrecy therefore deters vote buying and coercion, by stopping a vote buyer from confirming that he is getting what he wants – a voter can take the payment, or pretend to knuckle under to the coercion, while still voting however he likes. With weak secrecy, the buyer or coercer can demand proof.

In theory, our electoral system is supposed to provide strong secrecy, as a corrective to an unfortunate history of vote buying and coercion. But in practice, our system provides only weak secrecy.

The main culprit is voting by mail. A mail-in absentee ballot is only weakly secret, the voter can mark and mail the ballot in front of a third party, or the voter can just give the blank ballot to the third party to be filled out. Any voter who wants to reveal his vote can request an absentee ballot. (Some states allow absentee voting only for specific reasons, but in practice people who are willing to sell their votes will also be willing to lie about their justification for absentee voting.)

Strong secrecy seems to require the voter to cast his ballot in a private booth, which can only be guaranteed at an officially run polling place.

The trend toward voting by mail is just one of the forces eroding the secret ballot. Some e-voting technologies fail to provide even weak secrecy, for example by recording ballots in the order they were cast, thereby allowing officials or pollwatchers who record the order of voters’ appearance (as happens in many places) to connect each recorded vote to a voter.

Worse yet, even if a complex voting technology does protect secrecy, this may do little good if voters aren’t confident that the system really protects them. If everybody “knows” that the party boss can tell who votes the wrong way, the value of secrecy will be lost no matter what the technology does. For this reason, the trend toward complex black-box technologies may neutralize the benefits of secrecy.

If secrecy is being eroded, we can respond by trying to restore it, or we can decide instead to give up on secrecy or fall back to weak secrecy. Merely pretending to enforce strong secrecy looks like a recipe for bad policy.

(Thanks to Alex Halderman and Harlan Yu for helpful conversations on this topic.)

On Tuesday, the Technical Guidelines Development Committee (TGDC), the group drafting the next-generation Federal voting-machine standards, voted unanimously to have the standards require that new voting machines be software-independent, which in practice requires them to have some kind of paper trail.

(Officially, TGDC is drafting “guidelines”, but the states generally require compliance with the guidelines, so they are de facto standards. For brevity, I’ll call them standards.)

The first attempt to pass such a requirement failed on Monday, on a 6-6 vote; but a modified version passed unanimously on Tuesday. The most interesting modification was an exception for existing machines: new machines will have to be software-independent but already existing machines won’t. There’s no scientific or security rationale for treating new and old machines differently, so this is clearly a political compromise designed to lower the cost of compliance by sacrificing some security.

If you believe, as almost all computer scientists do, that paper trails are necessary today for security, you’ll be happy to see the requirement for new machines, but disappointed that existing paperless voting machines will be allowed to persist.

Whether you see the glass as half full or half empty depends on whether you see the quest for paper trails as mainly legal or mainly political, that is, whether you look to courts or legislatures for progress.

In court, the exception for existing machines will be strong, assuming it’s written clearly into the standard. It will be hard to get rid of the old machines by filing lawsuits, or at least the new standards won’t be useful in court. If anything, the new standards may be seen as ratifying the decision to stick with old, insecure machines.

In legislatures, on the other hand, the standard will be an official ratification of the fact that paper trails are preferable. The latest, greatest technology will use paper trails, and paperless designs will look old-fashioned. The exception for old machines will look like a money-saving compromise, and few legislators will want to be seen as risking democracy to save money.

As for me, I see legislatures more than courts, and politics more than lawyering, as driving the trend toward paper trails. Thirty-five states either have a paper trail statewide or require one to be adopted by 2008. The glass is already 70% full, and the new standards will help fill it the rest of the way.

In an important development in e-voting policy, NIST has issued a report recommending that the next-generation federal voting-machine standards be written to prevent (re-)certification of today’s paperless e-voting systems. (NIST is the National Institute of Standards and Technology, a government agency, previously called the National Bureau of Standards, that is a leading source of independent technology expertise in the U.S. government.) The report is a recommendation to another government body, the Technical Guidelines Development Committee (TGDC), which is drafting the 2007 federal voting-machine standards. The new report is notable for its direct tone and unequivocal recommendation against unverifiable paperless voting systems, and for being a recommendation of NIST itself and not just of the report’s individual authors.

[UPDATE (Dec. 2): NIST has now modified the document’s text, for example by removing the “NIST recommends…” language in some places and adding a preface saying it is only a discussion draft.]

The key concept in the report is software independence.

A voting system is software-independent if a previously undetected change or error in its software cannot cause an undetectable change or error in an election outcome. In other words, it can be positively determined whether the voting system’s (typically, electronic) CVRs [cast-vote records] are accurate as cast by the voter or in error.

This gets to the heart of the problem with paperless voting: we can’t be sure the software in the machines on election day will work as expected. It’s difficult to tell for sure which software is present, and even if we do know which software is there we cannot be sure it will behave correctly. Today’s paperless e-voting systems (known as DREs) are not software-independent.

NIST does not known how to write testable requirements to make DREs secure, and NIST’s recommendation to the STS [a subcommittee of the TGDC] is that the DRE in practical terms cannot be made secure. Consequently, NIST and the STS recommend that [the 2007 federal voting standard] should require voting systems to be [software independent].

In other words, NIST recommends that the 2007 standard should be written to exclude DREs.

Though the software-independence requirement and condemnation of DREs as unsecureable will rightly get most of the attention, the report makes three other good recommendations. First, attention should be paid to improving the usability and accessibility of voting systems that use paper. Second, the 2007 standard should include high-level discussion of new approaches to software independence, such as fancy cryptographic methods. Third, more research is needed to develop new kinds of voting technologies, with special attention paid to improving usability.

Years from now, when we look back on the recent DRE fad with what-were-we-thinking hindsight, we’ll see this NIST report as a turning point.