The big e-voting story from November’s election was in Sarasota, Florida, where a congressional race was decided by about 400 votes, with 18,412 undervotes. That’s 18,412 voters who cast votes in other races but not, according to the official results, in that congressional race. Among voters who used the ES&S iVotronic machines – that is, non-absentee voters in Sarasota County – the undervote rate was about 14%. Something went very wrong. But what?
Since the election there have been many press releases, op-eds, and blog posts about the undervotes, not to mention some lawsuits and scholarly studies. I want to spend the rest of the week dissecting the Sarasota situation, which I have been following closely. I’m doing this now for two reasons: (1) enough time has passed for the dust to settle a bit, and (2) I’m giving a joint talk on the topic next week and I want to work through some thoughts.
There’s no doubt that something about the iVotronic caused the undervotes. Undervote rates differed so starkly in the same race between iVotronic and non-iVotronic voters that the machines must be involved somehow. (For example, absentee voters had a 2.5% undervote rate in the congressional race, compared to 14% for iVotronic voters.) Several explanations have been proposed, but only two are at all plausible: ballot design and machine malfunction.
The ballot design theory says that the ballot offered to voters on the iVotronic’s screen was misdesigned in a way that caused many voters to miss that race. Looking at screenshots of the ballot, one can see how voters might miss the congressional race at the top of the second page. (Depressingly, some sites show a misleading photo that the photographer angled and lit to make the misdesign look worse than it really was.) It’s very plausible that this kind of problem caused some undervotes; and that is consistent with the reports of many voters that the machine did not show them the congressional race.
It’s one thing to say that ballot design could have caused some undervotes, but it’s another thing entirely to say it was the sole cause of so elevated an undervote rate. Each voter, before finalizing his vote, was shown a clearly designed confirmation screen listing his choices and clearly showing a no-candidate-selected message for the congressional race. Did so many voters miss that too? And what about the many voters who reported choosing a candidate in the congressional race, only to have the no-candidate-selected message show up on the confirmation screen anyway?
The malfunction theory postulates a problem or malfunction with the voting machines that caused votes not to be recorded. There are many types of problems that could have caused lost votes. The best way to evaluate the malfunction theory is to conduct a careful and thorough study of the machines themselves. In the next entry I’ll talk about the efforts that have been made toward that end. For now, suffice it to say that no suitable study is available to us.
If we had a voter-verified paper trail, we could immediately tell which theory is correct, by comparing the paper and electronic records. If the voter-verified paper records show the same high undervote race, then the ballot design theory is right. If the paper and electronic records show significantly different undervote rates, then something is wrong with the machines. But of course the advocates of paperless voting argued that paper trails were unnecessary – while also arguing that touchscreen systems reduce undervotes.
Several studies have tried to use statistical analyses of undervote patterns in different races, precincts, and machines to evaluate the two theories. Frisina, Herron, Honaker, and Lewis say the data support the ballot design theory; Mebane and Dill say the data point to malfunction as a likely cause of at least some of the undervotes. Reading these studies, I can’t reach a clear conclusion.
What would convince me, one way or the other, is a good study of the machines. I’ll talk next time about the fight over whether and how to look at the machines.
These E&S machines will be used in some places for the next french presidential elections, more here (in french):
http://guerby.org/blog/index.php/2007/01/28/143-vote-electronique-et-transparence
I’d love to get a working demo of vote spying (some radio device detecting remotely with high probability the voter choice). What that done? In France, vote secrecy must be strict, and it’s easy to demo, no mumbo-jumbo about voting booth procedure secure blah blah.
A quick look at the ballot makes it seem like you don’t want to be listed first as it almost seems like it is part of the heading. On the screen in question, my initial glance led me to believe that Christine Jennings was the sole candidate running. If she’s the sole candidate, then theoretically I don’t need to mark that one and move on. If we had a three party system rather than a two party system my hunch is that there wouldn’t have been the undervotes….
I agree with DMC’s last paragraph!
My point on ballot design was that voter misreading will have a random distribution yielding varing rates of undervote across a group of similar machines. Machine malfunction should be always the same yielding a constant rate of undervote. The Machine malfunction can have the appearance of random varition if the malfunction depends on a variable. For example, static electricity could be a cause of malfunction which could vary according to hourly temperature/humidity changes.
In any case as DMC said review of the voting data for each machine may yield strong evidence of machine malfunction and perhaps tampering!
Mickey
Wallace Green says:
I believe the Mebane and Dill paper contains a highly flawed analysis attempting to dismiss the ballot design theory.
Mebane and Dill’s paper states:
“The ballot design argument cannot in any simple way explain why different kinds of voters seem to have responded to the design very differently…â€
This is incorrect. A simple explanation is that Jennings…
Wallace: you say that there is a simple explanation, then you go on for several paragraphs. The length of the time you have taken to explain it belies the fact that it is not ‘simple’.
I would disagree with Mickey’s analysis…
To me it seems that Ballot Misdesign, being constant across all machines, would give rise to a fairly normal distribution of undervoting, across all voting machine types and all units of the same machine. (There’s no reason to think that confused voters would be skewed towards voting on any specific machine.)
On the other hand, significantly different undervote rates on different units within a single machine type would make me think that certain units had been tampered with. Significantly different rates across machine types would make me think that one or more machine types had been compromised.
apologies for the poor formatting of my above comment
I believe the Mebane and Dill paper contains a highly flawed analysis attempting to dismiss the ballot design theory.
Mebane and Dill’s paper states:
“The ballot design argument cannot in any simple way explain why different kinds of voters seem to have responded to the design very differently…”
This is incorrect. A simple explanation is that Jennings voters were
more likely to be low-SES (and therefore less educated), first-time
voters, or the elderly. All of these correlate, to varying extents,
with Democratic voting tendencies. This alone explains in a simple way
why Jennings voters were more affected. Differing instructions and levels
of assistance at the different polling places explain differences among
similar populations.
and, when attempting to compare the Buchanan/Jennings race to the
Hospital Board race:
“The race for Hospital Board Southern District Seat 1 had several of
those features: the two candidates running for that office were listed
at the top of the page followed by a bold heading in a teal box and
then the choices for other offices. But the same voters who undervoted
significantly more often for the CD-13 race undervoted significantly
less often for the Hospital Board race. To explain such variations in
terms of the ballot design, one would need to believe the same stimulus
provoked some voters to make errors on one page but not on another page.”
Again, incorrect. The “stimulus” was not the same, nor even very similar.
See the above link for the ballot.
Note the prominent red header on page 1, 3 lines, shaped as an inverted
trapezoid, with no voting choices, and with a solid line underneath.
On page 2, the Buchanan/Jennings race has a near-identically shaped header, the two lines of text for the race, and a solid line underneath. There is a blue-highlighted “CONGRESSIONAL” header above the Senate race. When the page is flipped between page 1 and 2, the Buchanan/Jennings race appears to be part of a header for the Governor’s race, with its similar header. In other words, the Buchanan/Jennings race is physically similar to a non-race header on the previous page. The natural instinct is to treat the Buchanan/Jennings race similarly, and not vote in it.
In contrast, Hospital Board race on page 6 follows a page 5 with three
easily read races, and no colored headers. The Hospital Board race text
is physically similar to the Review Board race in the same position on
page 5. The natural instinct is to treat the Hospital Board race
similarly, and vote in it.
In any case, this point is easily settled. A study could be conducted
with a near-identical ballot could be set up with only the names changed,
in a state other than Florida. I believe with high certainty that such
a study would show that ballot design explained the entire undervote.
But why waste time and money arguing? Do the study!
Ed,
After writing my comment I found the Charles Stewart declaration on line. I gather from it that there were over 160 machines involved and the election data from each machine is available. I suggest there is a way to test the 2 theories of undervote cause, ballot misdesign vs machine malfunction. The rate of undervote from ballot misdesign is based upon subjective reaction of voters and should have a degree of random variation. Machine misdesign is a fixed process and should have a relatively constant rate of undervote. So given reasonably good data set with each machine(I understand a group of 158 was programed on one day) a statistical test of the variation of the undervote rate in this group may point to either fixed rate(mechanical) or variable( voter reactions).
If you can point to machine malfunction the legal case is much stronger than one based upon voter error.
Mickey
Mickey,
There are statistically significant differences between the undervote rates on different machines. More precisely, the undervote data are inconsistent with the hypothesis that all voters and all machines are equally prone to random and independent undervote events.
I am wondering if there are significant differences in voting rates among individual machines. That is, how many separate machines are involved and are there subgroups of machines with non random variation in the vote rates?
Bill,
They are a whole lot of places around the country where elections officials don’t quite see eye-to-eye with computer scientists. For the most part, those election officials pro’lly just have a somewhat different perspective and emphasize priorities in a little bit different order. If you talk to them, they’re actually pretty reasonable people.
But when someone pisses all over your boot, and then looks you straight in the eye and tells you it’s raining—that’s a communications strategy. They’re sending you a message. And in this case, the message appears to be about relative political power.
The judge who issued this Order on Motions also sent a message.
As I started out by saying, there are a whole lot of other places around the country.
@Joe: “if the ballot design theory turns out to be the likely explanation: How do we avoid these kinds of problems in the future?”
One suggestion would be to have one and only one ballot per page, with one option labled “blank vote”, and requiring one item to be checked before progressing to the next page. This will force voters to actively do something to cast a blank vote. In addition, by placing the blank-vote option as the top item, the small bonus* of being the top placed candidate might diminish
I also agree that the paper trail would only be truly helpful if it was generated independently, but it should help distinguish most accidental failures from usability errors. However, if the flaw lies in the part of the system that registers votes, then neither the electronic nor the paper record would receive the signal that a vote was cast, and neither would show any record of it. (Interestingly enough, the active blank-vote scheme I propose above would help some in this area).
*There were some studies showing that undecided voters had a higher probability of chosing the top candidate. I can’t remember the correct citation for this right now; but I’m sure someone here can find it if it’s important 🙂
“Each voter, before finalizing his vote, was shown a clearly designed confirmation screen listing his choices and clearly showing a no-candidate-selected message for the congressional race. Did so many voters miss that too?”
While it appears there were other issues involved I wouldn’t underestimate the role of poor usability design here. How many times have you deleted an item from your desktop and just clicked straight through the “Are you sure?” pop-up only to realize as you were clicking the window away that, oops, not that one! Even if you haven’t done that it’s a common mistake. Confirmation screens are some of the least effective ways of presenting information as users often (almost predictably) pass over them without thorough review.
Ned,
I am saddened to see you discouraging Prof. Felten in his efforts to heal an open wound at the heart of our democratic system.
Felten is to be congratulated for putting his own reputation on the line to investigate an issue that many are unwilling to touch. Rather than raining on his parade, why not thank him for being among the rare brave soul?
Hal,
Theories involving malice aren’t getting much play. If you like, you can consider them a type of malfunction theory, where the malfunction happens on purpose rather than accidentally.
Isn’t there another possibility, that the failure to record was not a malfunction, but a malicious misprogramming of the machine? Is anyone advancing that theory?
[W]hat about the many voters who reported choosing a candidate in the congressional race, only to have the no-candidate-selected message show up on the confirmation screen anyway?
According to accounts I’ve read, the official explanation is that these individuals are all either lying for partisan advantage, or on drugs. Does it really matter?
Ed, you’ve got to maintain situational awareness. This game looks rigged. And if you read the judge’s opinion on the discovery issue, the whole jurisdiction looks corrupt. So just get out. Make any excuse, feign illness, whatever…
A voter-verified paper trail can help us distinguish between two cases: (1) the machine told the voter that it wasn’t going to record a vote, but the voter didn’t notice; (2) the machine told the voter it was going to record a vote but then didn’t record it. In case (1) the paper and electronic records will match; in case (2) they won’t match. And of course if the electronic confirmation screen differered from the paper record often, many (but not all) voters would have noticed the difference on election day.
“If we had a voter-verified paper trail, we could immediately tell which theory is correct, by comparing the paper and electronic records.”
Would we? The “paper trail” is generated by the machine… not the voter. It might reflect whatever the machines says… or it might not… or, in the worst-case scenario, it might reflect whatever someone wants the machine to say.
The reasoning for the papertrails seems flawed to me: if the machine on the final confirmation screen would miss the cast vote, it would be missing on the paper too, wouldnt it?