Expert opinions on in-person voting machines and vote-by-mail

In November 2018 I got opinions on voting machines and vote-by-mail from 17 experts on election verification, who have experience running/observing/studying elections in 17 states.

On the acceptability of these in-the-polling-place voting technologies, in the context of U.S. elections:

The consensus is that Direct Recording Electronic voting machines are unacceptable, even with a VVPAT (“voter verified paper audit trail visible to the voter under glass”).  Most experts are lukewarm to hand-counted paper ballots, presumably because they’re impractical for large elections with many contests on the ballot.  Most experts prefer hand-marked optical scan ballots, and all of these experts find hand-marked optical scan acceptable.  Most experts are willing to accept ballot marking devices (BMDs) that prepare “bubble ballots” to be scanned by optical scan machines, but only 17% find this preferable to hand-marked optical-scan ballots.  Opinion is mixed on BMDs that prepare bar-code ballots (with human-readable summaries) for tabulation by optical scanners, with most finding this  technology at least “barely acceptable.”  Almost no one prefers all-in-one machines that combine ballot marking and ballot scanning (but at least the voter can hold the ballot in her hand while inspecting it), with about a 50/50 split between “acceptable” and “barely acceptable”.

Most experts don’t prefer ballot-marking devices (BMDs) for these reasons:

1. If the paper jams, the power fails, or something else goes wrong with technology, voters using hand-marked paper ballots can still deposit their ballots in an emergency ballot for counting later; this is not an option with a BMD-only solution.
2. BMDs are more susceptible to fraud: if a BMD wrongly marks a paper ballot, (studies have shown that) most voters won’t notice.
3. BMDs cost $5000, pens cost 50c; it is expensive to supply enough BMDs for all voters, but it is feasible to supply BMDs sufficient for those voters unable to mark a paper ballot by hand.

A few experts (17%) prefer BMD-marked ballots to hand-marked ballots because (1) there’s no chance of ambiguous marks and (2) it’s easier to give voters feedback about undervotes/overvotes.

DETAILS OF THE RESPONSES:

In November 2018 I got expert opinions on voting machines and vote-by-mail from 17 experts on election verification, who have experience running/observing/studying elections in these states:
AZ,CA,CO,CT,DC,FL,IA,IN,MD,MN,NJ,NM,NY,OR,PA,SC,VA.

Referring to the bar-chart above on the acceptability of different polling-place voting technologies, 2 people prefer hand-counted paper ballots, 13 prefer hand-marked optically-scanned, 3 prefer BMD-marked optical-scan-bubble ballots, 3 prefer BMD-marked bar-code ballots with human-readable summary, 1 prefers all-in-one BMD/scanners.

Considered barely acceptable or unacceptable are: hand-counted (9 people), BMD-marked bubble ballots (5), BMD-marked bar-codes (10), All-in-one (12), DRE+VVPAT (15), and paperless DRE (everybody).

Comments on the choice of in-the-polling place voting machines:

• The paper ballots must be subject to a risk-limiting audit to be acceptable
• [Joe Hall]  I want universally-usable, universally-verifiable, high-capacity voting systems that keep no state per-voter and that support single-ballot risk limiting audits with a minimal set of side channels
• CCOS preferred to PCOS; also need independent (not reliant on voting system) tally of the number of pieces of paper, to create a “ballot manifest”
• [Doug Jones]  I downgrade all technologies where proofreading is required because proofreading, on paper or a screen, is something we know people are not good at. While I dislike bar codes, I grudgingly rank them equal to other BMDs because they leave hard evidence in the hands of any auditors when they print summaries that disagree with their bar codes. I downgraded the BMD/scanner combination because you said ‘can hold’ in a way that might lead to ‘license to cheat‘ logic.
• Having done a lot of recounts you have to have the ballot in your hands to determine the voter’s intent – nothing else works in a recount situation
• BMD are critical to eliminate voter intent errors due to misinterpretation of hand markings.
• [Luther Weeks]  At least in my State if seems out of the question to recruit enough officials to hand count all ballots in polling places after every election, let alone recruit sufficient volunteer observers. Also election officials in our state argue that people cannot accurately count ballots by hand and claim to prove that in every post-election audit. We also suffer from very poor ballot security. So polling place optical scanning followed by sufficient audits would make it more difficult a statewide attack on our voting system to succeed.
• [Jeremy Epstein]  BMD with bar codes would be OK if there’s audits that the bar codes match the selection (which is how I rated it), but would be barely acceptable otherwise.
• I have not studied the BMD options in any detail, so I base my responses mostly on what I read. I’m assuming that BMD in the BMD+scanner option is only there for people with disabilities. I base my response on that, though I don’t rule out the possible but unlikely situation in which a person without disabilities uses the BMD.
• I moved from PA to OR, and there’s lots to love about OR’s vote-by-mail. Hand-marked paper ballots, voters have lots of time and space to mark and review their ballots, etc. downsides include: easy coercion and invasion of privacy, e.g., of a victim of domestic abuse possible undetectable ballot tampering by insiders at the Board of Elections. One of the weakest security links is the custody of the ballots from casting to counting. From that point of view the ideal is precinct voting on paper, counted publicly at the precinct. After the counting of the ballots, the results are public and confirmable.
• Having served as a chief election judge (head pollworker) in MD since 2004, I have witnessed the long lines caused by having a limited number of voting stations. We had paperless DREs until 2016. Hand-marked paper ballots scanned by a scanner that notifies the voter if marks cannot be read correctly is the best solution because we can inexpensively add ballot-marking spaces to accommodate peak demand. This year we even had voters marking their ballots while standing in line waiting for a voting booth, using the folder we issue as a privacy sleeve to keep their ballots secret. When the scanner notifies a voter that their ballot could not be read, they can bring it to the chief judges to “spoil” it and be re-issued a new ballot. In the process they learn how to mark it correctly in the future. In many cases just filling in the bubble over top of their X or checkmark solved the problem. Any voters who have a hard time marking the ballot correctly by hand may always use the BMD to mark their ballot.
• There is no need for bar codes given the availability of OCR-friendly fonts and the limited set of valid inputs (we’re not trying to read journal articles here; we know exactly who is supposed to be on the ballot, so this is NOT the general OCR problem). Meanwhile, bar codes require either another layer of auditing (very few places have good audit regimes) or a dubious verification mechanism using vendor equipment, or voters to bring bar-code scanners to the polling place, which is bad because it will encourage people to take pictures of their ballots. Vendors pushing bar codes seem to be doing so based on the same key factor as last time when they were pushing DRE’s, namely their convenience. But since we’re buying voting machines instead of humidifiers, we should start with security and privacy goals and all “features” should be explicitly evaluated with respect to security and privacy (or else what the heck are we doing here?).

A breakdown of the reasons for these opinions, regarding in-person voting:

“(For nondisabled voters) Some form of BMD is preferable to hand-marked optical-scan because it avoids the problem of ambiguous marks”
4 strongly agree, 10 disagree or strongly disagree.

“(For nondisabled voters) BMDs are preferable to hand-marked opscan ballots because then, disabled and conventionally abled voters are all using the same equipment”
4 strongly agree, 9 disagree or strongly disagree.

(For nondisabled voters) Hand-marked opscan ballots are preferable to BMD-marked ballots because of the danger that the BMD will cheat or malfunction and the voter might not notice wrong marks”
5 strongly agree, 7 agree, 3 disagree or strongly disagree.

“With precinct-count opscan of hand-marked ballots, the election administrator should configure the opscanner to notify the voter about overvotes/undervotes”
10 strongly agree, 4 agree, 2 disagree

“BMDs are preferable to hand-marked paper ballots because it avoids the problem of preprinting opscan ballot forms”
3 strongly agree or agree, 10 disagree or strongly disagree

“Answer the previous question, especially considering a scenario WITH vote centers”
1 strongly agree, 6 agree, 5 disagree, 3 strongly disagree

“Answer the previous question, for a state without vote centers, every voter assigned to just one precinct”
3 strongly agree or agree, 12 disagree or strongly disagree

“The cost and logistics of preprinted opscan ballots is just not a big deal”
9 agree or strongly agree, 4 disagree or strongly disagree

“BMDs are preferable to ballot-on-demand printers because they can use cheaper, lighter-weight, less power-hungry technology”
3 agree, 6 disagree or strongly disagree

“Hand-marked preprinted opscan ballots are preferable to ballot-on-demand or BMDs because they still work if all the technology fails in the polling place”
11 strongly agree, 4 agree, 1 disagree

“In my experience running or observing elections, ballot-on-demand printers work well”
1 strongly agree, 2 agree, 1 disagree, 1 strongly disagree

“Precinct-count opscan is preferable to central count because you get results as soon as the polls close, and before there are chain-of-custody issues with ballot boxes”
15 strongly agree or agree, 2 disagree

“Central-count opscan is preferable to precinct-count because it’s cheaper and easier to manage”
3 agree, 11 disagree or strongly disagree

“Central-count opscan is preferable to precinct-count because (at least with technology on the market now) it’s easier to do ballot-comparison audits”
1 strongly agree, 6 agree, 6 disagree or strongly disagree

“Central-count opscan will likely be easier to audit than precinct-count for the foreseeable future”
4 agree or strongly agree, 7 disagree or strongly disagree

“If a BMD or VVPAT were to change some of the voters’ selections on the printout, most voters would notice”
2 agree, 12 disagree or strongly disagree

“If a voter notices that a BMD has wrongly marked the ballot, it’s clear to the voter what to do”
3 agree or strongly agree, 12 disagree or strongly disagree

“If a voter believes that a BMD has wrongly marked a ballot, the pollworker usually knows what to do”
5 agree, 7 disagree or strongly disagree

“A hacked BMD could get away with wrongly marking some votes on some ballots, because the worst that happens is that if detected, just those ballots will be remarked and the rest of the cheating survives”
6 strongly agree, 5 agree, 2 disagree

Comments on the questions above:

• [Joe Hall]  I don’t think scanners should notify voters on undervotes, but should on overvotes, so I disagreed there because you combined them. And your notion of a BMD seems to be one that prints the ballot? I think the question on BMD/VVPAT changing votes depends on the interaction design, but certainly research using available examples shows people only rarely check and correct.
• The rate of ambiguous marks is negligible; CCOS scanners are more cost effective and more accurate
• [Doug Jones]  I have slowly come to the conclusion that the myth of ‘universal design’ is just that, a myth. ‘Accessible BMDs’ may be a single device that allows both disabled and able voters to vote, but the audio ballot is a different voting interface from the screen, and if you have to navigate with a sip and puff device, that’s a third interface. In reality, that BMD is really a bundle of 3 separate user interfaces, some of which might be well designed and some miserable. Bundling them into one mechanism is not necessarily appropriate.
• The two reasons for precinct-count (fast results and chain of custody) are of very different value. I value precinct-count because of its extremely strong chain of custody value. Posting results for the public at the precinct, sending the ballots and the electronic record and the printed record back to the county and modeming the electronic record. These separate copies of the record of what happened make it very hard for someone to alter the records in transit without being detected.
• I’ve done research on voters’ proofreading skills. A significant fraction of voters will not notice top-of-the-ballot alterations (I switched votes for McCain and Obama and 1/3 didn’t notice it), and mid-ballot changes are missed by most voters (a study at Rice found 2/3 missed random changes). Studies of VVPATs in Nevada showed similar rates of proofreading. Furthermore, in my study, when voters did notice switching of McCain and Obama, they blamed themselves for making the mistake and did not blame the voting machine.
• You did not specify if the central count option was due to a vote by mail administration or for some other reason. If it is an all vote by mail jurisdiction then of course you have to have a central count and the question makes no sense to me. If you do not have a vote by mail jurisdiction then there is no good reason to gather up ballots in precincts and ship them off somewhere. I don’t really understand what the question means?? This survey does not have an option “I do not know” so that is a huge problem for me – never having any actual experience to make a judgement from. Also, once you click on a circle this survery does not allow for unclicking – a huge problem for me and I do not want any of my responses to the last question to be included – I happened to click on it by mistake and discovered this flaw in the process
• [Dan McCrea]  PCOS scanners should notify of overvotes only – not undervotes (too much time/noise); BMDs for non-disabled voters is a vendor’s pipe dream and entirely unnecessary. ($4,000 BMD vs. $5 pen + accrued portion of booth); I object fundamentally to playing CCOS off PCOS because of audits – It’s a flawed problem because where people want to conduct effective audits, both can be made to work acceptably well.
• Vendors will quickly develop compliant devices if federal guidelines & state standards require precinct-count ballot-comparison audits compatibility.
• Precinct scanned opt-scan is much preferable to central scanning because it gives the voter the chance to correct over and under votes, and removes an opportunity for fraud during transport. Our state does not use vote centers (except with early absentee voting), so usually ballot on demand is not a big deal – but there are times and places where it would be very useful if state law allowed. I’m concerned about the opportunity for fraud with ballot on demand printing – there would have to be very strict procedures to track ballots printed. We reduced lines by moving from DRE to opt-scan since many voters can mark paper ballots in parallel. Pushing BMDs for most voters would add cost and lines when most voters could just use an inexpensive pen to mark a ballot.
• [Jeremy Epstein]  The answers depend on whether cost is a consideration – I’d love to have enough BMDs that no one has to wait for one, but that’s ridiculously expensive. In my small precinct (1800 voters) I’d need 8-10 of them, and it’s only that small a number because Virginia usually has very short ballots. Similarly, the answers on BMD marked depend on whether there’s an assumption that audits take place. For notification of under/overvotes, those two are quite different – definitely yes for overvotes, but the scanner will constantly alert on undervotes especially in places with long ballots. (In our most complex elections in Virginia, which are short compared to other states, we probably have 30% undervote on downballot items, which would paralyze the polling place if they all alerted.)
• [Paul Stokes]  Re: Central-count opscan will likely be easier to audit than precinct-count for the foreseeable future; my answer (Disagree) is based on having done statistical, evidence-based audits since 2010, based on comparing hand-counted and machine counted precinct tallies. We don’t find the hand-counting of the precinct results difficult. The overall logistics for hand-counting precinct results don’t seem to me to require a lot more resources that single ballot auditing at central locations when you factor in the reasonable assumptions that can be made for precinct-based auditing that reduce the number in the sample by something like a factor of four. And auditing precinct results has the functional advantage of verifying the vote tallies as well as the ballot content.
• BMDs are too expensive to buy, store, and maintain to make it practical to have enough equipment to meet peak demand, thereby causing long wait times. Voting systems should be easily expandable to accommodate peak demand without causing voters to have long waits to vote. Most counties cannot easily afford to spend huge sums on equipment that gets such limited usage and will be obsolete in just a few election cycles.
• Under/overvote warning – – yes for over votes, yes for blank pages, yes for 3-4 top of the ticket contests, no for other contests. Voters may notice changes more on BMDs than VVPATS. Studies should be to answer analyze this.
• Vote centers are an availability menace if they require real-time network connectivity to prevent one voter from voting in two vote centers. At least in PA, polling places are required to have a reasonable quantity of pre-printed “emergency ballots”, so anybody talking about how expensive it is to pre-print paper ballots for each polling place instead of “just” sending lots of BMDs should be making a comparison including this baseline paper cost. I can imagine a sensible saving-paper architecture based on “printing centers” in a few places in a county that can demand-print ballots for polling places that run low. That way if one on-demand printer breaks nobody is in trouble, which is not the case if each polling place has a cheap printer.

Opinions regarding mail-in ballots

“By-mail voting improves voter turnout”
no consensus

“By-mail voting seriously compromises the secret ballot”
15 strongly agree or agree, 1 disagree

“It still matters a lot whether we have coercion-resistant secret balloting”
ALL strongly agree or agree

“I am confident that ballots are not interfered with between the time they leave the voters’ hands and the time they are counted”
3 strongly agree or agree, 10 disagree or strongly disagree

“Chain-of-custody issues for mail-in ballots are different, but not inherently worse than, chain-of-custody for in-precinct ballot boxes”
5 strongly agree or agree, 11 disagree or strongly disagree

“(Whether or not I’m confident now), security of mail-in ballots on their way to being counted _could_ be made adequately secure”
2 agree, 9 disagree or strongly disagree

“Public observation of _all_ the steps in handling a mail-in ballot, is essential”
14 strongly agree or agree, 1 disagree

“Public observation of those steps is actually done in a jurisdiction I’m familiar with”
1 strongly agree, 9 disagree or strongly disagree

Comments on the questions above:

• Mail out, return in dropbox is better than mail out, mail back
• For those of us where there is a mix of precincts and mail-in there are many different ways to answer these – I need that “it depends” option
• [Dan McCrea]  Public observation of all the steps in elections requires not just accommodating regulation but a motivated public to do the observing. The latter tends to be sporadic, partly dependent on controversy, often absent.
• I don’t see how there can be public observation of all steps in handling a mail-in ballot. For example, we can’t observe inside the post office continuously during the mail-in period.
• Voting by mail tends to disenfranchise an already vulnerable population: those without a stable address, who tend to be young or elderly, poor, and non-white. Vote centers do not always make it easy to vote because they are fewer and may be farther away than a neighborhood polling place, requiring long rides and multiple transfers on public transportation. There are many ways to fail at voting by mail, including not signing oaths, not returning the ballot on time, over- or under-voting, not receiving a ballot or receiving the wrong ballot style, ballots stolen from apartment mailrooms, etc. It is very difficult to reliably authenticate remote voters, and fraud is more easily possible. Vote-buying/selling is also much more easily accomplished.
• The cost and loginstics of preprinting ballots should not be a big deal, but we see again and again, like in this election in Maryland, that it can be paramount.
• I would be interested in credible academic studies showing that vote-by-mail permanently improves turnout, as opposed to arguably providing a brief spike. Several years ago I looked for studies and what I saw sure looked like “brief novelty spike”.

Other comments:

• Disenfranchisement is a bigger issue than invalid ballots as of now.
• Literally *every* time I have talked to an immigrant (to PA) from a vote-by-mail state, I ask “How do you know that your ballot is received when you mail it?” and *every* time I am told “Oh, there is some way to check” [telephone or online], and *every* time I ask “Have you ever checked?” the answer is no. But the hypothetical safety of vote-by-mail is predicated on a *high* rate of people checking–again and again, in every election. So I think it would be interesting if vote-by-mail jurisdictions would publicly announce how many people checked whether their ballots arrived–interesting, but only in a “yeah, fat chance!” kind of way. And here we are discussing mere *arrival*, not tampering.

The respondents were,