October 6, 2022

Archives for 2018

Pilots of risk-limiting election audits in California and Virginia

In order to run trustworthy elections using hackable computers (including hackable voting machines), “elections should be conducted with human-readable paper ballots. … States should mandate risk-limiting audits prior to the certification of election results.

What is a risk-limiting audit, and how do you perform one? An RLA is a human inspection of a random sample of the paper ballots (or batches of ballots)—using a scientific method that guarantees with high confidence that if the voting machines claimed the wrong winner, then the audit will declare, “I cannot confirm this election,” in which case a by-hand recount is appropriate.  This is protection against voting-machine miscalibration, or against fraudulent hacks of the voting machines.

That’s what it is, but how do you do it?  RLAs require not only a statistical design, but a practical plan for selecting hundreds of ballots from among millions of sheets of paper.  It’s an administrative process as much as it is an algorithm.

In 2018, RLAs were performed by the state of Colorado.  In addition, two just-published reports describe pilot RLAs performed by Orange County, California and Fairfax, Virginia.  From these reports (and from the audits they describe) we can learn a lot about how RLAs work in practice.

Orange County, CA Pilot Risk-Limiting Audit, by Stephanie Singer and Neal McBurnett, Verified Voting Foundation, December 2018.

Neal Kelley, Registrar of Voters of Orange County, ran an RLA of 3 county-wide races in the June 2018 primary, with assistance from Verified Voting.  About 635,000 ballots were cast; many ballots were 3 pages long (printed both sides), about 1.4 million sheets overall.  Of these, just 160 specific (randomly selected) ballot sheets  needed to be found and tabulated by human inspection.  How do you manage a million sheets of paper?

Orange County elections warehouse during the June 2018 risk-limiting audit

Like this!  Keep well organized ballot manifests that list each batch of ballots (that were initially counted by optical scanners), where they came from, how many ballots.  How do you know how many ballots are in each batch?  The optical scanners tell you, but you don’t want to trust the optical scanners (a hacked scanner could influence the audit by lying about how many ballots are in a batch).  So you weigh the batch on a high-precision scale, that tells you ±2 sheets.  And so on.   You can read the details in the report, which really helps to demystify the process.   Still, there are many ways of doing an RLA, and this report describes just one of them.  The audit was finished before the deadline for certifying election results.  The estimated salary cost of the staff of the Registrar of Voters, for the days running the audit, was under $4000.

City of Fairfax,VA Pilot Risk-Limiting Audit, by Mark Lindeman, Verified Voting Foundation, December 2018.

Brenda Cabrera, General Registrar of the City of Fairfax, ran a pilot RLA of the June 12th 2018 Republican primary Senate election, with assistance from Verified Voting.  There were 948 ballots cast, and the audit team ran the audit three ways, to test three different RLA methods.   The audit was scheduled to take two days but finished ahead of schedule.

Colorado ran statewide RLAs of its 2018 primary and general elections, after pilot projects in previous years.

From all these activities we continue to learn more about how to run trustworthy elections.  I encourage state and local election officials nationwide to try RLA pilots of their own.  The Verified Voting Foundation, Democracy Works, the Democracy Fund, Free and Fair, and other individuals and organizations are available to provide advice.

 

 

Why voters should mark ballots by hand

Because voting machines contain computers that can be hacked to make them cheat, “Elections should be conducted with human-readable paper ballots. These may be marked by hand or by machine (using a ballot-marking device); they may be counted by hand or by machine (using an optical scanner).  Recounts and audits should be conducted by human inspection of the human-readable portion of the paper ballots.

Ballot-marking devices (BMD) contain computers too, and those can also be hacked to make them cheat.  But the principle of voter verifiability is that when the BMD prints out a summary card of the voter’s choices, which the voter can hold in hand before depositing it for scanning and counting, then the voter has verified the printout that can later be recounted by human inspection.

 

ExpressVote ballot card, with bar codes for optical scanner and with human-readable summary of choices for use in voter verification and in recount or audit.

 

 

 

 

 

 

 

But really?  As a practical matter, do voters verify their BMD-printed ballot cards, and are they even capable of it?  Until now, there hasn’t been much scientific research on that question.

A new study by Richard DeMillo, Robert Kadel, and Marilyn Marks now answers that question with hard evidence:

  1. In a real polling place, half the voters don’t inspect their ballot cards, and the other half inspect for an average of 3.9 seconds (for a ballot with 18 contests!).
  2. When asked, immediately after depositing their ballot, to review an unvoted copy of the ballot they just voted on, most won’t detect that the wrong contests are presented, or that some are missing.

This can be seen as a refutation of Ballot-Marking Devices as a concept.  Since we cannot trust a BMD to accurately mark the ballot (because it may be hacked), and we cannot trust the voter to accurately review the paper ballot (or even to review it at all), what we can most trust is an optical-scan ballot marked by the voter, with a pen.  Although optical-scan ballots aren’t perfect either, that’s the best option we have to ensure that the voter’s choices are accurately recorded on the paper that will be used in a recount or random audit. [Read more…]

CITP Call for Visitors for 2019-20

The Center for Information Technology Policy is an interdisciplinary research center at Princeton University that sits at the crossroads of engineering, the social sciences, law, and policy.

CITP seeks applicants for various visiting positions each year. Visitors are expected to live in or near Princeton and to be in residence at CITP on a daily basis. They will conduct research and participate actively in CITP’s programs.

For all visitors, we are happy to hear from anyone working at the intersection of digital technology and public life, including experts in computer science, sociology, economics, law, political science, public policy, information studies, communication, and other related disciplines.

Visitors

All visitors must apply online through the links below. There are three job postings for CITP visitors: 1) the Microsoft Visiting Researcher Scholar/Professor of Information Technology Policy, 2) Visiting IT Policy Fellow, and 3) IT Policy Researcher.

Microsoft Visiting Research Scholar/Professor of Information Technology Policy

The successful applicant must possess a Ph.D. and will be appointed to a ten-month term, beginning September 1st. The visiting professor must teach one course in technology policy per academic year. Preference will be given to current or past professors in related fields and to nationally or internationally recognized experts in technology policy.

Full consideration of the Microsoft Visiting Research Scholar/Professor of Information Technology Policy position is given to those who apply by the end of December for the upcoming year.

Apply here to become the Microsoft Visiting Research Scholar/Visiting Professor of Information Technology Policy


Visiting IT Policy Fellow

A Visiting IT Policy Fellow is on leave from a full-time position (for example, a professor on sabbatical). The successful appliant must possess an advance degree and typically will be appointed to a nine-month term, beginning September 1st.

Full consideration for the Visiting IT Policy Fellow is given to those who apply by the end of December for the upcoming year.

Apply here to become a Visiting IT Policy Fellow


IT Policy Researcher

An IT Policy Researcher will have Princeton University as the primary affiliation during the visit to CITP (for example, a postdoctoral researcher or a professional visiting for a year between jobs). The successful applicant must possess a Ph.D. or equivalent and typically will be appointed to a 12-month term, beginning September 1st.

This year we are also looking for a postdoctoral fellow to work on bias in AI in collaboration with an interdisciplinary team: Arvind Narayanan and Olga Russakovsky at Princeton and Kate Crawford at the AI Now institute NYU. We are interested in developing techniques for recognizing, mitigating and governing bias in computer vision and other modern areas of AI that are are characterized by massive datasets and complex, deep models. If you are interested specifically in this opening, please mention it in your cover letter.

Full consideration for the IT Policy Researcher positions is given to those who apply by the end of December for the upcoming year.

Apply here to become an IT Policy Researcher


Applicants should apply to either the Visiting IT Policy Fellow position (if they will be on leave from a full-time position) or the IT Policy Researcher position (if not), but not both positions; applicants to either position may also apply to be the Microsoft Visiting Research Scholar/Professor if they hold a Ph.D.

All applicants should submit a current curriculum vitae, a research plan (including a description of potential courses to be taught if applying for the visiting professor), and a cover letter describing background, interest in the program, and any funding support for the visit. References are not required until finalists are notified. CITP has secured limited resources from a range of sources to support visitors. However, many of our visitors are on paid sabbatical from their own institutions or otherwise provide some or all of their own outside funding.

Princeton University is an Equal Opportunity/Affirmative Action Employer and all qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity or expression, national origin, disability status, protected veteran status, or any other characteristic protected by law.

All offers and appointments are subject to review and approval by the Dean of the Faculty.


If you have any questions about any of these positions or the application process, please feel free to contact us at .

Expert opinions on in-person voting machines and vote-by-mail

In November 2018 I got opinions on voting machines and vote-by-mail from 17 experts on election verification, who have experience running/observing/studying elections in 17 states.

On the acceptability of these in-the-polling-place voting technologies, in the context of U.S. elections:

The consensus is that Direct Recording Electronic voting machines are unacceptable, even with a VVPAT (“voter verified paper audit trail visible to the voter under glass”).  Most experts are lukewarm to hand-counted paper ballots, presumably because they’re impractical for large elections with many contests on the ballot.  Most experts prefer hand-marked optical scan ballots, and all of these experts find hand-marked optical scan acceptable.  Most experts are willing to accept ballot marking devices (BMDs) that prepare “bubble ballots” to be scanned by optical scan machines, but only 17% find this preferable to hand-marked optical-scan ballots.  Opinion is mixed on BMDs that prepare bar-code ballots (with human-readable summaries) for tabulation by optical scanners, with most finding this  technology at least “barely acceptable.”  Almost no one prefers all-in-one machines that combine ballot marking and ballot scanning (but at least the voter can hold the ballot in her hand while inspecting it), with about a 50/50 split between “acceptable” and “barely acceptable”.

Most experts don’t prefer ballot-marking devices (BMDs) for these reasons:

  1. If the paper jams, the power fails, or something else goes wrong with technology, voters using hand-marked paper ballots can still deposit their ballots in an emergency ballot for counting later; this is not an option with a BMD-only solution.
  2. BMDs are more susceptible to fraud: if a BMD wrongly marks a paper ballot, (studies have shown that) most voters won’t notice.
  3. BMDs cost $5000, pens cost 50c; it is expensive to supply enough BMDs for all voters, but it is feasible to supply BMDs sufficient for those voters unable to mark a paper ballot by hand.

A few experts (17%) prefer BMD-marked ballots to hand-marked ballots because (1) there’s no chance of ambiguous marks and (2) it’s easier to give voters feedback about undervotes/overvotes.

Regarding vote-by-mail:   There is no consensus on whether vote-by-mail increases voter turnout. Almost all the experts agree that vote-by-mail seriously compromises the secret ballot, and that it still matters whether we have coercion-resistant secret balloting.  Most experts are not confident that ballots are not interfered with between the time they leave the voters’ hands and the time they are counted, and are not confident the chain of custody for mail-in ballots could be made adequately secure.  The experts agree that it is essential to have public observation of all the steps in handling mail-in ballots, but almost none of the experts believes that there is adequate public observation in their own jurisdictions.That is not to say that the experts are against vote-by-mail; it’s just that there are some issues that ought to be discussed and improved.

[Read more…]

Florida is the Florida of ballot-design mistakes

Well designed ballot layouts allow voters to make their intentions clear; badly designed ballots invite voters to make mistakes.  This year, the Florida Senate race may be decided by a misleading ballot layout—a layout that violated the ballot design recommendations of the Election Assistance Commission.

In Miami Palm Beach, Florida in the year 2000, the badly designed “butterfly ballot” misled over 2000 voters who intended to vote for Al Gore, to throw away their vote.  (That’s a strong statement, but it’s backed up by peer-reviewed scientific analysis.)

In Sarasota, Florida in the year 2006, in a Congressional race decided by 369 votes, over 18,000 voters failed to vote in that race, almost certainly because of a badly designed touch-screen ballot layout.

In Broward County, Florida in the year 2018, it appears that a bad optical-scan ballot design caused over 26,000 voters to miss voting in the Senate race, where the margin of victory (as of this writing, not yet final) is 12,562 votes.

[Read more…]