October 6, 2022

Archives for November 2018

CITP Call for Visitors for 2019-20

The Center for Information Technology Policy is an interdisciplinary research center at Princeton University that sits at the crossroads of engineering, the social sciences, law, and policy.

CITP seeks applicants for various visiting positions each year. Visitors are expected to live in or near Princeton and to be in residence at CITP on a daily basis. They will conduct research and participate actively in CITP’s programs.

For all visitors, we are happy to hear from anyone working at the intersection of digital technology and public life, including experts in computer science, sociology, economics, law, political science, public policy, information studies, communication, and other related disciplines.


All visitors must apply online through the links below. There are three job postings for CITP visitors: 1) the Microsoft Visiting Researcher Scholar/Professor of Information Technology Policy, 2) Visiting IT Policy Fellow, and 3) IT Policy Researcher.

Microsoft Visiting Research Scholar/Professor of Information Technology Policy

The successful applicant must possess a Ph.D. and will be appointed to a ten-month term, beginning September 1st. The visiting professor must teach one course in technology policy per academic year. Preference will be given to current or past professors in related fields and to nationally or internationally recognized experts in technology policy.

Full consideration of the Microsoft Visiting Research Scholar/Professor of Information Technology Policy position is given to those who apply by the end of December for the upcoming year.

Apply here to become the Microsoft Visiting Research Scholar/Visiting Professor of Information Technology Policy

Visiting IT Policy Fellow

A Visiting IT Policy Fellow is on leave from a full-time position (for example, a professor on sabbatical). The successful appliant must possess an advance degree and typically will be appointed to a nine-month term, beginning September 1st.

Full consideration for the Visiting IT Policy Fellow is given to those who apply by the end of December for the upcoming year.

Apply here to become a Visiting IT Policy Fellow

IT Policy Researcher

An IT Policy Researcher will have Princeton University as the primary affiliation during the visit to CITP (for example, a postdoctoral researcher or a professional visiting for a year between jobs). The successful applicant must possess a Ph.D. or equivalent and typically will be appointed to a 12-month term, beginning September 1st.

This year we are also looking for a postdoctoral fellow to work on bias in AI in collaboration with an interdisciplinary team: Arvind Narayanan and Olga Russakovsky at Princeton and Kate Crawford at the AI Now institute NYU. We are interested in developing techniques for recognizing, mitigating and governing bias in computer vision and other modern areas of AI that are are characterized by massive datasets and complex, deep models. If you are interested specifically in this opening, please mention it in your cover letter.

Full consideration for the IT Policy Researcher positions is given to those who apply by the end of December for the upcoming year.

Apply here to become an IT Policy Researcher

Applicants should apply to either the Visiting IT Policy Fellow position (if they will be on leave from a full-time position) or the IT Policy Researcher position (if not), but not both positions; applicants to either position may also apply to be the Microsoft Visiting Research Scholar/Professor if they hold a Ph.D.

All applicants should submit a current curriculum vitae, a research plan (including a description of potential courses to be taught if applying for the visiting professor), and a cover letter describing background, interest in the program, and any funding support for the visit. References are not required until finalists are notified. CITP has secured limited resources from a range of sources to support visitors. However, many of our visitors are on paid sabbatical from their own institutions or otherwise provide some or all of their own outside funding.

Princeton University is an Equal Opportunity/Affirmative Action Employer and all qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity or expression, national origin, disability status, protected veteran status, or any other characteristic protected by law.

All offers and appointments are subject to review and approval by the Dean of the Faculty.

If you have any questions about any of these positions or the application process, please feel free to contact us at .

Expert opinions on in-person voting machines and vote-by-mail

In November 2018 I got opinions on voting machines and vote-by-mail from 17 experts on election verification, who have experience running/observing/studying elections in 17 states.

On the acceptability of these in-the-polling-place voting technologies, in the context of U.S. elections:

The consensus is that Direct Recording Electronic voting machines are unacceptable, even with a VVPAT (“voter verified paper audit trail visible to the voter under glass”).  Most experts are lukewarm to hand-counted paper ballots, presumably because they’re impractical for large elections with many contests on the ballot.  Most experts prefer hand-marked optical scan ballots, and all of these experts find hand-marked optical scan acceptable.  Most experts are willing to accept ballot marking devices (BMDs) that prepare “bubble ballots” to be scanned by optical scan machines, but only 17% find this preferable to hand-marked optical-scan ballots.  Opinion is mixed on BMDs that prepare bar-code ballots (with human-readable summaries) for tabulation by optical scanners, with most finding this  technology at least “barely acceptable.”  Almost no one prefers all-in-one machines that combine ballot marking and ballot scanning (but at least the voter can hold the ballot in her hand while inspecting it), with about a 50/50 split between “acceptable” and “barely acceptable”.

Most experts don’t prefer ballot-marking devices (BMDs) for these reasons:

  1. If the paper jams, the power fails, or something else goes wrong with technology, voters using hand-marked paper ballots can still deposit their ballots in an emergency ballot for counting later; this is not an option with a BMD-only solution.
  2. BMDs are more susceptible to fraud: if a BMD wrongly marks a paper ballot, (studies have shown that) most voters won’t notice.
  3. BMDs cost $5000, pens cost 50c; it is expensive to supply enough BMDs for all voters, but it is feasible to supply BMDs sufficient for those voters unable to mark a paper ballot by hand.

A few experts (17%) prefer BMD-marked ballots to hand-marked ballots because (1) there’s no chance of ambiguous marks and (2) it’s easier to give voters feedback about undervotes/overvotes.

Regarding vote-by-mail:   There is no consensus on whether vote-by-mail increases voter turnout. Almost all the experts agree that vote-by-mail seriously compromises the secret ballot, and that it still matters whether we have coercion-resistant secret balloting.  Most experts are not confident that ballots are not interfered with between the time they leave the voters’ hands and the time they are counted, and are not confident the chain of custody for mail-in ballots could be made adequately secure.  The experts agree that it is essential to have public observation of all the steps in handling mail-in ballots, but almost none of the experts believes that there is adequate public observation in their own jurisdictions.That is not to say that the experts are against vote-by-mail; it’s just that there are some issues that ought to be discussed and improved.

[Read more…]

Florida is the Florida of ballot-design mistakes

Well designed ballot layouts allow voters to make their intentions clear; badly designed ballots invite voters to make mistakes.  This year, the Florida Senate race may be decided by a misleading ballot layout—a layout that violated the ballot design recommendations of the Election Assistance Commission.

In Miami Palm Beach, Florida in the year 2000, the badly designed “butterfly ballot” misled over 2000 voters who intended to vote for Al Gore, to throw away their vote.  (That’s a strong statement, but it’s backed up by peer-reviewed scientific analysis.)

In Sarasota, Florida in the year 2006, in a Congressional race decided by 369 votes, over 18,000 voters failed to vote in that race, almost certainly because of a badly designed touch-screen ballot layout.

In Broward County, Florida in the year 2018, it appears that a bad optical-scan ballot design caused over 26,000 voters to miss voting in the Senate race, where the margin of victory (as of this writing, not yet final) is 12,562 votes.

[Read more…]

Two cheers for limited democracy in New Jersey

When I voted last week in Princeton, New Jersey, here were the choices I faced, all on one “page”:

I had to vote in 7 contests, total: for Senator, Congress(wo)man, County board, County board unexpired term, City council, Statewide referendum, School board.  Put another way, I had to select 13 choices out of 27 options (not counting write-in options).  The ballot is so short because: New Jersey elects its governor and legislature in odd-numbered years, does not have initiative-and-referendum by petition, does not elect judges.

In contrast, a voter in Los Angeles was given a 76-page packet, in which the 9-page ballot contained optical-scan bubbles to fill in.  The voter had to select 55 choices out of 229 options.

The founders of our democracy designed a Constitution in which (at the Federal level) voters elect representatives and executives who pass legislation, nominate and confirm judges, and so on.  That is, we have limited democracy, in the sense that it is representative.  But some States, especially California, ask voters to decide legislative questions (“propositions”) and elect judges.  Political scientists and informed citizens debate whether that’s a good idea.

Here I’ll consider a particular aspect of that constitutional difference:  Auditability of elections.  It is a clear scientific consensus, and it is becoming a consensus among the citizenry and the States, that we should vote with paper ballots that are recountable by human inspection, and we should have random audits by human inspection of (a random sample of) those paper ballots, just to make sure the voting machines are not malfunctioning or cheating.

Random audits take time, effort, and money–not an enormous cost, but not trivial either.  The cost and difficulty of risk-limiting audits surely increases as the number of contests on the ballot increases.  Auditing New Jersey’s 7 contests (more or less, in different towns) will not be very difficult.  Auditing Los Angeles’s 52 contests will be quite a chore.  Surely that’s one argument for keeping the ballot short.

In practice, risk-limiting audits of California elections are likely to be done in such a way that Federal, statewide, and countywide contests are checked to a specified risk limit (perhaps 5%), but on all ballots examined, all contests are audited.  That will audit many more contests to a risk limit that is not predetermined, and may or may not be small, but will at least be reported as a result of the audit.  In that manner, even LA’s long ballot can be audited.

Why only “two cheers” for New Jersey?  Well, in New Jersey we have “limited democracy” in a different form as well:  we use paperless direct-recording electronic (DRE) voting machines.  The computers in those machine get to decide what to report about the buttons we pressed, and there are no paper ballots to recount.  We have delegated our representation to whomever was the last to install a computer program in those machines, whether legimitately or illegitimately.  Surely that’s not what the founders intended by “representative democracy.”



When the optical scanners jam up, what then?

In the November 2018 election, many optical-scan voting machines in New York experienced problems with paper jams, caused by the rainy weather and excessive humidity.

Also, this was the first time New York used a 2-page ballot that the voter had to separate at the perforations.  This doubled the number of sheets of paper that the optical scanners had to process.

These two factors caused long lines, and voter frustration, at some polling places.  At some polling places, there were not adequate “emergency ballot boxes” for deposit of not-yet-scanned paper ballots.

New York, like many other states, uses a robust, trustworthy, and reliable means of balloting:  optical-scan paper ballots, hand-marked by the voters (except for those voters who choose to use a ballot-marking device), which the voter deposits directly into an optical scanner.  That is, “precinct-count optical scan” (PCOS).

No voting method is perfect, but PCOS is less imperfect than other methods.  Here are some important principles of precinct-count optical scan:

  1. Feedback: if the voter inadvertently overvotes the ballot (marks too many bubbles in the same contest), the scanner can alert the voter to this problem, giving the voter the chance to correct it by filling in a fresh ballot.
  2. Immediate count:  vote totals are reported as soon as the polls close.  Unofficial (but informative) precinct totals can be reported immediately to the county, to the news media, and to members of the public present at the polling place.  Also, there’s at least one count of the votes before any transportation or handling of ballot boxes.  The paper ballots are legally the ballot of record for recounts, with random audits of paper ballots necessary to detect and deter cheating via hacking of optical scanner software.  But if there is interference with the paper ballots in the chain of custody between the precinct and the audit or recount, the in-the-precinct totals are at least evidence that an investigator or court of law might find useful.
  3. Robustness:  if the power fails, or the optical scanner fails for some other reason, voters can still hand-mark their optical-scan ballots, and deposit them into a ballot box for later counting.

You might notice that “deposit into a ballot box for later counting“ conflicts with ”feedback” and “immediate count.”  What should we do about that?

[Read more…]