September 19, 2020

Archives for 2020

Election Audits in NJ 2020

It has been well understood for more than 15 years that computerized voting machines can be hacked to make them cheat (or might be misconfigured by accident), and therefore it is essential to have random audits of the ballots. That is: Human inspection of paper ballots that the voters marked, of a random sample of the ballots or ballot-boxes (batches); so that if the computers had been hacked (or misconfigured) to produce the wrong outcome, there is a good chance (statistically speaking) of catching and correcting the error.

New Jersey’s Legislature passed such an audit law in 2007, shortly after they passed a law requiring paper ballots. The paper ballot law was “suspended” in 2008–most counties still use paperless touchscreen voting machines–but the audit law remains in force. So, New Jersey’s law requires the audits of paper ballots that don’t exist.

In 2019, things started to change: three or four counties purchased voting machines with a paper trail, and the Division of Elections (correctly) determined that those counties must perform audits.

The gold standard for election audits is the Risk-Limiting Audit, a class of methods that combines high assurance with high efficiency. New Jersey’s law does not prescribe RLAs, because back in 2007 those were not well understood. New Jersey’s audit law was pretty good for its time, but RLAs are more effective and it would be a good idea to update the law.

In 2019, the Division of Elections did some “pilot audits,” using the RLA method on three or four county-level elections in those counties that had paper-ballot equipment. These were for the purpose of training local election administrators in how RLAs work, learning about the process, adapting RLAs to New Jersey’s equipment and regulations, and so on.

In 2020, things changed a lot more: the November 2020 general election will be conducted almost entirely with hand-marked paper ballots, as a public-health measure during the COVID-19 pandemic. The vast majority of ballots will be mail-in ballots (that can returned, in their signed envelopes, either by U.S. mail, in county drop-boxes, or at polling places). Voters who wish to may vote in-person on November 3rd at a polling place, using a provisional ballot (also a hand-marked paper ballot). (Provisional, to give election administrators the ability to ensure that nobody votes by mail and in person.) A tiny proportion of votes will be on paperless DREs that have disability accommodations, for voters who cannot mark a paper ballot and do not wish to vote with assistance on a paper ballot.

So, for the first time in more than a century, New Jersey will have a (practically) all-paper-ballot election, and you might think that the 2007 audit law will finally come into force. And apparently the Governor thinks so as well!

Governor Phil Murphy’s Executive Order 177, dated August 14th (in the 245th year of the Independence of the United States) outlines the many emergency procedures that will accommodate this year’s election to the pandemic. Some of the more major components of this EO were (2 weeks later) passed as laws by the Legislature.

Among the provisions of the Order are that “to allow enough time for results to be certified prior to the meeting of electors, N.J.S.A. 19:61-9(c)(8) is suspended, and counties may certify their election results prior to the commencement of the election audit required in N.J.S.A. 19:61-9.”

That is, normally the audit must be complete before election results are certified. (This is essential, to make audits meaningful.) This year, that deadline is suspended. But the rest of the Audit statute is not suspended, including, “Within a reasonable period of time after the final vote count after an election, the Attorney General, with the audit team, shall determine and then announce publicly the election districts in the State in which audits shall be conducted, and within 24 hours of that announcement, the audit shall be commenced.” That is, the audit must be commenced “within a reasonable period of time.”

Certification of election results must be done by November 20th, in part so that New Jersey can meet its Federal “safe harbor” deadline of December 8th to choose Electors. Because of the unprecedented (for New Jersey) all-by-mail-and-provisional-ballot election, county election officials may need all of those days between November 3rd and November 19th to finish counting the votes, and may not have time to do audits before November 20th.

The Secretary of State has informed county election officials that the audits shall take place between November 23 and December 4. Normally I would like to see the audits completed before results are certified, as the law requires. But this seems like a reasonable compromise in light of the pandemic. Completing the audits by December 4th means:

  • If the audit uncovered anything drastically wrong in the Presidential election, there’s still time to do something before December 8th.
  • If the audit uncovered anything drastically wrong in any other election, there are several weeks before Inauguration day (for Congressional offices), and recounts can be done.

New Jersey’s audit law could still use some updating, but audits using the current law are far better than no audits. I hope that in future years, New Jersey continues to use all-hand-marked paper ballots; that in future years, the timetable for counting votes allows audits before certification as the law requires; and that (based on experience with statutory audits and pilot RLAs) the Division of Elections recommends to the Legislature that the law be updated to more modern methods.

And one more thing: The statute requires that the chief election official shall appoint an independent audit team to design the audit, and the “procedures and assumptions shall be published prior to any given election, and the public shall have the opportunity to comment thereon.” If the audit team has been appointed, I urge them to comply with the law and publish their procedures soon.

Voting by mail in NJ 2020

For hundreds of years, New Jersey voters have voted in their local precinct polling places (800 registered voters per precinct), with only a tiny percentage voting absentee. This year, for reasons of public health in the pandemic, all voters will receive a mail-in ballot; a few polling places will be open on November 3rd for voters who need other accommodations or to vote by provisional ballot.

Thus, New Jersey has had to implement in 6 months what some western states did over a many-year period: switch to all vote-by-mail. Some of those states have developed procedures and know-how to do it very well; can New Jersey catch up, without being overwhelmed?

I spoke to Nicole DiRado, Administrator of the Union County (NJ) Board of Elections, whose office handles the mail-in ballots. Most years, that’s a few thousand ballots that they process on election day. This year it will be very different.

New this year in NJ are drop boxes.  Union County will ultimately have a drop box in every municipality.  Bipartisan teams of Board of Elections employees will collect ballots from every drop box, every day, in the 45 days before the election.  They will also collect their U.S. mail twice a day once voters start returning their ballots.

Each day’s collection is received at the BoE offices in Elizabeth, NJ, for “staging”.  Members of the public can, in principle, observe this process from the “public” side of the walk-up counter.  Staging includes:  sorting by municipality and ward/district;  comparing signatures with the State Voter Registration System.  For signature matching, BoE workers have access to a “signature history” from that voter, from DMV records and from voter-registration records, but not from every previous election (because NJ’s pollbooks are not electronic).    After staging, the ballot envelopes go into the vault.

New this year is ballot tracking offered on the NJ Division of Elections’ website.  The tracking numbers are not USPS tracking–they can’t tell you where inside the U.S. mail your ballot is–but the tracking system can tell the voter:  when the County Clerk cleared the absentee ballot for mailing to the voter; when it was received back from the voter by the BoE; whether the ballot was accepted or not.  (The tracking system does not seem to say when exactly the County Clerk mailed the ballot to the voter.)

When Election Board Commissioners reject a ballot (due to a deficiency in signature), the voter is contacted by U.S. mail.  (By law, the Commissioners include two Democrats and two Republicans.) The voter is mailed a form to fill out and sign (with the ability to provide other identifying information), and return by U.S. mail.   I asked, “can the voter drop that form into one of the drop boxes”?  Ms. DiRado responded, “I would certainly accept that”, but it didn’t seem to be a formal statewide policy.   She said she has accepted, through drop-boxes, voter registration forms and requests for absentee ballots.  (This year in NJ, absentee ballots will be mailed out even if the voter doesn’t request it.)  

Receiving and staging of ballots begins well over a month before November 3rd.  From time to time there are “inspection periods”, where members of the public can inspect the ballot envelopes to challenge a ballot.  The first such inspection period (in Union County) is October 9th.  After each inspection period (for example, on October 10th), Union County’s ballots are transported to a facility in Linden, NJ. Because the BoE needs a lot more space for a lot more workers to process the ballots, they have acquired additional space for that part of the process (see video).

First, the perforated tab with voter-identifying information is removed from the outer envelope — but the envelope is not yet opened.   Credentialed challengers can observe this process in person, other members of the public from a live-stream video.

In late August 2020, the NJ Legislature passed (and the governor signed) 3 bills regarding election procedures.   Now,  starting 10 days before November 3rd, the envelopes can be opened and run through the scanners.  Ms. DiRado said that her staff will open envelopes and flatten ballots, but will likely wait just a few days before election day to begin running them through the scanners.  Opening and prepping the ballots is far more labor-intensive and will take much longer than running them through the county’s high-speed scanners.  Members of the public can observe all of these processes at the Linden facility (as above: credentialed challengers in person, others on live-stream).  There will be Sheriff’s officers to ensure  that the challengers don’t interfere with the procedures.

The BoE will have several safeguards in place to avoid premature leaks of vote counts.  With every batch of ballots, the machine will report how many ballots are in the batch (and then the ballot papers, but not the votes on them, will be hand-counted to make sure it matches); but the vote totals will be retained in the machine’s memory and not reported until an explicit report is run, on November 3rd.  No one is authorized to run that report before November 3rd.  The optical scanners log any such reports, and the State will audit those logs after the election, to make sure no unauthorized reports were run.

I went online at  to track my ballot.   To log in to that site, voters need to provide their name, DoB, and a number.  Voters who registered after 2005 using a driver’s license as ID, can use their driver’s license number.  Voters who put a SSN on their voter registration, can use the last 4 digits.  Other voters will have to use their Voter ID number — but nobody knows their own Voter ID number, unless they happen to have saved old sample ballots or voter-registration cards.  This is going to be a problem!  The County Clerk will mail every voter a postcard with this info; or voters can contact their county election officials (link provided on the tracking-system login page) to for help with this.  Ms. DiRado helpfully looked up my number and provided it to me.   I logged into the system and it says my “General Election Mail-in Ballot Request received date” is 8/14/2020.  I didn’t request a ballot, so I assume that’s the date my County Clerk requested mail-in ballots for every voter in Mercer County.   The “Request processed date” is 8/30/2020.  There is no such field as “Ballot mailed to voter date” (and I think that would be worthwhile).  As of September 11th, I had not yet received a ballot in the mail.  The “Ballot received date” is N/A (which is good because I haven’t sent it back yet!) and “Ballot status” is N/A. I can’t tell whether “Ballot status” will track, in a timely way, whether the signature has been accepted.

In summary: I am optimistic that New Jersey is doing a good job in getting its act together in a hurry–from the Legislature and Governor down to the County Boards of Elections (at least Union County, anyway). What voters should do is use those dropboxes to return your ballots, or if you must mail in your ballot, do so as early as you can.

By the way, official government agencies such as the New Jersey Division of Elections shouldn’t use .org domain names like, they should use .gov instead. Anyone can get a .org domain name, but only authenticated governmental entities can get a .gov. Therefore, using would be more secure: fraudsters can try to fool voters by setting up, but they have a much harder time creating fake .gov domains.

GPT-3 Raises Complex Questions for Philosophy and Policy

GPT-3, a powerful, 175 billion parameter language model developed recently by OpenAI, has been galvanizing public debate and controversy. As the MIT Technology Review puts it: “OpenAI’s new language generator GPT-3 is shockingly good—and completely mindless”. Parts of the technology community hope (and fear) that GPT-3 could brings us one step closer to the hypothetical future possibility of human-like, highly sophisticated artificial general intelligence (AGI). Meanwhile, others (including OpenAI’s own CEO) have critiqued claims about GPT-3’s ostensible proximity to AGI, arguing that they are vastly overstated.

            Why the hype? GPT-3 is unlike other natural language processing (NLP) systems, the latter of which often struggle with what comes comparatively easily to humans: performing entirely new language tasks based on a few simple instructions and examples. Instead, NLP systems usually have to be pre-trained on a large corpus of text, and then fine-tuned in order to successfully perform a specific task. GPT-3, by contrast, does not require fine tuning of this kind: it seems to be able to perform a whole range of tasks reasonably well, from producing fiction, poetry, and press releases to functioning code, and from music, jokes, and technical manuals, to “news articles which human evaluators have difficulty distinguishing from articles written by humans”.

            GPT-3 raises a number of deep questions, which tie into long-standing debates in various subfields of philosophy (from epistemology and the philosophy of mind to aesthetics, and from moral, social, and political philosophy to the philosophy of language). In a recently published discussion symposium, nine philosophers (Amanda Askell, David Chalmers, Justin Khoo, Carlos Montemayor, C. Thi Nguyen, Regina Rini, Henry Shevlin, Shannon Vallor, and myself) explore the philosophical and policy implications of GPT-3.

            As I argue in my essay “If You Can Do Things with Words, You Can Do Things with Algorithms”, GPT-3 is indeed ‘shockingly good’ at performing some tasks, “but on the other hand, GPT-3 is predictably bad in at least one sense: like other forms of AI and machine learning, it reflects patterns of historical bias and inequity. GPT-3 has been trained on us—on a lot of things that we have said and written—and ends up reproducing just that, racial and gender bias included. OpenAI acknowledges this in their own paper on GPT-3,where they contrast the biased words GPT-3 used most frequently to describe men and women, following prompts like “He was very…” and “She would be described as…”. The results aren’t great. For men? Lazy. Large. Fantastic. Eccentric. Stable. Protect. Survive. For women? Bubbly, naughty, easy-going, petite, pregnant, gorgeous. This is not purely a tangibly material distributive justice concern: especially in the context of language models like GPT-3, paying attention to other facets of injustice—relational, communicative, representational, ontological—is essential.” As important earlier work on NLP tools—notably by Aylin Caliskan, Joanna Bryson and Arvind Narayanan—has shown, social norms and practices affect the ways in which linguistic concepts underpinning these tools are defined and operationalized.

            This problem space has important implications for policy-making in this area. I argue that “our aim should be to engineer conceptual categories that mitigate conditions of injustice rather than entrenching them further. We need to deliberate and argue about which social practices and structures—including linguistic ones—are morally and politically valuable before we automate and there by accelerate them.”

            Relatedly, GPT-3 and similar tools open up regulatory and policy challenges with respect to enabling free speech and informed political discourse, given that language generation tools can facilitate online misinformation at a massive scale. As philosopher of language Justin Khoo points out, “the marketplace [of ideas] is not well-functioning if bots are used to carry out large-scale misinformation campaigns thus resulting in sincere voices being excluded from engaging in the discussion. Furthermore, the use of bots to conduct such campaigns is not relevantly different from spending large amounts of money to spread misinformation via political advertisements. If, as the most ardent defenders of free speech would have it, our aim is to secure a well-functioning marketplace of ideas, then bot-speak and spending on political advertisements ought to be regulated.”

            Ultimately, productive policy-making around GPT-3 and related tools will require a clear-sighted assessment of its abilities and limitations. Philosopher Regina Rini subjects the hype around GPT-3 to critical scrutiny: “GPT-3 is not a mind, but it is also not entirely a machine. It’s something else: a statistically abstracted representation of the contents of millions of minds, as expressed in their writing. Its prose spurts from an inductive funnel that takes in vast quantities of human internet chatter: Reddit posts, Wikipedia articles, news stories. When GPT-3 speaks, it is only us speaking, a refracted parsing of the likeliest semantic paths trodden by human expression.”

            Indeed, as philosopher of consciousness David Chalmers argues: “GPT-3 does not look much like an agent. It does not seem to have goals or preferences beyond completing text, for example. It is more like a chameleon that can take the shape of many different agents. […] The big question is understanding. […] Can a disembodied purely verbal system truly be said to understand? Can it really understand happiness and anger just by making statistical connections? Or is it just making connections among symbols that it does not understand? I suspect GPT-3 and its successors will force us to fragment and re-engineer our concepts of understanding to answer these questions.”

            On this point, philosopher of technology and ethicist Shannon Vallor argues that “understanding is beyond GPT-3’s reach because understanding cannot occur in an isolated behavior, no matter how clever. Understanding is not an act but […] a lifelong social labor. […] This labor does something, without which intelligence fails, in precisely the ways that GPT-3 fails to be intelligent—as will its next, more powerful version. For understanding does more than allow an intelligent agent to skillfully surf, from moment to moment, the causal and associative connections that hold a world of physical, social, and moral meaning together. Understanding tells the agent how to weld new connections that will hold, bearing the weight of the intentions and goals behind our behavior. Predictive and generative models, like GPT-3, cannot accomplish this.”

            Read the full set of philosophical essays on GPT-3 here.