June 15, 2024

Unrecoverable Election Screwup in Williamson County TX

In the November 2020 election in Williamson County, Texas, flawed e-pollbook software resulted in voters inadvertently voting for candidates and questions not from their own districts but from others in the same county.  These voters were deprived of the opportunity to vote for candidates they were entitled to vote for—and their votes were wrongly counted in elections that they shouldn’t have voted in.  This wasn’t the voters’ fault, but it does mean that the results in elections for local offices were affected by this screwup by Tenex Software Solutions.  Tenex’s e-pollbook malfunctions call into question the results of the 2020 school district races, municipal elections, potentially a county commissioners race, and state legislative races in Williamson County. 

As more and more states use e-pollbooks in vote centers, election administrators should understand this failure, because it could potentially affect any kind of e-pollbook that prints ballots on demand.

I’ve written about other screwups caused by election software or hardware—in Antrim County MI, in Windham NH, in Mercer County NJ—but in all those cases, voters marked the paper ballots they were entitled to vote on, and election officials can and did recount those ballots to report accurate election results.  That is, all those screwups were recoverable, and election officials took immediate action to recount and recover—to get an accurate result.

But in Williamson County, the mistake was unrecoverable.  Once the voters have been given the wrong ballots, then no amount of recounting can recover a fully valid election result.  Because all these voters were in the same county, this screwup did not affect any countywide offices, statewide offices, or the Presidential election:  any countywide or statewide candidate would be on every ballot in the county, so all those votes could be cast and counted accurately.  But for local races and referendum questions, and for state legislative districts, voters registered in those localities did not seem to be the ones actually voting.

Here’s what happened: Williamson County used vote centers, both for early voting in person (October 13-30) and on Election Day (November 3).  Under the vote-center model, used in several states now, the voter can cast ballots at any vote center in the county.  But each voter must receive a ballot presenting the candidates and contests local to their own district (in addition to the county-wide and state-wide candidates). 

Polling places in the U.S. have a pollbook, a list of all the voters registered to vote at that location, with (typically) name, address, and signature.  The voter signs the book, and the poll worker enables the voter to cast the appropriate ballot.  That’s to make sure that only registered voters can vote, and prevent voters from voting twice.  Vote centers need e-pollbooks, a networked system of electronic tablet computers, to make sure the voter doesn’t travel to multiple vote centers and vote in each one.

Sure, go ahead and put a networked system of computers in the polling place, filled with layers of complicated software to configure—what could possibly go wrong?

From almost the beginning of the early voting period (October 18) through election day, voters and poll workers reported a problem: voters given the wrong ballot by the e-pollbook system.   County election officials discussed the problem in e-mails; the problem persisted.  The vendor applied a fix, but to me the fix looks fragile, so this problem may lurk in wait for any county nationwide that uses Tenex e-pollbooks with ES&S ExpressVote voting machines.

The problem is this:  Once the voter’s name and address are confirmed in the tablet computer running the Tenex e-pollbook software, the e-pollbook (when used with ExpressVote voting machines as in Williamson County) prints out a ballot card marked with a barcode indicating the ballot style (that is, the locality in which this voter is entitled to vote).  Other than the barcode, the ballot card is mostly blank.  The voter takes this card over to an ExpressVote ballot-marking device, inserts the card, and the ExpressVote displays on its touchscreen the contests to vote on.  After the voter has selected candidates, the ExpressVote prints onto that same ballot card those selections, both in barcode form and in plain-text form.  (Well, it prints those selections if the ExpressVote hasn’t been hacked to cheat; but there’s no evidence that the ExpressVotes were hacked in Williamson County.)

But in many cases, the Tenex e-pollbooks were printing the wrong barcode onto the ballot, enabling the voter to vote in the wrong district.  Based on affidavits from poll workers in different vote centers on different dates, and on e-mails exchanged by administrators in the Williamson Department of Elections, we have a pretty good idea what computer glitch caused the problem: the computer’s print queue.

When you print a document from your PC, it goes into a print queue, waiting for its turn to be transmitted over a network to your printer.  Maybe you’ve had the experience of, press Print, nothing happens. Press Print again, then eventually two copies of the document come off the printer.  In the Tenex/ExpressVote setup, each e-pollbook has its own attached printer, but the e-pollbook’s operating system still maintains a print queue.

In Williamson County, October-November 2020, sometimes the voter would be checked in at the pollbook, the poll worker would press Print to create the ballot card—then no ballot card would come out right away, so the poll worker would press Print again, and perhaps again after that, resulting in multiple copies.  The voter from Hutto would take their Hutto ballot card to the ExpressVote, and vote.  Then the next voter shows up, perhaps from Round Rock, be checked in, the poll worker presses Print, and inadvertently hands the Round Rock voter (the second copy of) the Hutto ballot, printed from that extra time the poll worker pressed Print.  Now suppose another voter arrives, from Bushy Creek.  That voter will get the third ballot printed, which is for Round Rock—the Bushy Creek voter will be voting in the Round Rock election!  For the rest of the day, every voter will get a previous voter’s ballot.

There’s no way to fix this with a recount.  A recount can only tally what’s printed on the paper, and what’s printed on the paper is from voters in the wrong districts.

Besides the print queue problem, there were other glitches with the Tenex e-pollbooks:  ballot cards with the wrong precinct-code (but the right candidates).   In the days after November 3, the Williamson County Elections Administrator Christopher J. Davis was not able to produce a precinct-by-precinct report of the vote totals (as required by Texas law).  Mr. Davis thought he could produce such a report by sorting and recounting the paper ballots by hand, so on November 10 he asked a district court judge to order such a recount.  It took eight months for the Williamson County Elections Department to complete this recount and produce such a report. 

When Mr. Davis went into court on November 10, he explained this other mistake in e-pollbook programming: failure to mark a precinct-number on some ballots. That mistake could be resolved by recounting the ballots. But he did not tell the court about the print-queue problem.

That’s a big problem.  In other cases I’ve written about (Antrim, Windham, Mercer), election officials (and the public) could be reasonably confident that the recount got the correct result.  But in Williamson County, if it’s really the case that thousands of voters got ballots for the wrong districts, then it would be impossible for Mr. Davis’s office to produce accurate and meaningful results for the November 3 election (for local contests smaller than countywide).    When he stated under oath on November 10, “Voters were being given the correct ballot style, they were voting on all the precise races and contests that they were eligible for when they voted in person during early voting” he had already sent an e-mail to his own staff on October 18 with a detailed analysis concluding,

Because if there was some kind of mishap with the previous voter at that station, it might be possible that the [ballot style] for them is repeated for the next voter. I have to wonder how many voters are voting an incorrect ballot…

Clearly Mr. Davis was in a bind.  If he stated clearly that the problem was impossible to recover from, then someone would have to ask a judge to order a new election for every local (less than countywide) office.  That would be highly embarrassing and expensive.  Could his eight-month delay in reporting results have been because it was difficult to reconcile precinct-by-precinct results that didn’t really add up right?

The Tenex e-pollbook malfunctions call into question the results of the 2020 school district races, municipal elections, potentially a County Commissioners Court race, and State Congressional races in Williamson County.  When e-pollbooks again gave some voters the wrong ballots in the March 2022 primary election, after redistricting had split the county between two congressional districts, it could have affected the U.S. Congressional election as well.

In October 2022 I noticed that Morris County NJ was using a similar combination of Tenex e-pollbooks with ES&S ballot cards.  I asked the Morris County Administrator of Elections whether he was aware of the print-queue problem, and he responded:

Our e-Pollbook vendor, Tenex, has stated the incident was the result of a timer on the popup that the Texas county did not have turned on to prevent the poll worker from pressing the print button multiple times.  In Morris County, the reprint button is actually grayed out for 10 seconds after the print button is pressed to give it time to process. 

       Tenex confirmed this setting is on for all e-Pollbook clients that use the ExpressVote printers and is not an issue for any of their clients. 

So it appears that the print-queue-wrong-ballot problem is really a thing, that Tenex patched the problem with a popup, that if one is not careful with installation and configuration then it could happen again.  And what if the printer takes more than 10 seconds to wake up?  I don’t have a lot of confidence in this band-aid fix, “gray out the reprint button for 10 seconds.”  Beware of using Tenex e-pollbooks in combination with ExpressVote touchscreens.

Conclusion:  a train wreck

Some ways of running Early Vote Centers are susceptible to a failure mode in which computer software error can lead to wrong and uncorrectable election results.  This is not simply about Tenex and ExpressVote.  If the software controls what ballot the voter is given, and if the poll worker cannot easily check, or does not routinely check, that it’s the right ballot for the voter’s address, then software error can cause many voters to vote the wrong ballot.  As long as software is fallible, we should not design election systems in which software error leads to unrecoverable election failure.

This article is based in large part on observation and analysis by Lori Gallagher, and on documents she obtained by using Texas Public Information Act requests.

Comments

  1. In 2019, Party precinct chairs chose Election Systems & Software (ES&S) to conduct our elections in Williamson County, TX, largely because it gave us a printed ballot card we could check for accuracy before feeding it into the DS200 ballot scanning/counting device.

    Former programmer turned attorney, Clint Curtis informed us at a recent meeting in the Texas Capitol Extension that the programmer can code the ballot printout to show your vote exactly as you chose, but program the barcode to follow and produce his 51% / 49% results in favor of candidates HE chose.

    At the same meeting, Mark Cook, an election machine hack-a-thon-er, showed us that any programmer with a flash drive, smart phone, or laptop can override the correct code and get the results they want.

    Is it possible that 3 Eastern Europeans with a legitimate, SOS-authorized news media permit to “observe” our elections on Election Night 11/3/2020 from their vehicles beyond the 100’ marker could use the Cedar Park ,TX, library’s wi-fi porch or parking lot wi-fi to alter the election results? According to Clint Curtis, yes. In fact, Clint adds, they could even change the results by plugging into an outlet near where the locally (not Internet) networked election machines are plugged into electricity. Or they can access SOS servers in any state remotely, just as online driver license renewal/updating & online voter registration updating software can access any state’s SOS servers. So two men inside the Cedar Park library after the polls closed with laptops plugged into the library research tables also could have changed the election 11/3/2020 without detection.

    Is there anyway for election administrators or poll clerks, judges & supervisors to detect the changes? According to Clint, “They’d never see it. Not without the source code,” which the machine companies claim is proprietary and not available to election poll workers and watchers.

    I am under the impression that a bill requiring the source code for auditing purposes may be filed this legislative session in Austin, TX, but Clint says a programmer can cover his tracks so no log of his intervention is recorded. So the election results are determined by the last one in to change the results (and remove his tracks) in favor of his candidates. Not even having the source code makes an election certifiable.

    Williamson County’s election ES&S machines & software is not a secure system. Whoever programs it determines the results. The results it produces are not certifiable. The printed ballot May or may not match the bar or. ES&S machines are no more secure than the Dominion machines we rejected in Texas.

    The machines have to go. We need:
    —Hard copy poll books printed at least 30 days ahead of Election Day for each voting precinct.
    —Voter ID with matching name, address, photo, date of birth, signature and PHOTO.
    —Did you know Wilco’s clerk check-in stations screens aren’t programmed to show the clerk don’t show on the clerk your photo or signature or alert the clerk when the license is expired. The tablets can be programmed to show all of these, but checking these would slow down the line, which would thwart Executive Order 13639’s “PCEA Report & Recommendations: The American Voting Experience January 2014.” Why do 70 or so of the most populated Texas counties (which are home to 80% of registered Texas voters, including Williamson County) conduct our elections so the line moves faster, not so the results show every legal vote?
    —One day of in-person voting (holiday) in one’s voting precinct.
    —Hand marked ballots, hand-counted on Election Day one day in each voting precinct, using secure ballot and poll book stock and kept in a secure place for two years.
    —Audited within two years of Election Day.
    —provisional voting available only in person on Election Day before the ballot board itself at the County Elections Office to cure one’s lack of proper voter ID matching the printed poll book.

    The Parties are telling us Poll-workers that the machines have to stay. The parties want to continue to cheat legal voters out of their vote? What self-respecting voter is going to work the polls knowing all this?

    • I was at the library when the Eastern Europeans were there. I witnessed them sitting in their car with laptops as they tentatively watched their screens after before and after the polls closed. I mentioned this the the Williamson County Elections Administrator and the GOP Chairman at an open meeting and they both claimed they didn’t know anything about it even though someone else had already brought it up to them days prior. I’m not sure why they were playing dumb, but it appears they were covering up for the Europeans that were monitoring our elections.

  2. Walter Daugherity says

    The 10-second pause is indeed an inadequate patch. A more robust solution would be to limit the print queue to one ballot, e.g., like this skeleton code to print a ballot:

    1. Wait till print queue is empty (with error messages and recovery as needed)
    2. Wait till printer is ready and idle (with error messages and recovery as needed)
    3. Send ballot image to print manager
    4. Wait till print queue is empty (with error messages and recovery as needed)
    5. Wait till printer is ready and idle (with error messages and recovery as needed)