May 6, 2024

Posts Comments

Thoughtcrime Experiments

July 2, 2009 by

Cosmic rays can flip bits in memory cells or processor datapaths. Once upon a time, Sudhakar and I asked the question, “can an attacker exploit rare and random bit-flips to bypass a programming-language’s type protections and thereby break out of the Java sandbox?”

Thoughtcrime Experiments

A recently published science-fiction anthology Thoughtcrime Experiments contains a story, “Single-Bit Error” inspired by our research paper. What if you could use cosmic-ray bit flips in neurons to bypass the “type protections” of human rationality?

In addition to 9 stories and 6 original illustrations, the anthology is interesting for another reason. It’s an experiment in do-it-yourself paying-the-artists high-editorial-standards open-source Creative-Commons print-on-demand publishing. Theorists like Yochai Benkler and others have explained that production costs attributable to communications and coordination have been reduced down into the noise by the Internet, and that this enables “peer production” that was not possible back in the 19th and 20th centuries. Now the Appendix to Thoughtcrime Experiments explains how to edit and produce your own anthology, complete with a sample publication contract.

It’s not all honey and roses, of course. The authors got paid, but the editors didn’t! The Appendix presents data on how many hours they spent “for free”. In addition, if you look closely, you’ll see that the way the authors got paid is that the editors spent their own money.

Still, part of the new theory of open-source peer-production asks questions like, “What motivates people to produce technical or artistic works? What mechanisms do they use to organize this work? What is the quality of the work produced, and how does it contribute to society? What are the legal frameworks that will encourage such work?” This anthology and its appendix provide an interesting datapoint for the theorists.

Filed Under: Uncategorized Tagged With:

NJ Voting-machine Trial: Defense Witnesses

May 22, 2009 by

I’ve previously summarized my own testimony and other plaintiffs’ witnesses’ testimony in the New Jersey voting machines trial, Gusciora v. Corzine.

The defendant is the State of New Jersey (Governor and Secretary of State). The defense case comprised the following witnesses:

Defense witness James Clayton, the Ocean County voting machine warehouse supervisor, is a well-intentioned official who tries to have good procedures to secure the Ocean County voting machines. Still, it became apparent in his testimony that there are security gaps regarding transport of the machines, keys to the machines, and security at polling places before and after election day.

Richard Woodbridge is a patent attorney who has chaired the NJ Voting Machine Examination Committee for more than 20 years. It’s not clear why the defendants called him as a witness, because they conducted only a 15-minute direct examination in which he didn’t say much. On cross-examination he confirmed that his committee does not conduct an independent analysis of software and does not consult with any computer security experts.

Robert Giles, Director of Elections of the State of New Jersey, testified about experimenting with different forms of seals and locks that New Jersey might apply to its AVC Advantage voting machines. On cross examination, it became clear that there is no rhyme or reason in how the State is choosing seals and other security measures; that they’re not getting expert advice on these matters. Also he admitted that there are no statewide control or even supervision of the procedures that counties use to safeguard the voting machines, the results cartridges, keys, and so on. He confirmed that several counties use the cartridges as the official tally, in preference to paper printouts witnessed and signed (at the close of the polls) by election workers.

Edwin Smith testified as an expert witness for the State defendants. Mr. Smith is vice-president and part owner of Sequoia Voting Systems. He stands to gain financially depending on the verdict in this trial: NJ represents 20% of Sequoia’s market, and his bonuses depend on sales. Mr. Smith testified to rebut my testimony about fake Z80 processors. (Wayne Wolf, who testified for plaintiffs about fake Z80s, testified after Mr. Smith, as a rebuttal witness.) Even though Mr. Smith repeatedly referred to replacement of Z80s as “science fiction”, he then offered lengthy testimony about methods to try to detect fake Z80s. This gave credence to the fact that fraudulent CPUs are not only a possibility but a real threat.

Mr. Smith also confirmed that it is a security risk to connect WinEds computers (that prepare electronic ballot definitions and tabulate results) to the Internet, and that those counties in NJ that do so are making a mistake.

Paul Terwilliger testified as a witness for the defense. Mr. Terwilliger is a longtime employee and/or contractor for Sequoia, who has had primary responsibility over the development of the AVC Advantage for the last 15 years. Mr. Terwilliger admitted that in 2003 the WIPO found that he’d acted in bad faith by cybersquatting on the Diebold.com domain name at the request of Sequoia. Mr. Terwilliger testified that it is indeed possible to program an FPGA to make a “fake Z80” that cheats in elections. But, he said, there are some methods for detecting FPGAs installed on AVC Advantage voting machines instead of the legitimate (Some of these methods are impractical, others are ineffective, others are speculative; see Wayne Wolf’s report.) This testimony had the effect of underscoring the seriousness of the fake-Z80 threat.

Originally the defendants were going to rely on Professor Michael Shamos of Carnegie Mellon University as their only expert witness. But the Court never recognized him as an expert witness. The Court ruled that he could not testify about the security and accuracy of the AVC Advantage, because he had not offered an opinion about security and accuracy in his expert report or his deposition.

The Court did permit him to testify in general terms. He said that in real life, we have no proof that a “hacked election” has ever occurred; and that in real life, such a hack would somehow come to light. He offered no studies that support this claim.

Professor Shamos attempted to cast doubt in the Court’s mind about the need for software independence, and disparaging precinct-based optical scan voting (PCOS). But he offered no concrete examples and no studies regarding PCOS.

On many issues, Professor Shamos agreed with the plaintiffs’ expert: it’s straightforward to replace a ROM chip, plastic-strap seals provide only a veneer of protection, the transformed machine can cheat, and pre-election logic-and-accuracy testing would be ineffective in detecting the fraud. He does not dispute many of the bugs and user-interface design flaws that we found, and recommends that those should be fixed.

Professor Shamos admitted that he is alone among computer scientists in his support of paperless DREs. He tried to claim that other computer scientists such as Ted Selker, Douglas W. Jones, Joseph Lorenzo Hall also supported paperless DREs by saying they supported parallel testing–implying that those scientists would consider paperless DREs to be secure enough with parallel testing–but during cross-examination he backed off a bit from this claim. (In fact, as I testified in my rebuttal testimony, Drs. Jones and Hall both consider PCOS to have substantially stronger security, and to be substantially better overall, than DREs with parallel testing.)

Parallel testing is Professor Shamos’s proposed method to detect fraudulent software in electronic voting machines. In order to catch software that cheats only on election day, Professor Shamos proposes to cordon off a machine and cast a known list of test votes on it all day. He said that no state has ever implemented a satisfactory parallel testing protocol, however.

Summary of the defendant’s case

One of the plaintiffs’ most important claims–which they demonstrated on video to the Court–is that one can replace the firmware of the AVC Advantage voting machine with fraudulent firmware that changes votes before the polls close. No defense witness contradicted this. To the extent that the defense put up a case, it hinged on proposed methods for detecting such fraudulent firmware, or on proposed methods for slowing down the attack by putting tamper-evident seals in the way. On both of these issues, defense witnesses contradicted each other, and plaintiffs presented rebuttal witnesses.

Filed Under: Other Topics Tagged With:

NJ Voting-machine trial: Plaintiffs' witnesses

May 21, 2009 by

Both sides in the NJ voting-machines lawsuit, Gusciora v. Corzine, have finished presenting their witnesses. Briefs (in which each side presents proposed conclusions) are due June 15 (plaintiffs) and July 15 (defendants), then the Court will eventually issue a decision.

In summary, the plaintiffs argue that New Jersey’s voting machines (Sequoia AVC Advantage) can’t be trusted to count the votes, because they’re so easily hacked to make them cheat. Thus, using them is unconstitutional (under the NJ state constitution), and the machines must be abandoned in favor of a method that provides software independence, for example precinct-count optical-scan voting.

The plaintiffs’s first witness was Stephanie Harris, who testified for half an hour about her experience voting on an AVC Advantage where the pollworker asked her to go back and recast her ballot for a total of three or four times, because the pollworker wasn’t sure that it registered. Ms. Harris testified that to this day she’s not sure whether her vote registered 0 times, or 1, or 2, or 3, or 4.

I testified second, as I’ve described. I testified about many things, but the most important is that you can easily replace the firmware of an AVC Advantage voting machine to make it cheat in elections (but not cheat when it’s being tested outside of elections).

The third witness was Ed Felten, who testified for about an hour that on several different occasions he found unattended voting machines in Princeton, on weekends before elections, and he took pictures. (Of course, as the Court was well aware by this time in the trial, a hacker could take advantage of an unattended voting machine to install vote-stealing firmware.) Ed wrote about this on Freedom-to-Tinker here, here, and here; he brought all those pictures with him to show the Court.

Next were Elisa Gentile, Hudson County voting machine warehouse supervisor, and Daryl Mahoney, Bergen County voting machine warehouse supervisor. Mr. Mahoney also serves on the NJ Voting Machine Examination committee (which recommends certification of voting machines for use in NJ). These witnesses were originally proposed by the defense, but in their depositions before trial, they said things so helpful to the plaintiffs that the plaintiffs called them instead! They testified about lax security with regard to transport and storage of voting machines, lax handling of keys to the voting machines, and no security at polling places where the machines are delivered several days before the election. They didn’t seem to have a clue about information security and how it affects the integrity of elections conducted using computers.

Next the plaintiffs called County Clerk of Union County, Joanne Rajoppi, who had the sophistication to notice a discrepancy in the results report by AVC Advantage voting machine, the integrity to alert the newspapers and the public, and the courage to testify about all the things that have been going wrong with AVC Advantage voting machines in her county. Ms. Rajoppi testified about (among other things):

  • Soon after the February 5, 2008 Super Tuesday presidential primary, she noticed inconsistencies in AVC Advantage results-reports printouts (and cartridge data): the number of votes in some primaries was higher than the number of voters. (See Section 56 of my report, or Ed Felten’s analysis on Freedom-to-Tinker)
  • She brought this to the attention of State election officials, but the State officials made no move at all to investigate the problem. She arranged for Professor Felten of Princeton University to examine the Union County voting machines, but she stopped when she was threatened with a lawsuit by Edwin Smith, vice president of Sequoia Voting Systems.
  • In a different election, the Sequoia AVC voting system refused to accept a candidate’s name with a tilde over the ñ. Sequoia technicians produced a hand-edited ballot definition file; she was uneasy about turning control of the ballot definition file over to Sequoia.
  • Results Cartridges get locked in the machines sometimes (when pollworkers forget to bring them back from the polling places for tabulation). (During this time they are vulnerable to vote-changing manipulation; see Section 40 of my report.)
  • Union County considers the vote data in the cartridges to be the official election results, not the vote data printed out at the close of the polls (and then signed by witnesses). (This is unwise for several reasons; see Sections 40 and 57 of my report.)

The defendant (the State of New Jersey) presented several witnesses. I’ll summarize them in my next post. After the defense witnesses, the plaintiffs called rebuttal witnesses.

Plaintiffs’ rebuttal witness Roger Johnston is an expert on physical security at the U.S. government’s Argonne National Laboratory (testifying as a pro bono expert on his own behalf, not representing the views of the U.S. government). Dr. Johnston testified that supposedly tamper-evident seals and tape can be defeated; that it does no good to have seals without a rigorous protocol for inspecting them (which NJ does not have); that such a protocol (and the training it requires) would be very expensive to implement and execute; that AVC Advantage’s design makes it impractical to really secure using seals; and that in general New Jersey’s “security culture” and its proposed methods for securing these voting machines are incoherent and dysfunctional. He demonstrated for the Court one defeat of each seal, and testified about other defeats of these kinds of seals.

The last plaintiffs’ witness was Wayne Wolf, professor of Electrical Engineering at Georgia Tech. Professor Wolf testified (and wrote in his expert report) that it’s straightforward to build a fake computer processor chip and install it to replace the Z80 computer chip in the AVC Advantage voting machine. (See also Section 12 of my report.) This fake chip could (from time to time) ignore the instructions in the AVC Advantage ROM memory about how to add up votes, and instead transfer votes from one candidate to another. It can cheat just like the ROM-replacement hack that I testified about, but it can’t be detected by examining the ROM chips. Professor Wolf also testified about the difficulty (or impossibility) of detecting fake Z80 chips by some of the methods proposed by defense witnesses.

Filed Under: Other Topics

NJ Voting-machine trial update

February 26, 2009 by

Earlier this month I testified in Gusciora v. Corzine, the trial in which the plaintiffs argue that New Jersey’s voting machines (Sequoia AVC Advantage) can’t be trusted to count the votes, because they’re so easily hacked to make them cheat.

I’ve previously written about the conclusions of my expert report: in 7 minutes you can replace the ROM and make the machine cheat in every future election, and there’s no practical way for the State to detect cheating machines (in part because there’s no voter-verified paper ballot).

The trial started on January 27, 2009 and I testified for four and a half days. I testified that the AVC Advantage can be hacked by replacing its ROM, or by replacing its Z80 processor chip, so that it steals votes undetectably. I testified that fraudulent firmware can also be installed into the audio-voting daughterboard by a virus carried through audio-ballot cartridges. I testified about many other things as well.

Finally, I testified about the accuracy of the Sequoia AVC Advantage. I believe that the most significant source of inaccuracy is its vulnerability to hacking. There’s no practical means of testing whether the machine has been hacked, and certainly the State of New Jersey does not even attempt to test. If we could somehow know that the machine has not been hacked, then (as I testified) I believe the most significant _other_ inaccuracy of the AVC Advantage is that it does not give adequate feedback to voters and pollworkers about whether a vote has been recorded. This can lead to a voter’s ballot not being counted at all; or a voter’s ballot counting two or three times (without fraudulent intent). I believe that this error may be on the order of 1% or more, but I was not able to measure it in my study because it involves user-interface interaction with real people.

In the hypothetical case that the AVC Advantage has not been hacked, I believe this user-interface source of perhaps 1% inaccuracy would be very troubling, but (in my opinion) is not the main reason to disqualify it from use in elections. The AVC Advantage should be disqualified for the simple reason that it can be easily hacked to cheat, and there’s no practical method that will be sure of catching this hack.

Security seals. When I examined the State’s Sequoia AVC Advantage voting machines in July 2008, they had no security seals preventing ROM replacement. I demonstrated on video (which we played in Court in Jan/Feb 2009) that in 7 minutes I could pick the lock, unscrew some screws, replace the ROM with one that cheats, replace the screws, and lock the door.

In September 2008, after the State read my expert report, they installed four kinds of physical security seals on the AVC Advantage. These seals were present during the November 2008 election. On December 1, I sent to the Court (and to the State) a supplemental expert report (with video) showing how I could defeat all of these seals.

In November/December the State informed the Court that they were changing to four new seals. On December 30, 2008 the State Director of Elections, Mr. Robert Giles, demonstrated to me the installation of these seals onto the AVC Advantage voting machine and gave me samples. He installed quite a few seals (of these four different kinds, but some of them in multiple places) on the machine.

On January 27, 2009 I sent to the Court (and to the State) a supplemental expert report showing how I could defeat all those new seals. On February 5th, as part of my trial testimony I demonstrated for the Court the principles and methods by which each of those seals could be defeated.

On cross-examination, the State defendants invited me to demonstrate, on an actual Sequoia AVC Advantage voting machine in the courtroom, the removal of all the seals, replacement of the ROM, and replacement of all the seals leaving no evidence of tampering. I then did so, carefully and slowly; it took 47 minutes. As I testified, someone with more practice (and without a judge and 7 lawyers watching) would do it much faster.

Filed Under: Other Topics, Privacy & Security Tagged With: ,

Optical-scan voting extremely accurate in Minnesota

January 21, 2009 by

The recount of the 2008 Minnesota Senate race gives us an opportunity to evaluate the accuracy of precinct-count optical-scan voting. Though there have been contentious disputes over which absentee ballot envelopes to open, the core technology for scanning ballots has proved to be extremely accurate.

The votes were counted by machine (except for part of one county that counts votes by hand), then every single ballot was examined by hand in the recount.

The “net” accuracy of optical-scan voting was 99.99% (see below).
The “gross” accuracy was 99.91% (see below).
The rate of ambiguous ballots was low, 99.99% unambiguous (see below).

My analysis is based on the official spreadsheet from the Minnesota Secretary of State. I commend the Secretary of State for his commitment to transparency in the form of making the data available in such an easy-to-analyze format. The vast majority of the counties use the ES&S M100 precinct-count optical-scanners; a few use other in-precinct scanners.

I exclude from this analysis all disputes over which absentee ballots to open. Approximately 10% of the ballots included in this analysis are optically scanned absentee ballots that were not subject to dispute over eligibility.

There were 2,423,851 votes counted for Coleman and Franken. The “net” error rate is the net change in the vote margin from the machine-scan to the hand recount (not including change related to qualification of absentee ballot envelopes). This was 264 votes, for an accuracy of 99.99% (error, one part in ten thousand).

The “gross” error rate is the total number of individual ballots either added to one candidate, or subtracted from one candidate, by the recount. A ballot that was changed from one candidate to the other will count twice, but such ballots are rare. In the precinct-by-precinct data, the vast majority of precincts have no change; many precincts have exactly one vote added to one candidate; few precincts have votes subtracted, or more than one vote added, or both.

The recount added a total of 1,528 votes to the candidates, and subtracted a total of 642 votes, for a gross change of 2170 (again, not including absentee ballot qualification). Thus, the “gross” error rate is about 1 in 1000, or a gross accuracy of 99.91%.

Ambiguous ballots: During the recount, the Coleman and Franken campaigns initially challenged a total of 6,655 ballot-interpretation decisions made by the human recounters. The State Canvassing Board asked the campaigns to voluntarily withdraw all but their most serious challenges, and in the end approximately 1,325 challenges remained. That is, approximately 5 ballots in 10,000 were ambiguous enough that one side or the other felt like arguing about it. The State Canvassing Board, in the end, classified all but 248 of these ballots as votes for one candidate or another. That is, approximately 1 ballot in 10,000 was ambiguous enough that the bipartisan recount board could not determine an intent to vote. (This analysis is based on the assumption that if the voter made an ambiguous mark, then this ballot was likely to be challenged either by one campaign or the other.)

Caveat: As with all voting systems, including optical-scan, DREs, and plain old paper ballots, there is also a source of error from voters incorrectly translating their intent into the marked ballot. Such error is likely to be greater than 0.1%, but the analysis I have done here does not measure this error.

Hand counting: Saint Louis County, which uses a mix of optical-scan and hand-counting, had a higher error rate: net accuracy 99.95%, gross accuracy 99.81%.

Filed Under: Uncategorized Tagged With: