May 26, 2020

Archives for November 2010

Finding the best ideas in the world

Policy-makers often strive to solicit ideas from the public, but making this process effective is very, very hard. Americans need only think back to the health-reform town hall meetings last summer, many of which devolved into chaos. Online approaches, such as President Obama’s virtual town hall meeting, have also ran into problems. So, how can policy-makers discover the best ideas from the public?

The Organization for Economic Cooperation and Development (OECD)–an intergovernmental think tank–recently faced this challenge. They were planning a global summit for education leaders that is taking place in Paris today (November 4th), and they wanted to bring fresh thinking from the public to this group. To achieve this goal, the OECD created an “idea marketplace” at, a website that I have been developing with a team of students and web-developers. An idea marketplace allows groups to collect and prioritize ideas in a way that is democratic, transparent, and open. It’s certainly a lot easier than inviting everyone to Paris for the meeting.

Here’s how it worked for the OECD, more specifically Joanne Caddy, Julie Harris, and Cassandra Davis. They “seeded” the website with about 50 ideas (e.g., “Give teachers regular sabbaticals so that they can be trained outside of regular school time or during holidays.”). Then visitors to their idea marketplace were presented with the question “Which is the more important action we need to take in education today?” and two ideas from the pool. The visitors voted for one of the ideas, and then another pair of ideas was presented. This process of pairwise voting continued for as the long as the visitor wished. Also, at any time the visitor could upload an idea which would then go into the pool of ideas to be voted on by others. In this way the idea marketplace allowed the OECD to collect ideas from the community and have the community prioritize them. Both of these steps–collection and prioritization–are needed for a successful “crowdsourcing” of ideas. Here is what the voting looked like to the visitor:

So, how did it work in practice? In just one month, people from more than 90 countries collectively cast more than 27,000 votes.

Equally impressive, these participants uploaded 325 new ideas, again from all over the world.

By combining the voting process with the open uploading of new ideas, this community of stakeholders was able to generate a ranked list that more truly reflected the opinions of the group. The top five ideas (shown below) will be presented to the education leaders today at the summit in Paris, and critically four of these five ideas were uploaded by visitors. In other words, these were ideas that the OECD did not include on their initial list. This example nicely shows how idea marketplaces enable new ideas to bubble-up, allowing groups to learn about things they didn’t even know to ask about.

The top five ideas were:

  1. Teach to think, not to regurgitate.
  2. Commit to education as a public good and a public responsibility.
  3. Focus more on creating a long-term love of learning and the ability to think critically than teaching to standardised tests.
  4. Ensure all children have the opportunity to discover their natural abilities and develop them.
  5. Ensure that children from disadvantaged background and migrant families have the same opportunity to quality education as others.

In addition to the OECD, more than 600 idea marketplaces have been created for free at These have collected about 250,000 votes and 5,000 uploaded ideas. Other policy-related uses have come from student governments (for example, at Princeton and Columbia) and the New York City Government, which is currently using for two projects: is an open-source research project that has received generous funding from CITP and Google. You can keep up on the project by reading our blog or following us on Twitter or Facebook.

Paper vs. Electronic Voting in Today's Election in Houston

(Cross-posted at the Computing@Rice blog at the Houston Chronicle.)

Back in late August, Harris County (Houston)’s warehouse with all 10,000 of our voting machines, burned to the ground. As I blogged at the time, our county decided to spend roughly $14 million of its $40 million insurance settlement on purchasing replacement electronic voting machines of the same type destroyed in the fire, and of the same type that I and my colleagues found to be unacceptably insecure in the 2007 California Top-to-Bottom Report. This emergency purchase was enough to cover our early voting locations and a smattering of extras for Election Day. We borrowed the rest from other counties, completely ignoring the viral security risks that come with this mixing and matching of equipment. (It’s all documented in the California report above. See Section 7.4 on page 77. Three years later, and the vendor has fixed none of these issues.)

Well, the county also spent the money to print optical-scan paper ballots (two sheets of 8.5″ x 17″, printed front and back), and when I went to vote this morning, I found my local elementary school had eight eSlate machines, all borrowed from Travis County (Austin), Texas. They also had exactly one booth set up for paper ballot voting.

After I signed in, the poll worker handed me the four-digit PIN code for using an eSlate before I could even ask to use paper. “I’d like to vote on paper.” “Really? Uh, okay.” Apparently I was only the second person that day to ask for paper and they were in no way making any attempt to give voters the option to vote on paper.

How did it work? They had a table with three blank ballots (each a stack of two sheets of paper), of which I could choose one. Both sheets shared a long serial number on the left column, which appears to serve two functions. First, it allows the two sheets to be kept together (notably, allowing the straight ticket voting option on the first sheet to apply to the second sheet). Also, these serial numbers, by virtue of being large and hopefully random, would act to prevent ballot stuffing (assuming the county kept records of which numbers were valid). Additionally, there was a signature from one of the poll workers at the bottom of the ballot, which I presume to be an additional anti-ballot-stuffing measure.

I was handed a Bic pen and pointed to a rickety standing table with a privacy partition. At the same time, my wife voted on a standard eSlate. I decided to ask a poll worker the question of how a straight ticket on the first sheet would apply to the second sheet. The first poll worker, who was operating the eSlates, said “sorry, I was only trained on the eSlates” and made me wait until the head guy came over. The head guy then proceeded to give me an extended tutorial in the ways of our straight ticket system, requiring me to interrupt him and say, “yeah, but all I want to know is how my tick of the straight ticket box on the first sheet is carried over to the second sheet.” We ultimately concluded that it must be due to the matching serial numbers.

Anyway, despite all this fun and excitement, I still managed to finish my ballot a solid minute faster than my wife. Also, by that time, a queue of maybe six people was waiting to vote while all the eSlates were busy. I asked the poll workers at the sign-in table if they were planning to offer paper ballots to anybody in line and they looked at me as if I was insane. I also mentioned that I finished voting faster than my wife and one poll worker went as far as to say “don’t tell anybody!” as if that might (gasp!) cause people to want to vote on paper.

What’s going on here? I blame our lame-duck election administrator, who has been urging voters to use the eSlate, and doing her best to ignore the paper ballot option that she was compelled to offer as a consequence of the warehouse fire. If there’s no emphasis on paper, from the leadership on top, one could hardly expect poll workers to behave any differently.

What’s happening next?

One way or another, Harris County will have a new elections administrator after our incumbent one retires, and the next one will be responsible for rebuilding our election systems. Curiously, Travis County recently announced that they’re retiring their eSlates after the 2012 election, replacing them with paper ballots that are scanned in the precinct. This gives Harris County the chance to buy their used gear at a fraction of the price of new equipment, should we choose to go that route, or we could instead follow Travis County’s lead and ditch our eSlates entirely (save for keeping one in each precinct for accessibility purposes). Either way, we would save literally millions of dollars, relative to the costs of purchasing new eSlates from scratch, and of course the new paper ballot systems are more secure and (gasp!) faster and easier to use.

Sidebar: Are these paper ballots really private?

The Texas Election Code actually has a requirement that ballots be “numbered”, which I understand is generally taken to mean that there must be mechanisms in place to prevent tampering and ballot stuffing. (You would require a very broad interpretation of that statute in order to have allowed traditional lever voting machines, used widely in Texas prior to 2000, where there is nothing approximating individual ballot numbers in the machine.) The sparse and hopefully unguessable serial numbers on our paper ballots appear to follow the letter of the law as well as offering the ability to have ballots larger than a single sheet of paper. That’s the good news, but let’s consider what it would mean in the case where somebody was attempting to bribe or coerce my vote and they had access to the output of the central ballot scanner, which presumably includes these ballot numbers.

Of course, the poll worker who puts out the blank ballots can track who gets which ballot. Furthermore, I could simply write down my own ballot number. Because these numbers are sparse, and thus hard to guess, somebody bribing or coercing me would have some serious leverage on me if I produced an invalid ballot number. If I sneakily remembered one of the other two ballot numbers from the table, I could present my coercer with one of those numbers instead, but then I would have no knowledge of how (or even if) that other ballot was cast, and could thus get in trouble with my coercer.

How can this coercion risk be mitigated? One simple option is to render the ballot numbers only as barcodes. Very few of us can visually read a barcode, much less the newer two-dimensional barcodes. So long as we ban smartphones or other cameras, we’re in good shape. Concerned voters or auditors, who want to ensure the same number exists on both ballot sheets could hold them up to a bright light, lining them up together, to make sure that they match up.

Oh, and ballots aren’t private with the current eSlate either. See the California report, linked above, “issue 25” on page 58. See also Section 7.1 which starts on page 72.

E-Voting Links for Election Day

Today, of course, is Election Day in the U.S. Many of our U.S. readers will be casting their votes electronically.

CITP has been front and center on the e-voting issue. Here’s a quick set of CITP e-voting links:

Finally, in keeping with tradition here, on Election Day I post photos of unguarded voting machines in the Princeton area. Here are two photos taken over the weekend:

[This entry was updated at 1:00 PM Eastern on 2 Nov 2010, to add the Checkoway et al. item.]