November 24, 2024

What an expert on seals has to say

During the New Jersey voting machines lawsuit, the State defendants tried first one set of security seals and then another in their vain attempts to show that the ROM chips containing vote-counting software could be protected against fraudulent replacement. After one or two rounds of this, Plaintiffs engaged Dr. Roger Johnston, an expert on physical security and tamper-indicating seals, to testify about New Jersey’s insecure use of seals.

In his day job, Roger is a scientist at the Argonne National Laboratory, working to secure (among other things) our nation’s shipments of nuclear materials. He has many years of experience in the scientific study of security seals and their use protocols, as well as physical security in general. In this trial he testified in his private capacity, pro bono.

He wrote an expert report in which he analyzed the State’s proposed use of seals to secure voting machines (what I am calling “Seal Regime #2” and “Seal Regime #3”). For some of these seals, he and his team of technicians have much slicker techniques to defeat these seals than I was able to come up with. Roger chooses not to describe the methods in detail, but he has prepared this report for the public.

What I found most instructive about Roger’s report (including in version he has released publicly) is that he explains that you can’t get security just by looking at the individual seal. Instead, you must consider the entire seal use protocol:


Seal use protocols are the formal and informal procedures for choosing, procuring, transporting, storing, securing, assigning, installing, inspecting, removing, and destroying seals. Other components of a seal use protocol include procedures for securely keeping track of seal serial numbers, and the training provided to seal installers and inspectors. The procedures for how to inspect the object or container onto which seals are applied is another aspect of a seal use protocol. Seals and a tamper-detection program are no better than the seal use protocols that are in place.

He explains that inspecting seals for evidence of tampering is not at all straightforward. Inspection often requires removing the seal—for example, when you pull off an adhesive-tape seal that’s been tampered with, it behaves differently than one that’s undisturbed. A thorough inspection may involve comparing the seal with microphotographs of the same seal taken just after it was originally applied.

For each different seal that’s used, one can develop a training program for the seal inspectors. Because the state proposed to use four different kinds of seals, it would need four different sets training materials. Training all the workers who would inspect the State’s 10,000 voting machines would be quite expensive. With all those seals, just the seal inspections themselves would cost over $100,000 per election.

His report also discusses “security culture.”


“Security culture” is the official and unofficial, formal and informal behaviors, attitudes, perceptions, strategies, rules, policies, and practices associated with security. There is a consensus among security experts that a healthy security culture is required for effective security….

A healthy security culture is one in which security is integrated into everyday work, management, planning, thinking, rules, policies, and risk management; where security is considered as a key issue at all employee levels (and not just an afterthought); where security is a proactive, rather than reactive activity; where security measures are carefully defined, and frequently reviewed and studied; where security experts are involved in choosing and reviewing security strategies, practices, and products; where the organization constantly seeks proactively to understand vulnerabilities and provide countermeasures; where input on potential security problems are eagerly considered from any quarter; and where wishful thinking and denial is deliberately avoided in regards to threats, risks, adversaries, vulnerabilities, and the insider threat….

Throughout his deposition … Mr. Giles [Director of the NJ Division of Elections] indicates that he believes good physical security requires a kind of band-aid approach, where serious security vulnerabilities can be covered over with ad hoc fixes or the equivalent of software patches. Nothing could be further from the truth.

Roger Johnston’s testimony about the importance of seal use protocols—as considered separately from the individual seals themselves—made a strong impression on the judge: in the remedy that the Court ordered, seal use protocols as defined by Dr. Johnston played a prominent role.

The trick to defeating tamper-indicating seals

In this post I’ll tell you the trick to defeating physical tamper-evident seals.

When I signed on as an expert witness in the New Jersey voting-machines lawsuit, voting machines in New Jersey used hardly any security seals. The primary issues were in my main areas of expertise: computer science and computer security.

Even so, when the state stuck a bunch of security seals on their voting machines in October 2008, I found that I could easily defeat them. I sent in a supplement expert report to the Court, explaining how.

Soon after I sent in my report about how to defeat all the State’s new seals, in January 2009 the State told the Court that it was abandoning all those seals, and that it had new seals for the voting machines. As before, I obtained samples of these new seals, and went down to my basement to work on them.

In a day or two, I figured out how to defeat all those new seals.

  • The vinyl tamper-indicating tape can be defeated using packing tape, a razor blade, and (optionally) a heat gun.
  • The blue padlock seal can be defeated with a portable drill and a couple of jigs that I made from scrap metal.
  • The security screw cap can be defeated with a $5 cold chisel and a $10 long-nose pliers, each custom ground on my bench grinder.

For details and pictures, see “Seal Regime #3” in this paper.

The main trick is this: just to know that physical seals are, in general, easy to defeat. Once you know that, then it’s just a matter of thinking about how to do it, and having a pile of samples on which to experiment. In fact, the techniques I describe in my paper are not the only way to defeat these seals, or the best way—not even close. These techniques are what an amateur could come up with. But these seal-defeats were good enough to work just fine when I demonstrated them in the courtroom during my testimony, and they would almost certainly not be detected by the kinds of seal-inspection protocols that most states (including New Jersey) use for election equipment.

(In addition, the commenters on my previous post describe a very simple denial-of-service attack on elections: brazenly cut or peel all the seals in sight. Then what will the election officials do? In principle they should throw out the ballots or data covered by those seals. But then what? “Do-overs” of elections are rare and messy. I suspect the most common action in this case is not even to notice anything wrong; and the second most common is to notice it but say nothing. Nobody wants to rock the boat.)

Super Bust: Due Process and Domain Name Seizure

With the same made-for PR timing that prompted a previous seizure of domain names just before shopping’s “Cyber Monday,” Immigration and Customs Enforcement struck again, this time days before the Super Bowl, against “10 websites that illegally streamed live sporting telecasts and pay-per-view events over the Internet.” ICE executed seizure warrants against the 10, ATDHE.NET, CHANNELSURFING.NET, HQ-STREAMS.COM, HQSTREAMS.NET, FIRSTROW.NET, ILEMI.COM, IILEMI.COM, IILEMII.COM, ROJADIRECTA.ORG and ROJADIRECTA.COM, by demanding that registries redirect nameserver requests for the domains to 74.81.170.110, where a colorful “This domain name has been seized by ICE” graphic is displayed.

This domain name has been seized

As in a previous round of seizures, these warrants were issued ex parte, without the participation of the owners of the domain names or the websites operating there. And, as in the previous rounds, there are questions about the propriety of the shutdowns. One of the sites whose domain was seized was Spanish site rojadirecta.com / rojadirecta.org, a linking site that had previously defeated copyright infringement claims in Madrid, its home jurisdiction. There, it prevailed on arguments that it did not host infringing material, but provided links to software and streams elsewhere on the Internet. Senator Ron Wyden has questioned the seizures, saying he “worr[ies] that domain name seizures could function as a means for end-running the normal legal process in order to target websites that may prevail in full court.”

According to ICE, the domains were subject to civil forfeiture under 18 U.S.C. § 2323(a), for “for illegally distributing copyrighted sporting events,” and seizure under § 981. That raises procedural problems, however: when the magistrate gets the request for seizure warrant, he or she hears only one side — the prosecutor’s. Without any opposing counsel, the judge is unlikely to learn whether the accused sites are general-purpose search engines or hosting sites for user-posted material, or sites providing or encouraging infringement. (Google, for example, has gotten many complaints from the NFL requesting the removal of links — should their domains be seized too?)

Now I don’t want to judge one way or the other based on limited evidence. Chilling Effects has DMCA takedown demands from several parties demanding that Google remove from its search index pages on some of these sites — complaints that are themselves one-side’s allegation of infringement.

What I’d like to see instead is due process for the accused before domain names are seized and sites disrupted. I’d like to know that the magistrate judge saw an accurate affidavit, and reviewed it with enough expertise to distinguish the location of complained-of material and the responsibility the site’s owners bear for it: the difference between direct, contributory, vicarious, and inducement of copyright infringement (for any of which a site-owner might be held liable, in appropriate circumstances) and innocent or protected activity. As Joe Hall has written here, domain names can’t defend themselves.

In the best case, the accused gets evidence of the case against him or her and the opportunity to challenge it. We tend to believe that the adversarial process, judgment after argument between the parties with the most direct interests in the matter, best and most fairly approaches the truth. These seizures, however, are conducted ex parte, with only the government agent presenting evidence supporting a seizure warrant. (We might ask why: a domain name cannot disappear or flee the jurisdiction if the accused is notified — the companies running the .com, .net, and .org registries where these were seized have shown no inclination to move or disregard US court orders, while if the name stops resolving, that’s the same resolution ICE seeks by force.)

If seizures must be made on ex parte affidavits, the magistrate judges should feel free to question the affiants and the evidence presented to them and to call upon experts or amici to brief the issues. In their review, magistrates should beware that a misfired seizure can cause irreparable injury to lawfully operating site-operators, innovators, and independent artists using sites for authorized promotion of their own materials.

I’d like to compile a set of public recommendations to the magistrate judges who might be confronted with these search warrants in the future, if ICE’s “Operation In Our Sites” continues. This would include verifying that the alleged infringements are the intended purpose of the domain name use, not merely a small proportion of a lawful general-use site.

Predictions for 2011

As promised, the official Freedom to Tinker predictions for 2011. These predictions are the result of discussions that included myself, Joe Hall, Steve Schultze, Wendy Seltzer, Dan Wallach, and Harlan Yu, but note that we don’t individually agree with every prediction.

  1. DRM technology will still fail to prevent widespread infringement. In a related development, pigs will still fail to fly.
  2. Copyright and patent issues will continue to be stalemated in Congress, with no major legislation on either subject.
  3. Momentum will grow for HTTPS by default, with several major websites adding HTTPS support. Work will begin on adding HTTPS-by-default support to Apache.
  4. Despite substantial attention by Congress to online privacy, the FTC won’t be granted authority to mandate Do Not Track compliance.
  5. Some advertising networks and third-party Web services will begin to voluntarily respect the Do Not Track header, which will be supported by all the major browsers. However, sites will have varying interpretations of what the DNT header requires, leading to accusations that some purportedly DNT-respecting sites are not fully DNT-compliant.
  6. Congress will pass an electronic privacy bill along the lines of the principles set out by the Digital Due Process Coalition.
  7. The seemingly N^2 patent lawsuits among all the major smartphone players will be resolved through a grand cross-licensing bargain, cut in a dark, smoky room, whose terms will only be revealed through some congratulatory emails that leak to the press. None of these lawsuits will get anywhere near a courtroom.
  8. Android smartphones will continue gaining market share, mostly at the expense of BlackBerry and Windows Mobile phones. However, Android’s gains will mostly be at the low end of the market; the iPhone will continue to outsell any single Android smartphone model by a wide margin.
  9. 2011 will see the outbreak of the first massive botnet/malware that attacks smartphones, most likely iPhone or Android models running older software than the latest and greatest. If Android is the target, it will lead to aggressive finger-pointing, particularly given how many users are presently running Android software that’s a year or more behind Google’s latest—a trend that will continue in 2011.
  10. Mainstream media outlets will continue building custom “apps” to present their content on mobile devices. They’ll fall short of expectations and fail to reverse the decline of any magazines or newspapers.
  11. At year’s end, the district court will still not have issued a final judgment on the Google Book Search settlement.
  12. The market for Internet set-top boxes like Google TV and Apple TV will continue to be chaotic throughout 2011, with no single device taking a decisive market share lead. The big winners will be online services like Netflix, Hulu, and Pandora that work with a wide variety of hardware devices.
  13. Online sellers with device-specific consumer stores (Amazon for Kindle books, Apple for iPhone/iPad apps, Microsoft for Xbox Live, etc.) will come under antitrust scrutiny, and perhaps even be dragged into court. Nothing will be resolved before the end of 2011.
  14. With electronic voting machines beginning to wear out but budgets tight, there will be much heated discussion of electronic voting, including antitrust concern over the e-voting technology vendors. But there will be no fundamental changes in policy. The incumbent vendors will continue to charge thousands of dollars for products that cost them a tiny fraction of that to manufacture.
  15. Pressure will continue to mount on election authorities to make it easier for overseas and military voters to cast votes remotely, despite all the obvious-to-everybody-else security concerns. While counties with large military populations will continue to conduct “pilot” studies with Internet voting, with grandiose claims of how they’ve been “proven” secure because nobody bothered to attack them, very few military voters will cast actual ballots over the Internet in 2011.
  16. In contrast, where domestic absentee voters are permitted to use remote voting systems (e.g., systems that transmit blank ballots that the voter returns by mail) voters will do so in large numbers, increasing the pressure to make remote voting easier for domestic voters and further exacerbating security concerns.
  17. At least one candidate for the Republican presidential nomination will express concern about the security of electronic voting machines.
  18. Multiple Wikileaks alternatives will pop up, and pundits will start to realize that mass leaks are enabled by technology trends, not just by one freaky Australian dude.
  19. The RIAA and/or MPAA will be sued over their role in the government’s actions to reassign DNS names owned by allegedly unlawful web sites. Even if the lawsuit manages to get all the way to trial, there won’t be a significant ruling against them.
  20. Copyright claims will be asserted against players even further removed from underlying infringement than Internet/online Service Providers: domain name system participants, ad and payment networks, and upstream hosts. Some of these claims will win at the district court level, mostly on default judgments, but appeals will still be pending at year’s end.
  21. A distributed naming system for Web/broadcast content will gain substantial mindshare and measurable US usage after the trifecta of attacks on Wikileaks DNS, COICA, and further attacks on privacy-preserving or anonymous registration in the ICANN-sponsored DNS. It will go even further in another country.
  22. ICANN still will not have introduced new generic TLDs.
  23. The FCC’s recently-announced network neutrality rules will continue to attract criticism from both ends of the political spectrum, and will be the subject of critical hearings in the Republican House, but neither Congress nor the courts will overturn the rules.
  24. The tech policy world will continue debating the Comcast/Level 3 dispute, but Level 3 will continue paying Comcast to deliver Netflix content, and the FCC won’t take any meaningful actions to help Level 3 or punish Comcast.
  25. Comcast and other cable companies will treat the Comcast/Level 3 dispute as a template for future negotiations, demanding payments to terminate streaming video content. As a result, the network neutrality debate will increasingly focus on streaming high-definition video, and legal academia will become a lot more interested in the economics of Internet interconnection.

2010 Predictions Scorecard

We’re running a little behind this year, but as we do every year, we’ll review the predictions we made for 2010. Below you’ll find our predictions from 2010 in italics, and the results in ordinary type. Please notify us in the comments if we missed anything.

(1) DRM technology will still fail to prevent widespread infringement. In a related development, pigs will still fail to fly.

We win again! There are many examples, but one that we predicted specifically is that HDCP was cracked. Guess what our first prediction for 2011 will be? Verdict: Right.

(2) Federated DRM systems, such as DECE and KeyChest, will not catch on.

Work on DECE (now renamed UltraViolet) continues to roll forward, with what appears to be broad industry support. It remains to be seen if those devices will actually work well, but the format seems to have at least “caught on” among industry players. We haven’t been following this market too closely, but given that KeyChest seems to mostly be mentioned as an also-ran in UltraViolet stories, its chances don’t look as good. Verdict: Mostly wrong.

(3) Content providers will crack down on online sites that host unlicensed re-streaming of live sports programming. DMCA takedown notices will be followed by a lawsuit claiming actual knowledge of infringing materials and direct financial benefits.

Like their non-live bretheren, live streaming sites like Justin.tv have received numerous DMCA takedown notices for copyrighted content. At the time of this prediction, we were unaware of the lawsuit against Ustream by a boxing promotional company, which began in August 2009. Nonetheless, the trend has continued. In the UK, there was an active game of cat-and-mouse between sports teams and live illegal restreaming sources for football (ahem: soccer) and cricket, which make much of their revenue on selling tickets to live matches. In some cases, a number of pubs were temporarily closed when their licenses were suspended in the face of complaints from content providers. In the US, Zuffa, the parent company for the mixed martial arts production company Ultimate Fighting Championship, sued when a patron at a Boston bar connected his laptop to one of the bar’s TVs to stream a UFC fight from an illicit site (Zuffa is claiming $640k in damages). In July, Zuffa subpoenaed the IP addresses of people uploading its content. And last week UFC sued Justin.tv directly for contributory and vicarious infringement, inducement, and other claims (RECAP docket). Verdict: Mostly right.

(4) Major newspaper content will continue to be available online for free (with ads) despite cheerleading for paywalls by Rupert Murdoch and others.

Early last year, the New York Times announced its intention to introduce a paywall in January 2011, and that plan still seems to be on track, but didn’t actually happen in 2010. The story is the same at the Philly Inquirer, which is considering a paywall but hasn’t put one in place. The Wall Street Journal was behind a paywall already. Other major papers, including the Los Angeles Times, the Washington Post, and USA Today, seem to be paywall-free. The one major paper we could find that did go behind a paywall is the Times of London went behind a paywall in July, with predictably poor results. Verdict: Mostly right.

(5) The Supreme Court will strike down pure business model patents in its Bilski opinion. The Court will establish a new test for patentability, rather than accepting the Federal Circuit’s test. The Court won’t go so far as to ban software patents, but the implications of the ruling for software patents will be unclear and will generate much debate.

The Supreme Court struck down the specific patent at issue in the case, but it declined to invalidate business method patents more generally. It also failed to articulate a clear new test. The decision did generate plenty of debate, but that went without saying. Verdict: Wrong.

(6) Patent reform legislation won’t pass in 2010. Calls for Congress to resolve the post-Bilski uncertainty will contribute to the delay.

Another prediction that works every year. Verdict: Right.

(7) After the upcoming rulings in Quon (Supreme Court), Comprehensive Drug Testing (Ninth Circuit or Supreme Court) and Warshak (Sixth Circuit), 2010 will be remembered as the year the courts finally extended the full protection of the Fourth Amendment to the Internet.

The Supreme Court decided Quon on relatively narrow grounds and deferred on the Fourth Amendment questions on electronic privacy, and the Ninth Circuit in Comprehensive Drug Testing dismissed the lower court's privacy-protective guidelines for electronic searches. However, the big privacy decision of the year was in Warshak, where the Sixth Circuit ruled strongly in favor of the privacy of remotely stored e-mail. Paul Ohm said of the decision: “It may someday be seen as a watershed moment in the extension of our Constitutional rights to the Internet.” Verdict: Mostly right.

(8) Fresh evidence will come to light of the extent of law enforcement access to mobile phone location-data, intensifying the debate about the status of mobile location data under the Fourth Amendment and electronic surveillance statutes. Civil libertarians will call for stronger oversight, but nothing will come of it by year’s end.

Even though we didn’t learn anything significant and new about the extent of government access to mobile location data, the debate around “cell-site” tracking privacy certainly intensified, in Congress, in the courts and in the public eye. The issue gained significant public attention through a trio of pro-privacy victories in the federal courts and Congress held a hearing on ECPA reform that focused specifically on location-based services. Despite the efforts of the Digital Due Process Coalition, no bills were introduced in Congress to reform and clarify electronic surveillance statutes. Verdict: Mostly right.

(9) The FTC will continue to threaten to do much more to punish online privacy violations, but it won’t do much to make good on the threats.

As a student of the FTC’s Chief Technologist, I’m not touching this one with a ten-foot pole.

(10) The new Apple tablet will be gorgeous but expensive. It will be a huge hit only if it offers some kind of advance in the basic human interface, such as a really effective full-sized on-screen keyboard.

Gorgeous? Check. Expensive? Check. Huge hit? Check. Advance in the basic human interface? The Reality Distortion Field forces me to say “yes.” Verdict: Mostly right.

(11) The disadvantages of iTunes-style walled garden app stores will become increasingly evident. Apple will consider relaxing its restrictions on iPhone apps, but in the end will offer only rhetoric, not real change.

Apple’s iPhone faced increasingly strong competition from Google’s rival Android platform, and it’s possible this could be attributed to Google’s more liberal policies for allowing apps to run on Android devices. Still, iPhones and iPads continued to sell briskly, and we’re not aware of any major problems arising from Apple’s closed business model. Verdict: Wrong.

(12) Internet Explorer’s usage share will fall below 50 percent for the first time in a decade, spurred by continued growth of Firefox, Chrome, and Safari.

There’s no generally-accepted yardstick for browser usage share, because there are so many different ways to measure it. But Wikipedia has helpfully aggregated browser usage share statistics. All five metrics listed there show the usage share falling by between 5 and 10 percent over the last years, with current values being between 41 to 61 percent. The mean of these statistics is 49.5 percent, and the median is 46.94 percent. Verdict: Right.

(13) Amazon and other online retailers will be forced to collect state sales tax in all 50 states. This will have little impact on the growth of their business, as they will continue to undercut local bricks-and-mortar stores on prices, but it will remove their incentive to build warehouses in odd places just to avoid having to collect sales tax.

State legislators continue to introduce proposals to tax out-of-state retailers, but Amazon has fought hard against these proposals, and so far the company has largely kept them at bay. Verdict: Wrong.

(14) Mobile carriers will continue locking consumers in to long-term service contracts despite the best efforts of Google and the handset manufacturers to sell unlocked phones.

Google’s experiment selling the Nexus One directly to consumers via the web ended in failure after about four months. T-Mobile, traditionally the nation’s least restrictive national wireless carrier, recently made it harder for consumers to find its no-contract “Even More Plus” plans. It’s still possible to get an unlocked phone if you really want one, but you have to pay a hefty premium, and few consumers are bothering. Verdict: Right.

(15) Palm will die, or be absorbed by Research In Motion or Microsoft.

This prediction was almost right. Palm’s Web OS didn’t catch on, and in April the company was acquired by a large IT firm. However, that technology firm was HP, not RIM or Microsoft. Verdict: Half right.

(16) In July, when all the iPhone 3G early adopters are coming off their two-year lock-in with AT&T, there will be a frenzy of Android and other smartphone devices competing for AT&T’s customers. Apple, no doubt offering yet another version of the iPhone at the time, will be forced to cut its prices, but will hang onto its centralized app store. Android will be the big winner in this battle, in terms of gained market share, but there will be all kinds of fragmentation, with different carriers offering slightly different and incompatible variants on Android.

Almost everything we predicted here happened. The one questionable prediction is the price cut, but we’re going to say that this counts. Verdict: Right.

(17) Hackers will quickly sort out how to install their own Android builds on locked-down Android phones from all the major vendors, leading to threatened or actual lawsuits but no successful legal action taken.

The XDA Developers Forum continues to be the locus for this type of Android hacking, and this year it did not disappoint. The Droid X was rooted and the Droid 2 was rooted, along with many other Android phones. The much-anticipated T-Mobile G2 came with a new lock-down mechanism based in hardware. HTC wasn’t initially forthcoming with the legally-mandated requirement to publish their modifications to the Linux source code that implemented this mechanism, but relented after a Freedom to Tinker post generated some heat. The crack took about a month, and now G2 owners are able to install their own Android builds. Verdict: Right.

(18) Twitter will peak and begin its decline as a human-to-human communication medium.

We’re not sure how to measure this prediction, but Twitter recently raised another $200 million in venture capital and its users exchanged 250 billion tweets in 2010. That doesn’t look like decline to us. Verdict: Wrong.

(19) A politican or a candidate will commit a high-profile “macaca”-like moment via Twitter.

We can’t think of any good examples of high-profile cases that severely affected a politician’s prospects in the 2010 elections, like the “macaca” comment did to George Allen’s 2006 Senate campaign. However, there were a number of more low-profile gaffes, including Sarah Palin’s call for peaceful muslims to “refudiate” the “Ground Zero Mosque” (the New Oxford American Dictionary named refudiate its word of the year), then-Senator Chris Dodd’s staff mis-tweeting inappropriate comments and a technical glitch in computer software at the U.S. embassy in Beijing tweeting that the air quality one day was “crazy bad”. Verdict: Mostly wrong.

(20) Facebook customers will become increasingly disenchanted with the company, but won’t leave in large numbers because they’ll have too much information locked up in the site.

In May 2010, Facebook once again changed its privacy policy to make more Facebook user information available to more people. On two occasions, Facebook has faced criticism for leaking user data to advertisers. But the site doesn’t seem to have declined in popularity. Verdict: Right.

(21) The fashionable anti-Internet argument of 2010 will be that the Net has passed its prime, supplanting the (equally bogus) 2009 fad argument that the Internet is bad for literacy.

Wired declared the web dead back in August. Is that the same thing as saying the Net has passed its prime? Bogus arguments all sound the same to us. Verdict: Mostly right.

(22) One year after the release of the Obama Administration’s Open Government Directive, the effort will be seen as a measured success. Agencies will show eagerness to embrace data transparency but will find the mechanics of releasing datasets to be long and difficult. Privacy– how to deal with personal information available in public data– will be one major hurdle.

Many people are calling this open government’s “beta period.” Federal agencies took the landmark step in January by releasing their first “high-value” datasets on Data.gov, but some advocates say these datasets are not “high value” enough. Agencies also published their plans for open government—some were better than others—and implementation of these promises has indeed been incremental. Privacy has been an issue in many cases, but it’s often difficult to know the reasons why an agency decides not to release a dataset. Verdict: Mostly right.

(23) The Open Government agenda will be the bright spot in the Administration’s tech policy, which will otherwise be seen as a business-as-usual continuation of past policies.

As we noted above, the Obama administration has had a pretty good record on open government issues. Probably the most controversial tech policy change has been the FCC’s adoption of new network neutrality rules. These weren’t exactly a continuation of Bush administration policies, but they also didn’t go as far as many activist groups wanted. And we can think of any other major tech policy changes. Verdict: Mostly right.

Our score: 7 right, 8 mostly right, 1 half right, 2 mostly wrong, 4 wrong.