January 11, 2025

Two Stories about the Comcast/Level 3 Dispute (Part 1)

Like Steve and a lot of other people in the tech policy world, I’ve been trying to understand the dispute between Level 3 and Comcast. The combination of technical complexity and commercial secrecy has made the controversy almost impenetrable for anyone outside of the companies themselves. And of course, those who are at the center of the action have a strong incentive to mislead the public in ways that makes their own side look better.

So building on Steve’s excellent post, I’d like to tell two very different stories about the Level 3/Comcast dispute. One puts Level 3 in a favorable light and the other slants things more in Comcast’s favor.

Story 1: Level 3 Abuses Its Customer Relationships

As Steve explained, a content delivery network (CDN) is a network of caching servers that help content providers deliver content to end users. Traditionally, Netflix has used CDNs like Akamai and Limelight to deliver its content to customers. The dispute began shortly after Level 3 beat out these CDN providers for the Netflix contract.

Smart electrical meters and their smart peripherals

When I was a college undergraduate, I lived in a 1920’s duplex and I recall my roommate and I trying to figure out where our electrical bill was going. He was standing outside by the electrical meter, I was turning things on and off, and we were yelling back and forth so we could sort out which gadgets were causing the wheel to spin faster. (The big power sinks? Our ancient 1950’s refrigerator and my massive-for-the-day 20-inch computer monitor.) Needless to say, this was more difficult than it should have been.

More recently, I got myself a Kill-a-Watt inline power meter which you can use at any power outlet, but it’s a pain. You have to unplug something to measure its usage. People with the big bucks will spring for a Ted 5000 system, which an electrician installs in your breaker box. That’s fantastic, but it’s not cheap or easy.

Today, I’m now the proud new owner of an LS Research “RateSaver”, which speaks ZigBee wireless to the “smart meter” that CenterPoint Energy installed on all the houses in our area. How did I get this thing? I went to a League of Women Voters “meet the candidates” event back in October and CenterPoint Energy had a display there. I asked the guy if I could get one of these things and he said he’s look into it for me. Fast forward two months later, and a box arrived in the mail. New toy!

So what exactly is it? It’s a battery-powered light-weight box with a tolerably readable two-inch monochrome LCD display. As I’m sitting here typing, it’s updating my “current usage” every few seconds and is giving me a number that’s ostensibly accurate to the watt. In the last minute, after I pressed the proper button, it’s been alternating between reading 650-750 watts, and 1400-1500 watts. (Hmm… maybe my fridge consumes 700 watts.) If you leave it alone, the refresh rate slows down to maybe once a minute. Also, it’s sometimes reading “0.000 kW” which is clearly incorrect but it returns to the proper number when I press the button. Wireless range is quite good. I’m on the opposite side of the house as our electrical meter and it’s working fine.

The user interface is all kinds of terrible. In addition to slow button response, the button labels are incorrect. LS Research is apparently just rebranding a Honeywell Home Energy Display (for which the Honeywell manual was included). LS Research apparently rearranged the button labels without changing the corrresponding software. Bravo! Thankfully, the Honeywell manuals have the proper labeling. Also amusing: there’s a message in the system saying that “non-peak price starts at 7:00 PM. Save Money by waiting” when in fact my electrical pricing deal is for a flat rate (which floats with market conditions and is presently $0.0631 per kWh).

Update: I’ve since learned that Honeywell acquired LS Research, so this is something of a transitional screw-up. Welcome to the world of beta products.

Since I’m a security guy, I assumed I’d have to go through some kind of protocol to get the thing activated, and the manual from inside the box describes an activation procedure where you make a phone call to your energy company, giving them the hardware ID numbers of the outdoor smart meter and the indoor display box. Conflicting instructions were also included with my display, describing setup which was as simple as “turn it on and hit the connect button” so I went with the easy instructions. Time passed and the box started working without requiring any additional input from me. I hope that my display box was pre-configured to work exclusively with my house, but this does lead me to wonder about whether they got the security right. (I experimentally turned lights on and off while watching the meter updates and validated that I am, in fact, looking at the usage of my own house.)

At the end of the day, I and everybody else here is now required to pay a $3.24 “advanced meter charge” in order to have all this functionality (which, incidentally, saves the electric company money since it no longer needs human meter readers). Is it worth it? Presumably, at some point I’ll have some kind of variable-priced electricity and I could then hack my refrigerator and air conditioning system to pay attention to the spot price of electricity. If electricity got extra cheap during a five minute window of the hot summer, the controller could then crank the A/C and drop the house an extra few degrees. Of course, if everybody was following this same algorithm, you’d either have insane demand swings, when everybody jumps on to consume cheaper electricity when it’s available, or you’d have to carefully engineer the pricing system such that you had stable demand. Presumably, if you got somebody who understood control theory to design this properly, you could end up incentivizing both demand and pricing to be fairly stable across the space of any given hour of the day.

Probably the biggest benefit of these smart meters will come the next time we have a major hurricane that comes through and knocks out power. Hurricane Ike left my house without power for ten days. At the time, CenterPoint Energy had a vague and useless web site that would give you an idea what neighborhoods were being repaired. Since it was too hot to stay in our house, we stayed instead with a friend who had power and drove by our place every day to see if it had power. This was very frustrating. (I unplugged all my computer equipment, since I didn’t want flakey power to nuke my equipment. Consequently, I couldn’t just do something simple like ping my home computer.) Today, I can log into CenterPoint Energy’s web site and see the power consumption of my house, in 15-minute intervals, and so can the people coordinating the repairs. If they integrated that with a mapping system, they’d have real-time blackout maps, which have obvious value to emergency planners and repair operations coordination.

I just hope they have somebody with a clue looking over the security of their system. (If somebody from CenterPoint reads this: people like me are more than happy to do private security evaluations, red-team exercises, and so forth.)

Future work: there’s a mini USB port on the side of the box. Now I just have to find some documentation. It’s probably bad form for me to go reverse-engineer it myself.

Unpeeling the mystique of tamper-indicating seals

As computer scientists have studied the trustworthiness of different voting technologies over the past decade, we notice that “security seals” are often used by election officials. It’s natural to wonder whether they really provide any real security, or whether they are just for show. When Professor Avi Rubin volunteered as an election judge (Marylandese for pollworker) in 2006, one of his observations that I found most striking was this:


Avi Rubin


“For example, I carefully studied the tamper tape that is used to guard the memory cards. In light of Hursti’s report, the security of the memory cards is critical. Well, I am 100% convinced that if the tamper tape had been peeled off and put back on, nobody except a very well trained professional would notice it. The tamper tape has a tiny version of the word “void” appear inside it after it has been removed and replaced, but it is very subtle. In fact, a couple of times, due to issues we had with the machines, the chief judge removed the tamper tape and then put it back. One time, it was to reboot a machine that was hanging when a voter was trying to vote. I looked at the tamper tape that was replaced and couldn’t tell the difference, and then it occurred to me that instead of rebooting, someone could mess with the memory card and replace the tape, and we wouldn’t have noticed. I asked if I could play with the tamper tape a bit, and they let me handle it. I believe I can now, with great effort and concentration, tell the difference between one that has been peeled off and one that has not. But, I did not see the judges using that kind of care every time they opened and closed them. As far as I’m concerned, the tamper tape does very little in the way of actual security, and that will be the case as long as it is used by lay poll workers, as opposed to CIA
agents.”

Avi is a first-rate expert in the field of computer security, in part because he’s a good experimentalist—as in, “I asked if I could play with the tamper tape.” To the nonexpert,
security seals have a mystique: there’s this device there, perhaps a special tape or perhaps a thing that looks like a little blue plastic padlock. Most of us encounter these devices in a context where we can’t “play with” them, because that would be breaking the rules: on voting machines, on our electric meter, or whatever. Since we don’t play with them, we can’t tell whether they are secure, and the mystique endures. As soon
as Avi played with one, he discovered that it’s not all that secure.

In fact, we have a word for a piece of tape that only gives the appearance of working:

band-aid: (2) a temporary way of dealing with a problem that will not really solve it (Macmillan Dictionary)

In the last couple of years I’ve been studying security seals on voting machines in New Jersey. For many decades New Jersey law has required that each voting machine be “sealed with a numbered seal”, just after it is prepared for each election (NJSA 19:48-6). Unfortunately it’s hard for legislators to write into the statutes exactly how well these seals must work. Are tamper-indicating seals used in elections really secure? I’ll address that question in my next few articles.

CITP Seeks Visitors for 2011-2012

The Center for Information Technology Policy (CITP) seeks candidates for positions as visiting faculty members or researchers, or postdoctoral research associates for one year appointments for the 2011-2012 academic year. Please see our website for additional information and requirements at http://citp.princeton.edu/call-for-visitors/.

If you are interested, please submit a CV and cover letter, stating background, intended research, and salary requirements, to jobs.princeton.edu/applicants/Central?quickFind=60250.

Princeton University is an equal opportunity employer and complies with applicable EEO and affirmative action regulations.

What happens when there's no recount possible?

Greetings,

This is my first posting on Freedom To Tinker, so a brief introduction first. I’ve been involved in electronic voting technology issues for about five years, as founder of Virginia Verified Voting where I wrote the law that prohibited purchase of more DREs (i.e., paperless voting machines), as a researcher on the EAC Voting System Risk Assessment, a consultant to the Kentucky Attorney General, an advisor to the DC City Council, and (since I joined SRI International almost two years ago) as one of the investigators on the NSF ACCURATE program. It’s an honor to join this blog.

On to the real topic of this post.

I live in Fairfax County, Virginia. Virginia uses mostly paperless DREs (no VVPAT paper trail). Thanks to the law I helped write (see above), Fairfax County now has both DREs and optical scanners in polling places, and gives voters the choice which they want to use. As a pollworker on election day, my experience was that 80% pick DREs – the reason is subject for a future blog posting.

One of the closest races in the country in last week’s election was for House of Representatives in the Virginia 11th District where I live – Gerry Connolly (D) is the one-term incumbent, and Keith Fimian (R) is the primary challenger, along with five minor-party candidates. Connolly beat Fimian by 12 points in 2008, but everyone knew this race would be closer.

In fact, the unofficial totals as of this writing show Connolly ahead by about 900 votes out of 225,000, or about 0.40%. Close, but certainly not the closest I’ve seen. (The 2005 Attorney General’s race was decided by about 0.015% – yes, just over one one-hundred of a percent.) Virginia law allows for a recount when the margin is under one percent, so a recount would seem obvious.

However, The Washington Post reports that Fimian has decided not to seek a recount. Why? Well, there’s the official reason, but the real reason is two-fold.

First, a large majority of the votes in the 11th district race were cast on paperless DREs, spanning Fairfax County, Prince William County, and Fairfax City (a separate jurisdiction from Fairfax County). I don’t have the numbers yet, but my estimate is that 90% were on paperless DREs. So there’s nothing really to recount.

Second, Virginia law is very restrictive on recounts. For DREs, all you can do is look at the total tapes from election night, and add up the totals again. If the total tape is unreadable, you can print it again. But that’s it, without a judge’s order. For optical scans, you reprogram the optical scanner to count just the race in question, retest it, rerun the ballots, and use the total tape to add to the DRE total tape. If the scanner rejects any ballots (e.g., for write-ins), you can examine those by hand – but you can’t examine any that the scanner accepts, again without a judge’s order. And since the law gives the judge no guidance on why they should allow looking at anything other than the total tapes, it’s unlikely that a judge would make up his/her own rules.

So the result is that a recount is fairly meaningless – it’s really a retally of the totals, not an examination of the ballots, as a recount is meant to be.

Time for Virginia to both replace its DREs with optical scanners AND update its antiquated recount laws, as the Washington Examiner noted in their editorial yesterday.