As most people know by now, Hewlett-Packard was recently caught spying on its directors and employees, and some reporters, using methods that are probably illegal and certainly unethical. Throughout the scandal, we’ve heard a lot from HP spokesman Mike Moeller. This got my attention because Mike was my next-door neighbor in Palo Alto during my sabbatical five years ago. Mike and I spent more than a few evening and weekend hours chatting over the fence.

Now it is reported that one of the targets of HP’s spying was … Mike Moeller. An HP internal email turned over to investigators says, “New monitoring system that captures AOL Instant Messaging is now up and running and deployed on Moeller’s computer”. The company also reportedly had a detective follow Mike at a trade show, and they acquired his private phone records.

I wouldn’t have figured Mike as the type to leak boardroom secrets to the press, and indeed the spies found he had done nothing improper.

What’s interesting is that he is still serving as spokesman for HP. I’m not sure what to make of this. He must have been unhappy about being targeted; who wouldn’t be? But the essence of the spokesman’s job is to stay on message – the company’s message, not your own. Resigning in anger is not the spokesmanlike thing to do, and can’t be a good career move.

Heads have rolled at HP over the spying incident – as they should have – but the investigation is far from over. Executives claim not to have known what was going on, and not to have known it might be illegal, but those claims are hard to believe. Why would the company’s lawyers have allowed this to happen without getting careful legal opinions in advance? The most plausible reason is that they didn’t want to find out whether the spying tactics were legal, just as the executives probably didn’t want to find out how the information they received had been collected.

Obviously HP is not the only organization that did this. The investigators HP hired had plenty of other customers, and they are only part of a larger industry of private spies. Obtaining others’ phone records by identity theft is common enough to have its own euphemism: “pretexting”.

After the fallout at HP, expect more revelations about spying by other organizations. People will be more alert for spying, and they’ll know that revealing it can bring down the mighty. Meanwhile, law enforcement will be prying open the records of the “investigators,” finding more examples of reputable organizations that wanted information but didn’t want to be told where it came from.

Eventually the scandal at HP will blow over, and Mike Moeller’s job will return to normal. But maybe he’ll think twice before sending that next email or instant message to his family.

Today’s New York Times has an interesting article by Katie Hafner on AOL’s now-infamous release of customers’ search data.

AOL’s goal in releasing the data was to help researchers by giving them realistic data to study. Today’s technologies, such as search engines, have generated huge volumes of information about what people want online and why. But most of this data is locked up in the data centers of companies like AOL, Google, and eBay, where researchers can’t use it. So researchers have been making do with a few old datasets. The lack of good data is certainly holding back progress in this important area. AOL wanted to help out by giving researchers a better dataset to work with.

Somebody at AOL apparently thought they had “anonymized” the data by replacing the usernames with meaningless numbers. That was a terrible misjudgement – if there is one thing we have learned from the AOL data, it is that people reveal a lot about themselves in their search queries. Reporters have identified at least two of the affected AOL users by name, and finding and publishing embarrassing search sequences has become a popular sport.

The article quotes some prominent researchers, including Jon Kleinberg, saying they’ll refuse to work with this data on ethical grounds. I don’t quite buy that there is an ethical duty to avoid research uses of the data. If I had a valid research use for it, I’m pretty sure I could develop my own guidelines for using it without exacerbating the privacy problem. If I had had something to do with inducing the ill-fated release of the data, I might have an obligation to avoid profiting from my participation in the release. But if the data is out there due to no fault of mine, and the abuses that occur are no fault of mine, why shouldn’t I be able to use the data responsibly, for the public good?

Researchers know that this incident will make companies even more reluctant to release data, even after anonymizing it. If you’re a search-behavior expert, this AOL data may be the last useful data you see for a long time – which is all the more reason to use it.

Most of all, the AOL search data incident reminds us of the complexity of identity and anonymity online. It should have been obvious that removing usernames wasn’t enough to anonymize the data. But this is actually a common kind of mistake – simplistic distinctions between “personally identifiable information” and other information pervade the policy discussion about privacy. The same error is common in debates about big government data mining programs – it’s not as easy as you might think to enable data analysis without also compromising privacy.

In principle, it might have been possible to transform the search data further to make it safe for release. In practice we’re nowhere near understanding how to usefully depersonalize this kind of data. That’s an important research problem in itself, which needs its own datasets to work on. If only somebody had released a huge mass of poorly depersonalized data …