[We’re pleased to introduce Luis Villa as a guest blogger. Luis is a law student at Columbia Law School, focusing on law and technology, including intellectual property, telecommunications, privacy, and e-commerce. Outside of class he serves as Editor-in-Chief of the Science and Technology Law Review. Before law school, Luis did great work on open source projects, and spent some time as “geek in residence” at the Berkman Center. — Ed]
[A big thanks to Ed, Alex, and Tim for the invitation to participate at Freedom To Tinker, and the gracious introduction. I’m looking forward to my stint here. — Luis]
A couple weeks ago at the Web 2.0 Expo NY, I more-or-less stumbled into a speech by Clay Shirky titled “It’s Not Information Overload, It’s Filter Failure.” Clay argues that there has always been a lot of information, so our modern complaints about information overload are more properly ascribed to a breakdown in the filters – physical, economic, and social- that used to keep information at bay. This isn’t exactly a shockingly new observation, but now that Clay put it in my head I’m seeing filters (or their absence) everywhere.
In particular, I’m seeing lots of great examples in online politics. We’ve probably never been so deluged by political information as we are now, but Clay would argue that this is not because there is more information- after all, virtually everyone has had political opinions for ages. Instead, he’d say that the old filters that kept those opinions private have become less effective. For example, social standards used to say ‘no politics at the dinner table’, and economics used to keep every Luis, Ed, and Alex from starting a newspaper with an editorial page. This has changed- social norms about politics have been relaxed, and ‘net economics have allowed for the blooming of a million blogs and a billion tweets.
Online political filtering dates back at least to Slashdot’s early attempts to moderate commenters, and criticism of them stretches back nearly as far. But the new deluge of political commentary from everyone you know (and everyone you don’t) rarely has filtering mechanisms, norms, or economics baked in yet. To a certain extent, we’re witnessing the birth of those new filters right now. Among the attempts at a ‘new filtering’ that I’ve seen lately:
- The previously linked election.twitter.com. This is typical of the twitter ‘ambient intimacy‘ approach to filtering- everything is so short and so transient that your brain does the filtering for you (or so it is claimed), giving you a 100,000 foot view of the mumblings and grumblings of a previously unfathomably vast number of people.
- fivethirtyeight.com: an attempt to filter the noise of the thousands of polls into one or two meaningful numbers by applying mathematical techniques originally developed for analysis of baseball players. The exact algorithms aren’t disclosed, but the general methodologies have been discussed.
- The C-Span Debate Hub: this has not reached its full potential yet, but it uses some Tufte-ian tricks to pull data out of the debates, and (in theory) their video editing tool could allow for extensive discussion of any one piece of the debate, instead of the debate as a whole- surely a way to get some interesting collection and filtering.
- Google’s ‘In Quotes’: this takes one first step in filtering (gathering all candidate quotes in one place, from disparate, messy sources) but then doesn’t build on that.
Unfortunately, I have no deep insights to add here. Some shallow observations and questions, instead:
- All filters have impacts- keeping politics away from the dinner table tended to mute objections to the status quo, the ‘objectivity’ of the modern news media filter may have its own pernicious effects, and arguably information mangled by PowerPoint can blow up Space Shuttles. Have the designers of these new political filters thought about the information they are and are not presenting? What biases are being introduced? How can those be reduced or made more transparent?
- In at least some of these examples the mechanisms by which the filtering occurs are not a matter of record (538’s math) or are not well understood (twitter’s crowd/minimal attention psychology). Does/should that matter? What if these filters became ‘dominant’ in any sense? Should we demand the source for political filtering algorithms?
- The more ‘fact-based’ filters (538, inquotes) seem more successful, or at least more coherent and comprehensive. Are opinions still just too hard to filter with software or are there other factors at work here?
- Slashdot’s nearly ten year old comment moderation system is still quite possibly the least bad filter out there. None of the ‘new’ politics-related filters (that I know of) pulls together reputation, meta-moderation, and filtering like slashdot does. Are there systemic reasons (usability, economics, etc.?) why these new tools seem so (relatively) naive?
We’re entering an interesting time. Our political process is becoming both less and more mediated– more ‘susceptible to software’ in Dave Weinberger’s phrase. Computer scientists, software interaction designers, and policy/process wonks would all do well to think early and often about the filters and values embedded in this software, and how we can (and can’t) ‘tinker’ with them to get the results we’d like to see.
We saw a huge amount of emails and even text messages about Mitt Romney coming from a company called LDS Living. How are they able to send text message to our phones if we dont give out our cel phone address? LDS Living says on their site that they have 600,000 subscribers. I just don’t recall ever giving my cel phone number to LDS Living or anyone else.
I think in this day and age its up to the user to provide their own filters!
Regarding the remark “So I’ve come up, in some sense, through a meritocracy …”, I was remiss in not linking to the essay
“Guys Don’t Link”
http://burningbird.net/connecting/guys-dont-link/
I would suggest there are perhaps more bloggers (of various caliber and writing on various subjects) than like-minded individuals have time to read or even converse with, aside from “matching failures”. One “gatekeeper” group are clearly search engines ranking by popularity (positive feedback?), another is sparse social associations (word of mouth), which also have positive-feedback characteristics.
Positive feedback implies success is to a large extent a function of lucky timing (being in the “right place” at the “right time” – or its variant, doing the “right thing” at the “right time” — both usually are only apparent in hindsight), and the ability to effectively work social connections (and being favorably connected to boot).
It’s similar to labor markets — more contenders (genuinely qualified as well as posers) than positions in any given specialty, and success being more a matter of good networking and matching the right employer “preferences” than adding to the deluge of resumes; failing that the chance factor of “walking in the door” at the right time.
In shorter words, even when a society (or large enough group) is not strictly hierarchical, interaction will be structured around a relatively sparse network of “influential” individuals/subgroups. Positive feedback will make sure of that.
Does this make sense to you?
Andrew: that is interesting, great to know. Particularly salient in light of 538’s own attack on the non-transparent methodology of Real Clear Politics.
Regarding 538.com not releasing their source code — Prof. Sam Wang and I have always kept the source for http://election.princeton.edu completely open, both the 2004 version in just Matlab, and the current version in Python, Matlab, and shell script. In fact, the directories visible from http://election.princeton.edu/code/ are symbolic links to the live directories, and if you refresh during an update, you can see all the temporary files being generated and moved around. The whole process for each update is documented at http://election.princeton.edu/for-fellow-geeks/
Since our source code is open to the public, we’ve had a lot of excellent and informed feedback from readers. Some have suggested more efficient ways to do the calculations, while others have explored (and debated) the specific details of how we calculate confidence intervals, etc. Truly, the only way to perform objective “science” is out in the open.
Luis, you’re arguing something I call the Lottery Winner’s Fallacy – that you won the lottery doesn’t prove lotteries are a new form of economic justice. What about all the other people who didn’t win?
I could give links for a while, but I suspect it wouldn’t do any good – recursively, because you’re “up there” and I’m “down here”, which sort of proves the point. It doesn’t matter if I’m right, because I’ll never be heard.
The bogosphere shorthand for the limits of meritocracy is “WHERE ARE THE WOMEN”
(the extent of gatekeeping is very evident then!)
It’s a law of the universe that for any communication channel, someone will try to subvert that channel to their own advantage, and there will be a power struggle between the people trying to get their message across and the gatekeepers trying to ownzor the medium.
My email box is a constant battleground of messages versus filters — a widespread problem. Probably 90% of the messages sent to me are complete rubbish, but there are a small number of incredibly interesting messages.
Why do I post on “Freedom to Tinker” ? Certainly not because of some A-list popularity contest. This is hardly a mainstream blog by any measure. Of the fringe blogs dealing in the public ramifications of cryptography and election technology, freedom-to-tinker probably stands out (in a very small crowd, not many people discuss these issues). Mostly, I post here because I read here. Ideas pop up around here that I wouldn’t have thought of myself, and these particular topics don’t get much airplay. It seems common sense to be cooperative and post a related response next to the original idea (even when pointing out obvious problems).
Let’s consider a practical example of news and filtering: should I buy gold? I want information on the price of gold and something that I can use to predict future prices (relative to inflation of course). Clearly, the major banks and especially the US government (right before election) don’t want to see panic buying of gold, so they are working hard to keep the price down (yes, that’s market manipulation, it’s an exercise of power, powerful people do that kind of stuff, get shocked, then get over it). On the other hand, the intrinsics of the situation are pushing the price up (risk of inflation, uncertainty about war, other investments tanking). Thus, we have conflicting forces (as per normal), and additional information can sway the balance for individual decision-making.
Where do I go looking for information? If I read the mainstream press, I can be sure that they will follow the party line and never advocate the buying of gold (check it yourself), some of these propaganda articles are worth a laugh but no new information. On the other hand, places like the “Free Market News Network”, do have articles advocating that I buy gold (and also a bunch of other gold-investor websites). However, if you roll back the archives, FMNN always push for people to buy gold, and they are Ron Paul supporters to boot.
So the cat never barks, and the dog never purrs, and a predictable outcome is no news at all.
Here we have a real-world example where both sides of the coin (presumably a debased coin) are filtered in such a way as to block the opposing point of view. I would argue that this is the most common case for politically sensitive topics. In the gold market, what you really need is the raw data on supply (refinery production, mine activity, prospecting, etc) plus an inside view into what the major banks are doing (what their reserves are like, how much they buy and sell). Some of this critical information is available in bits and pieces, most is not available. Trying to factor in public perception, you might as well take a guess from gut estimate. However, knowing that various sources are filtered and (more importantly) knowing WHY those sources are filtered puts you a long way ahead of the average punter who takes things on face value.
Matt: doing the filtering yourself is certainly empowering, and it is great that we have that option where we didn’t before. But the reality is that it doesn’t scale very well- very few of us have the time to do a good job of it. So we’ll likely end up delegating that filtering job to others; we just now have a much broader range of filters to choose from.
Seth: I understand the frustration; I’d certainly be one of the first to say that none of this is perfect at all, and of course some of it is (more or less) mass opiates. But I do think that there are new opportunities for new voices that there weren’t previously. The reason I have the opportunity here is because of my previous posting on my own blog, and commenting on the blogs of others. So I’ve come up, in some sense, through a meritocracy that wouldn’t have been open to me 10 or 15 years ago. Yes, there are gatekeepers involved, but like I said, that is only one facet of the broader concept, and the gatekeepers are wildly different than they were even very recently.
No word is a perfect fit.
But there is a reason you are posting this on “freedom-to-tinker.com” rather than your own blog. There is a gatekeeper, with all the social implications that implies, rather than a piece of electronic equipment.
No personal offense intended. But from the bottom, hearing the fantasy over and over from “on high”, it is very frustrating.
Have another link:
“The Great Unread”
http://www.roughtype.com/archives/2006/08/the_great_unrea.php
I always saw the new lack of filters as a good think which forces people to enact their own filters. Obviously you cannot trust what every website says about everything, so you need to develop criteria that helps you tell the waste form the web. Sure twitter has an election section, but you can still go and search for every tweet that has been posted about the election. In other words the information is still there they are just putting it in one place. Thats what makes Google’s approach so appealing. You say they don’t do anything after they assemble the quotes what would you want them to do? Do you want them to give analysis? I don’t think thats Google’s job. I think in this day and age its up to the user to provide their own filters
Not all filters are gatekeepers, as some are not people but more passive entities (social norms, economics, etc.). So ‘gatekeepers’ doesn’t cover what Clay (or I) were trying to get across. But thanks as always for your civil and constructive tone, Seth. 😉
“economics used to keep every Luis, Ed, and Alex from starting a newspaper with an editorial page”
I am sooo tired of seeing this nonsense. You have no ability for ” starting a newspaper”. Unless a person has some sort of connection to an existing gatekeeping system
YOU WILL NOT BE HEARD. Anything else is the cruel delusion-peddling of blog-evangelism.
The better word is “gatekeepers”. A new system leads to new gatekeepers.
Here’s an excellent essay on the phenomena:
Jon Garfunkel: “The New Gatekeepers”
http://civilities.net/TheNewGatekeepers