Writing in this week’s New Yorker, Stacy Schiff takes a look at the Wikipedia phenomenon. One sign that she did well: The inevitable response page at Wikipedia is almost entirely positive. Schiff’s writing is typical of what makes the New Yorker great. It has rich historical context, apt portrayals of the key characters involved in the story, and a liberal sprinkling of humor, bons mots and surprising factual nuggets. It is also, as all New Yorker pieces are, rigorously fact-checked and ably edited.
Normally, I wouldn’t use FTT as a forum to “talk shop” about a piece of journalism. But in this case, the medium really is the message – the New Yorker’s coverage of Wikipedia is itself a showcase for some of the things old-line publications still do best. As soon as I saw Schiff’s article in my New Yorker table of contents (yes, I still read it in hard copy, and yes, I splurge on getting it mailed abroad to Oxford) I knew it would be a great test case. On the one hand, Wikipedia is the preeminent example of community-driven, user-generated content. Any coverage of Wikipedia, particularly any critical coverage, is guaranteed to be the target of harsh, well-informed scrutiny by the proud community of Wikipedians. On the other, The New Yorker’s writing is, indisputably, among the best out there, and its fact checking department is widely thought to be the strongest in the business.
When reading Wikipedia, one has to react to surprising claims by entertaining the possibility that they might not be true. The less plausible a claim sounds, the more skepticism one must have when considering it. In some cases, a glance at the relevant Talk page helps, since this can at least indicate whether or not the claim has been vetted by other Wikipedians. But not every surprising claim has backstory available on the relevant talk page, and not every reader has the time or inclination to go to that level of trouble for every dubious claim she encounters in Wikipedia. The upshot is that implausible or surprising claims in Wikipedia often get taken with a grain or more of salt, and not believed – and on the other hand, plausible-sounding falsehoods are, as a result of their seeming plausibility, less likely to be detected.
On the other hand, rigorous fact-checking (at least in the magazine context where I have done it and seen it) does not simply mean that someone is trying hard to get things right: It means that someone’s job depends on their being right, and it means that the particularly surprising claims in the fact-checked content in particular can be counted on to be well documented by the intense, aspiring, nervous young person at the fact checker’s desk. At TIME, for example, every single word that goes in to the magazine physically gets a check mark, on the fact-checkers’ copy, once its factual content has been verified, with the documentation of the fact’s truth filed away in an appropriate folder (the folders, in a holdover from an earlier era, are still called “carbons”). It is every bit as grueling as it sounds, and entirely worthwhile. The same system is in use across most of the Time, Inc. magazine publishing empire, which includes People, Fortune, and Sports Illustrated and represents a quarter of the U.S. consumer magazine market. It’s not perfect of course – reports of what someone said in a one-on-one interview, for example, can only ever be as good as the reporter’s notes or tape recording. But it is very, very good. In my own case, knowing what goes in to the fact-checking process at places like TIME and The New Yorker gives me a much higher level of confidence in their accuracy than I have when, as I often do, I learn something new from Wikipedia.
The guarantee of truth that backs up New Yorker copy gives its content a much deeper impact. Consider these four paragraphs from Schiff’s story:
The encyclopedic impulse dates back more than two thousand years and has rarely balked at national borders. Among the first general reference works was Emperor’s Mirror, commissioned in 220 A.D. by a Chinese emperor, for use by civil servants. The quest to catalogue all human knowledge accelerated in the eighteenth century. In the seventeen-seventies, the Germans, champions of thoroughness, began assembling a two-hundred-and-forty-two-volume masterwork. A few decades earlier, Johann Heinrich Zedler, a Leipzig bookseller, had alarmed local competitors when he solicited articles for his Universal-Lexicon. His rivals, fearing that the work would put them out of business by rendering all other books obsolete, tried unsuccessfully to sabotage the project.
It took a devious Frenchman, Pierre Bayle, to conceive of an encyclopedia composed solely of errors. After the idea failed to generate much enthusiasm among potential readers, he instead compiled a “Dictionnaire Historique et Critique,†which consisted almost entirely of footnotes, many highlighting flaws of earlier scholarship. Bayle taught readers to doubt, a lesson in subversion that Diderot and d’Alembert, the authors of the Encyclopédie (1751-80), learned well. Their thirty-five-volume work preached rationalism at the expense of church and state. The more stolid Britannica was born of cross-channel rivalry and an Anglo-Saxon passion for utility.
Wales’s first encyclopedia was the World Book, which his parents acquired after dinner one evening in 1969, from a door-to-door salesman. Wales—who resembles a young Billy Crystal with the neuroses neatly tucked in—recalls the enchantment of pasting in update stickers that cross-referenced older entries to the annual supplements. Wales’s mother and grandmother ran a private school in Huntsville, Alabama, which he attended from the age of three. He graduated from Auburn University with a degree in finance and began a Ph.D. in the subject, enrolling first at the University of Alabama and later at Indiana University. In 1994, he decided to take a job trading options in Chicago rather than write his dissertation. Four years later, he moved to San Diego, where he used his savings to found an Internet portal. Its audience was mostly men; pornography—videos and blogs—accounted for about a tenth of its revenues. Meanwhile, Wales was cogitating. In his view, misinformation, propaganda, and ignorance are responsible for many of the world’s ills. “I’m very much an Enlightenment kind of guy,†Wales told me. The promise of the Internet is free knowledge for everyone, he recalls thinking. How do we make that happen?
As an undergraduate, he had read Friedrich Hayek’s 1945 free-market manifesto, “The Use of Knowledge in Society,†which argues that a person’s knowledge is by definition partial, and that truth is established only when people pool their wisdom. Wales thought of the essay again in the nineteen-nineties, when he began reading about the open-source movement, a group of programmers who believed that software should be free and distributed in such a way that anyone could modify the code. He was particularly impressed by “The Cathedral and the Bazaar,†an essay, later expanded into a book, by Eric Raymond, one of the movement’s founders. “It opened my eyes to the possibility of mass collaboration,†Wales said.
After reading this copy, and knowing how The New Yorker works, one can be confident that a devious Frenchman named Pierre Bayle once really did propose an encyclopedia comprised entirely of errors. The narrative is put together well. It will keep people reading and will not cause confusion. Interested readers can follow up on a nugget like Wales’ exposure to the Hayek essay by reading it themselves (it’s online here).
I am not a Wikipedia denialist. It is, and will continue to be, an important and valuable resource. But the expensive, arguably old fashioned approach of The New Yorker and other magazines still delivers a level of quality I haven’t found, and do not expect to find, in the world of community-created content.
The New Yorker ran a long and relatively positive piece about Wikipedia–I argued that the old-media method of laboriously checking each fact was superior to the wiki model, where assertions have to be judged based on their plausibility. I claimed that personal experience
Online Games
What an interesting way to get people interested in reading! Book trailers are like movie trailers, but for books! You can find them all over the internet now, but here is a site that’s featuring them on YouTube. http://www.youtube.com/booktrailers
Your blog regarding The New Yorker Covers Wikipedia looks very interesting to me. I found it doing a search for computer consulting indiana.
The extract from the New Yorker piece does not seem to me to be exhaustively checked for facts. The historical chapters are a very much potted history. The second paragraph, on Pierre Bayle, should come before the material on the eighteenth-century Germans. And as to the encyclopedia of errors, the source for the article must have had Sir Thomas Browne in mind. From the Wikipedia article on him, we read:
In 1646, Browne published Pseudodoxia Epidemica, or, Enquiries into Very many Received Tenets, and commonly Presumed Truths, whose title refers to the prevalence of false beliefs and “vulgar errors.” A sceptical work that debunks a number of legends circulating at the time in a paradoxical and witty manner, it displays the Baconian side of Browne—the side that was unafraid of what at the time was still called “the new learning.” The book is significant in the history of science.
Wikipedia is now my own first stop for serious research, followed if necessary by broader Internet searching. Paper books are now a last resort, for two reasons: cost and lack of full-text search. (No library within walking range here, so even library borrowing would incur travel costs — fuel or bus fare or taxi fare — and again when the thing had to be returned…) Of course, I own copies of some reference works in important areas, which have indices, but even so…
There is a great opportunity for the Wiki constructs to support a new form of corporate knowledge management, a distributed one whereby the model consist in the corporation providing server space and basic facilities (including an intranet search engine) with all “knowledge workers” can contribute — This model is fundamentally different to the current one which tend to be extremely restrictive as to what can be posted, the format and taxonomy that can be used etc…This article makes the case that some governance is still required to achieve a balance between allowing creativity unbridled and imposing suffocating normalization in the harvesting and publication of knowledge in the enterprise.
I don’t completely trust wikipedia, but I don’t trust the New Yorker either. In some recent articles on topics I know well, I’ve found numerous basic factual errors (streets misnamed, claiming that major public events took place on the wrong day, etc.).
The lesson is that everything should be verified. Preferably twice.
Yeah, I agree…Wikipedia has to be seen as what it is-a great place to look up a pop culture topic, and to see what info is out there, but it can’t be seen as a place to use as a reference for a paper, or as a scholarly website.
A bigger point perhaps is that a lot of “research” is now done by reference to on-line content and resources. It follows that fact checking for efficient news papers must move in the same direction (or invent a new business model to justify the inefficiency), so does that mean fact checking will deteriorate or things like wikipedia will improve ?.
In any case it’s well understood that news papers are declining as a source of news for most people, and this blog reads as though it could have been written by a business trying to support a failing business model (similar to the feel of previous blogs about DRM being nothing to worry about).
I wonder how they checked the “fact” that Jimmy Wales’ male-oriented Internet portal had only 1/10 of its revenue from pornography. From my experience and understanding, this kind of material plays a much more dominant role in male-oriented recreational activities. And if this “fact” came from Mr. Wales himself, he would have a natural motivation to downplay the degree to which he owes his wealth to a career as a pornographer.
Also, one issue you fail to mention in comparison is the issue of scale. Although the New Yorker/Time process is one of rigorous fact checking, they are not obliged to report about every single topic in the horizon. For a magazine article on a foucsed topic, this rigorous fact checking is possible. But to do this (or quality control of even a comparably low order of magnitude) for a project of the scale of an encyclopedia would be difficult for just a pool of editors. This is where the community driven encyclopedia is bound to scale to higher levels of quality.
I agree with what you’re saying about The New Yorker and Time, but I think the fact-checking process you describe is limited to prestige magazines. I used to edit computer trade publications, and there was basically no fact-checking process. Quotes and facts would only be checked in egregious cases. Friends who work at daily newspapers have said similar things – the emphasis is getting the paper out the door, rather than getting everything exactly right.
A lot of the debate about WIkipedia seems to hinge on professional standards versus amateur production; newspapers seem to have a vested interest in protecting their institutional credibility, so they run stories critical of the information available in Wikipedia and blogs. While I doubt many journalists would deliberately run false information like some Wikipedia editors, I think the fact-checking issue is probably a canard in talking about Wikipedia. If anything, as Scott points out, Wikipedia provides more opportunities than traditional publishing for verifying information in the long run.
As an aside, there is a wiki deliberately full of errors, Uncyclopedia.
“With enough eyeballs, all bugs are shallow.” Or however the saying goes.
Maybe Wikipedia gets to be as good as it is because it allows so easily for everyone to contribute and fix things. Human systems tend to contain error and distortion — it’s possible that our best hope is for a system that allows more people to correct the errors.
I don’t deny there are challenges with this open model, but I love how Wikipedia is growing and with each article I contribute to and each link I make to it from my own web site, I hope it continues to flourish and maintain (and even improve) quality. (If for no other reason than saving me the effort of going back and updating all those links if the the thing tanks.)
The only “error” that I recall seeing in the New Yorker was one less of facts, and more of not understanding statistical methods. Paul Brodeur wrote an article or two about electronic fields and cancer, and though the facts were correct, an understanding of statistical methods was totally lacking. The two examples I remember were
(1) the story of a cancer cluster in a California School, where the statisticians (rightly) expanded their comparison sample each time they were told “but what about that cancer over there (in another part of the school”. This was spun as a nefarious dilution of the sample.
(2) a straightforward reporting of a Scandinavian power line study, where the researchers accidentally data dredged (too many hypotheses for their sample size, or else post hoc search for an arbitrary number of “patterns”).
This made me a lot more careful about New Yorker stories, and downright paranoid about everything else I read. Given the totally-wrong things I sometimes read in the paper (never mind advertising and political speech) it’s a wonder that Wikipedia is as good as it is.