April 20, 2014

avatar

Secrecy in Science

There’s an interesting dispute between astronomers about who deserves credit for discovering a solar system object called 2003EL61. Its existence was first announced by Spanish astronomers, but another team in the U.S. believes that the Spaniards may have learned about the object due to an information leak from the U.S. team.

The U.S. team’s account appears on their web page and was in yesterday’s NY Times. The short version is that the U.S. team published an advance abstract about their paper, which called the object by a temporary name that encoded the date it had been discovered. They later realized that an obscure website contained a full activity log for the telescope they had used, which allowed anybody with a web browser to learn exactly where the telescope had been pointing on the date of the discovery. This in turn allowed the object’s orbit to be calculated, enabling anybody to point their telescope at the object and “discover” it. Just after the abstract was released, the Spanish team apparently visited the telescope log website; and a few days later the Spanish team announced that they had discovered the object.

If this account is true, it’s clearly a breach of scientific ethics by the Spaniards. The seriousness of the breach depends on other circumstances which we don’t know, such as the possibility that the Spaniards had already discovered the object independently and were merely checking whether the Americans’ object was the same one. (If so, their announcement should have said that the American team had discovered the object independently.)

[UPDATE (Sept. 15): The Spanish team has now released their version of the story. They say they discovered the object on their own. When the U.S. group's abstract, containing a name for the object, appeared on the Net, the Spaniards did a Google search for the object name. The search showed a bunch of sky coordinates. They tried to figure out whether any of those coordinates corresponded to the object they had seen, but they were unable to tell one way or the other. So they went ahead with their own announcement as planned.

This is not inconsistent with the U.S. team's story, so it seems most likely to me that both stories are true. If so, then I was too hasty in inferring a breach of ethics, for which I apologize. I should have realized that the Spanish team might have been unable to tell whether the objects were the same.]

When this happened, the American team hastily went public with another discovery, of an object called 2003UB313 which may be the tenth planet in our solar system. This raised the obvious question of why the team had withheld the announcement of this new object for as long as they did. The team’s website has an impassioned defense of the delay:

Good science is a careful and deliberate process. The time from discovery to announcement in a scientific paper can be a couple of years. For all of our past discoveries, we have described the objects in scientific papers before publicly announcing the objects’ existence, and we have made that announcement in under nine months…. Our intent in all cases is to go from discovery to announcement in under nine months. We think that is a pretty fast pace.

One could object to the above by noting that the existence of these objects is never in doubt, so why not just announce the existence immediately upon discovery and continue observing to learn more? This way other astronomers could also study the new object. There are two reasons we don’t do this. First, we have dedicated a substantial part of our careers to this survey precisely so that we can discover and have the first crack at studying the large objects in the outer solar system. The discovery itself contains little of scientific interest. Almost all of the science that we are interested in doing comes from studying the object in detail after discovery. Announcing the existence of the objects and letting other astronomers get the first detailed observations of these objects would ruin the entire scientific point of spending so much effort on our survey. Some have argued that doing things this way “harms science” by not letting others make observations of the objects that we find. It is difficult to understand how a nine month delay in studying an object that no one would even know existed otherwise is in any way harmful to science!

Many other types of astronomical surveys are done for precisely the same reasons. Astronomers survey the skies looking for ever higher redshift galaxies. When they find them they study them and write a scientific paper. When the paper comes out other astronomers learn of the distant galaxy and they too study it. Other astronomers cull large databases such as the 2MASS infrared survey to find rare objects like brown dwarves. When they find them they study them and write a scientific paper. When the paper comes out other astronomers learn of the brown dwarves and they study them in perhaps different ways. Still other astronomers look around nearby stars for the elusive signs of directly detectable extrasolar planets. When they find one they study it and write a scientific paper….. You get the point. This is the way that the entire field of astronomy – and probably all of science – works. It’s a very effective system; people who put in the tremendous effort to find these rare objects are rewarded with getting to be the first to study them scientifically. Astronomers who are unwilling or unable to put in the effort to search for the objects still get to study them after a small delay.

This describes an interesting dynamic that seems to occur in all scientific fields – I have seen it plenty of times in computer science – where researchers withhold results from their colleagues for a while, to ensure that they get a headstart on the followup research. That’s basically what happens when an astronomer delays announcing the discovery of an object, in order to do followup analyses of the object for publication.

The argument against this secrecy is pretty simple: announcing the first result would let more people do followup work, making the followup work both quicker and more complete on average. Scientific discovery would benefit.

The argument for this kind of secrecy is more subtle. The amount of credit one gets for a scientific result doesn’t always correlate with the difficulty of getting the result. If a result is difficult to get but doesn’t create much credit to the discoverer, then there is an insufficient incentive to look for that result. The incentive is boosted if the discoverer gets an advantage in doing followup work, for example by keeping the original result secret for a while. So secrecy may increase the incentive to do certain kinds of research.

Note that there isn’t much incentive to keep low-effort / high-credit research secret, because there are probably plenty of competing scientists who are racing to do such work and announce it first. The incentive to keep secrets is biggest for high-effort / low-credit research which enables low-effort / high-credit followup work. And this is exactly the case where incentives most need to be boosted.

Michael Madison compares the astronomers’ tradeoff between publication and secrecy to the tradeoff an inventor faces between keeping an invention secret, and filing for a patent. As a matter of law, discovered scientific facts are not patentable, and that’s a good thing.

As Madison notes, science does have its own sort of “intellectual property” system that tries to align incentives for the public good. There is a general incentive to publish results for the public good – scientific credit goes to those who publish. Secrecy is sometimes accepted in cases where secret-keeping is needed to boost incentives, but the system is designed to limit this secrecy to cases where it is really needed.

But this system isn’t perfect. As the astronomers note, the price of secrecy is that followup work by others is delayed. Sometimes the delay isn’t too serious – 2003UB313 will still be plodding along in its orbit and there will be plenty of time to study it later. But sometimes delay is a bigger deal, as when an astronomical object is short-lived and cannot be studied at all later. Another example, which arises more often in computer security, is when the discovery is about an ongoing risk to the public which can be mitigated more quickly if it is more widely known. Scientific ethics tend to require at least partial publication in cases like these.

What’s most notable about the scientific system is that it works pretty well, at least within the subject matter of science, and it does so without much involvement by laws or lawyers.

Comments

  1. Tom Slee says:

    The best example of preserving secrecy I have heard of was during the high temperature superconductor early days, when Paul Chu submitted an important paper to Phys Rev Lett about a compund YbBCO (Yb = Ytterbium) and then changed the Yb to Y (Yttrium) at, I believe, the galley proof stage.

    There is more at http://www.physics.emich.edu/mthomsen/thom2.htm

  2. Liudvikas Bukys says:

    The best example of scholars preserving scarcity (the discovery was known and not secret, scholarly access was limited to a select few) was the Dead Sea Scrolls, with unpublished scroll fragments hoarded for decades by scholars optimizing their personal influence instead of general knowledge. It came to a head in 1991 when one group reverse-engineered a 1988 concordance, then another group published photographs of the secret materials. There’s an excellent summary in Wikipedia.

  3. Hal says:

    Another problem is that this habit of secrecy carries over into other areas of science. Many times labs are unable to replicate other reported results because some of the techniques and tricks necessary to make things work are unpublished. This gives the reporting lab an advantage in publishing follow-on work. In practice the only way that the truth gets out is via exchange of personnel, as grad students, post-docs and visiting researchers move around from lab to lab, carrying the secrets with them. But this is all under the control of the originating lab and they will often be credited as co-authors with joint work, etc. It just adds to the power of senior scientists.

    On the one hand, the bottom line is that science works and has been extremely successful. But on the other hand, there is a disconnect between the idealistic outside view of science as a grand community of shared information, and the insider’s reality of hoarding, empire building and secrecy. One specific negative impact is on students who go into the sciences, many of whom become disillusioned as they find out how ugly the process is behind closed doors. People leave science as a result, and so we do not have as effective a scientific community as we might if science did a better job of honoring its ideals.

  4. WhyNow? says:

    Nice article! I completely agree with your sentiment that the system works pretty well without bylaws and lawyers.

    Curiously, I checked out Michael Brown’s website at Caltech some time back. I could swear that these statements appeared on his web site: “There is no question that the Spanish group is rightly credited with discovery. Even if they had found the object only this year and announced its existence, they would still be considered the rightful discovers. We took a chance that no one else would find it while we were awaiting our observations from the Spitzer Space Telescope. We were wrong! And we congratulate our colleagues on a very nice discovery.”

    I could no longer find them, but an article in Universe Today also quotes Michael Brown’s statement (see http://www.universetoday.com/am/publish/10th_planet_controversy.html). Google’s cache also indicates as much (unless I am misinterpreting it).

    I was really impressed by this stance and humility. But reading this today, I guess he correlated access to their ‘obscure’ website with the announcement and has changed his mind? (see http://www.gps.caltech.edu/~mbrown/planetlila/#hack).

  5. Hal says:

    I’ve done a little more reading about this case. The Spanish team which apparently peeked at the Caltech group’s observing logs has taken down their announcement page, but they missed a copy which is online here, http://www.iaa.es/~psantos/

    You can see on this page that the Spanish team did in fact image the object in 2003. They apparently had a huge backlog of data and had to go back and analyze this specific sequence of images in order to find the object. The question is whether they used the information from Caltech’s observing logs to decide where to look.

    As an amateur astronomer, I am a little skeptical about this. The Caltech observations were made May 6, 2004. The Spanish observations were March 7-10, 2003. The question is, just from information about telescope pointing in the Caltech observations, could someone deduce orbital elements precise enough to tell them where to look in their backlog of data from the year before? In general, I think this would be quite difficult. The telescope is not pointed directly at the object, rather it takes in a large field of view and the object is somewhere in it. You can see that in the animation on the Spanish URL I posted above. Just knowing the region of the sky it’s in is not going to tell you how fast or in what precise direction it is moving.

    Now, given that it’s a Kuiper belt object, they would nevertheless have a rough idea of its direction and speed. The orbits of all such objects have some similarity, but there is a lot of variation as well. The question is whether it is plausible that just knowing the telescope’s direction of pointing in the Caltech discovery photograph would be enough of a clue to let the Spanish group find this needle in the haystack. It took just 40 hours from the time the Caltech logs were viewed until the Spanish team sent out the discovery email. It’s not obvious to me that this timeline works.

    In the other direction, if the Spanish team had legitimately found the object and was checking to see if Caltech had the same thing, that would certainly be possible. From their 2003 observations they could derive orbital elements that would tell them exactly where it would be on May 6, 2004. Then they could see if the Caltech observations were looking at that spot. On seeing that Caltech had found the same object, the Spanish team might then rush out their announcement.

    My conclusion is that it is not yet clear what has happened here. Hopefully the investigation in progress will shed light on the truth. But we should not jump to conclusions about who is right and wrong.

  6. Gonzalo says:

    The web of P2P program in ERROR!!!!!!!

    Please inform the correct URL in my e-mail

    ERROR in the http://www.freedom-to-tinker.com/tinyp2p.py

    500 Internal Server Error
    The server encountered an internal error or misconfiguration and was unable to complete your request.
    Please contact the server administrator, webmaster@freedom-to-tinker.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.

    More information about this error may be available in the server error log.

    Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

  7. Franck Marchis says:

    Dr. Ortiz sent a good and detailed response to the accusations of Dr. M. Brown.
    http://groups.yahoo.com/group/mpml/message/15693

    He states that the object they found was so bright that they decided to check if the one reported by M. Brown (and described in an abstract submitted to the DPS conference and available on the web) was in fact the same one. They therefore did not steal the data as claimed in the NYTime article, they merely checked if they was not about to claim a false discovery.

    Regards
    F.

  8. Edward W. Felten says:

    I updated the original post in response to the information Frank Marchis pointed to.

  9. Neo says:

    It’s actually like copyright with short terms.