Criteria for a Proper Risk Assessment
Wednesday, June 11th, 2008I have been thinking about your question about criteria. There are two things I would like to see in order to be happy.
I would like to see good reasons to think that the probability of “trouble” (a nicer expression than “destruction of Earth”) is quite low. For me that probability does not need to be as low as the very low probability required by [Adrian Kent "A critical look at risk assessments for global catastrophes," Risk Analysis, Vol. 24, No. 1, 2004] This is because there is also a low probability of a transcendent discovery that could save humanity. However, it is difficult to demonstrate very low probabilities, and even more difficult to balance them. Any demonstration of a very low probability is suspect, since the probability that the model is wrong is usually higher. At least we need to study the issue very carefully.
In order to study the issue carefully, I would like to see a fair and appropriate methodology for addressing the issue. Saying “black holes will not eat you” and setting up a committee to “monitor speculations” does not inspire confidence. However, Mangano seemed to be serious about his job, although his preliminary talk at Berkeley had good and bad aspects and is not yet convincing. Thought experiments involving compact matter are generally considered speculative, requiring verification from observation. I await the report of his committee. Meanwhile peer review is hardly an appropriate as a way of vetting that report. Physicists anxious to publish may come to see peer review as a gold standard, but it is only a method of keeping obvious errors from publication and it does that only moderately well. CERN’s avoidance of more appropriate protocols for vetting the report suggests that they know they will lose if the game is not fixed. In this case CERN appears to be stacking the deck with peer reviewers known to think that colliders are safe. Meanwhile, outside experts are kept in the dark about report contents, and apparently will not get a chance to comment until CERN has made up its mind. Real environmental assessment protocols include solicitation of public comments, with adequate time for the public to study the issue. In addition, best practices include use of experts from more than one discipline–in this case it would be helpful to have ethicists, risk assessment experts, and astronomers who are experts in neutron stars. Experts in these subjects are not just window dressing–there appear to be risk assessment issues that physicists in general and Mangano in particular do not understand. I fear that CERN will have a rubber stamp peer review committee declare the LHC to be safe, and then fire it up before others have a chance to study the issue. This is not an appropriate way to handle an issue of this importance.
Science can handle these things well. As models, I point to the Asilomar process and the precautionary principle. (At Asilomar, biologists agreed to limits on dangerous experiments, limits that allowed most experiments to continue.) It is disappointing when science falls short of these models.