Prescribing the Right Dose of Peer Review for the Endangered Species Act

Abstract

Environmental law stands out among all fields of law as the one most concerned with the physical world around us. This is both its great appeal and its heavy burden. If environmental law has done anything in the epistemological sense, it has taught us how little we know about the physical world and, even more so, how little we know about how to improve the physical world through law. Alas, environmental law seems puny and confused compared to its intended beneficiary, and we have made many mistakes with it.

Lately, however, we hear much about science coming to the rescue of environmental law. The so-called sound science movement promises to improve decision making under environmental law by infusing the field with better and more practice of science. Of course, I am not about to argue against sound science, whatever it means, as it is one of those loaded terms that almost begs a fight. Who is for unsound science? Not I, but I am for a sound approach to sound science. Maybe sound science is good for environmental law, but we all know that too much of a good thing can be bad for you. So, I ask, could sound science, depending on how it is dosed out to environmental law, be counterproductive?

I am by no means the first to ask this question, or to suggest, as I do, that the answer is yes. But I wish to focus on the one component of sound science that is most often held out as the panacea by some and the problem by others - peer review. So, to be more precise, what I examine here is whether peer review, depending on how it is dosed out, could be counterproductive for environmental law.

The peer review question has not received as much discrete attention in environmental law literature, but it is truly the sleeping dog of the sound science movement. Understanding why requires some background on science and administrative law. The sound science movement, as its name suggests, advocates that environmental law decisions be made based principally on scientific information and conclusions that have been derived through the rigorous, unbiased practice of science. Science is generally regarded as a formalized system for gathering and evaluating information about the world in which prescribed methods of observation, communication, informed criticism, and response must be carefully followed. If these steps work for science, so goes the argument, they should work for environmental law as well.

One difficulty the sound science movement faces, however, is that many of the components of science are already firmly embedded in environmental law, not only through occasional specific references to science in environmental laws, but generally so through basic standards of administrative law prescribed under the Administrative Procedure Act (APA). Thus, just as a scientist would be accused of practicing unsound science in research if he or she declared that some set of relevant data was ignored or altered in reaching the research conclusion because the original data did not support the conclusion, an agency would also be chastised for doing the same in reaching the decision of any rulemaking or adjudication. That would be arbitrary and capricious behavior any court acting on judicial review of the decision would know to strike down as a violation of the APA. A court would not need sound science to get there. Thus, although the Endangered Species Act (ESA) ominously requires that many decisions called for under the statute be based on the best scientific data available, there is no evidence that this standard has made a bit of difference in terms of how agency decisions are examined in judicial review settings. Courts do not implement the ESA standard in ways that add anything beyond the demands that are already placed on the decision maker under the APA.

The sound science movement thus has a difficult time articulating exactly what it seeks when it comes to matters such as data quality, because many of the sound practices of science already are required of agencies in some degree or another. What is sound science adding? To be sure, one can envision requiring additional procedural steps agencies must take to assure the public about such matters as data quality - in essence, enforcing sound science by mandating more science - but it is simply not the case that the substantive demand for data quality is something the sound science movement brings new to the table for environmental law.

By contrast, peer review is a practice of science that is neither specifically required by most environmental laws nor generally required through administrative law doctrines. Peer review is generally described as a scientifically rigorous review and critique of a study's methods, results, and findings by others in the field with requisite training and expertise who are independent of the persons conducting the study. The sound science movement holds out peer review as an immutable principle of environmental decision making through the following syllogism:

Premise 1: The practice of sound science is an essential component of many decisions required by environmental law.

Premise 2: Peer review is an essential component of the practice of sound science.

Conclusion: Therefore, peer review is an essential component of those environmental law decisions for which sound science is an essential component.

On its surface this argument appears unassailable - if peer review is part of science, which it is, and science is part of environmental decision making in many instances, which it is, ought not peer review be a part of environmental decision making? But the answer is, not necessarily. What the argument fails to reveal is that, even with respect to environmental decisions for which sound science is an essential component, sound science is not the only essential component. This is because science, even sound science, usually does not lead to compelling answers about the questions posed in environmental law. Indeed, it frequently leads in no particular direction at all. Sound science could produce a mountain of relevant data of the highest quality and still provide no clues as to what to do for purposes of the environmental law decision. In at least those cases something else will be needed in order to reach decisions, such as sound judgment by agency decision makers, sound procedures for weighing alternatives, sound understanding of social and economic impacts, sound methods of providing public information and soliciting public views, and even sound politics.

There are, in other words, many ingredients to sound decision making in environmental law, with sound science being only one among them. But advocates of peer review are likely to observe that this argument alone does not refute the case on behalf of including peer review, for it does not demonstrate why peer review should ever be left out of the sound science package. Particularly because peer review is the one element of sound science not already incorporated at some level through environmental law, they may argue that it should be the last to be excluded from the mix.

Nevertheless, the complexity of environmental decision making will rarely allow us to practice all of the ingredients of decision making to their soundest degrees. Even where time and money would allow the possibility of engaging in any one of the ingredients to the highest standards, there will be cases in which attempting to do so for all at the same time would pose conflicting constraints between the respective ingredients. For example, at some point the quest for more relevant and reliable data may add so much time onto a decision making process, such as the decision whether a species is endangered, that the effectiveness of the decision is impeded - what a hollow victory for sound science it would be to spend so much time ensuring the reliability of the data proving the species is endangered that the species is already extinct by the time the decision to declare it endangered is made. Adding time and budget constraints to the picture amplifies the prospect and potential intensity of these conflicting constraints, requiring that we make trade-offs between the components of decision making in order to get the job of decision making done. Sound decision making, in other words, may require that we practice all of the decision making components, including sound science, at, say, only 80 percent of their respective soundest levels.

Taking this phenomenon of conflicting constraints to heart, the central thesis of this Article is that, of all the components of sound science, peer review presents one of the highest potentials for triggering conflicting constraints with other components of sound decision making. Using the Endangered Species Act as a case study, in Part II of the Article I describe the three conflicts unrestrained doses of peer review pose for environmental decision making. First, because its advocates regularly overstate what peer review will accomplish for environmental law decision making, mandating it across the board raises unrealistic expectations of the quality of agency decisions and weakens the position of other important components of decision making. Second, inflexibly mandating rigorous peer review adds substantial demands on agency resources, potentially draining resources from other decision making components and, in many cases, impeding decision making altogether. Finally, peer review is subject to abuse if it is adopted in ways that allow agencies to manipulate the process and thereby rig outcomes so as to justify agency decisions that would not withstand legitimate peer scrutiny.

Peer review, in other words, can be prescribed at overdose levels even though it is an essential component of sound science. In Part III of the Article I then suggest two ways peer review could be incorporated into environmental laws in ways that retain its value while minimizing its potential adverse effects. One approach is to identify criteria which, when present, suggest that a particular decision could benefit from rigorous peer review, with the idea being that the criteria would screen out most cases to produce a limited set of instances justifying substantial expenditure of resources for peer review. Peer review in these cases would be rigorous - the soundest attainable. Another approach is to develop an independent peer review swat team that would conduct random, less intensive peer reviews of agency decisions, with the idea being to derive much of the decision making quality incentives peer review can produce without suffocating overall agency practice. These uses of peer review substantially reduce the potential for conflicting constraints while still allowing environmental law to reap most of its benefit. More potent doses of peer review, I contend, offer little incremental benefit but risk significant adverse effects.

Arguing against broad use of peer review in environmental law invites accusations of trying to hide the flaws of agency practice in the shadow of agency expertise. Peer review is the light that will expose those flaws, its advocates contend. But while that is indeed the purpose of peer review in science, I question whether those who advocate it unyieldingly in environmental law really have that purpose in mind. The optimal use of peer review in environmental law, I contend, is not all the way all of the time, but at the right dose at the right time. The sound use of sound science demands nothing less than a careful approach to writing that prescription.