Assessing regulatory impact analyses: the failure of agencies to comply with Executive Order 12,866.

• Author: Hahn, Robert W.

1. INTRODUCTION

   Although regulations often have no direct fiscal impact, they pose real costs to consumers as well as businesses. Regulations aimed at protecting health, safety, and the environment alone cost over two hundred billion dollars annually -- about two-thirds as much as outlays for federal, nondefense discretionary programs.(1) Yet, the economic impacts of federal regulation receive much less scrutiny than the budget.(2)

   To encourage the development of more effective and efficient regulations, Presidents Reagan, Bush, and Clinton have directed agencies to perform economic analyses of major regulations that show whether a regulation's benefits are likely to exceed its costs and whether alternatives to that regulation are more effective or less costly. Each president also attempted to increase agency accountability for decisions by requiring that the President's Office of Management and Budget ("OMB") review all major regulations. More recently, Congress embraced regulatory reform and inserted accountability provisions(3) and analytical requirements into laws such as the Safe Drinking Water Act Amendments of 1996, the Small Business Enforcement and Fairness Act of 1996, and the Unfunded Mandates Reform Act of 1995.(4)

   The most prominent and far-reaching of these regulatory reform efforts are President Reagan's Executive Order 12,291 and President Clinton's Executive Order 12,286. Both require agencies to prepare a Regulatory Impact Analysis ("RIA") for all major federal regulations.(5) Agencies have prepared RIAs for almost twenty years in accordance with the executive orders and guidelines for economic analysis provided by the OMB.(6)

   This Article suggests that the impact of RIAs has fallen short of the expectations of regulatory reform advocates in part because agencies do not fully comply with OMB's guidelines.(7) The RIAs typically do not provide enough information to enable regulatory agencies to make decisions that will maximize the efficiency or effectiveness of a rule.(8)

   This conclusion is based on the results of an evaluation of forty-eight major environmental, health, and safety regulations and their associated RIAs.(9) The authors completed a "regulatory scorecard" for each of the forty-eight regulations, which includes a checklist of the requirements for a good economic analysis outlined in the Executive Order and the OMB guidelines.(10) The study of RIAs shows that agencies only quantified net benefits -- the dollar value of expected benefits minus expected costs -- for 29 percent of the forty-eight rules, even though the Executive Order directs agencies to show that the benefits of a regulation "justify" the costs.(11) The agencies also did not adequately evaluate alternatives to the proposed regulation, another element of the Executive Order. Agencies failed to discuss alternatives for 27 percent of the rules and quantified the costs and benefits of alternatives for only 31 percent. In addition, the agencies often failed to present the results of their analysis clearly. Agencies provided executive summaries for only 56 percent of the rules.

   This Article also offers specific suggestions for improving the quality of RIAs, which will in turn improve the allocation of regulatory resources. These include: (1) the use of clear executive summaries; (2) the provision of on-line RIAs; (3) improved evaluation of regulatory alternatives; and (4) improved assessment of net benefits.

   Part II of the paper describes the methodology of the study. Part III presents the results. Part IV describes in detail the policy recommendations to improve RIAs.

2. METHODOLOGY

   This study builds on previous efforts to evaluate the quality of RIAs.(12) Whereas previous studies evaluated a few RIAs in great detail, this study assesses the quality of forty-eight RIAs published from April 1996 to July 1999.(13) This approach is advantageous because it is possible to identify common strengths and weaknesses among many RIAs, a task that no previous study has undertaken.

   The authors included only major rules, also known as "economically significant" rules, in the study because they typically have annual costs or benefits in excess of one hundred million dollars per year. These rules have the largest impact on society and agencies should scrutinize them more than other rules.(14) Also, with a few exceptions, agencies produce RIAs for all major rules. The study excludes so-called "transfer" rules, or rules designed to move resources from the federal government to designated segments of the population, because agencies generally do not assess the costs and benefits of transfer rules.(15) The study only includes "non-transfer" rules, which are rules that address market failures and focus on achieving regulatory objectives, such as improving air quality.

   The study further assumes that agency numbers presented in RIAs are accurate and complete. This approach allows third parties to easily reproduce the study's results. At the same time, this approach precludes critical evaluation of the agency estimates, which other authors suggest are often biased in support of the regulation or are compromised by analytical flaws.(16)

   The study examines the extent to which agency RIAs meet the government's own standards for economic analysis, as described in the Executive Order and the OMB guidelines.(17) The Executive Order states, for example, that agencies shall provide "an assessment, including the underlying analysis," of benefits and costs expected from a regulation and, "to the extent feasible," provide a quantification of those benefits and costs.(18) The OMB Guidelines further direct agencies to express benefits and costs in monetary terms "to the fullest extent possible."(19) The Executive Order also states that "agencies should assess all costs and benefits of available regulatory alternatives, including the alternative of not regulating."(20) According to the Executive Order, the RIA must provide sufficient information to demonstrate that the agency is selecting the regulatory approach that maximizes net benefits, unless the approach is prohibited by statute.(21) The OMB Guidelines further provide agencies with a recommended approach for evaluating alternatives.(22)

   The authors developed a "regulatory scorecard," based on the Executive Order and the OMB Guidelines, summarized in the Appendix. Each item listed on the scorecard represents an essential element of a good economic analysis. The researcher evaluating the RIA filled out the scorecard based on an evaluation of the Federal Register notice, the agency's formal description of the rule that is available to the public, and the RIA.(23) Another researcher then would validate the first researcher's findings by reviewing the same documents. If the findings of the two researchers differed for any part of the scorecard, the researchers resolved the differences by discussion.

   Generally, there was little disagreement between researchers because completing the scorecard did not require researchers to subjectively assess the agency's compliance with the Executive Order and the OMB Guidelines.(24) Determining whether the agency "discussed alternatives," for example, is relatively easy because an agency must simply mention the existence of alternative regulatory approaches.(25) The most prominent exception is the scorecard item that measures whether the agency "considered the most important alternative approaches" to the regulation.(26) Although this is an important component of a good economic analysis, the authors did not include it in the summary presented in this Article because of concerns about subjectivity.(27)

3. RESULTS

   This Part describes the aggregate results of our study of agencies' economic analyses. In general, we find that most economic analyses do not meet the expectations set forth in the Executive Order and the OMB Guidelines, and a significant percentage clearly violate them. Specifically, agencies frequently do not provide the kind of information in the analyses necessary to select the best regulatory alternative or to show that the agency should proceed with the regulation.

   This Part breaks the discussion of the results of the study into the following categories: estimation of costs, estimation of benefits, comparison of benefits and costs, evaluation of alternatives, clarity of presentation, and consistent use of analytical assumptions. It then discusses conclusions arising from the analysis. Three agencies have finalized more than five rules included in the database: the Department of Transportation ("DOT"), the Environmental Protection Agency ("EPA"), and the Department of Health and Human Services ("HHS"). This study presents the results from these agencies separately and grouped results from the remaining agencies together, simply because no other single agency finalized enough rules for meaningful summary statistics.(28)

   The reader needs to interpret the statistics presented in this section with care. Some agencies noted, for example, that regulations have costs in addition to direct compliance costs and administrative costs. It would be misleading to suggest that these agencies performed a lower quality analysis simply because they noted the existence of some indirect costs of the regulations, but did not attempt to quantify them. In fact, the acknowledgment of indirect costs is arguably an indication of a more thorough analysis on the part of agencies.

   1. Estimation of Costs

      Comprehensive estimates of regulatory costs allow decision makers to compare regulatory alternatives and identify the impact of a regulation on different groups to address distributional concerns. We found that agencies could present the results of their cost analyses more clearly and identify the impact on different groups more frequently, but in general, agencies attempt to evaluate the costs of regulation.

      Agencies always define categories of costs associated with a...