Editorial favoritism in economics?

• Author: Medoff, Marshall H.

1. Introduction

   In 1990, the Journal of Economic Literature listed the contents from over 300 economics journals. The journals ranged from general interest (e.g., American Economic Review, Journal of Political Economy, Quarterly Journal of Economics) to those specializing in specific areas (e.g., business, finance, law, real estate, trade). Economic knowledge is now disseminated primarily through a journal-dominated system. The audience is predominantly professional academic economists who screen the research of each other in order to certify its quality. Publication in these journals is a necessary condition for tenure, promotion, influence, reputation, and mobility. By chance or choice, economics journals hold the keys to success for academic economists.

   Not surprisingly, given the rewards from publishing, there has been substantial research on economics journals. A quality hierarchy has been found to exist among economics journals based on such indicators as surveys, citations, and institutional affiliations of authors. The more visible or highly cited economics journals tend to be the more prestigious or higher-quality journals (Moore 1972; Hawkins, Ritter, and Walter 1973; Laband and Piette 1994b).

   Considerably more attention has focused on the quality-control appraisal system used in the publication process: peer review. In order to publish in virtually any academic economics journal, an author must receive a favorable evaluation from an anonymous peer saying that the paper is potentially publishable and, if necessary, what further work is needed to make the paper worthy of publishing. The paper is not approved for publication unless and until any suggested revisions are made to the satisfaction of the editor and referee. The peer review process has been examined in terms of (i) what the characteristics and functions of these anonymous referees are and (ii) whether they are fair and objective in their evaluation.

   Hamermesh (1994) examined the characteristics of referees at four general and three specialty economics journals and found that referees are overwhelmingly male, with 16 years of experience since the receipt of their Ph.D. degree, and are typically of higher quality than the author of the paper they are reviewing. Higher-quality journals have higher-quality referees who are not systematically assigned to review higher-quality authors. Mackie (1998) surveyed referees at seven economics journals and found that referees use a set of highly subjective and interpretive criteria in assessing the significance or quality of a research paper. The criteria included originality, novelty, creativity, innovativeness, advances in existing economic knowledge, and relevance to real economic problems.

   Laband (1990) found that referees have two functions. First, they screen the quality of the research conducted by their professional peers in order to determine ff the paper meets a minimum quality standard. Second, referees, through their comments and suggested revisions, increase the quality of a potential publication (as measured by the subsequent number of citations a paper receives in the six years after publication). However, this relationship is only statistically significant. The numerical impact of a referee's comments on the quality of a paper is virtually negligible. Using Laband's figures, a referee's comments increase the number of citations a published paper receives by less than 0.25 per year over the subsequent six-year period.

   The second issue of concern about peer review is whether the evaluation process is fair. Peer review takes two forms: single blind (the author does not know the identity of the referee but the referee knows the author's identity) and double blind (neither the author nor the referee knows the identity of the other). Blank (1991) conducted a unique controlled experiment to analyze the effects of single-blind versus double-blind refereeing on papers submitted to the American Economic Review between 1987 and 1989. She found that under the double-blind system, acceptance rates are lower and referee comments more critical. Acceptance rates of authors at the top five ranked universities were not affected by the type of review system used. Authors at the near-top universities (ranks 6-50) had lower acceptance rates under double-blind reviewing. These results, however, provide no conclusion regarding the fairness of the peer review process. (1)

   Very little research has focused on the behavior of the key journal decision makers in the review process: the editor/coeditors. Editors must decide whether research submissions are of sufficient quality to warrant publication in their limited number of journal pages. An editor's objective, presumably, is to produce a journal of the highest possible quality. Journal editors compete with each other to attract papers that will make the greatest scientific contributions. Because of the active competition between journals, editors attempt to persuade authors to submit their high-quality papers to them in exchange for a reduction in the transaction costs involved in the reviewing/publication process.

   Critics of the editorial review process contend that the absence of any clearly defined criteria of what constitutes a significant high-quality contribution produces editorial favoritism in the review process (Folster 1995; Mackie 1998). It is argued that publication decisions are swayed by an author's personal or institutional connections to the editor or coeditors. The consequence is that nonscientific considerations influence editorial decisions.

   There exists considerable anecdotal evidence regarding the perception that editorial favoritism exists in the review process. Bhagwati, editor of the Journal of International Economics. noted that he published a paper from a former student (Paul Krugman) despite the fact that there were two adverse referee reports from very distinguished experts and he did not normally publish his own students' work (Shepherd 1995, p. 89). Clower, when he was editor of the American Economic Review, frequently accepted research papers for publication without submitting them for peer review (Shepherd 1995, p. 99). While editor of the Review of Economics and Statistics, Houthakker read every incoming manuscript and summarily rejected papers without sending them out for formal review (Shepherd 1995, p. 107).

   These examples of editorial discretion am not determinative since it cannot be inferred whether these editorial decisions am systematic or random. The crucial question is whether favoritism influences the choices editors make in the prepublication appraisal process. The methodological problem is that editorial favoritism is difficult to directly detect from acceptance rates for several reasons. First, most researchers do not have access to journal submissions that are needed in order to compare the characteristics of published and rejected authors. Second, an editor's choice of referees may predetermine the publication decision about a research paper. Editors may assign a paper to referees who are ideologically biased (in either direction) toward an author. Third, authors' decisions about what journal to submit their paper to may be influenced by their concerns, positively or negatively, about editorial favoritism.

   It is possible to indirectly detect the presence of editorial favoritism from published articles. If editorial favoritism exists, one would expect to find...