Beyond Citation Counts: Reassessing Top Criminologists’ “Influence” With Altmetric Scores

• Published date: 01 August 2023
• DOI: http://doi.org/10.1177/10439862231170971
• Author: Whitney S. Sanders,Jessica Corey,John L. Worrall
• Date: 01 August 2023

https://doi.org/10.1177/10439862231170971

Journal of Contemporary Criminal Justice

2023, Vol. 39(3) 387 –404

© The Author(s) 2023

Article reuse guidelines:

sagepub.com/journals-permissions

DOI: 10.1177/10439862231170971

journals.sagepub.com/home/ccj

Article

Beyond Citation

Counts: Reassessing Top

Criminologists’ “Influence”

With Altmetric Scores

Whitney S. Sanders1, Jessica Corey1,

and John L. Worrall1

Abstract

Criminal justice and criminology (CCJ), like many academic disciplines, conducts

its share of rankings. Citation-based ranks of individual scholars are particularly

popular, and they tend to consistently identify the field’s supposedly “top” scholars

and “academic stars.” Whether citations equate with “influence,” however, is up for

debate. At the least, citation-based metrics are unidimensional and fail to capture

attention outside academia. Accordingly, we drew on the work of Cohn etal. and re-

ranked top-cited scholars using the Google Chrome “Altmetric it!” bookmarklet. As

expected, the Altmetrics methodology fundamentally altered past rankings. The most

influential scholars in our rankings, Terrie E. Moffitt and Avshalom Caspi, received

higher Altmetric scores than all the remaining ranked scholars combined.

Keywords

altmetrics, scholarly influence, article rankings, citations

Introduction

As with most academic disciplines, criminology/criminal justice (CCJ) is replete with

ranking studies. With few exceptions, quantitative ranking studies1 fall into four catego-

ries: people rankings, program rankings, article rankings, and journal rankings. People

rankings often seek to identify CCJ’s “academic stars” (Copes et al., 2012), typically

1The University of Texas at Dallas, Richardson, USA

Corresponding Author:

Whitney S. Sanders, Public Policy and Political Economy, The University of Texas at Dallas, 800 W

Campbell Rd., Richardson, TX 75080-3021, USA.

Email: Whitney.Sanders@utdallas.edu

1170971CCJXXX10.1177/10439862231170971Journal of Contemporary Criminal JusticeSanders et al.

research-article2023

388Journal of Contemporary Criminal Justice 39(3)

those scholars with the highest citation counts. Program rankings generally sum up peo-

ple rankings in some fashion to determine which have the most scholarly influence, typi-

cally measured by faculty publication productivity (e.g., Kleck & Mims, 2017; Lebron

et al., 2023).2 Article rankings tend to search out the “most cited” scholarship (e.g.,

Graham et al., 2019), presumably identifying research (and, indirectly, and authors) that

has made the most significant “mark” on the field. Finally, journal rankings tend to be

built on the so-called “impact factor,” the average number of times articles from a jour-

nal published in the past 2 years have been cited in the impact factor year.3

At the university or school/department/program level, rankings are used to set pri-

orities and goals, not to mention attract resources, students, and favorable press. At the

individual level, rank information can inform tenure and promotion cases and factor

into individual recognition (e.g., awards) and mobility prospects. By some accounts,

the “ranking industry,” which is built on the assumption that all participants will want

to emulate “the best,” has taken on a life of its own. Critics argue that academic rank-

ings stifle innovation and originality; “. . . ranking encourages universities to associate

themselves with whatever the leader incarnates as defined by the criteria of the rank-

ing” (Kauppi, 2018). In addition, an entire literature has emerged that criticizes objec-

tive ranking metrics such as the impact factor (e.g., Arnold & Fowler, 2011; Seglen,

1997). Whatever their limitations, ranking studies—and the indicators used to develop

them—are here to stay. One might go so far as to describe ranking indicators as the

currency of the academic profession.

Focusing squarely on individual scholar rankings, two key patterns have emerged in

the CCJ literature. First, rankings for the most cited scholars have remained remarkably

consistent for at least the past decade. For example, Robert Sampson was the most-cited

scholar in three top criminology journals in three different periods (2001–2005, 2006–

2010, and 2011–2015) (see Cohn et al., 2020, p. 6). Similarly, Francis Cullen was the

first or second most-cited scholar in three top criminal justice journals during the same

periods (Cohn et al., 2020). Of course, results vary by journal, and rankings predictably

shift based on how long a scholar has been employed in the field, but there is remarkable

consistency, nonetheless. Second, citation counts appear to be the main, if not sole, met-

ric used to calculate such rankings. The reason is straightforward: Citation counts are

readily accessible through the Social Sciences Citation Index and other prominent

sources, such as Google Scholar and Elsevier’s Scopus. However, each source has its

limitations, prompting some to advocate for raw citation accounts, that is, scouring arti-

cle references manually and counting citations to exclude self-citations and correct for

other potential errors in mainstream databases (see, for example, Farrington et al., 2019).

A relatively novel and largely untapped ranking metric, the Altmetric Attention

Score, overcomes many of the known limitations of citation-based metrics. Altmetric,4

a company founded in 2011 by Euan Adi, works with various sources and companies

to “. . . track and analyze the online activity around scholarly research outputs” (https://

www.altmetric.com/about-us/). Altmetric scores capture citations and other indicators

of attention, dissemination, influence, and impact. These can include discussion on

research blogs, activity on social media, and coverage in mainstream media outlets,

among many other indicators. While citations generally indicate academic attention,