Citation Data and Analysis: Limitations and Shortcomings

• Published date: 01 August 2023
• DOI: http://doi.org/10.1177/10439862231170972
• Author: John L. Worrall,Ellen G. Cohn
• Date: 01 August 2023

https://doi.org/10.1177/10439862231170972

Journal of Contemporary Criminal Justice

2023, Vol. 39(3) 327 –340

© The Author(s) 2023

Article reuse guidelines:

sagepub.com/journals-permissions

DOI: 10.1177/10439862231170972

journals.sagepub.com/home/ccj

Article

Citation Data and

Analysis: Limitations

and Shortcomings

John L. Worrall1 and Ellen G. Cohn2

Abstract

As a means for measuring scholarly influence, citation analysis has several limitations

and shortcomings. We first review the main sources of citation data (Web of Science,

Scopus, Google Scholar, and information collected directly from reference lists) and

discuss the shortcomings of each source. Next, we review five significant limitations

of citation analysis as a methodology (academic over popular interest, various

motivations for citing, manipulation potential, failure to account for author ordering,

and citations only appearing in “indexed” journals). The issues we touch on set the

stage for the remainder of the articles in this special issue.

Keywords

citation, influence, Web of Science, Scopus, Google Scholar

Citation analysis has long been employed as a method for determining scholars’, jour-

nals’, and university departments’ impact on the field. It has also been used to gauge the

influence of specific works (journal articles, books, etc.) on later research products, as

researchers tend to cite works that they find impactful and valuable. There are two pri-

mary assumptions made by scholars using citation analysis: that the most-cited works

are those that were most important to the research and that citations indicate influence

(Meadows, 1974). The primary advantage of citation analysis is that it is objective,

quantitative, and replicable, and is less likely to be influenced by personal bias or spe-

cial interest than other measures of measuring scholarly influence and prestige, such as

peer rankings and prestige surveys or the receipt of scholarly awards and prizes.

1The University of Texas at Dallas, Richardson, USA

2Florida International University, Miami, USA

Corresponding Author:

John L. Worrall, Criminology and Criminal Justice, The University of Texas at Dallas, Richardson, TX 75080,

USA.

Email: worrall@utdallas.edu

1170972CCJXXX10.1177/10439862231170972Journal of Contemporary Criminal JusticeWorrall and Cohn

research-article2023