All journal articles are not created equal: guidelines for evaluating medical literature.

• Author: Socha, Charles Q.

Discovering junk science, "honorary" authors, and just plain ghosts may be difficult, but defense counsel can learn to do so with the right questions

DEFENSE counsel are concerned with the periodical medical and health science literature, which consists primarily of journal articles. Between 3,000 and 4,000 journals in the medical and health sciences fields are indexed in the Index Medicus of the National Library of Medicine and in Medline, a biomedical electronic database. These journals are regarded by the medical and scientific communities as a primary channel for both communicating knowledge and as arbitrators of the authenticity or legitimacy of that knowledge. They represent the principal means of formal communication among scientists through which research is made public and through which it is evaluated and authenticated by other experts.(1)

As a consequence, most of the medical and health science literature defense counsel advance or confront as proof of premises for an expert opinion, pursuant to Rules 703 and 803(18) of the Federal Rules of Evidence or similar state court rules, comes in the form of a journal article. Evaluation of medical journal articles is not taught in law school. But, especially since the U.S. Supreme Court decision in Daubert v. Merrell Dow Pharmaceuticals Inc.,(2) it has become a necessary, fundamental skill. This article concerns the interpretation and critical evaluation of the medical and health science periodical literature.(3)

PERIODICAL MEDICAL AND HEALTH SCIENCE LITERATURE

1. Kinds of Medical Journal Articles

   The following kinds of medical articles are of interest to defense counsel:

   1. Epidemiologic studies in human beings offer statistical analysis of data, gathered from samples of human populations, permitting inferences and conclusions about the relationship between various characteristics of the study subjects and disease or other health effect.

   2. Laboratory studies involving living organisms, either human or animal (in vivo studies).

   3. Laboratory studies involving human or animal tissue (in vitro studies).

   4. Anecdotal articles, which may consist of case reports, case series, or reviews of clinical experience, often provide the basis for more sophisticated research. In a case report or case series, authors describe their experience with a case or series of cases thought to exhibit unusual features, which may be useful to practitioners. The review of clinical experience is a more expansive and sophisticated case series in which authors may review all cases of a particular type or exhibiting common features over a period of months or years. The experiences of a group of patients are organized and summarized according to specific characteristics of interest, with relevant commentary by the author.

   5. Review articles, in which authors summarize and review other articles devoted to a common subject.

   6. Editorials and letters to the editor comment on matters of controversy or interest and often contain helpful critiques of original research. Occasionally a letter to the editor is used as a vehicle for providing information that supplements previously published original research. Letters and editorials are not necessarily peer reviewed. It is worth checking to see if an upcoming expert witness has published either an editorial or letter to the editor that contains information that can be utilized in cross-examination.

2. Format and Style

   1. Format

   The format of a medical journal article is described in the following table.(4)

   Section: Look for the Following: Abstract/Summary Overview or summary of the work. Highlights of results. General statement of significance. Introduction Background information: history, pathophysiology, clinical presentation. Review of the work of others, rationale for present study. Method/Materials Study design. Subject selection procedures. Methods of measurement. Description of analytic techniques. Results What happened. Graphics: tables, charts, figures that summarize findings. Discussion, Comment, Meaning, significance of work. Critique Conclusion of study: discussion of limitations as well as strengths, further analysis. Comparison with work of others. Disclaimers, equivocation, apologies, chest thumping, speculation, instruction, fantasy, etc. References/Bibliography Evidence that work of others has been considered. Leads to further exploration of the subject. 2. Style

   Approximately 500 medical journals subscribe to the Uniform Requirements for Manuscripts Submitted to Biomedical Journals.(5) A review of these requirements is recommended. They contain standards for authorship and use of statistics in medical journal articles, in addition to format and citation instructions. A supplement, published in the Annals of Internal Medicine, contains a statement on conflict of interest.(6)

3. Authorship

   According to the uniform requirements:

   All persons designated as authors should qualify for authorship.... Each author should have participated sufficiently in the work to take public responsibility for the content. Authorship credit should be based only on Substantial contributions to (a) conception and design, or analysis and interpretation of data; and to (b) drafting the article or revising it critically for important intellectual content; and on (c) final approval of the version to be published. Conditions (a), (b) and (c) must all be met.... Any part of an article critical to its main conclusions must be the responsibility of at least one author.(7) Any regular reviewer of the medical and health science literature is aware that the original research article usually has multiple authors. Notwithstanding the uniform requirements, the prevalence of "authors" who do little or nothing to earn authorship credit is significant. For instance, a group that included the editor and some staffers of JAMA, the journal of the American Medical Association, surveyed 809 articles published in 1996 in six peer-reviewed medical journals--three large and three small circulation--and concluded, "A substantial proportion of articles in peer-reviewed medical journals demonstrate evidence of honorary authors or ghost authors."

   The surveyors defined "honorary authorship (guest or gift authorship)" as "naming, as an author, an individual who does not meet authorship criteria." "Ghost authorship" was defined as "failure to name, as an author, an individual who has made substantial contributions to the research or writing of the article." They found 156 articles showing evidence of honorary authorship, 93 of ghost authorship, and 13 of both.

   The surveyors commented:

   Misappropriation of authorship (i.e. awarding honorary authorship and concealing ghost authorship) is incompatible with the principles, duties and ethical responsibilities involved in scientific publication. In this study, approximately 1 in 4 articles demonstrated misapplication of authorship criteria and inappropriate assignment of authorship.(8) Research and review articles were found to have both ghost and honorary authors, although the practice appeared to be less extensive, but still significant, with respect to the research article. For example, the New England Journal of Medicine, one of the top journals, exhibited ghost authorship in 26 percent and honorary authorship in 16 percent of research articles. The well-recognized authority whose name is included in the authorship credits might have had little or nothing to do with the content of the medical journal article. What appears, at first glance, to have been written by an authority, may have been written by a ghost.

   Order of authorship does not necessarily provide a clue as to authorship contribution. There is no standard governing order of authorship. This is a matter generally left to the co-authors.(9)

   HUMAN EPIDEMIOLOGICAL STUDIES

1. Design

   Critical evaluation of human epidemiological studies requires an ability to identify study design. An investigator's methodology underlies what can legitimately be inferred with respect to a relationship between an exposure and a health effect.

   Human epidemiologic studies fall into three broad categories of study design: (1) descriptive, (2) randomized and nonrandomized clinical trials, and (3) observational or explanatory. The purpose of clinical trials and observational studies is to gather and compare information in such a way that the results, in conjunction with other information, will permit inferences about the presence or absence of a causal relation between an exposure (the putative cause) and an outcome or health effect.

   1. Descriptive Studies

      Descriptive studies describe how particular characteristics are distributed in human populations. An example would be a community health survey. These studies may provide information about the incidence or prevalence of health effects. "Incidence" refers to the number of new cases of a health effect that occur within a specified period of time; "prevalence" refers to the total number of cases that exist at a specified time. Descriptive studies do not permit inferences about causal relationships.

      Anecdotal data (case reports, case series, reviews of clinical experience) are sometimes included in the category of descriptive studies, although describing a case report or case series as a "study" is an exaggeration. Case reports, regardless of the number in the series, lack any basis for comparison. "Without comparisons, descriptive studies cannot explain causes of disease."(10) Descriptive studies often suggest hypotheses to be investigated by analytic techniques employing comparison groups.

   2. Randomized and Non-randomized Clinical Trials

      The randomized clinical trial is a true experiment in which the investigator randomly assigns study participants to one or more kinds of medical intervention or no intervention (a placebo) and then compares the experience of the study groups. The most sophisticated of such studies are not only randomized but double...