Introduction

Governments, funding agencies and promotion committees often use citation data of the ISI, a company in Philadelphia formerly known as the Institute for Scientific Information and now owned by the Thomson Corporation of Toronto, to perform evaluations of individual scientists, research groups, departments, universities and even countries due to the lack of reliable measurements of research quality [1]. The argument is that important papers will be cited more frequently and therefore a popular method for measuring the impact of an article is to count its citations. The number of citations (i.e., how many times a given article is counted in the reference lists of subsequent articles) is then seen as a direct measure of the recognition that this publication has had in its scientific field.

Although citation statistics have been criticized on many accounts [2, 3, 4, 5], the analysis of citation rates may allow for the identification of advances in a specialty and may provide a historical perspective on its scientific progress. Therefore, various specialties have recently summarized their so-called “citation classics” [6, 7, 8, 9]. In addition, various journals have published their own citation classics [10, 11, 12, 13, 14, 15]. However, a systematic analysis of top-cited articles in the field of critical care medicine is not yet available. The purpose of the present study was therefore to identify and examine the characteristics, such as ranking, year of publication, publishing journal, type of article, institution of origin, country, state, topic and authorship of the most frequently cited articles published in critical care journals. In addition, the most frequently cited articles related to critical care medicine published in other biomedical journals are also presented.

Materials and methods

All seventeen journals specializing in critical care medicine, according to the Journal Citation Report (JCR) 2002 under the subject category Critical Care Medicine, were analyzed (Table 1). The most frequently cited articles in critical care journals were identified using the database of the Science Citation Index Expanded (SCI-EXPANDED, 1945 to present). The Science Citation Index Expanded is a multidisciplinary database powered by the ISI and its Web of SCIENCE with searchable author abstracts covering the journal literature of the sciences; it indexes more than 5,700 major journals across 164 scientific disciplines. The Cited Reference Search option of the Web of SCIENCE returns a site-configured maximum number of results, which is limited to 500. To find more than the maximum number of results, the data can be searched in segments, as, for example, by searching one year or one journal for data at a time. For journals publishing articles with high citation rates, even this segmented search for one year of data yielded more than the maximum number of results. In these cases, each author’s name, according to the author index of the respective journal and year, was “hand-searched” using the Web of SCIENCE interface.

Table 1 List of screened critical care journals

To perform such a “hand-search” for other general medicine and biology journals such as the New England Journal of Medicine, Nature, Science, Lancet, Cell, Chest, Journal of Clinical Investigation, Thorax, European Respiratory Journal, American Review of Respiratory Disease etc. is almost impossible due to the large number of articles published in these journals. To retrieve highly cited articles related to the field of critical care medicine which were originally published in non-critical care journals, we performed a search using SciSearch with the broadest key word available, i.e., “Critical Care”, since 1974 and combined it with the names of all other biomedical journals.

Data stored for each reference in the database include, among other categories, the title, author names, institutions, addresses, journal, abstract and key words. The most frequently cited articles in critical care journals and the most frequently cited articles related to critical care medicine in all other biomedical journals were read and reviewed online (PubMed) by both authors; when relevant information was not available online, the articles were obtained in a printed format by direct library access. After excluding articles from the study with no direct relevance to the field of critical care medicine, the 74 top-cited articles in critical care journals and the 45 top-cited articles related to the field of critical care medicine in all other biomedical journals were analyzed and the data tabulated according to their specific features.

Results

Citation classics in critical care journals

Four hundred and eighteen articles were retrieved that were published in critical care journals and that were cited 100 times or more. Using the Web of SCIENCE update of July 27, 2003, we selected the top 74 most frequently cited articles with a direct relation to the field of critical care medicine from this list for further analysis and ranked them according to the number of citations they received (Table 2). The most cited article received 3402 citations and the four least cited articles received 170 citations. The mean number of citations per article was 341. The majority (55 articles) received more than 200 citations. The top 74 citation classics were published between 1968 and 1999. The decade from 1990 to 1999 produced the most citation classics with 38 articles followed by the decade from 1980 to 1989 with 30 articles. The most classic papers published within a given year were 7 articles each in 1989 and 1994. Sixty percent of the classic articles were published after 1989. The top-cited articles were all published in six high-impact critical care journals, led by Critical Care Medicine (37 articles) and followed by the Journal of Trauma (21 articles), American Journal of Respiratory and Critical Care Medicine (9 articles), Intensive Care Medicine (4 articles), Journal of Neurotrauma (2 articles) and Shock (1 article). Of the 74 articles, 70 articles were original publications (45 human studies, 9 animal studies and 16 laboratory investigations), 2 were review or guideline articles and 2 were editorials. While 36 clinical human studies and 9 physiological-experimental studies received a mean citation rate of 246 and 244 citations, respectively, per article, the 12 studies related to critical care scoring systems achieved 845 citations per article.

Table 2 Top seventy-four citation classics in critical care journals ranked in order of citations received

The 74 top-cited articles originated from 12 countries, with the United States (US) contributing 51 (69%) articles, followed by five countries (Germany, Canada, Netherlands, France, New Zealand) with three articles each (Table 3). Only 15 articles originated from non-English speaking countries (Table 3). Within the US, Ohio, California and Washington, DC, lead the list of citation classics with six articles each. Eight states contributed three or more classic articles (Table 3). Of the total 74 articles, 21 originated from multi-institutional collaboration, of which 10 were from multinational collaborations and 53 from individual institutions. Sixty different institutions produced the 74 top-cited articles. Ten institutions produced more than one citation classic article (Table 4). The Department of Surgery of the Washington Hospital Center, Washington, DC, and the Department of Surgery of the Denver General Hospital, Denver, Colorado, share the lead in this list with four classic articles each.

Table 3 Country and state of origin and number of articles identified as citation classics in critical care journals
Table 4 Institution of origin of first author and number of articles identified as citation classics in critical care journals

The first author was affiliated with an academic department in all classic articles. The number of authors of the top-cited articles ranged from one to 50. Nine articles were authored by a single author and nine articles by two authors. Forty-eight persons authored two or more of the top-cited articles. Table 5 presents a list of “frequent authors” with three or more classic articles. This list is led by Roger C. Bone, who authored or co-authored six classic papers; he is also the only person who is first author of four such articles. Among the first authors is only one woman; Susan P. Baker’s papers on the injury severity score were cited 1978 and 438 times (Table 2, ranks 2 and 10). Three classic papers were not authored by individual authors but by a consortium or committee of authors such as the American-European Consensus Committee on ARDS (Table 2, rank 3), American College of Chest Physicians and Society of Critical Care (rank 6) and the American Association for the Surgery of Trauma (rank 43). In addition, many papers present the results of multicenter studies organized by research groups such as the Sepsis Study Group (ranks 8, 23 and 54), ARDS Study Group (rank 53) or the International Sepsis Trial Study Group (rank 69b).

Table 5 List of authors who contributed more than two citation classics in critical care journals

Pathophysiology of the lung (18 articles) is the main topic covered by these highly cited articles. Considerable attention was also given to topics such as sepsis (13 articles), trauma (13 articles) and scoring systems (12 articles). Other topics include weaning procedures, antibiotic therapy and monitoring.

Citation classics related to critical care medicine published in other biomedical journals

The most cited critical care article published in a non-critical care journal received 2860 citations with a mean number of 810 citations per article for the 45 most cited articles (Table 6). To compare, the mean number of citations of the top 45 articles published in critical care journals was 440. Thirteen different non-critical care journals published the 45 classic critical care articles; the New England Journal of Medicine leads this list with 11 such articles, followed by JAMA and Lancet (6 articles each), and American Review of Respiratory Disease and Annals of Internal Medicine (5 articles each). The US also leads the list of citation classics in non-critical care journals with 40/45 (89%) articles, followed by the UK with three articles. Six articles were the result of a multinational cooperation. The topics covered are also similar to the classic articles published in critical care journals; the main topics were lung physiology and injury (12 articles), followed by sepsis (10 articles), cardiac disease (6 articles) and critical care scoring systems (5 articles). From the list of “frequent authors” in critical care journals (Table 5), five authors also contributed to classic critical care articles in other biomedical journals (W.A. Knaus, R.C. Bone, C.J. Fisher, R.P. Dellinger, W.S. Copes). Roger C. Bone and William A. Knaus both were first authors of three such articles.

Table 6 Top forty-five citation classics in non-critical care journals ranked in order of citations received

Discussion

The lists of the top-cited critical care articles (Tables 2 and 6) identify authors and topics that reflect advances in critical care medicine and provide an insight into the history and development of this specialty. The number of citations received was significantly higher for the 45 top-cited articles published in non-critical care journals compared with the 45 top-cited articles published in the specialty journals (810 versus 440 citations per article). It is known that the true impact and fame of an article often cannot be accurately assessed for at least two decades [7, 10, 13, 14, 15]. The sum of the citations of an article is logically dependent on its publication year, since citations accumulate over time. Scientific papers usually are not cited until 1 or 2 years after their publication and generally reach a maximum after 3–10 years, at which time they continue to be cited but at a lower rate [16]. The decline of the citation rate is described as the half-life of a publication and is defined as the time when the citation rate has dropped to half of the maximum. This normal life span of a publication shows that evaluating the rank and significance of recent publications is, at best, limited; thus, articles that are not listed here will eventually deserve classic status.

Another limitation of the present method of calculating the impact of publications is the so-called “obliteration by incorporation,” i.e., original seminal work is absorbed in current knowledge and it is no longer explicitly cited; the absolute number of citations an article has accumulated, therefore, cannot be used as a sole measurement of its “importance”. On the other hand, articles describing critical care scoring systems achieve high citation rates in the present study (mean citation rate of 845 citations per such article in the specialty journals and 918 in all other biomedical journals) and are, with approximately 14%, also over-represented in the lists of top-cited articles (12/74 such articles in the specialty journals and 5/45 in all other biomedical journals). This is probably related to the circumstance that the reasons for citing specific articles may not be entirely appropriate; authors do not necessarily reference the work that have influenced them most significantly in their own work and it seems that citing is not simply giving credit where credit is due but, instead, is a complex social and psychological response with many different motives [17]. This may lead to the finding that some scientists are over-cited by, for example, a substantial number of ceremonial citations [18].

We have been able to make several observations concerning the authors, and their institutions, of the articles included in our study. The vast majority (69% and 89%, respectively) of citation classics in critical care and other biomedical journals originated in the US. This figure is comparable with the origin of citation classics in general surgical journals (78% US) [6], in clinical dermatologic journals (75% US) [7] and in otolaryngology-head and neck surgery journals (84% US) [8]. In the Citation Classics of JAMA, 95% of all articles originated in the US [10], thus confirming the overwhelming influence that the US has on medical research due to its large population and the financial resources available to the scientific community. One must keep in mind that biomedical research productivity world-wide is largely dependent on each country’s per capita gross national product and the expenditure allotted for research and development [19]. Financial support, however, is not the only reason why this US dominance is so strong, as there is also a tendency for US authors to cite local papers and for European authors to publish in American journals [6, 8].

Another observation is rather surprising. In many cross-discipline studies of top-cited articles, methodological publications and review articles predominate [7, 10]. We, therefore, did not expect to find that original research articles accounted for 95% of all citation classics in the specialty journals, although only one of the top-ten articles was a clinical prospective study (Table 2, rank 8); the other top-ten cited articles were either reviews or consensus papers or those describing the development and validation of various scoring systems. Finally, a large gender gap exists in our results. Only one woman is represented among the first authors of top-cited articles in critical care journals; this finding is similar to the results of an earlier study which also identified only six women as first authors of citation classics in dermatologic journals [7]. However, this under-representation of women is rather surprising because critical care medicine is a modern and relatively young specialty of medicine, as reflected by the fact that 60% of the classic articles were published after 1989 in critical care journals.

Although we have tried to eliminate potential flaws in our citation analysis, some limitations were inevitable and are linked to the inherent problems of citation analysis as previously described [2, 3, 17, 18, 20]. It has also to be noted that ISI’s databases and information products were primarily developed for bibliographic use and not for bibliometric analysis. Other problems include biased citing, such as self-citation, citing high-impact journals and review articles, and national or language preferences [21]. In addition, it has been documented that citation statistics at the level of individuals, research groups, journal and countries are strongly affected by sloppy referencing, editorial characteristics, referencing conventions, language problems, author-identification problems, unfamiliarity with foreign author names and ISI data-capturing conventions. Strongly affected entities are consortium papers, journals with dual volume-numbering systems or combined volumes, and journals published in different versions or languages applying different article-numbering systems [5]. The search for classic critical care articles in general biomedical journals was dependent on the key word “Critical Care”; other search terms would yield different data. In addition, key words are often incorrectly or incompletely entered in databases.

Keeping the above limitations in mind and using a high level of suspicion and sophistication, directly counting citations to papers of various researchers, as performed in our study, remains the most obvious measure of the general interest in their work [1]. The founder of ISI, Eugene Garfield, stated that we tend to remember those works that receive the greatest public recognition [10]. The lists of classic papers in Tables 2 and 6, therefore, represent an index related to how often a specific article has been used. These classic articles have influenced many people and should help to bring to our attention important advances in critical care medicine. Citation analysis as a bibliometric indicator does not reflect scientific quality; however, the number of citations an article receives over the years tells us something about the impact on the scientific community of that article [2, 5].