In the international scientific community, there is broad consensus that the number of citations and other derived indicators (impact indicators) cannot be used to compare subject areas or different specialties for various reasons. On one hand, the size of the field of research influences the number of citations that an article (or journal) may achieve: in a community of 500 researchers, the absolute frequency of citations will be smaller than in a community of 5000 researchers.1,2 On the other, each field of science exhibits different citation patterns. For example, while articles in the fields of biochemistry or social sciences include approximately 30 references each, articles in engineering tend to have 10 references and articles in mathematics 5, and therefore the probability of being cited in the biochemical literature is 6 times greater than the probability of being cited in the mathematical literature. Similar patterns can be found in the medical specialty literature, especially in fields with a smaller “research mass” or in which clinical practice predominates over basic research.3–5

Some of the agencies that evaluate bibliometric performance do not take into account the existing variability between specialties and include all of them in the broader umbrella of Medicine, using citation and impact indicators to compare them, which can be seriously and unfairly damaging to researchers in small or medium-size fields that compete at a disadvantage, as the instruments that evaluate their performance are not adjusted to the characteristics of their fields.

Numerous studies have been published that analyse the current situation, evolution and trends of paediatric research worldwide6–8 and in Spain,9 some of which have focused on paediatric subspecialties and others on specific journals.9–14 A search we did on the subject in the PubMed database in October 2019 with the terms ("Pediatrics"[Mesh]) AND "Bibliometrics"[Mesh]) retrieved 181 articles. However, we did not find studies comparing bibliometric indices in paediatrics with those of other medical specialties, a crucial aspect that we believe should be analysed and discussed, always taking into account the particular characteristics of each field.

The aim of our study was two-fold: on one hand, we aimed to identify and characterize indicators for the output, citation, impact and collaboration in the Journal Citation Reports Pediatrics category, and on the other, to characterize the position of the journal Anales de Pediatría in the context of the Spanish journals in the other 20 medical specialties/categories.

The source we used to obtain indicators were: the Science Citation Index-Expanded (SCI) database of the Web of Science (WoS) for output indicators, the Journal Citation Reports (JCR) for impact indicators, and the Scimago Journal & Country Rank (SJR) (sites that offer information on scientific indicators of journals indexed in the SCI and Scopus databases, respectively) for collaboration indicators. The period we selected to present the indicators in annual or cumulative distributions was the 2009–2018 decade, and we restricted the sample to original articles.

We selected the subject categories related to Paediatrics from the website of the Asociación Española de Pediatría (Spanish Association of Paediatrics [AEP], https://www.aeped.es/especialidades), choosing medical fields associated to any of the 24 paediatric specialty associations affiliated to the AEP. In the end, our study included the subject of Pediatrics and the 20 subject categories listed in Table 1.

We structured the study in 2 parts. The first part consisted in the analysis of the Paediatrics subject category in the context of the other 20 medical specialties. In the second part, we analysed Anales de Pediatría in comparison with the other Spanish journals indexed in the SCI.

We classified indicators into:

• 1

  Output indicators: number of documents, number of journals per subject and number of documents per journal.

• 2

  Citation indicators: number of citations, average number of citations per journal, average number of citations per article and number of uncited articles.

• 3

  Impact indicators: mean impact factor, aggregate impact factor and h-index. We calculated the mean impact factor by dividing the impact factor of all journals in a given category in one year by the number of journals in the category. We calculated the aggregate impact factor in the same way, but using the number of citations and the number of articles in the journals in the category.

• 4

  Indicators of the most frequently cited articles by subject category.

• 5

  Indicators of international collaboration. To calculate these indicators, we selected the 5 journals with the highest impact factors (IFs) in 2018 in each JCR Science Edition subject category for the year, and obtained the percentage of international collaboration for each journal from the SJR, which reflects the proportion of documents signed by authors from more than 1 country. We then calculated the mean values for each journal, taking into account the years for which data was available, either for the full 2009–2018 decade or the years following indexation of the journal in the database. Lastly, we calculated the mean of the 5 journals to obtain the indicator for each subject category. We also calculated the interquartile range between the journals with the highest and lowest percentages of international collaboration.

We included Spanish journals based on the hits obtained in a search of the JCR database in 2018 of the 21 subject categories (Pediatrics and the 20 categories listed in Table 1). The search yielded 24 journals, and we obtained indicators similar to those obtained for the subject categories and the number of funded articles. We conducted the searches on September 16, 2019 using computer terminals of the Universitat de València.

We performed the statistical analysis with the software Statgraphics Centurion XVI, calculating the correlations between variables. We also performed logarithmic transformation to normalize the data and obtain regression curves as needed.

This study allowed us to establish the ranking of global and Spanish paediatrics publications in absolute terms and in relation to other subjects based on a selected set of output, citation, impact and collaboration indicators. The sources used for the purpose were the WoS SCI and JCR databases, supplemented with information from the SJR. We obtained some of the indicators directly from these sources and calculated the rest. To make comparisons with other fields, we chose the 20 subjects covered by the “specialty associations” affiliated to the Asociación Española de Pediatría (https://www.aeped.es/especialidades).

As we previously discussed, in absolute terms the subject category of Pediatrics held an intermediate position in relation to the other categories, both in the mean number of journals and in its output (ranking 9th in both) and in terms of citations (ranking 12th). However, taking into account other relative indicators, such as the mean number of citations per journal or the percentage of uncited articles, it ranks 17th and 19th, respectively. We found a similar trend in the mean and aggregate impact factors, which placed Pediatrics in the 18th and 17th positions, respectively.

In most citation-related indicators, the Pediatrics category usually ranked higher than 2 other categories (Emergency Medicine and Primary Health Care), and it also outperformed another 2 categories in other indicators (Dentistry, Oral Surgery & Medicine and Rehabilitation). The analysis of the most cited documents yielded a perspective that was similar to the perspective obtained with other citation-based indicators, placing Pediatrics above the same specialties.

A finding worth highlighting is that the h-index for Pediatrics ranked this category above not only the 4 specialties we already mentioned, but also above others such as Cardiac & Cardiovascular System, Allergy and Rheumatology. However, this indicator is not the most suitable to compare different research fields, as it does not take into account the differences in the patterns of publication and citation between fields. The h-index is high in subjects with a long research tradition.15,16

Collaboration is a key factor in research. The multidisciplinary, technical and complex nature of biomedical research requires teamwork, and its maximum expression is the formation of international teams for the development of collaborative international projects such as multicentre trials.17–19 In the category of Pediatrics, international collaboration occurred in nearly 18% of the articles, a percentage that once again outperformed Emergency Medicine and Primary Health Care, and, in this case, also Oncology. There is evidence that clinical trials conducted through collaborative networks give rise to articles with high numbers of citations and published in high-impact journals.20,21 The regression analysis showed that international collaboration was associated with a higher mean number of citations per article.

Compared to other Spanish journals indexed in SCI, Anales de Pediatría is one of the most productive journals and occupies an intermediate position based on the number of citations. The mean number of citations per article is not high, but it is above the mean for specialties such as Urology & Nephrology, Endocrinology & Metabolism and Clinical Neurology.

One of the possible explanations of the position of Pediatrics relative to other subject categories is the dearth of published paediatric randomised controlled trials. Clinical trials in children are affected by numerous methodological, ethical and economic barriers in meeting quality standards, which hinders their performance. On the other hand, the marketing authorisation of drugs distributed in Europe included evidence from paediatric clinical trials in only 30% of drugs.22 Initiatives such as the Global Research in Paediatric (GRiP) project, aimed at the research and development of safe and effective drugs for children and launched under the Seventh Framework Programme of the European Commission, may contribute to improving this situation. In this context, Salim et al. found that, following the trend observed in other medical specialties, there has been a significant increase in recent years in the publication of systematic reviews in paediatric surgery, although their quality has been poor.23

Paediatric diseases are often rare and may only affect a small number of patients, and therefore it may be necessary to develop research networks and perform multicentre studies to obtain significant results. In this regard, there has been a substantial increase in collaborative projects and in the development of paediatrics research networks in recent years, but it is still too soon to evaluate the results and necessary to continue to develop these initiatives.

On the other hand, a high percentage of paediatricians carry out their work at the outpatient level outside hospitals with scarce resources and far from larger health care facilities that are better equipped, which means they produce less research, as their circumstances are less advantageous compared to medical professionals in other fields that are mainly hospital-based or involve basic research.

To improve the quality of paediatrics research and with it citation and impact indicators, it would be worth promoting training in research methods to provide these specialists with the skills required to design and participate in paediatric clinical trials.24–26 Holland et al. found that each additional year of training in research methods in a national child health institute in the United States was associated with a 15% increase in the h-index.27 A key point in this process is to facilitate the collaboration and pooling of resources between academics, health care providers, regulatory agencies and industry at both the national and international levels. At the same time, governments and educational institutions should provide an appropriate framework to reward companies that invest in paediatric research.

There is no question that since this is a multifactorial problem, it is challenging to identify every necessary improvement strategy in order to improve the standing of the field of Pediatrics relative to other subjects. Years ago, we performed a strengths, weaknesses, opportunities, and threats (SWOT) analysis of the Spanish paediatric research publication output (based on a study of science output metrics for the 2006–2010 period conducted by the Asociación Española de Pediatría),9,28,29 and some of their conclusions as to the strengths, weaknesses, opportunities, and threats can apply to our current discussion.

One of the limitations of this study is that some of the indicators that we used, especially the impact factor, have drawbacks when used as tools for assessing research that have been extensively debated in the literature. Some of the most relevant drawbacks, as discussed above, is that these indicators exhibit significant variability depending on the field of research, that they may be manipulated by editors and the lack of transparency as regards the data used for their calculation.30–32 In this regard, the San Francisco Declaration on Research Assessment recommends against using journal-based metrics, such as the impact factor, as a measure of the quality of individual research articles, to assess the contribution of an individual researcher or to guide funding, appointment, and promotion decisions.33

The citation and impact indicators in Pediatrics tended to be lower compared to other subject categories, with the exception of Emergency Medicine, Primary Health Care, Dentistry, Oral Surgery & Medicine and Rehabilitation. The various factors mentioned above may contribute to this situation.

Whatever the reason, the final conclusion is clear and was one of the aims of this study: if the agencies that evaluate scientific output do not take this inter-specialty variability into account and place all of these journals in the broader subject of Medicine, the category of Pediatrics (and, therefore, paediatricians) is at a disadvantage when it comes to applying for grants and competing for national and international funding, which constitutes a vicious cycle that will be difficult to overcome.