Elsevier

Journal of Pediatric Surgery

Volume 52, Issue 11, November 2017, Pages 1732-1735
Journal of Pediatric Surgery

Review Articles/Meta Analyses
Quality of systematic reviews and meta-analyses published in pediatric surgery

https://doi.org/10.1016/j.jpedsurg.2017.07.022Get rights and content

Abstract

Aims

Systematic reviews (SRs) and meta-analyses (MAs) studies may influence and direct surgical practice. Against this background we have analyzed the quality of systematic reviews and meta-analyses published in the specialty field of pediatric surgery using the Assessment of Multiple Systematic Reviews 11-item tool (AMSTAR).

Methods

MEDLINE and EMBASE databases and the three major journals in pediatric surgery were searched for SRs and MAs in pediatric surgery. Studies involving predominantly adult populations were excluded. Two reviewers independently scored included studies against AMSTAR criteria and disagreements were resolved by consensus. A total rating of 4 or less was considered ‘poor’ methodological quality, 5–8 as ‘fair to good’ and 9 or greater as ‘good’.

Results

Original searching retrieved 1,281 articles. 126 articles were included for final analysis. Examining recent trends, 4 studies were published in 1995–2000 compared to 78 in 2011–2014. Using AMSTAR scoring criteria, 35 reviews (28%) were regarded as ‘poor’ in terms of methodological quality, 59 (47%) ‘fair’, and 32 (25%) ‘good’ quality. We observed no improvement in AMSTAR score before and after the development of the tool (mean score pre-2008 6.8, post-2008 5.9, p = 0.136).

Conclusions

Despite an increase in the number of SRs / MAs published in pediatric surgery, a quarter of all studies were considered poor in terms of their quality and scientific validity. Journals must define and apply minimum criteria to ensure pediatric surgeons seeking to publish high quality SRs / MAs achieve these requirements.

Section snippets

Literature search

MEDLINE and EMBASE databases were searched using key words “systematic review” or “meta-analysis” and “pediatric” or “pediatric”. We also searched the table of contents of three major pediatric surgery journals, namely Journal of Pediatric Surgery, European Journal of Pediatric Surgery and Pediatric Surgery International. The reference listings of relevant studies were also hand searched. A study had to be identified as either a systematic review or meta-analysis to be included for the

Characteristics of included studies

The original literature search retrieved 1281 articles. After removing duplicates, 857 titles were screened and 181 full-text articles were reviewed for eligibility. One hundred and twenty six articles were included for final analysis, which comprised 72 systematic reviews and 54 systematic reviews with accompanying meta-analysis (Fig. 1). These studies were published during the 20 year period covering 1995–2014. The list of studies included and excluded at the stage of full paper review is

Discussion

Despite the influence of an increasing number of systematic reviews informing clinical practice, aiding guideline developers and research agendas the quantity of systematic reviews in the specialty field of pediatric surgery remains low compared to many other specialties. The MEDLINE database indexed over 2500 systematic reviews in 2007 of which 20% were included in Cochrane reviews [8]. In Neonatology for example there were 61 Cochrane reviews available [12]. We were only able to identify 126

References (16)

There are more references available in the full text version of this article.

Cited by (13)

  • The methodological quality of surgical randomized controlled trials: A cross-sectional systemic review

    2022, Asian Journal of Surgery
    Citation Excerpt :

    A guidance recommended to consider and describe interventions in systematic review has been published in 2017, pertaining to enhancing the usability and reproducibility.36 Our finding is consistent with others empirical studies in diverse disciplines that addressed low quality and major flaws are very common.13–20 Meanwhile, we provided detailed information on each item instead of overall score, which has helped review authors, surgeons, and clinicians identify critical weaknesses and suggest some ways to improve the process.

  • A systematic survey showed important limitations in the methods for assessing drug safety among systematic reviews

    2020, Journal of Clinical Epidemiology
    Citation Excerpt :

    The extent to which systematic reviews and meta-analyses are useful for informing drug safety largely depends on how well the studies are designed, conducted, analyzed, and reported [1,5]. A number of previously published surveys [6–19] have applied A Measurement Tool to Assess systematic Reviews or other scales to investigate the methodological quality of systematic reviews in various fields of clinical medicine and shown that methodological quality was often inconsistent. Four studies [20–23] examined methodological details about adverse effects among systematic reviews.

  • Assessing the methodological and reporting quality of clinical systematic reviews and meta-analyses in paediatric urology: can practices on contemporary highest levels of evidence be built?

    2020, Journal of Pediatric Urology
    Citation Excerpt :

    Although the least commonly included PRISMA parameter in this study, the presence of review registration also significantly correlated with higher PRISMA scores (P < 0.0001). Salim et al. [23] attempting to evaluate all paediatric general surgical literature (n = 44) using the AMSTAR tool also highlighted publication bias as a poorly addressed criteria, with 80% of studies failing to do so adequately, with literature searched being the most commonly scored item. To the study authors' knowledge, neither AMSTAR-2 nor PRISMA has been formally validated, and there are no studies demonstrating that these scores have a significant correlation with bias or treatment effect.

  • Systematic reviews and meta-analytic techniques

    2018, Seminars in Pediatric Surgery
    Citation Excerpt :

    These temptations have led many authors to publish hastily created reviews that fail to meet the appropriate level of scientific rigour. Indeed, evaluations of the quality of systematic review reporting in the field of pediatric surgery have generally not been favourable.26,27 Thus, despite their traditional position at the top rung of the pyramid of evidence, many reviews amount to bad science and should be treated with suspicion – the term ‘systematic review’ is not necessarily equivalent to trustworthy.

View all citing articles on Scopus

Funding: No funding was sought or received for this work.

1

Equal contributing authors.

View full text