The principal investigators contacted the specialised neonatal and paediatric transport units that existed at the time in Spain to set up a coordinating committee in which each of them would be represented, to the end of defining a project that would be relevant to each of the existing care delivery models.

After setting up the committee, its members developed an initial list of indicators based on a review of the international literature on the subject and their own personal experience as practicing physicians specialised in neonatal and paediatric transport.

The committee developed a list of voluntary collaborators, selected from the pool of clinicians that staffed the participating units, whose task would be to assess the different indicators proposed in the initial list. The committee also asked the collaboration of the working groups on health care transport of the SECIP and the Sociedad Española de Neonatología (Spanish Society of Neonatology, SENeo) to recruit collaborators in transport units other than those represented in the committee.

Indicators were selected by the Delphi method, starting with collaborators assessing the indicators through an online questionnaire in Microsoft Office Excel® format. Each indicator was rated in terms of its feasibility and relevance on a scale of 1–9 (1 indicating lowest value and 9 the highest value). In addition, participants had to give their opinion on the pertinence of including each indicator in the definitive list. Lastly, participants were directed to provide comments to justify their position, if they deemed it fitting, and were allowed to propose additional indicators for inclusion in the list.

After this initial assessment, the results and comments were submitted to all participants. A second round of assessments was conducted following the same procedure.

In the first round, the indicators selected for inclusion in the list were those with a consensus greater than 95% in support of their inclusion and the highest ratings in relevance as well as feasibility. In the second round, the indicators selected for inclusion were those with a consensus greater than 75% in support of their inclusion and the highest ratings, until the desired number of indicators (12–15) had been selected to adequately address every quality domain defined by the National Academy of Medicine (NAM, previously known as Institute of Medicine) of the United States6 and the Donabedian model.7

In the last phase of the project, the committee developed the final list of indicators, given the best ratings by participants. The exact definition of each indicator was also established through a consensus process by the members of the coordinating committee, so that any units that wished to use the indicators would have no doubts as to their interpretation.

To our knowledge, this is the first time that a list of quality indicators developed by consensus for assessment of neonatal or paediatric transport is published in Spain. The development process started with a search for references in the international literature. The results of the search included a consensus statement published in 2013 by the Ohio Neonatal/Paediatric Transport Quality Collaborative (USA)14 that proposed and defined 23 indicators at the state level, although the authors did not believe they could be extrapolated to the national level; a Delphi project published in 2015 by the American Academy of Pediatrics (AAP) Section on Transport Medicine15 that yielded 12 indicators, which have since been used to monitor the activity of transport teams at the national level; and most importantly the Ground & Air Medical Quality in Transport (GAMUT) database of the Air Medical Physician Association (AMPA),16 created in 2014, which includes 27 indicators that are applicable to both adult and paediatric transport teams.

The initial selection of indicators made by the coordinating committee included the indicators of the cited sources that the committee deemed most relevant for paediatric and neonatal transport in Spain and excluded those that were not considered pertinent, such as indicators that are mostly relevant in the adult care setting.

Later on, the remaining participants included some indicators considered particularly relevant with nearly unanimous consensus, such as the absence of medical equipment failures or accidental extubation, and eliminated other indicators that would be difficult to modify because the team could not address them directly (such as presence of nursing staff or specialised technicians) or that reflected events that were not frequent enough that their monitoring would be relevant (such as the need of resuscitation or transport crew injuries).

As has been the case in similar projects, the most limiting challenge was the lack of gold standards that would serve as a reference with which to assess potential indicators, which is the reason why we resorted to a method based on expert opinion in lieu of the scientific evidence that is not currently available.

The Delphi methods is a widely used process for the development of quality indicators in the health care field, involving a systematic approach to generating consensus, with development of content through a series of rounds or questionnaires.17 One of its strengths is its considerable feasibility, allowing participation, possibly on an anonymous basis, of a large number of evaluators who are geographically apart and without need of meeting in person. In our case, since we selected evaluators that were active practitioners, the method offered the added advantage that the indicators were developed by the same professionals meant to use the resulting tool.

We believe that this consensus-based list meets the objective of including indicators addressing the three domains of health care quality included in the model published by Donabedian in 1966: structure (elements required for care delivery, such as infrastructure, equipment, staff, education etc.), process (what is done in care delivery, including processes in diagnosis, treatment, prevention etc.) and outcome (individual health outcomes and public health outcomes)7.

The selection also covers the six key aims in quality improvement of the NAM model, explained in its publication Crossing the Quality Chasm: a New Heath System for the 21st Century6 (2001): safety, effectiveness, equitability, timeliness, efficiency and patient-centredness.

There are limitations to our study, including the fact that the Delphi method does not have a fully defined design, leaving the conditions required for consensus or the number of rounds to the judgment of the coordinators. In addition, the selection of indicators by consensus does not guarantee their validity or usefulness, as it is not based on an objective approach but depends on the subjective opinion of the panel of experts, as mentioned above.