Some sedation scales associate different levels of consciousness with behavioural responses to auditory or somatosensory stimuli.

For a sedation scale to be considered ideal, it must have the following characteristics1: easy to use, applicable to every possible clinical situation and age group, reliable, capable to discriminate between different levels of sedation (especially between moderate and deep sedation) and feasible (that is, must be based on few, simple and easy to measure parameters).

The most significant limitations of these scales are their subjectivity, that they are reliable only at minimal and moderate depths of sedation and the use stimuli to obtain a score.2 Stimuli can be disruptive during procedures and there may be variance between observers.

The Niño Jesús Hospital Procedural Sedation Scale (known as the ESPHNJ) was published by A. Valdivielso Serna in 19983 and uses 4 categories to classify the level of consciousness of patients during procedures: level 1 corresponds to an awake, alert and oriented, and level 4 to a patient that does not respond to physical stimuli, although possibly capable of perceiving pain if the stimulus is strong enough (Table 1).

This scale offers several advantages: it is simple, takes only a few seconds and is easy to reproduce, all of which makes it widely applicable. It has been widely used in Spain in children of varying ages to assess the depth of sedation in procedures performed outside the operation theatre and in the paediatric intensive care unit (PICU) setting.4–11 A study published in 2015 reviewed a series of 13737 cases of paediatric sedation and analgesia during painful procedures performed at the Hospital Universitario Niño Jesús (HUNJ).4 The main limitation of this scale is that it has not been validated.

The bispectral index (BIS) estimates the level of consciousness of a patient through a mathematical analysis of electroencephalographic frequency and amplitude data. It is validated for assessment of for children aged more than 1 year undergoing procedural sedation,2,12–16 and some authors have found it useful in children aged more than 6 months.12,17,18

The University of Michigan Sedation Scale (UMSS)19 was designed specifically to assess procedural sedation in paediatric patients and has been validated (Table 1).

The Ramsay Scale (RS)20 is the scale used most widely at the international level for assessment of procedural sedation. It has not been validated in the paediatric age group (Table 1).

The aim of our study was to assess the validity, reliability and feasibility of the ESPHNJ to assess sedation used for invasive procedures in paediatric patients through comparison with the BIS (a validated objective instrument), the UMSS (a validated subjective instrument) and the RS.

We conducted a prospective analytical study in 2 hospitals to validate the ESPHNJ.

Enrollment by consecutive sampling of all children that met the inclusion criteria. Patients in the sample underwent invasive diagnostic or therapeutic procedures and received sedation or analgesia from paediatricians at the HUNJ (in the analgesia and sedation unit) and the Hospital General La Mancha Centro (HGMC) (in the special tests unit, the paediatric intensive care unit [PICU], the emergency department or the paediatric ward) between April 2016 and June 2017. The exclusion criteria were: age less than 6 months, use of general anaesthesia, administration of ketamine (as it decreases the level of consciousness without an associated decrease in the BIS readings), use of mechanical ventilation, administration by a route other than the intravenous route, severe compromise, moderate or severe cognitive impairment, or performance of a magnetic resonance scan (as BIS monitoring is not possible during this procedure).

We used the ESPHNJ, the UMSS19 and the RS.20

We also use BIS monitoring. Values of 80–100 indicate absence of sedation or mild sedation, values of 60–80 moderate sedation, values of 40–60 deep sedation and values of less than 40 a deep hypnotic state.21

All sedation and analgesia procedures followed the established protocol in each of the hospitals, which had similar protocols. Propofol and/or midazolam were used for sedation and fentanyl for analgesia.

Propofol and fentanyl protocol: propofol, 2mg/kg (up to a maximum of 100mg) over 3min, followed by 1mg/kg (maximum 50mg) over 3min, and 10mg/kg/h for maintenance. Fentanyl, 2μg/kg (maximum 100μg) over 3min, followed by 1μg/kg (maximum 50μg) over 3min. From this point, titration by 0.5μg/kg increments (maximum 25μg) until desired analgesia is achieved.

Midazolam and fentanyl protocol: midazolam 0.2mg/kg over 3min (maximum 10mg), followed by 0.1mg/kg (maximum 5mg) over 3min. From this point, titration by 0.05mg/kg increments (maximum 2.5mg) to maintain deep sedation. Fentanyl same as previous protocol.

The physicians that managed the patients were blinded to the BIS values.

All patients were monitored with BIS (BIS VISTA®; Aspect Medical Systems, Norwood, United States) through 10min after the procedure ended. The principal investigator recorded the BIS values displayed in the BIS view monitor.

We recorded video for all procedures. We edited out segments with insufficient BIS quality (signal quality index [SQI] <50 and/or electromyography [EMG] value >50), after which there were 1188min of video recording left. We split video recordings in 30-s segments (so that observers would be blind to the moment when sedatives and analgesics were delivered). Subsequently, we coded the segments and selected 150 of them at random.

Four observers evaluated the level of sedation using the three scales. These observers did not receive specific training on the scales, instead applying them based on their own experience. The observers were 1 paediatrician, 1 PICU nurse, 1 anaesthesiologist and 1 surgical nurse with experience in paediatric sedation and analgesia. All were blinded to the timing of drug delivery and to the BIS values. Each observer scored all 150 videos using the UMSS. After this, the paediatrician and the PICU nurse (group 1A) scored a random selection of 75 of the 150 videos (video set 1) with the RSS, independently and blinded to each other. The anaesthesiologist and the surgical nurse (group 1B) scored the remaining 75 videos (video set 2) with the RSS, independently and blinded. Later on, group 1B scored the videos they had not seen (video set 1) with the ESPHNJ, and group 1A did the same with video group 2 (Fig. 1).

Figure 1.

Flowchart of the process of validation of the Niño Jesús Hospital Procedural Sedation Scale and the Ramsay Scale.

(0.65MB).

To assess the test-retest reliability and intraobserver agreement, 50 videos from sets 1 and 2 selected at random were scored again by the same observers with the ESPHNJ and the RS. Lastly, we assessed interobserver variation by comparing the different members of group 1.

To correlate the value recorded per minute of BIS monitoring with a 30-s fragment extracted from each minute of recording, we used the BIS value recorded during that minute, as has been done in most previous studies.12,13,22–27

Since the scoring scheme differed between the UMSS (5 categories), the ESPHNJ (4 categories) and the RS (6 categories), we performed z transformations of the scores obtained with these instruments.

We calculated the required sample size for an expected correlation coefficient of 0.6, a power of 90% and a 95% level of confidence, which was of 25 patients. The final sample included 65 patients enrolled by consecutive sampling.

We have summarised quantitative data as mean and standard deviation (SD) or median and interquartile range (IQR) depending on the distribution of the variable. We summarised qualitative data as absolute and relative frequencies.

In the assessment of the construct validity of the ESPHNJ, we used the Wilcoxon signed-rank test to describe changes in scores over time following sedative administration.

To assess criterion validity, we analysed the correlation between ESPHNJ scores and BIS values, UMSS scores and RS scores by calculating the Spearman or the Pearson correlation coefficient as applicable based on the distribution of the variable. We used the guidelines proposed by Cohen to assess the obtained correlation values,28 considering coefficients of 0.6 or greater indicative of an adequate correlation.

We assessed interobserver agreement by means of the intraclass correlation coefficient (ICC), consider values of the ICC of 0.75 or greater indicative of good reliability. We also assessed this aspect with Kendall's τ. In addition, we assessed the internal consistency of the scale using Cronbach's α, defining adequate consistency as a value greater than 0.7. We assessed test-retest reliability by means of the Spearman correlation coefficient.

To assess the precision of the BIS score in predicting the depth of sedation, we generated receiver operating characteristic (ROC) curves for the ESPHNJ score, dichotomising the BIS score for each of the levels of sedation.

All the statistical analyses were performed with the software SPSS® version 18. We defined statistical significance as a p-value of less than 0.05.

The sample included 65 patients with a mean age of 8.1 years (standard deviation [SD]: 4.4), 49% were male, all had an ASA (American Society of Anesthesiologists) physical status classification I or II.29 Thirty-seven (56%) underwent sedation/analgesia at the HUNJ (sedation and analgesia unit) and 28 (44%) at the HGMC (15 [23%] in the special tests unit, 7 [11%] in the PICU, 6 [9%] in the emergency department and 1 [1%] in the paediatrics ward). Table 2 summarises the characteristic of the patients included in the study.

The 65 patients underwent a total of 79 procedures associated with 70 sedation/analgesia procedures. The most frequent sedative was propofol, used in 87% of the patients (mean dose, 4.2mg/kg; SD, 1.6), while midazolam was used in 13% (mean dose, 0.31mg/kg; SD, 0.18). Fentanyl was used in 85% of the patients (mean dose, 3.1μg/kg; SD, 0.95).

Table 3 presents the median BIS values and median scores obtained with the UMSS, the ESPHNJ and the RS for the 150min of recording included in the analysis.

Validity

When it came to the construct validity of the ESPHNJ, we found a significant increase in the sedation scores relative to baseline 5, 10 and 15min after administration of sedation/analgesia (p<.001). The level of sedation increased from a median of 1 (range, 1–1) at baseline to 3 (2.8–4) at 5min and 4 (4–4) at 10 and 15min. For the criterion validity, we found a negative correlation between the ESPHNJ and the BIS (r=−0.166), compared to a positive and stronger correlation between the BIS and the UMSS (r=0.497) and the BIS and the RS (r=0.405).

Fig. 2 presents the scatterplot for the ESPHNJ and BIS scores of the 150 paired measurements in the 65 patients. The ROC curves showed a low power of the BIS for prediction of sedation levels 2, 3 and 4 in the ESPHNJ. The sensitivity and specificity were 56% and 65%, respectively, for the prediction of level 4, and 40% and 94%, for the prediction of level 3 (Fig. 3).

Figure 2.

Scatterplot of the BIS and ESPHNJ scores of the 150 paired measures obtained in the sample of 65 patients.

(0.08MB).

Figure 3.

(A) ROC curve for the performance of the BIS score (optimal cutoff ≤76.5) in predicting level 3 of sedation in the ESPHNJ. (B) ROC curve for the performance of the BIS score (optimal cutoff ≤65.5) in predicting level 4 of sedation in the ESPHNJ.

(0.1MB).

Lastly, in the analysis of the interobserver agreement, we found a correlation coefficient of 0.428 for the agreement between the paediatrician and PICU nurse, and a coefficient of 0.388 for the agreement between the anaesthesiologist and the surgical nurse.

Although the ESPHNJ is the scale used most widely in Spain to assess the level of sedation in procedures performed in paediatric patients, we are not aware of any studies performed to assess the validity and the reliability of the ESPHNJ compared to other clinical scales.

Our results show that the ESPHNJ is a valid, reliable and applicable instrument for assessing the level of sedation during invasive procedures with deep sedation/analgesia in paediatric clinical practice.

The results of our study demonstrate that the ESPHNJ is a valid, reliable and applicable instrument for monitoring sedation during invasive procedures with deep sedation and analgesia in paediatric patients based on the comparison to other sedation scales. The ESPHNJ performed less well compared to the UMSS and the RSS.

The authors have no conflicts of interest to declare.

The authors thank Dr. Ramón Garrido Palomo, Dr. Pilar Saavedra Hernández, Ms. Miguela García-Cervigón Jaime and Ms. Belén Moreno Gutiérrez for being the observers who have scored the scales.