The objective assessment of disease severity is one of the areas of interest in medicine, especially in the field of paediatrics, given the specific characteristics of children.1 In the paediatric intensive care setting, classification schemes are used to attempt to objectively quantify overall severity in critically ill patients; most of them establish severity by estimating the risk of mortality based on the patient's clinical condition. Other classification systems are based on therapeutic interventions, estimating severity based on the number and complexity of procedures performed on the patient.2

The Therapeutic Intervention Scoring System (TISS) was developed in 1974 by Cullen et al.3 and was initially described as a severity index. It measures the intensity of therapeutic interventions and therefore the nursing workload in everyday practice and calculates the appropriate number of nursing staff required to care for the patient. Although the workload can be assessed with several systems,3,4 the TISS is the most widely used and has been proposed as the international gold standard for this measurement.

The TISS was revised and modified in 19834 and has remained unchanged since. It has several drawbacks, such as reliability problems due to the excessive number of items, the variability in the possible interpretation of some of the items and the time required to calculate the score, which can take experienced raters 2–5min.5,6 The TISS-76 has been validated for use in several intensive care unit (ICU) settings, including the Paediatric Intensive Care Unit (PICU).

In 1996, Miranda et al.6 published a simplified version of the original comprising 28 items and known for that reason as the simplified TISS (TISS-28). The authors reduced the number of items and provided specific guidance and directions to facilitate its application. Completing this version takes approximately 2min, less time than the TISS-76, and is less complicated than the previous version.

Previous studies, conducted only in adult patients, found a strong correlation between the TISS-28 and the TISS-76, including the original study by Miranda et al.6 in ICUs in the Netherlands. Moreno and Morais7 published data obtained in a Portuguese ICU and also found a good correlation between both scales. Sánchez-Velázquez et al.8 reported similar results of a multicentric study conducted in Mexico, and Castillo-Lorente et al.9 similar results of a study of ICUs in Spain.

The primary objective of this study was to analyse the correlation between the TISS-28 and the TISS-76 in a PICU in Spain and the substitution of the TISS-28 for the TISS-76, as previous studies have only been conducted in adult patients. The secondary objective was to assess the correlation between each of those 2 scores and disease severity in patients.

We conducted a prospective observational study. We collected data between October 1, 2009 and October 1, 2012. We included all patients admitted to the PICU of a teaching hospital. The exclusion criteria were a length of stay in the PICU of less than 2h and admission to PICU for performance of procedures under sedation.

We collected data on the following variables: date of birth, length of stay (days), age (months), sex (male/female), weight (kg), discharge diagnosis, type of disease based on the classification of the American Academy of Pediatrics10 (respiratory, cardiovascular, neurologic, haematology/oncology, renal/metabolic/endocrine, gastrointestinal, surgical, infectious, trauma, monitoring, poisoning, sedation and other). We documented the treatments and techniques used in the first 4 days of hospitalisation using the TISS-76 and TISS-28, so we obtained 4 pairs of measurements per patient.

The physician in charge of the patient calculated both TISS scores daily, and before discharge, the results were reviewed by 2 different members of the medical team trained in the use of these scales.

We classified patient outcomes in 3 categories (no sequelae, with sequelae and deceased) based on the Pediatric Overall Performance Category scale.11

We performed all the statistical analyses with the R software (www.r-project.org). We summarised continuous variables as mean and standard deviation, median, minimum and maximum, and compared them using the Welch t test if the data were normally distributed and otherwise with the nonparametric Kruskal–Wallis test. We described categorical variables as absolute and relative frequencies, and compared them using the Fisher exact test of independence. We assessed the size of differences by calculating odds ratios (ORs) with the corresponding 95% confidence intervals (CIs) for the crude OR and the adjusted OR, when the latter measure was deemed necessary to account for potential confounders. We calculated crude and adjusted by means of logistic regression. We analysed the association between the mean TISS-76 and TISS-28 scores by simple linear regression. We recorded the obtained Pearson correlation coefficient and the coefficient of determination (R2). We also analysed the consistency of both scales by calculating the intraclass correlation coefficient (ICC) for each pair of scores and for the mean scores in the sample. The analysis also included generation of scatter plots and Bland–Altman plots. We defined statistical significance as a p-value of less than 0.05. We plotted receiver operating characteristics (ROC) curves and calculated the area under the curve (AUC), estimating the optimal threshold using the Youden index.

The study was approved by the Regional Clinical Research Ethics Committee of the Principality of Asturias.

During the study, we collected data for 935 patients. As can be seen in Table 1, there was a predominance of the male sex (58.9%). The median age was 33 months (range, 0–256). The most frequent group of diagnoses corresponded to the respiratory system (30.2%). The mean length of stay was 5.25 days. The mean scores of the Pediatric Risk of Mortality III (PRISM III) and the Pediatric Index of Mortality 2 (PIM 2) were 2.8 and 1.36, respectively. The overall mortality in the PICU was 1.9% and the corresponding in-hospital mortality was 0.2%.

The mean TISS-76 scores for days 1–4 of admission were 18.27, 14.61, 13.71 and 13.12, respectively. The mean TISS-28 scores for the same days were 18.02, 15.93, 15.26 and 14.78. Table 2 presents the scores for the different scales (PRISM III, PIM 2, TISS-76 and TISS-28) for each of the patient outcome groups.

We found a good correlation between the TISS-28 and the TISS-76 scores. The Pearson correlation coefficient for these scores in days 1–4 of the stay in the PICU were 0.912, 0.558, 0.919 and 0.918, respectively. Fig. 1 shows the regression line for the comparison of the TISS-76 versus the TISS-28 scores in day 1 of the PICU stay, along with the linear regression equation. Fig. 2 shows the Bland–Altman plot for the comparison of the TISS-76 and TISS-28 scores in day 1 at the PICU. The ICC for day 1 was 0.90 (95% CI, 0.89–0.91). We found similar correlations for the remaining days under study, save for day 2, in which the correlation was weaker, with an ICC of 0.52 (95% CI, 0.47–0.57). We calculated the overall correlation of the TISS-28 and TISS-76 scores for the entire 4-day period, with a Pearson correlation coefficient of 0.91, the equation TISS-76=−1.74+1.05×TISS-28 (R2=0.824) and an ICC of 0.75 (95% CI, 0.74–0.77).

Figure 1.

Regression line for the comparison of the TISS-76 and the TISS-28 scores in the first day of admission in the 935 patients under study. The linear regression equation was T76=−0.53828+104398×T28 (R2=0.832).

(0.1MB).

Figure 2.

Bland–Altman plot of the TISS-76 and TISS-28 scores in the first day of hospitalisation. The intraclass correlation coefficient was 0.90 (0.89–0.91).

(0.11MB).

The predictive value of the TISS-76 to discriminate between patients without sequelae and patients with sequelae or that died was good and did not improve in subsequent days. The predictive value of the TISS-28 was similar, and the differences were not statistically significant. Fig. 3 presents the ROC curves for the TISS-76 and the TISS-28 scores for day 1. The AUC for the TISS-76 score on day 1 was 0.803 (0.728–0.872) compared to 0.759 (0.670–0.842) for the TISS-28 score. The cut-off point for differentiating patients without sequelae from patients with sequelae or that died for scores obtained in day 1 were 27 in the TISS-76 (sensitivity of 55.2% and specificity of 90.4%) and 24 in the TISS-28 (sensitivity of 58.3% and specificity of 87.2%). We prioritised specificity over sensitivity in the selection of cut-off points.

Figure 3.

ROC curve for the TISS-76 (dark line) and TISS-28 (light line) scores in the first day of admission for the disclination of patients without sequelae from patients with sequelae or deceased.

(0.07MB).

Mortality risk scores are commonly used in intensive care to classify patients and care units and to be able to compare the latter, as well as to guide the allocation and management of resources. However, they are not commonly used for clinical decision-making, which should be an objective in the improvement of both the scales themselves and their use by health care professionals. The PRISM III12,13 and the PIM 213–17 are the scales most widely used in the PICU setting. The TISS scales3,4 have exhibited a good correlation with these mortality scores, despite being mainly based on the care workload. The TISS-76 was published first,3 and it has been validated for use in adult and paediatric ICUs. The updated, simplified version, the TISS-28,4 has been validated in adult care settings but not in the paediatric population. Given the differences in the types and severity of disease and in the bed occupancy rate between the adult and paediatric populations, it is essential that scales be validated in the latter.

Our study validated the TISS-28 in a single PICU to be able to use this simplified scale and to compare the data obtained with both scales, the TISS-76 and the TISS-28, to patient outcomes.

The scores obtained in both scales in the first 4 days in the PICU were very similar: on day 1, the TISS-76 score was 18.27 and the TISS-28 score was 18.02. Our findings were similar to those of studies conducted in other countries, such as Mexico and Ecuador.8 In subsequent days, the scores in both scales decreased, although they continued to be quite similar. This means that the predictive value of the scales does not increase significantly with passing days, and that the values obtained in the early hours following admission can provide a fairly accurate estimate and good predictions about patient outcomes.

We found a good correlation between the scales. The overall correlation coefficient calculated for the first 4 days in the PICU was 0.90. It is a good value, higher than the coefficient of 0.85 reported by Castillo-Lorente et al.,9 although slightly lower than the coefficient of 0.93 reported by Miranda et al.,6 Moreno and Morais7 and Sánchez-Velázquez et al.8

In our study, the TISS-28 explained 82.4% of the variance in the TISS-76 score. This percentage was similar to those reported in the original sample in the Netherlands (86%)6 and the percentage in Mexico and Ecuador8 (86.4%), and higher compared to the percentage reported in the Portuguese ICU study7 (72%). Table 3 compares our results to those of previous studies.

In our sample, the difference of the mean TISS-76 and TISS-28 scores (difference of means, 0.25 points) was lower compared to the sample in the Netherlands (difference of means, 4.6 points)6 and the study conducted in Portuguese ICUs (difference of means, 1.3 points).7

The correlation coefficients found for days 1–4 of the stay in the PICU were 0.90, 0.52, 0.91 and 0.90, respectively. We did not find data in the literature to compare these results, as articles on the subject have only reported data for the first 24h,7,9 the mean of the values obtained throughout the stay or even, in some cases, the TISS score obtained in the shift preceding discharge from the ICU.8

It is worth noting that the correlation for day 2 was lower compared to the other days under study. In days 1, 3 and 4 the correlation was approximately 0.90, while on day 2 it dropped to 0.52. We were unable to compare the results of this specific factor with other studies for reasons noted above, although we could attribute this difference to the absence of some of the TISS-76 items in the TISS-28.

In previous studies,6,8 TISS-76 and TISS-28 scores were used to classify patients in 4 groups based on the care workload. Since disease severity was lower in our patients, we established cut-off points to attempt to predict patient outcomes, discriminating between patients without sequelae and patients that either died or developed sequelae. The cut-off points that we found were 27 for the TISS-76 and 24 for the TISS-28. With these values, the prediction of patient outcomes in our sample was quite accurate. More importantly, we were able to find a cut-off point in the TISS-28 under which there were no errors in the classification of deceased patients.

The main limitations of our study were the small sample size, the low mortality and the fact that it was not a multicentric study, in addition to data collection ending in 2012, which is important given how quickly intensive care evolves. Furthermore, we did not document 2 of the most criticised characteristics of the TISS-76, which are the amount of time it takes to complete and the vagueness in the definition of some of its items. The strength of this study is that, to our knowledge, this is the first study devoted to the validation of the TISS-28 in a PICU setting.

The TISS-28 administered in the first 24h from admission has a similar predictive value to the TISS-76, and the predictive value does not improve in subsequent days. Therefore, we propose the use of the TISS-28 in the first day of hospitalisation for quantitative assessment of the intensive care labour effort in critically ill children.

Our results show that in our region, the TISS-28 could be used instead of the TISS-76 with no loss in the obtained information.

The authors have no conflicts of interest to declare.