Since it is impossible to include the entire population that has a given disease in a study, it is very important that the selected sample be as similar as possible to this immeasurable universe (and that it manifests the spectrum of disease that we encounter in clinical practice).

The report must specify the criteria applied to ascertain that the included patients had the disease whose prognosis was being investigated and how the sample was selected, as the findings may be distorted by biases in these processes. The selected patients should all be at a similar stage in the course of disease, which needs to be clearly defined in the report,15 and the sample size must be adequate for the study of the outcome of interest.

In case-control studies, determining whether the exposure occurred prior to the target outcome is key. It is important to assess whether the researchers implemented strategies to minimize recall bias or interviewer bias. In the case of cohort studies and clinical trials, the reader must carefully consider whether strategies were implemented to ensure that the occurrence of the outcome has been ascertained through the same method and with equal probability in both groups.

The follow-up must be sufficiently long to allow time for the outcome of interest to occur and sufficiently complete in all patients. However, losses to follow-up are common. The number of losses (which can affect the accuracy of the risk estimate) and its relationship to the proportion of patients who experience the outcome of interest need to be taken into account. If the reasons for the losses are not related to the phenomenon under study and the characteristics of patients lost to follow-up are similar to those of the patients who completed it, the results are probably valid.

If the answer to any of the above questions is “no”, one can be fairly certain that the study will not provide accurate prognostic estimates, whereas if the answer to all of them was “yes,” one can be fairly confident that the study will provide good prognostic information.

• b)

  Secondary criteria:

The authors should have defined the outcomes of interest clearly before the start of the study: these outcomes may be objective and easily measurable (death) or subjective and difficult to measure (pain intensity). It is important that outcome assessors are blinded in order to avoid observer bias.

When data from patient subgroups are reported, the reader must verify that adjustments were made for other known relevant prognostic factors. Adjustments can be simple (stratified analysis) or more complex and powerful (multivariate analysis, in which the authors decide which variables to include). One of these methods must be applied to be able to tentatively conclude that the prognosis is better or worse in a given subgroup. We say “tentatively” because the statistical techniques used to determine the prognosis of subgroups are predictive, not explanatory.

The group in which the putative prognostic factor is first identified is called the training set (derivation set). When multiple prognostic factors are evaluated, it is possible that some may have been identified by chance alone, so these findings must be validated in an independent group of subjects (the validation set).

The quantification of results is based on the number of events that occur during the follow-up of a cohort and is usually expressed as a cumulative frequency at a given time point or the median survival. There are three methods for estimating this (Table 4).

Even when it is valid, a prognostic study only provides an estimate of the true risk. The next step is to appraise the precision of this estimate by means of the confidence interval (CI), which is a measure of the uncertainty due to random variability. The greater the precision, the more reliable the results. In most survival curves, earlier follow-up periods tend to include results from a larger number of patients compared to later periods (decreasing number of subjects available for follow-up, as patients are not included in the study at the same time); this means that survival curves are usually more precise in earlier follow-up periods, with narrower CIs at the left hand side of the curve.

One of the aspects that doctors, patients and families are most interested in is which factors can affect the prognosis. Once a diagnosis has been established, multiple factors related to the patient’s previous health status, disease, treatment or socioeconomic circumstances can have an impact on the final outcome. Clinicians, in particular, are most interested in determining whether they can intervene to modify these factors (through changes in diet, lifestyle, etc.). Prognostic factors are usually reported with the corresponding relative risk (RR) in cohort studies or odds ratio (OR) in case-control studies. The RR and the OR represent the change in the probability of the unfavorable outcome. Negative values of the RR or OR indicate that the factor has a protective effect.

First and foremost, clinicians need to assess the degree of similarity between their own patients and the subjects included in the study and whether the differences are large enough to doubt whether they can apply the results to make adequate prognoses for their patients.

Prognostic data can be very helpful in guiding therapeutic decision-making. In addition, valid, precise and generalizable results supporting a uniformly good prognosis for a disease or condition can be very useful in reassuring affected patients. On the other hand, prognostic results supporting a uniformly poor prognosis provide clinicians a starting place for discussion with the patient and family and planning care. The results of a prognostic study are applicable to the extent that the study subjects are similar to the local patients and that they are clinically significant.

While RCTs provide the highest-quality evidence of a cause-effect relationship, they can rarely be used to evaluate potential harmful exposures due to the ethical issues it would entail (however, if a clinical trial observes an association between an exposure and an adverse effect, the result is very trustworthy).

Cohort studies are often used as an alternative, in which case it is very important that the comparability of the cohorts has been evaluated to ensure that the subjects exposed to the putative harmful factor are similar to the unexposed subjects. If differences are detected in any variables with a known association with the outcome of interest, a stratified or multivariate analysis should be performed to control for potential confounding due to their influence; but it is important to keep in mind that, these adjustments notwithstanding, there could always be confounding due to an unknown or uncontrolled factor, so the results of a cohort study should always be interpreted with caution.

When the harmful effect is very rare or takes a long time to occur, case-control studies may be used, but they are more susceptible to bias and confounding, so their results should be interpreted with even greater caution. Harmful effects can also be reported in isolated case reports or case series, but evidence from these reports is very weak. As is the case of other aspects of health care, the best evidence is obtained through the systematic review of all pertinent studies.

In case-control studies, the ascertainment of previous exposure is the key issue; readers should look for any strategies used to minimize recall bias or biases related to the researchers. In contrast, in cohort studies and RCTs, readers must pay attention to the measures taken to ensure that the methods and circumstances of exposure ascertainment were similar and the risk of exposure the same in both groups.

In RCTs and cohort studies, the follow-up should be long enough to allow for the development and detection of the harmful effect.

• b)

  Secondary criteria:

It is unlikely that all the criteria for causation are met. The more that are met, the greater the confidence with which a causal relationship can be inferred. Clinicians should determine whether the association between the exposure and the harmful effect meets at least some of the tests for reasonable inference of causation, which are outlined in Table 5.

Interventions other than the experimental intervention under study (cointerventions), when implemented differentially in treatment and control groups, can affect the comparability of the groups and pose an even greater risk of bias when the study design is not double-blind or when the use of additional treatments that can affect the toxicity of the intervention under study is allowed.

The main measure that indicates that a therapeutic intervention causes an adverse outcome is the strength of the association between the implementation of the intervention and the occurrence of a harmful effect. Different methods are used in different types of study to estimate the strength of the association. The most commonly used measure of association is the RR, which indicates the extent to which the adverse outcome is more likely to develop in the exposed group compared to the unexposed group. In case-control studies, since the incidence cannot be calculated because subjects are selected after the outcome has already occurred, the appropriate measure of association is the OR. Another measure that can be used in cohort studies and RCTs is the difference in incidence or risk difference (RD) (an absolute measure of the excess risk of the exposed compared to the unexposed attributable to the exposure). Its inverse is a measure that indicates the number of people who need to be exposed for harm or an adverse outcome to occur, known as the number needed to harm (NNH). Another measure, similar to the relative risk reduction (RRR), is the attributable fraction in the exposed (AFe) (proportion of the events observed in the exposed group that can be attributed to the exposure). Fig. 2 details the significance and magnitude of these measures.

Figure 2.

Measures of association used in studies on harmful effects and their magnitude.

In most cases, interpreting the OR as the RR does not affect the qualitative interpretation of the results, although it may lead to a degree of overestimation of the magnitude of the effect.

BR, baseline risk (frequency of the outcome of interest in the reference population); OR, odds ratio (odds of exposure [probability of being exposed vs not exposed] in case group relative to the odds in the control group); RR, relative risk (incidence in the exposed group divided by incidence in the unexposed group); SR, systematic review.

The CI offers a measure of the precision with which the population parameter has been estimated based on the simple point estimate obtained from a sample of patients. Confidence intervals can be calculated for most statistical estimates or comparisons (OR, RR, RRR, ARR, NNT, NNH). Although readers can calculate CIs from the results presented in the study, it is preferable that the authors provide them in the report. If the CI contains the null value (1 in the case of the RR and the OR), the estimate of risk may be disregarded.

The first key issue is assessing whether our patients differ from studied patients in terms of morbidity, age, race, previous interventions or exposures or other important characteristics, in order to determine whether the study results can be extrapolated to our clinical practice.

The next step is determining the patient’s individual risk of experiencing the harmful effect. A simple way to do this is to express it as a fraction of the risk of the study subjects, assigning the patient a subjective value based on the patient’s clinical condition.

Once all the information has been obtained, clinicians must decide on the appropriate course of action based on the strength of the inference, the magnitude of the risk if exposure continues, the availability of alternatives, and, obviously, the patient's values and preferences, combined with the expertise of the provider that is making the decision.