Paediatricians play a key role in the start and duration of breastfeeding. Although it is known that they lack formal education on this subject, there are currently no validated tools available to assess paediatrician knowledge regarding breastfeeding.
ObjectiveTo develop and validate a Breastfeeding Knowledge and Skills Questionnaire for Paediatricians.
Material and methodsOnce the knowledge areas were defined, a representative sample of them was chosen to carry out the survey. After pilot testing, non-discriminating questions were removed. Content validity was assessed by 14 breastfeeding experts, yielding 22 scorable items (maximum score: 26 points). To estimate criterion validity, it was hypothesised that a group of paediatricians with a special interest in breastfeeding (1) would obtain better results than paediatricians from a hospital without a maternity ward (2), and the latter would obtain a higher score than paediatric residents receiving training in the same hospital (3). The questionnaire was also evaluated before and after a basic course in breastfeeding.
ResultsBreastfeeding experts have an index of agreement of >0.90 for each item. The 3 groups (n=82) were compared, finding significant differences between group (1) and the rest. Moreover, an improvement was observed in the participants who attended the breastfeeding course (n=31), especially among those with less initial knowledge. Reliability, internal consistency (KR-20=0.87), interobserver agreement, and temporal stability were examined, with satisfactory results.
ConclusionsA practical and self-administered tool is presented to assess paediatrician knowledge regarding breastfeeding, with a documented validity and reliability.
Los pediatras desempeñan un papel fundamental en el establecimiento y mantenimiento de la lactancia. Aun conociendo su carencia de formación en este campo, no se dispone actualmente de herramientas validadas para documentarla.
ObjetivoCrear y validar un cuestionario sobre conocimientos y habilidades en lactancia para pediatras.
Material y métodosSe definen las áreas de conocimiento, escogiéndose una muestra representativa de estas. Se realiza una prueba piloto y se eliminan las preguntas no discriminantes. La validez de contenido se establece por 14 expertos en lactancia, obtienéndose 22 ítems puntuables (26 puntos máximos). Para aproximarse a la validez de criterio, se hipotetizó que un grupo de pediatras con especial interés en lactancia (1) obtendría mejores resultados que pediatras (2) de un hospital sin maternidad y estos que los residentes (3) de pediatría del mismo hospital. También se analizaron las notas antes y después de un curso básico en lactancia.
ResultadosLos jueces tienen un índice de acuerdos >0,90 para cada ítem. Se comparan los 3 grupos (n=82), hallando diferencias entre el grupo con interés en lactancia y el resto; y se observa mejoría de la puntuación en los alumnos del curso (n=31), especialmente en aquellos que partían de peores conocimientos. Respecto de la fiabilidad se examinan la consistencia interna (Kuder-Richardson 20=0,87), concordancia interobservadores y estabilidad temporal, con resultados satisfactorios.
ConclusionesSe ofrece una herramienta para evaluar el conocimiento de los pediatras en lactancia, de la que se documentan validez y fiabilidad, adaptada a nuestro contexto, breve y autoadministrable en papel u online.
The WHO and the Spanish Paediatric Association (Asociación Española de Pediatría, AEP) recommend exclusive breastfeeding for 6 months before introduce complementary foods, continuing breastfeeding for at least 2 years. However, few mothers follow these recommendations.1 Low breastfeeding rates are a public health issue, with overwhelming evidence in the literature of the risks of artificial feeding,2,3 in addition to the economical burden for the community implied by early weaning.2,4 Paediatricians play an essential role in the initiation, maintenance and duration rates of breastfeeding.5,6 However, it is widely acknowledged that their training is deficient.7–11 There are various questionnaires on this topic,11–13 but, as far as we know, there are no validated tools to assess the breastfeeding knowledge of professionals. Our objective is to develop and validate a questionnaire about breastfeeding knowledge and skills for paediatricians. Quantifying knowledge should aid towards planning and justifying steps to improve paediatric training.
Material and methodsDefinition of construct and questionnaire developmentObjective: to quantify breastfeeding knowledge and skills.
Target population: paediatricians and resident paediatricians.
Concepts to be measured: basic breastfeeding knowledge, together with skills in daily handling of breastfeeding issues. Some knowledge areas are proposed (Table 1) based on earlier suggestions.11,14,15 After defining these, it was decided that not all of them would be represented, since to be applicable the test duration could not exceed 15–20min.7,12,16
Table 1.Basic areas of breastfeeding knowledge and skills.
Area Breakdown Physiology Breastfeeding in normal conditions Prematurity Extended breastfeeding Public health Official recommendations Risks of artificial feeding Issues during the first days Hypoglycaemia Cracks and pain while breastfeeding Mastitis Dehydration Jaundice Late issues Inadequate production of milk/poor weight gain Crying Breastfeeding crisis Return to work. Supplementary feeding Assessment of feeding Maternal factors Drugs, illnesses, toxic substances, prior surgery Attitudes Promotion of breastfeeding. Empathy and counselling Item selection: a preliminary prototype was designed and tested in 54 questionnaires (before and after a breastfeeding course), after which 3 non-discriminating questions (correct by more than 95% of the sample) were removed. Following this, a second prototype was proposed, tested on 10 paediatric residents, and examined by a panel of judges, after which the definitive questionnaire was generated.
Composition: the questionnaire, entitled “Questionnaire about breastfeeding knowledge” (Encuesta sobre conocimientos en lactancia, ECoLa), comprises 22 questions with a maximum score of 26 points (anexo 1). There are also questions – from 23 onwards – that are not scored, although they reflect variables which may have an influence on the study objective. Items 18 and 21 deserve special mention since they allow free text, limited to very few words:
Item 18 asks for 5 aspects of breastfeeding (one point for every correct answer). Those shown in the WHO breastfeeding observation form were deemed correct.17 This list was expanded by the judges (Annexe 2). An answer is correct if it is included in the checklist, although at the discretion of the evaluator an answer not included in the list may be deemed correct. This should only occur exceptionally and be consistent with current breastfeeding knowledge.
Item 21 assesses the ability to obtain quick and reliable information about drugs and breastfeeding. Adalimumab was chosen, which, while being safe,18–20 is listed in its data sheet as incompatible, even up to 5 months after the last injection.21 E-lactancia19 and LactMed18 were defined as valid resources,22 or any other website with direct links to either of the above. A correct response, accepting a minimum transcription error, is defined as one which upon being entered in Google shows either of the 2 correct websites among the first five hits. Other sources such as PubMed are not deemed useful because of the time needed to perform searches in these databases.23 Google is unsuitable due to the huge number of pages it retrieves, not all of which contain pertinent information.
Convenience sampling was use, with subjects selected from two groups: (a) paediatricians and resident paediatricians at the Hospital Infantil Niño Jesús (HNJ), a high complexity hospital without International Humanitarian Assistance Network (IHAN) accreditation, where there is no maternity ward and residents need to rotate to another hospital during their 2nd or 3rd year of residency. Paper questionnaires were delivered following a general session, and then sent via email (2 emails with a 15-day interval) to (a) 49 paediatricians and 48 residents, to include those who were not present during the session; and (b) paediatricians in the virtual forum LactanciaMat, an email distribution list for RedIris users, comprising 365 professionals with special interest in breastfeeding, specifically polling paediatricians (unknown number). Forty students from a basic AEP course for paediatricians and paediatric residents were also questioned about breastfeeding knowledge (May 2014). The questionnaire was delivered on paper, before and after the course.
Validation requirementsValidityThis is the extent to which the instrument measures what it is supposed to measure, and is therefore fit for purpose.24 Validity considers various aspects:
- 1.
Content validity refers to whether the questionnaire is an indicator of what is intended to be measured.24,25 To examine content validity, it was submitted for examination by 14 judges+3 consultant breastfeeding-expert paediatricians, and members of the Breastfeeding Committee of the AEP (anexo 3). The group of experts was enlarged during this work, adding 5 new members who joined the Committee during this period.26
Judges were asked to assess 3 aspects of each item: the correct response; the importance of the item, rated on a scale of 1–5; and whether a conflict might exist about the correct answer. In addition, all judges were encouraged to voice their opinions about each item, competence areas, and the questionnaire as a whole.
To quantify their responses, the agreement index was used (number of agreements/total number of responses). Values of >0.80 were deemed valid.27 Aiken's V coefficient was not used since non-ordinal variables were included.28
Items 18 and 22 are mentioned separately:
Item 18 asks for 10 criteria about breastfeeding assessment, included in the checklist. Respondents were asked whether any of the elements might be open to dispute, if there were other relevant criteria other than those considered, and the degree of subjectivity involved in amending the question. They were asked to grade their answers on a Likert scale.
Item 22: since this is about the self-perception of confidence while observing a breastfeed, it is not considered correct but rather desirable.
- 2.
Criterion validity is about the extent to which one particular answer estimates the value of a gold standard which we assumed is exactly what we want to measure.24,25 Since there is no such standard, an approach was made by examining the starting hypotheses: (a) knowledge should be greater among LactanciaMat specialists, followed by hospital specialists, and less extensive among residents; and (b) students should improve their knowledge after completing the course, especially those with lower initial knowledge.
This is the degree of accurate, error free measurement by an instrument. It is a reflection of its capability to repeatedly offer true and consistent results under similar circumstances.24
- 1.
Internal consistency is the degree to which items are related to each other, and may therefore be accumulated in a global score. Internal consistency was examined with the Kuder–Richardson 20 (KR20) formula,29,30 since all items were codified as dichotomous (correct/incorrect).
- 2.
Inter-observer concordance was assessed with the Kappa coefficient in items 18 and 21, independently classified by both authors as correct or incorrect. Blank responses were excluded, since they were automatically coded by the spreadsheet.
- 3.
Temporal stability (test–retest) was assessed with 10 subjects with a difference of 2–3 weeks, calculating the intraclass correlation coefficient (ICC).
Since open questions may be difficult to interpret and score, a short version of the test that excludes these items may be used. Therefore, the validity and reliability of the questionnaire without items 18 and 21 will be evaluated (over a total of 20 points).
Data analysisData was processed and analysed with the free software Libre Office and R with the interface RKWard (Available for free download at: http://www.libreoffice.org/yhttp://rkward.sourceforge.net/). Scoring was classified as grades for easier interpretation: <55% very insufficient, 55–70% insufficient, 70–85% good (10), >85% excellent. Confidence intervals (CI) were used for group comparisons, instead of hypotheses contrast.31
Ethical considerationsNo ethical conflicts were identified, in line with prior works of similar nature.32,33 Questionnaires were completed voluntarily and anonymously, and the project was endorsed by the Breastfeeding Committee of the AEP.
ResultsTwenty three paper questionnaires and 25 online questionnaires were completed at the hospital; 6 questionnaires were rejected since they did not belong to the study group, with an approximate response rate of 46% among residents and 41% among attendant doctors (the exact rate cannot be calculated since the overlap between face to face and email subjects is unknown.) Sixty questionnaires were received via email from LactanciaMat, 20 of which were rejected since they were not from paediatricians. Seventy six paper questionnaires were completed before and after the breastfeeding course, of which 62 were paired questionnaires completed by 31 individuals.
Validity- 1.
Content validity. The summary of the judges’ opinion is shown in Table 2. The items that did not pass the test were openly discussed until an agreement was reached to redraft or removed them entirely. Items 5 and 7 were added at the suggestion of the judges, and they were likewise subject to discussion until a consensus was reached. A third item was added later at their suggestion, and was later removed as non-discriminating. With regard to items 18 and 21:
Item 18 (from 1 –lowest– to 5 –highest–): potentially controversial criteria: average 1.6; possibility of existence of other clinically relevant criteria besides those considered: average 2.3; and subjectivity risk upon assessing this question: average 1.6.
Item 21: although there was wide agreement on the correct response (13/14 recommended e-lactancia), many believed it was not the only alternative, and 6 also suggested LactMed.18 After specifying the assessment criteria, a unanimous decision was reached to leave the question unchanged.
Table 2.Results of examination by judges.
No. Summarised questiona Relevancy (1–5) Agreement about correct option (n=14) I.A. about correct optionb Agreement about absence of conflict I.A. about lack of conflictc Resulting action Final I.A. about correct option Final I.A. about lack of conflict – Screening for neonatal hypoglycaemia 4.3 11 0.79 8 0.57 Modifiede Not applicable Not applicable – More reliable signs to appreciate intake 4.4 13 0.93 11 0.79 Modifiede Not applicable Not applicable – The main cause of hypernatremic dehydration in a newborn is the delay start of lactation 4.4 11 0.79 7 0.50 Deleted Not applicable Not applicable 1 Mastitis 4.8 14 1 14 1 Unchanged 1 1 2 Feeding as needed 4.6 14 1 14 1 Unchanged 1 1 3 Risks of formula 4.8 14 1 14 1 Unchanged 1 1 4 Need for bottle with caesarean-section 4.9 14 1 14 1 Unchanged 1 1 6 Phototherapy and breastfeeding 4.6 14 1 14 1 Unchanged 1 1 8 Choice of breastfeeding with large prematures 4.8 14 1 9 0.64 Modified 1 1 9 “Calm” infant at risk 4.4 12 0.86 10 0.71 Modified 1 0.93 10 Extended breastfeeding 4.4 14 1 12 0.86 Modified 1 1 11 Poor weight gain 4.9 13 0.93 10 0.71 Modified 1 1 12 Breastfeeding and supplementary feeding 4.6 14 1 12 0.86 Modified 1 1 13 Watery milk 4.2 13 0.93 13 0.93 Unchanged 0.93 0.93 14 Poor attachment 4.9 13 0.93 12 0.86 Unchanged 0.93 0.86 15 Recommendations 4.9 14 1 13 0.93 Modified 1 1 16 Cracks 4.9 14 1 12 0.86 Unchanged 1 0.86 17 Case of growth spurts 4.6 12 0.86 9 0.64 Modified 1 1 19 Pregnant woman hesitant about bottle feeding 4.4 10 0.71 6 0.43 Modified 1 0.93 20 ICMBSd 4.9 13 0.93 12 0.86 Modified 1 1 Questions 5, 7, 18, 21 and 22 are discussed in the text.
- 2.
Criterion validity. Group scores are shown in Fig. 1. The 95% CI of the LactanciaMat median does not overlap with the others, but there is overlapping among hospital specialists and residents.
Figure 1.Group scoring. Box and whisker plot. For each group, a box represents the interquartile range (IQR), the centre line represents the median, and the notches the 95% confidence interval of the median. The upper and lower whiskers reach respectively the maximum or minimum value at a distance of 1.5*IQR from the box. Extreme values are represented individually. The thick horizontal line at 70% represents sufficient.
(0.07MB).
Regarding the percentage of correct answers from course participants, both before and after the course, an improvement in scoring is shown in Fig. 2, with less dispersion and no overlapping in the CI of medians. Fig. 3 compares the increased score of each student relative to their initial score.
Scores before and after breastfeeding course. Box and whisker plot. For each group, a box represents the interquartile range (IQR), the centre line represents the median, and the notches, 90% confidence interval of the median. The upper and lower whiskers reach respectively the maximum and minimum values. The thick horizontal line at 70% represents sufficient.
- 1.
Internal consistency. KR20=0.87.
- 2.
Inter-observer concordance:
Item 18: n=627, kappa 0.82 (CI 95%: 0.75–0.87), maximum kappa 0.98, prevalence index 0.66 and bias index 0.01.
Item 21: n=144, kappa 0.44 (CI 95%: 0.10–0.78), maximum kappa 0.60, prevalence index 0.91, bias index 0.03, prevalence and bias adjusted kappa 0.90. Given the difficult interpretation of kappa in this case, a contingency table is provided (Table 3).34
Table 3.Observations of the two evaluators in item 21.
Evaluator 2 Correct Incorrect Evaluator 1 Correct 134 (135) 1 (0) 135 Incorrect 6 (5) 3 (4) 9 140 4 144 The theoretical maximums throwing a maximum kappa of 0.60 are between brackets. Note the high frequency of correct responses (prevalence index 0.91).
- 3.
Temporal stability (test–retest). Global scoring: ICC=0.82 (CI 95%: 0.44–0.95), and grouped as grades: ICC=0.70 (CI 95%: 0.20–0.91).
Fig. 4 shows scores grouped as grades from the short version, together with those from the full version.
- 1.
Criterion validity. The median of the LactanciaMat group (Me=95 [95% CI: 92.5–97.5]) did not overlap with other groups, which did overlap each other (specialists Me=75 [95% CI: 70.6–79.4], residents Me=65 [95% CI: 56.6–73.4]).
- 2.
Internal consistency. KR20=0.76.
- 3.
Temporal stability. ICC=0.76 (95% CI: 0.35–0.94) and grouped as grades ICC=0.72 (0.24–0.92).
The poor breastfeeding training of many paediatricians has been documented both internationally9,10,12 as well as in Spain,11 and also in relation to specialists from other fields.7,35 Despite their poor training, however, doctors may have confidence in their skills.16
However, we have no record of any validated questionnaire documenting this lack of training. A questionnaire in English is claimed to be validated,36 but its validation is insufficiently documented.12 ECoLa is, to our knowledge, the first questionnaire that has complied fully with the validation process, is reasonably brief, and can be self-administered both on paper and online.
The requirement that questionnaires must be analysed by a panel of judges has been of great benefit in this case. Not only has it required agreement about the correct response,27 but also about potential conflict, thereby placing high demand for each item. Amendments enabled us to reach almost unanimous agreement about the correct answer (>0.90) with a minimum amount of conflict (only 2 items are between 0.85 and 0.90, the others exceeding 0.90 of agreement).
We offer a tool that quantifies the breastfeeding knowledge and skills of paediatricians. Even though it does not encompass the entire range of basic required knowledge, it is a good starting point for further tests. Likewise, although its objective is to assess basic knowledge and skills, it clearly reveals the attitudes of the respondent: question 21 assesses knowledge of documentation sources, but also deals with unconsulted weaning if a drug is compatible with breastfeeding; or the question (scoreless) about whether it is appropriate to breastfeed in public. Question 19, about advertising aimed at pregnant women, shows whether most respondents in the study group blame the mothers or underestimate the importance of breastfeeding. However, the questionnaire is not designed to assess attitudes, and the statistical model confirms the internal consistency of the questionnaire, with KR20=0.87, higher than the recommended 0.7025 or 0.85,30 and lower than 0.90–0.95 which would suggest redundancy.25,29
The objective of this work has been to validate the questionnaire, not to document the state of knowledge of our sample. Therefore, we cannot draw conclusions about the rate of response, since the equation denominators have not been globally determined.
The hypothesis that a group with interest in breastfeeding has better scores is confirmed. However, no differences were shown between hospital paediatricians and residents. Heterogeneity between residents is notable, and has led to a wide median and interquartile range confidence interval, in which not even a minimum as low as 19% or a maximum as high as 85% are considered extremes. Attending doctors have a low median (69%) similar to that of residents, although there is less dispersion since there are fewer extremely low scores, with a score of 58% being the lowest end of the whisker; therefore lower values are deemed eccentric (Fig. 1).
Another approach to criterion validity, in the absence of a gold standard for benchmarking, is the second hypothesis, which is confirmed in its two aspects: students improve their scores after the course, especially those who start with little knowledge. The more basic knowledge the student has, the less improvement is observed in the questionnaire score which is consistent with a basic knowledge course and a test assessing basic knowledge.
The reduced sample size is a limitation: statistical analysis was satisfactory, although the kappa and the ICC have a wide CI, including values that might be unacceptable. ICC results may not only be due to the small sample size, but also to the nature of the knowledge questionnaire. Encouraged by the test itself, subjects may seek the answers to certain questions after completing the first questionnaire, thus acquiring new knowledge before completing the second test,30 and reducing the test–retest value.
With regard to the inter-observer test, a good kappa index has been obtained in item 18. In item 21, even if we consider the degree of agreement as high, the kappa has a low absolute value (0.44), affected by a high prevalence index (0.91), in what has been called one of the kappa paradoxes. Additional information has been supplied to facilitate interpretation of this phenomenon, in which the maximum kappa (0.60), prevalence-adjusted and bias-adjusted kappa (0.90) and the contingency table (Table 3) should be noted, as recommended in the literature.34 This scenario is foreseeable, since it is an open question which usually has few incorrect answers: if the respondent knows the answer he will answer correctly, and he will not answer if he does not. It should be noted that blank responses have been automatically tabulated, excluding them from the analysis and penalising the kappa. This question raises the issue that the test must be manually corrected and requires Internet connection. Internet is not available in some offices, and we consider this to be a limitation for the professional providing care to breastfeeding women.
Since validity is not an inherent feature of the test, rather it is related to the sample to which it is administered, it would be advisable to re-examine the psychometric properties of the questionnaire in other samples.37
A reduced version is provided, which presents the advantage that it may be corrected by untrained personnel, or automatically with a spreadsheet. Even though it does not seem advisable to us, it could be used in particular cases; although in our sample all the subgroups improved their scores, especially those whose scores were closer to the “pass” cut off point (Fig. 4). Thus, in our sample of hospital paediatricians, only 45% pass with the full version, and 5% are excellent; however, with the reduced version 85% pass and 35% are excellent. This version overrates those with a good theoretical level, but who are unfamiliar with essential practical aspects, such as the observation of a breastfeed. To use it, it would be convenient to review the cut off points suggested for the full version.
We believe it is important to document the lack of breastfeeding training among paediatricians, to justify the implementation of measures to counteract this deficit, both at the official level and locally.36,38 Therefore, we have developed and validated this questionnaire, which may be the basis for versions in other languages or for different medical specialities.
Conflict of interestThe authors state there is no conflict of interest.
We would like to thank the members of the breastfeeding committee of the AEP, and in particular: Dr Josefa Aguayo Maldonado, Dr Susana Ares Segura, Dr Marta Costa Romero, Dr Marta Díaz Gómez, Dr Blanca Espínola Docío, Dr Beatriz Flores Antón, Dr Ana Gimeno Navarro, Dr Adolfo Gómez Papí, Dr María Teresa Hernández Aguilar, Dr Ana Jiménez Moya, Dr Paula Lalaguna Mallada, Dr Juan José Lasarte Velillas, Dr Jesús Martín-Calama Valero, Dr José Manuel Martín Morales, Dr Carmen Rosa Pallás Alonso, Dr José María Paricio Talayero and Dr Laura Sanfeliciano Martín.
The following are the supplementary data to this article:
Please cite this article as: Gómez Fernández-Vegue M, Menéndez Orenga M. Validación de un cuestionario sobre conocimientos y habilidades en lactancia materna. An Pediatr (Barc). 2015;83:387–396.
