Original Articles

Indian Pediatrics 1999; 36: 881-885

Magnitude of recall bias in the estimation of immunization coverage and its determinants

R. Ramakrishnan, T. Venkata Rao, L. Sundaramoorthy and Vasna Joshua

From the ICMR Institute for Research in Medical Statistics, Mayor Ramanathan Road, Chennai 600 031, India.
Reprint requests: Director, Institute for Research in Medical Statistics, Mayor Ramanathan Road, Chetput, Chennai 600 031, India.
Manuscript received: December 28, 1998; Initial review completed: March 5, 1999; Revision accepted: March 18, 1999.

Objective: To study the magnitude of recall bias in the estimation of immunization coverage and to identify its determinants. Design: A follow-up study of cohort of children for one year; followed by a cross-sectional recall survey. Subjects: All live births in two contiguous PHC areas in Villupuram district, Tamil Nadu. Methods: The prospective data collected from mothers of 774 children was considered as `Gold Standard' and the retrospective recall data from them was compared with it. This was carried out for individual immunization schedules, namely, DPT, OPV, BCG and Measles and for the combined immunization status. Measures such as sensitivity, specificity, positive and negative predictive values were computed. Logistic Regression technique was employed for evaluating the determinants of agreement. Results: Less than 50% of mothers had immunization cards with them and more than 70% of the cards did not have complete information. The sensitivity of the recall method was 41.3% and the specificity was 79.5% when the complete immunization status was considered. For individual immunization schedules, the sensitivity ranged from 95% for BCG to 53% for Measles vaccination and the specificity ranged from 30% for BCG to 68% for Measles vaccination. Mothers' age emerged out as a significant determinant in the agreement of two methods. Maintenance of immunization cards were very poor. Conclusions: Method of obtaining immunization status through recall survey is not sensitive. Proper maintenance of immunization cards and ensuring the availability of them with mothers for inspection are recommended for obtaining accurate estimation of vaccine coverages.

Key words: Immunization coverage, Recall bias.

With the emergence and re-emergence of host of infectious diseases including resistant organisms, prophylactic measures to prevent them are important. Immunization against certain major diseases has been a thrust area for public health administrators. Estimation of immunization coverage has been routinely carried out to monitor the immunization programme. Presently, the WHO 30 cluster survey method(1) has been widely used to obtain rapid estimates of immunization coverage.Information is obtained from mothers of children aged 12-23 months regarding the immunization status of their children. The method of obtaining the information was through oral interrogation and wherever available, verification of them from immunization cards provided by the health agencies(2). In rural areas most mothers do not possess these cards and even in mothers who preserve it, the information may be incomplete and inaccurate. Under these circumstances, the estimation of immunization coverages are based mainly on the mother's abi-lity to recall the immunization schedules received for her child, the length of recall period being a few months to more than one year. The ability to recall vaccination status has been investigated using vaccination card in a developing country(3). There is a paucity of similar data from our setting. We therefore evaluated the magnitude of recall bias in the estimation of immunization coverage and its determinants.

Subjects and Methods

The data for the present study is from a community trial conducted to study the impact of vitamin A supplementation on Acute Respiratory Infection (ARI) and diarrheal diseases among infants. The trial was conducted in two rural contiguous Primary Health Center (PHC) areas in Villupuram district, Tamil Nadu, covering a population of about 53,000 from 11,484 households in 51 villages(4). Of the 909 children recruited for the trial, 774 had complete information for the entire period of infancy. The analyses are based on this sample.

Trained field investigators enrolled the newly delivered mothers and made periodic home visits to elicit information on morbidity conditions of infants during infancy. These home-visits were undertaken once a fortnight in the early stages of the study (first nine months, for operational reasons), but intensified to once a week subsequently (about a year and a half). Certain additional information on the feeding habits of the infants and immunization details were also obtained during those visits. This prospective data on immunization would give the `true' immunization status of the infants. The mothers were contacted within seven days after the completion of one year of their children by an investigator, who did not know the `true' immunization status of the children and details of immunization particulars were collected retrospectively by oral interrogation. Availability of immunization card and the information in it were also obtained.

Definitions

The following definitions were used:

A child was considered to be immunized if all the three doses of DPT, OPV were given apart from BCG and Measles vaccinations with the following time schedules: (i) BCG: Admi-nistered any time before the child completes one year; (ii) DPT/POLIO: First dose given at any time after six weeks of birth and subsequent two doses with an interval of at least four weeks between successive doses and all the three doses administered before the child had completed one year of age; and (iii) Measles: Administered after the completion of nine months but before the child had completed one year of age.

Statistical Analyses

The vaccination status evaluated by the prospective survey was considered as `Gold Standard' and the results of the retrospective survey were compared. The measures used for the comparison are sensitivity, specificity, efficiency, positive and negative predictive values.

A logistic regression model was employed to find out the determinants of agreement of the two methods of obtaining immunization status. In this model the dependent variable was the agreement in the reporting of immunization status and it takes the value `1' if the status in the retrospective method agrees with the status obtained by prospective method and `0' otherwise. Various socio-demographic variables were used as predictor variables in the model.

Results

The classification of immunization status of children of 774 mothers, by the prospective data (`gold standard') and retrospective data (recall method) is set out in Table I. The recall method was found to be 41% sensitive and 80% specific as compared to the prospective data. The positive predictive value of the test was as low as 18%. It is to be noted that 22% of the infants were reported as fully immunized by the mothers in response to retrospective interrogation, while it is actually less than 10%. The agreement between the methods was also assessed using McNemar's chi-square test. There was significant difference between the two methods (p <0.0001).

The agreement between the prospective and the retrospective methods was assessed for individual vaccination and the results are given in Table II. For the BCG vaccination, the sensitivity was high (95%) but the specificity was only 30%. In the case of DPT and Polio vaccination, the sensitivity was around 60% and specificity was around 50% while for Measles vaccine, the sensitivity was 53% and specificity was 68%.

The coverage obtained for BCG vaccination through prospective data collection was 68% while it was 87% by retrospective recall method. In the case of DPT, Polio and Measles vaccinations the corresponding coverages were 45%, 52% and 37% by prospective method and 55%, 55% and 40% by the retrospective method.

The results of the logistic regression to identify the determinants of agreement are set out in Table III. Agreement by retrospective method with prospective method was 1.7 times higher in younger mothers (aged £ 25 yrs) as compared to older mothers (p = 0.03). None of the other factors considered in the analysis, namely, father's age, sex of the child, place of dwelling, parity, mother's education, family size, previous sibling status and mother's occupation were significant determinants of agreement by the two methods.

Table I^__ Comparison Between the Prospective and Retrospective Methods for Obtaining Overall Immunization Status.

Sensitivity = 41.3%; Specificity = 79.5%; Efficiency = 75.8%; Positive Predictive Value = 17.8%; Negative Predictive Value = 92.7%; McNemar's x² = 51.37; p <0.0001.

Table II^__ Comparison Between the Prospective and Retrospective Method for Obtaining Vaccine-Specific Immunization Status

Table III^__ Influence of Socio-Demographic Variables on the Agreement of Two Methods of Obtaining Immunization Status.

* Significant at 5% level.

Discussion

Maternal recall error of child vaccination status is an important issue and has been addressed by many investigators(3,5). Vaccination cards were used as the accurate count of vaccination and the maternal recall of vaccination was compared to it in a study in Costa Rica(3). Costa Rica's successful vaccination card policy made feasible the use of these cards for comparison with the recall data. However, in our study area, the immunization cards did not provide much information as most of the mothers did not have them and even the mothers who were possessing the card, the information in them were incomplete and inaccurate. To cite an example, for some children the date of immunization preceded the date of birth. The use of prospective data collected at weekly/fortnightly interval throughout the period of infancy is a very reliable source of information regarding the accuracy of vaccination status and this information is used as a `Gold Standard' for the comparison of recall data. In the Costa Rica study, recall error was measured as the difference between the number of remembered vaccinations and the number of recorded vaccinations. They have also computed the Spearman's correlation between the number of doses from the vaccination cards and from maternal recall. The methods of analysis employed in the present investigation give more information about the nature and extent of disagreement between the prospective and the restrospective data in terms of sensitivity, specificity and predictive values. Also, none of the socio-demographic factors, except the mother's age were determinants of agreement between the two methods. In the case of identifying significant determinants of agreement between two methods, it is to be noted that the maternal education was not a significant determinant. Also, against the general belief of gender bias, sex of the child does not seem to influence the mother to remember about the immunization status of the child.

The retrospective data collection immediately after the completion of one year did not agree well with the prospective data. These mothers were highly sensitized to provide information as they were repeatedly and frequently visited for one year and interrogated. In contrast, in the usual immunization coverage surveys, the recall period is longer (mother's of children aged 12 to 23 months are considered) and the respondents are visited only once. Hence, in these situations the expected agreement would be still lower than the present study. In our study, more than 50% of the mothers were not having the immunization cards which were supposed to be given by the health workers. The available cards were also incomplete and inaccurate. Efforts must be taken to ensure the issuing of cards to the mothers with correct and full information in them.

Mothers should be suitably motivated to preserve these cards at least for a period of two years. Availability of these cards with the mothers at the time of survey would help to sort out the problem of recall bias.

Acknowledgement

The authors are grateful to Dr. S.K. Sureshkumar and all field staff for their assistance in data collection. Paul A. Tamby, A. Elangovan and their team are thankfully acknowledged for their support in data processing. Our sincere thanks to Dr. M.D. Gupte, Director, IRMS for his valuable suggestions and encouragement.

References

1. Henderson RH, Sundaresan T. Cluster sampling to assess immunization coverage: A review of experience with a simplified sampling method. Bull WHO 1982; 60: 253-260.

2. Murthy BN, Radhakrishna S, Nair NGK, Ezhil R, Venkatasubramanian S. Estimation of immunization coverages in children by WHO 30-cluster survey. Indian J Med Res 1993, 97(A): 234-238.

3. Joseph Valadez J, Leisa Weld H. Maternal recall error of child vaccination status in a developing nation. Am J Public Health 1992; 82: 120- 122.

4. Venkata Rao T, Ramakrishnan R, Nair NGK, Radhakrishna S, Sundaramoorthy L, Mohammad Koya PK, et al. Effect of vitamin A supplementation to mother and infant on morbidity in infancy. Indian Pediatr 1996; 33: 279-286.

5. Le CT. Parental knowledge of their own immunization of poliomyelitis. JAMA 1985, 254: 608-609.