Journal of Health & Medical Economics

Keywords

Health survey; Household health; Health economics; Developing country

Introduction

Household surveys: Overview

Surveys are a useful tool for data analysis in several areas like economics, health, sociology, innovation, marketing and trade. They are specially relevant in developing countries where the availability of registration systems and data records is limited. They have experienced a relevant progress during the last decade of the twentieth century due to the innovations in computation which allow among other things, to carry on the laptops into the field and to record and edit data as they are provided by the respondents.

In particular, household health surveys are nowadays the usual source of information for the social and health research studies in developing regions. Composition of the household, hygienic conditions, health, education and habits are the main contents of the questionnaires. All this information contributes noticeably to socioeconomic analysis and this is one of the reasons why household surveys are essential tools in policy planning.

Household Sample Surveys in developing and transition countries should help to explain not only what is happening but also why it is happening. Yet, these are often the most important questions because they seek to understand the impact of current policies or programs, and perhaps even hypothetical future policies or programs, on the circumstances and characteristics of households. Economists and other social scientists do not always agree on how to answer these questions, and sometimes they may not even agree that it is possible to answer a particular question. Cross-section data obtained through surveys are adequate for descriptive statistics but less so for quantitative analysis to explore the causal relationships between the socio-demographic features and health indicators identified throughout the interviews. Then, there are limitations to estimate econometric models about health indicators like child mortality, or child health. If causal relationships are important to the survey designers, very thorough design is essential. In this sense, some guidelines and reports have been published.

For example, The Handbook of Household Surveys [1] is a guideline that offers an overview of issues related with the design and implementation of household surveys in general.

More specifically, Grosh et al. [2], provide very detailed information on the design of household surveys for developing countries and the United Nations Document Household Sample Surveys in Developing and Transition Countries [3] contains information on several important aspects of conducting household surveys in developing and transition countries, like sample design, survey implementation, non-sampling errors, survey costs, and analysis of survey data. The following section focuses the attention in the Demographic and Health Surveys (DHS), one of the most relevant collections of household surveys in developing countries.

Demographic and Health Surveys: Limitations and remedies

DHS have been conducted in more than 85 countries since 1984. Standard DHS questionnaires have large sample sizes (usually between 5,000 and 30,000 households) and are conducted about every 5 years, to allow comparisons over time. National surveys conducted in the context of the Demographic and Health Surveys (DHS) program (USAID) constitute a source of information on family health indicators. Target topics are child health, family planning, infant and child mortality, malaria, maternal health and maternal mortality. Comparability is identified as one of the main advantages of the DHS survey data together with free availability for researchers [4,5]. From 1984 until now special questions and indicators have been introduced in the questionnaires, but, still some challenges and limitations remain [6].

Next we point out the most common limitations found by researchers related to these surveys and some possible solutions for them. The exposition is organized as "demand side" and "supply side" information limitations.

General limitations

Omission of relevant variables is the most common limitation in econometric analysis. Also econometric analysis with DHS data bases face this potential limitation that could be solved collecting additional data or merging data with other sources like the census and other studies. Another common problems of econometric models specification is endogeneity. This additional complication will require the use of instrumental variables to correct for it and will require more explanatory variables to include in the data.

For example, Shultz poses that endogeneity between health status and health care demand is a limitation in the estimation of econometric models to identify the determinants of child health. Then, the instrumental variables required will be independent of the biological endowments of the children, but will account for some of the variation in health input demands. Assuming the economic expectation that regional prices and health programs influence health input demands, but do influence the biological endowments of the children, he uses prices and the existence of health programs as instruments to estimate the determinants of child health [7].

Demand side limitations

The following limitations have been classified as "demand side" limitations, given that they are related to the population object of study.

First of all, omission of births and deaths is the most serious problem generating incomplete samples in household health surveys. In addition, some populations of interest cannot be covered in a survey of households, like street children and nomads.

Omission of births and deaths is a technical and cultural aspect that requires investment in education and registration systems.

Otherwise, recall bias could be solved with prospective studies using panel data, asking other family members about the child, or repeating questions in different ways to check the consistency of the answers.

Regarding recall biases, birth histories collected in the surveys like birth dates of the children are past events that could be biased due to confusion of the respondent; this kind of missclassification bias is another frequent problem affecting diseases and symptoms records. As long as the errors of measurement are fairly random, the aggregate estimates of indicators will be sufficiently adequate, but individual-level data will have to be interpreted more carefully, especially when making causal interpretations.

Surveys also have limitations to identify illnesses whose identification requires many questions. Nutrition patterns, malaria, tuberculosis and AIDS are risk factors and illnesses for which no complete set of questions is available, and require objective tests to be confirmed. Determination of causes of death through verbal autopsy gives a rough idea of the importance of selected causes of death, but is not precise enough for evaluation of the impact of health interventions or assessment of trends in cause-specific mortality. The inclusion of the results of tests are a partial solution of this problem because this increases the price of the field works and limits the sampling to those cases where the tests are available, but it gives more reliable information.

Other common limitations are related to the income of the household in relative terms; determination of economic status is limited to a short list of durable goods, and specific questions are not always answered. Market prices, salaries, household budget, household belongings, and other concrete indicators would identify the real wealth of the families.

All of the described "demand side" issues are affected by changes over time that is not usually included in the questionnaires. For example, source of drinking water in the household is described in the moment of the interview, so it is adequate for the children born in this period, but it could be the case that source of drinking water in the same household was worse or better some years ago, so children who were born some years before the interview could have grown up in different conditions that the current conditions described in the survey. Then, the demand side features -including household features- existing during the survey are applied to the whole birth history.

The inclusion of specific questions for children born in different periods would reduce this problem. Nevertheless, panel data is the adequate to deal with heterogeneity issues because time effects are captured along the panel.

Another inherent problem in these questionnaires is the unobserved heterogeneity. The unobserved biological endowment of the mother is one of them. Panel data would avoid this problem because, as well as time effects, fixed effects of children from the same mother are also captured using the panel structure.

Obviously, panel data are costly in time and money; even unfeasible in developing countries due to deficient registration systems. Then, when Government agencies and organizations want to know whether the living conditions of households are improving or deteriorating they use data from two or more surveys that are separated by a considerable length of time to answer this type of questions, with the data of interest being collected consistently in each survey. As explained in Deaton [8], even slightly different ways of collecting information can result in data that are not comparable and thus potentially misleading.

Supply side limitations

We consider supply limitations of questionnaires to basically those issues related with health care delivery. DHS do not always report availability, utilization and frequency of utilization of health services and health care delivery¹ as well as health services accessibility. In addition, no data on quality of care are collected. In this sense, disaggregation to district level is desirable, since a district is the usual unit of implementation of health programs and health care services are easier to study if complementary information is available.

Information of basic markets, especially those destined to feeding needs, is common supply side information that is not explicit in the household questionnaires and could be obtained from other sources.

What has improved?

Since the 80´s Demographic and Health Surveys questionnaires have evolved gradually, maintaining the core objectives and questions. The systematic review of Short et al. [9] shows that studies on certain topics have increased since more modules and questions have been added to the core questions. However, this systematic review does not analyze improvements in technical issues related with changes in the questionnaires that could have solved some of the limitations pointed out above enhancing the econometric analysis. To study this issue, questionnaires from different periods and countries have been reviewed and comparisons between 80´s surveys with the most recent questionnaires of 2013 have been done for different countries². Findings are summarized in Table 1, where limitations (first column), proposed solutions (second column) and the application of these proposed solutions "improvements" (third column)³ are showed.

Table 1 Findings.

Regarding limitations on demand side features, the questionnaires have been improved by including results of objective tests for some diseases like malaria and including indirect approximations to households´ income. Nevertheless, specific information of household features for every child has not experienced any change, so the declared situation during the survey is still applied to all the maternity history.

With respect to limitations in supply side features, frequency of utilization of health services is included in the current questionnaires, although accessibility, availability and quality are not well identified through the questions. Neither are well identified those issues related with coverage of government policies and programs as well as changes over time of supply side features.

Conclusions

Surveys in developing countries are useful tools for health planning due to the scarcity and limited reliability of records and registration systems. DHS Surveys are a referent in this field. From the 80´s until now, DHS questionnaires have improved their contents, enhancing not only the descriptive analysis, but the causality relationships analysis between socioeconomic and biological conditions and health, that is, allowing econometric models estimated with these data bases.

After comparing questionnaires of the 80´s with last decade questionnaires, we have pointed out that some relevant limitations remain, especially those related with time changing conditions. Further research should focus on papers where econometric models and analysis have been done based on DHS, in order to review how different authors have dealt with the remaining limitations.

¹The report of visits is usually focused on pregnancy and postnatal care and specific diseases targets of the study.

²The DHS Program, USAID. https://www.dhsprogram.com/ Accessed: 15 October 2014.

³Note that the solution of some of these issues does not depend of the questionnaire design but of how is the survey is conducted.

References

1. Handbook of Household Surveys (1984) Department of International Economic and Social Affairs. Statistical Office, Studies in Methods Series F, United Nations, New York.
2. Grosh M, Glewwe P(2000) Designing Household Survey Questionnaires for Developing Countries. Lessons from 15 years of the Living Standards Measurement Study, The World Bank, Washington.
3. Household Sample Surveys in Developing and Transition Countries (2005)Department of Economic and Social Affairs. Statistics Division Series,United Nations, New York.
4. Corsi DJ, Neuman M, Finlay JE, Subramanian SV (2012) Demographic and health surveys: a profile. International Journal of Epidemiology 1-12.
5. Vaessen M (1997) The potential of the demographic and health surveys (DHS) for the evaluation and monitoring of maternal and child health indicators. Demographic Evaluation of Health Porgrammes. Khlat M (eds.) Paris, CICRED 65-74.
6. Boermaa JT, Sommerfield AE (1993)Demographic and health surveys (DHS): contributions and limitations. World Health Statistics Quarterly 46:222-226.
7. Schultz P(1984) Studying the Impact of Household Economic and Community Variables on Child. Population and Development Review 10: 215-235.
8. Deaton A (1997) The Analysis of Household Surveys. A Microeconometric Approach to Development Policy. The World Bank, The Johns Hopkins University Press, Baltimore and London.
9. Short M, YoonJoung C, Birda S (2012) A systematic review of Demographic and Health Surveys: data availability and utilization for research. Bulletin of the World Health Organization 90:604-612.