By Suz Redfearn
Suz Redfearn is a freelance writer specializing in health issues.
When money gets allocated for a health initiative in a developing country, the pressure builds quickly to implement the initiative immediately. And yet rushing in and setting up intervention programs too fast can jeopardize the ability to learn, in the end, whether the project really helped those who needed the help.
Thus, in recent years, the tide has turned toward first setting up systems to evaluate the effect of a program as it unfolds.
“To truly know whether a project is working, you need to know what was happening in the beginning with a baseline in both the program and nonprogram areas, in the middle with a midline evaluation, and then at the end, tracking the changes as you go,” said Sian Curtis, project director of Monitoring and Evaluation to Assess and Use Results (MEASURE), a U.S. Agency for International Development (USAID) program run by the Carolina Population Center at the University of North Carolina at Chapel Hill.
MEASURE runs more than 120 projects in more than 20 developing countries to strengthen health monitoring, evaluation and information systems.
MEASURE faces major obstacles: Many poor countries lack not only the capability to collect health statistical data, but also systems that register births and deaths accurately. One of MEASURE’s ongoing efforts is the Measurement, Learning and Evaluation (MLE) Project, which focuses on the Urban Reproductive Health Initiative aimed at improving the health of the urban poor in India, Kenya, Nigeria and Senegal. MLE helps the initiative gather data that allow it to truly see which of its efforts is working and which is not.
DO BEDNETS WORK?
Dismissing measurement and evaluation of a health program can lead to waste of funds and a failure to provide care in areas where it is needed, said Emmanuela Gakidou, director of education and training at the Institute for Health Metrics and Evaluation (IHME) at the University of Washington.
As an example, she points to malaria in Zambia. Child mortality in that country has plummeted in the last decade, while the distribution of bednets has skyrocketed. (Bednets keep malaria-carrying mosquitoes away from people as they sleep.) Many observers directly correlate the two. But because many other health interventions were undertaken at the same time that might have contributed to Zambia’s drop in infant mortality, the IHME believes that linking the two directly may not be appropriate.
“It’s an easy leap to make, but it’s not scientifically accurate,” Gakidou said. As no baseline measurements were taken prior to massive bednet distribution or efforts to spray houses with pesticides, the IHME is combing through census and other data. It tries to piece together baseline numbers retroactively, as well as data on what effect each intervention had in the hope that a clearer picture will emerge about exactly which efforts have improved child mortality numbers. The undertaking is called the Malaria Control Policy Assessment Project, and is soon to wrap up in Zambia but is just getting under way in Uganda, where a similar situation exists.
METRICS MATTER
Thankfully, Gakidou said, gathering baseline data before launching health initiatives and midline data later is gaining acceptance as a necessary element of health programs. So evaluating them in the future is likely to become easier. President Obama included a focus on health program measurement and evaluation in his 2010 policy directive on development; such evaluation is also listed as part of the 2009 Global Health Initiative’s core principles.
In encapsulating the role of careful evaluation and accurate metrics in global health efforts, Curtis said, “It is also about trying to promote a culture to use information to produce better health outcomes.”
