2016 Nigeria: Evaluation of the Maternal, Newborn and Child Health Week in Nigeria
Author: Liverpool School of Tropical Medicine Centre for Maternal and Newborn Health
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.
Over the past decades, Nigeria has achieved good progress in improving maternal and child health outcomes, although insufficient to achieve its Millennium Development Goals targets. According to recent estimates, the under-five mortality rate has declined from 213 per 1,000 live births in 19903 to 128 per 1.000 live births in 20134; the MDG 4 target of reducing under-five mortality to 71 per 1,000 live births by 2015 remains far from reach. Trends in neonatal mortality also show good progress: the neonatal mortality rate has reduced from 52 per 1.000 live births to 37 per 1.000 live births during the period 1990-2013.
Nigeria has also successfully reduced the maternal mortality ratio (MMR): in Nigeria, the MMR was estimated at 1,200 per 100,000 live births in 19905 and at 576 per 100.000 live births in 2013. The MMR has more than halved during the period, although Demographic and Health Survey (DHS) data indicate that no progress has been achieved during the period 2008-2013, and further investments are still needed to achieve the MDG 5 target for MMR, set at 300 per 100,000 live births.
This slow progress in MDG 4 and 5 was recognised by the Government of Nigeria at the 53rd National Council on Health in March 2010, and therefore the Maternal and Child Health Week (MNCHW) was introduced-amongst other measures, as a priority and strategic action to accelerate the reduction of child mortality and improvement of maternal health. This intervention was primarily intended to improve access to essential, quality MNCH services, consistently with the objectives of the 2007 Integrated MNCH Strategy. The MNCHW is organised to deliver an integrated package of highly cost-effective MNCH services/interventions. These services are primarily delivered to strengthen the routine PHC services.
The objectives of the impact evaluation are as follows:
- Assess the extent to which MNCH weeks has been adapted to meet the needs of the targeted clientele, and partners.
- Assess the extent to which the MNCH weeks have been implemented as intended
- Assess whether the intended outcomes were achieved, and whether there were any unintended outcomes
- Provide a descriptive cost analysis for the intervention
- Identify lessons learned, exploring what has worked well, what has not worked as well and make recommendations to strengthen the MNCH Weeks
An evaluability report was produced following a scoping visit (Important background information about the programme was collected), this report paved the way the way for a full independent evaluation based on a theory based approach in the absence of a baseline and challenges using an experimental evaluation design.
The inception phase resulted in detailed evaluation plan, revised evaluation questions (EQs), identification of sources of data, determination of resources required to complete the evaluation, clarity of roles and responsibilities. During the inception phase LSTM ensured that ESC had a shared understanding of how the evaluation will proceed.
The Theory based approach used contribution analysis for the evaluation. The main evaluation question used to guide the impact evaluation was “Has the MNCHW contributed to improve the health status of women and children in Nigeria, by increasing coverage of key maternal, newborn and child health interventions?” additional evaluation questions linked to the revised Theory of Change were formulated.
A mixed data collection approach was used, primary data was collected from a household survey, key informant interviews and focus group discussions. Secondary data was analysed from relevant reports, data sets and publications. A household survey involving 5, 389 households in 320 clusters with 5, 139 children under 24 months of age, 2, 531 children between 24 and 59 months, and 5, 180 women of reproductive age. The survey was conducted in 12 states and FCT Abuja.
Secondary data was reviewed, planning of the 2nd round of the 2015 MNCHW at National level and FCT were observed and a case study of one state was under taken.
Findings and Conclusions:
The evaluation concludes that despite its lack of impact on MNCH (no significant contribution to coverage or improved health outcomes), the programme has potential of significantly increasing coverage of key MNCH interventions through efficient social mobilisation that creates awareness and participation. This can only be possible through effective partnership, adequate and timely release of funds and complete commitment by state government.
The objectives and related outcomes of the MNCHW are consistent with the priorities of Nigeria, and still relevant.
- The design of the MNCHW activities is partially consistent with its intended effects and impacts
- No evidence is found that the MNCHW has significantly contributed to coverage of essential MNCHW interventions in Nigeria.
- MNCHW achieves a significant population reach, at least for selected interventions
- MNCHW is not implemented consistently across States and over time
- Attendance to MNCHW is suboptimal
- The current model of the MNCHW is not fit to reach the most marginalized.
- The quality, inclusiveness and timeliness of the coordination and planning functions needs substantial improvement
- Equipment, supplies and medical items are inconsistently available across States and across different MNCHW rounds
- The effectiveness of the current training and deployment model is unclear
- Social mobilization does not reach targets groups sufficiently
- The MNCHWs are perceived as a one stop shop for valuable health care for women and children, as well as promoting the use of routine health care services.
- The M&E framework of the MNCHW presents design issues
- There are strong monitoring tools in place, but their actual use for real time analysis and decision making could be improved
- Reporting and documentation are inadequate and information is not accessible
The recommendations at the end of the evaluation were derived from the evidence generated, discussed in the findings and conclusion chapters of this report. Consultations with NPHCDA, UNICEF and MNCHW ESC have contributed to the development of the recommendations. The stakeholders have recommended presentation of this report to National Council on Health and the National Council of State. Some recommendations include;
- NPHCDA should develop an accountability framework in collaboration with partners and state governments at central and state levels to monitor, input, activities and outputs. This is also to improve coordination.
- NPHCDA should constitute a national steering committee with clear ToR developed in collaboration with UNICEF and other partners.
- SPHCDA/SMOH should constitute MNCHW coordination committees with clear ToR and KPIs, Committee at State and local government levels.
- NPHCDA in collaboration with the states and partners, should redesign social mobilization strategy for MNCHW. This is ensure rebranding of MNCHW to improve accountability and awareness
- NPHCDA in collaboration with implementing partners should consider central funding mechanism (matching funds from partners) through a single funder manager.
- Improved transparency about funding by all partners, timely reports, reconciliation available as open access documents
- NPHCDA should ― Improve quality of training, special team to monitor the quality of this, set standards to achieve this. in addition, consider reducing frequency of training and savings can be used to improve social mobilization
The evidence suggests that MNCH weeks provides increased awareness for PHC services however it is not efficiently run. Implementation is not as per guidelines, poor planning, weak monitoring and lack of a clear exit plan are key problems with the current implementation approach.
Political commitment and funding: Based on data available for this evaluation from NPHCDA, increasingly most of the funding for the MNCHWs comes from non-governmental sources such as UNICEF. Delays in funding or lack of funding put the objectives of the MNCHWs at risk.
Implementation not as intended: Successful implementation depends on good planning and the MNCHW guidelines has provided directions for planning, coordination, training, monitoring and review of the MNCHWs. Observation of the planning for November 2015 showed that there is poor adherence to the guidelines.
Disaggregated coverage data: The intervention coverage data would also have a more detailed picture if it were stratified by age groups and other demographic parameters, such as economic and social parameters. This is not currently the case from the data we have for the previous MNCHWs. The use of RapidSMS for data collation limits the amount of details that can be shared, this platform can be updated to improve its robustness and functionality.
Reduced number of interventions and implementation in all PHCs: The evaluation found a unique approach to implementation of MNCHWs from the Kaduna state case study. A smaller number of interventions are implemented based on state coverage numbers and rather than implementing in a few PHCs, implementation is in all PHCs. This strategy is to ensure equity in access by the target population, preventing the need for added transportation cost to the family, in order to participate in the MNCHWs.
More lessons in the report.
Full report in PDF
PDF files require Acrobat Reader.