Accelerating evaluation synthesis analysis: A case study
A case-study on data extraction from 631 UNICEF evaluation reports shows that evaluation synthesis analysis can be accelerated through text mining, searching, and highlighting
About
Through this working paper, we present how we analyzed UNICEF’s evaluation reports to assess their impact and identify areas for improvement. Given the large volume of evaluations conducted by UNICEF each year, a fully manual review would have been too time-consuming. To address this, we adopted a semi-automated approach to accelerate the analysis while maintaining quality and consistency.
Using computational tools, we processed 631 evaluation reports aligned with 64 outcomes from UNICEF’s 2022–2025 Strategic Plan. Text was converted into machine-readable format, AI was applied to extract key sentences, and advanced filtering software helped isolate the most relevant insights.
This method reduced the text volume by 92% while maintaining high accuracy in identifying pertinent outcomes. The semi-automated approach enabled a faster, more efficient analysis without compromising the rigour or reliability of the results.
Overall, automation significantly streamlined the review process, allowing us to extract meaningful insights at scale while upholding scientific integrity.
Click here to access the working paper.
The complete Global Evaluation Evidence Synthesis report is available at the following link.
AUTHOR(S): Lena Schmidt, Pauline Addis, Erica Mattellone, Hannah O’Keefe, Kamilla Nabiyeva, Uyen Kim Huynh, Nabamallika Dehingia, Dawn Craig, Fiona Campbell
