An operational framework for Machine Learning in evaluation
This document offers practical guidance on how to apply Machine Learning (ML) analytical approaches in evaluations.
About
This document offers practical guidance on how to apply Machine Learning (ML) analytical approaches in evaluations. It presents an operational framework that was developed after a thorough literature review and UNICEF Evaluation Office’s (EO) experiments with application of ML methods. Additionally, it discusses potential use cases of these methods, along with associated risks and limitations.
Summary
Machine Learning (ML) analytical approaches have significantly advanced our capacity to process large amounts of data, allowing for the systematic analysis of both quantitative and qualitative data in a timely and cost-effective manner. Over the past decade,
these methodologies have found widespread utilization in development research, including in evaluations. In this paper, we present an operational framework for applying ML approaches in evaluation studies. The framework is based on a literature review and
UNICEF Evaluation Office’s (EO) experience with using ML in a few recent global evaluations. The framework identifies three broad categories of data types, and corresponding ML methods that can be used to answer evaluative questions. This report offers practical guidance to evaluation managers who are interested in applying these innovative approaches to their projects.
The report was drafted by Nabamallika Dehingia, with inputs from Eduard Bonet Porqueras, Uyen Huynh, Miguel Almanzar, Francesco Iacoella, and Zlata Bruckauf.
