Researching Innovation Labs 4
Measuring the Impact of the lab
As the humanitarian innovation movement grows, there is increasing impetus to understand how the outcomes of innovation practice may be measured and proven to work. Measuring social change and thus the impact of humanitarian intervention is neither straight-forward nor clearly defined.
A number of innovation labs are aware of the need to better measure what they are doing from an early stage. Given the flexible and failure-friendly nature of innovation lab work, monitoring impact against traditional criteria often used in bureaucratic organisations can pose problems and has the potential to stifle innovation. In a sense, innovation spaces are being forced to innovate, using trial and error, in the way they measure impact and learn from their work.
In its first two years, UNHCR Innovation has already been successful in obtaining buy-in within the organisation and is now overwhelmed with requests for input into projects across the agency. Due to the increased demand, UNCHR Innovation is considering its approaches and formalising its mechanisms for selecting which projects to work on. The internal demand is an indication that the innovation unit is impacting the agency as a whole and providing a unique space for staff to engage in the development of ideas. Since its inception in 2012 the innovation team has documented activity, using it to reflect, and adapt what they do. However, UNHCR Innovation is now seeking to develop more strategic and structured ways of measuring its work, and the team will need to create new metrics and measures of impact to reflect the unique nature of their work.
The authors of Labcraft reflect that four different levels of impact measurement would be of use to social innovation labs at large: “1. impact at the level of the lab itself, 2. the spin-off labs that we generate, 3. the innovations and innovators we cultivate and support, and 4. an emerging new narrative.” In other words, it is important to question what would have happened if the lab did not exist, what the impacts of the lab are, especially on end-users, and whether there is an emerging new narrative whereby the lab is influencing the wider societal landscape.
Some projects that have been developed through UNICEF Innovations Lab Kosovo’s social entrepreneurship incubator have gone on to secure further grant funding and scale up. The impact in these cases is measurable in terms of the amount of financial support obtained and the extent to which the initiatives are scaled-up. However, if success is measured by the degree of scaling up, then how should that scale be measured? The answer will depend on the specific project and context. Even if scale can be measured, the ultimate impact of projects on society is often very difficult to measure. For example, the Innovations Lab Kosovo is faced with the challenge of accurately measuring an increase in confidence amongst young people who have been involved with the Lab’s Youth Advocacy Platform. The Innovations Lab Kosovo draws on a mix of quantitative and qualitative (for example ethnographic) methods of measurement, recognising that both are necessary to give a more complete picture of progress. Importantly, the Lab very deliberately prioritises constant reflection and measurement – projects are being analysed, criticised and modified at all stages and by all members of the Lab, rather than waiting until the end of a project or strategic agenda period to realise that aspects could have been improved.
It is important to note that impact will take a different form for the direct and indirect innovation imperatives (See blog #3). Tools used for measuring how far organisational change has been achieved will not look the same as those used to measure how far a community’s own innovation has been supported and achieved change at the grassroots. Remaining accountable to the community is something that will be key in innovation work. Allowing communities to define the measures of impact is something that has been advocated in humanitarian evaluation, so labs might be able to help bridge the relationship gap and work hand-in-hand with communities in re-defining what humanitarian impact really means.
To see the full account of what we found, including case studies of a few labs, read our full paper here.
The final blog in this series will unpack the recommendations that arise from our research for how labs may continue to advance humanitarian and development work.
About the Authors:
Louise Bloom is a Research Officer at the Humanitarian Innovation Project based at the Refugee Studies Centre at the University of Oxford with a focus on understanding innovation in humanitarianism.
@loubloom and @HiprojectOx
Romy Faulkner is a Research Assistant at the Oxford Humanitarian Innovation Project, has just completed a Masters of Law at Harvard Law School and is also working on corporate engagement at the International Committee of the Red Cross in Geneva.
@RomyFaulkner
