2013 Thailand: Evaluation of the UNICEF Child Protection Monitoring and Response System (CPMRS) in Thailand
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 4’ of the report.
The CPMRS project, launched by UNICEF in 2006 in association with key government and university partners, was intended to develop an integrated child protection system in the six tsunami-affected provinces in southern Thailand. The project was expected to: increase public awareness of children’s rights to protection; track the magnitude of child protection issues at local, provincial and national levels in Thailand; and provide adequate response mechanisms for identifying children in need of special protection and for delivering suitable child protection services as early as possible.
UNICEF Thailand, in association with the Royal Thai Government (RTG), contracted the Universalia Management Group Limited (hereafter “Universalia”) and its associate Child Frontiers to conduct an evaluation of the CPMRS and to provide an overview of the current child protection system in Thailand. Thammasat University School of Social Work was contracted to support the field data collection and analysis.
The objectives were:
1. To assess the actual and potential contribution of CPMRS to the national child protection system;
2. To determine the extent to which CPMRS has met its objectives;
3. To determine the relevance, efficiency and sustainability of the CPMRS as an approach to strengthen the child protection system, as well as the extent to which it has incorporated gender, human rights-based and equity-focused approaches; and
4. To provide recommendations for the refinement and potential scaling up of the CPMRS approach to the national level.
The evaluation is intended to provide input to RTG and UNICEF decision making on how to strategically advance the national child protection agenda and strengthen current systems towards a comprehensive and more holistic national child protection system.
The intended primary users of the evaluation are the Ministry of Social Development and Human Security (MSDHS) and other relevant Ministries of the RTG, such as the Ministry of Public Health (MoPH), Ministry of the Interior (MoI) and the Ministry of Justice (MoJ), as well as the UNICEF Thailand Country Office (TCO).
The assignment was carried out in conformity with international evaluation standards including OECD DAC, UNEG and UNICEF principles and guidelines. The evaluation was based on the Terms of Reference, the Inception Report, and an evaluation matrix that identified the main questions, sub-questions, sources of data, and data collection methods. The evaluation of the project’s effectiveness was based on a reconstructed results framework as the project did not have one.
The evaluation used a mix of data collection methods including a desk review of documents; key informant interviews with 87 stakeholders at the national, provincial and local levels; field visits to four provinces in Thailand (Suphanburi, Ranong, Ubon Ratchathani, and Trang) for interviews and group discussions with community members, leaders and service providers; and a questionnaire administered in those provinces to CPMRS frontline workers from a range of agencies (such as TAO, OSCC, PSDHS, Education Services Area Office, police and Children’s Reception Home staff). Careful attention was paid to ensure the capacity of national researchers to facilitate the discussions in a sensitive and safe way, and to respond appropriately to potential disclosure of child protection cases. The evaluation team did not conduct group discussions with children. Given the resources available, the evidence generated was largely illustrative, providing an overview of the functioning of the CPMRS within the overall national system, rather than a representative national sample.
Findings and Conclusions:
Overall, CPMRS objectives were very relevant given global conventions such as the CRC, the spirit of child protection legislation in Thailand, the needs of children in Thailand, the latest thinking in child protection at the time of project development, UNICEF’s systems-approach to child protection, as well as the strategic objectives of the UNICEF Thailand Country Programme in 2007-11. They were also relevant given recent promising actions by OPP and MoI to highlight child protection on the national agenda.
It is not likely that CPMRS results will be sustained without the investment of considerable additional effort and resources.
Due to a variety of challenging contextual factors and some unrealistic assumptions in the project design, the overall objective of CPMRS was not realised during the course of the project. On the basis of information collected during the evaluation, there is as yet no established “model” system in place in Thailand that integrates the child protection monitoring and response systems. The CPMRS project model was not introduced or implemented as intended and the overall objectives or results as defined in the CPD 2007-11 were not realised.
While CPMRS was intended to generate an integrated model child protection and response system, this did not occur. CPMRS was implemented and managed by project component without a sustained focus on the project’s overall purpose and objective. This effectively prevented the emergence of an integrated child protection monitoring and response system model.
CPMRS design and some aspects of its implementation took gender and equity issues into account. Notable shortcomings include insufficient involvement of children and youths (and attention to them in project planning), and modest attention to gender issues in reviewed training materials.
Recommendations to UNICEF
Recommendation 1: In designing projects in the future, UNICEF Thailand should incorporate a process in the design phase to develop and articulate a common understanding of the project with national partners, and should establish criteria for assessing whether there is sufficient government ownership of the project objectives to justify start-up and continued investment over time.
Recommendation 2: In light of the needs for child protection support at multiple levels in Thailand, UNICEF Thailand should consider the need for both upstream and downstream programming approaches.
Recommendation 3: In the future, UNICEF Thailand should adhere to UNICEF pilot project guidelines in designing and implementing projects.
Recommendation 4: In designing and implementing projects in the future, UNICEF Thailand should ensure that UNICEF’s results-based planning and reporting guidelines are respected.
Recommendation 5: In designing projects in the future, UNICEF Thailand should pay considerably more attention to monitoring the sustainability of project outputs and particularly outcomes.
Recommendations to RTG
Recommendation 6: MSDHS, on behalf of the RTG, should initiate a process to develop a national vision for the development of a child protection system as well as a strategic plan for its implementation.
Recommendation 7: In developing the national policy and system for child protection in Thailand, the RTG should clearly identify the roles and responsibilities of the agency with primary responsibility and accountability for the protection of children in Thailand, as well as those of partner agencies at national, provincial and TAO levels. It should also establish needed inter-ministerial coordination mechanisms.
Throughout the data collection process, the evaluation team compiled lessons learned, both operational and developmental, emerging from UNICEF’s support of CPMRS. These can serve to further develop the child protection system in Thailand, and to assess the pertinence of developing similar CPMR systems elsewhere.
Government involvement in and meaningful input to project strategies and implementation is critical, particularly for initiatives designed to serve as a government child protection system.
The likelihood of sustainability of project results increases when the project is anchored in a clear national policy framework and when there is national commitment to and ownership of the project objectives.
A project design that clearly identifies expected results and includes a plan to monitor them over time provides a guide for stakeholders to know where they are going and if they are getting there.
A project that is intended to test or demonstrate the value of a new approach should be designed, managed and monitored as such (i.e., like a pilot test). It is important to maintain focus on the overall purpose and expected learning of the pilot, rather than on the details of implementation.
Full report in PDF
PDF files require Acrobat Reader.