2013 Jamaica: Evaluation of the Caribbean Child Research Conference
Author: Paulette Griffiths
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report."
This evaluation was undertaken by Caribbean Child Research Conference (CCRC) Steering Committee to measure the planned and unplanned outcomes of the annual Caribbean Child Research Conference. It is intended that the results will inform the committee’s work plan to ensure long-term effectiveness of the conference. The main output of the CCRC, namely the conference, has been executed effectively since 2006. This evaluation sought to assess the extent to which the short to medium term objectives of the CCRC have been realized. Consequently, an assessment of the results achieved to date was carried out in order to determine optimal strategies for continuation of the conference and providing direction regarding the efficiency of implementation, as well as for ensuring its continued relevance and effectiveness. The conference objectives (outlined below) and implementation modalities were examined in light of their relevance, efficiency, effectiveness and sustainability.
1.Promote a culture of research among students in the High Schools in the Caribbean by facilitating the presentation by students of their research and awarding a prize to an Outstanding Child Researcher;
2.Promote a culture of research among teachers in the High Schools in the Caribbean.
3.Recognize the work of child researchers in child-related research;
4.Disseminate findings on child-related research;
5.Strengthen the networking links among researchers of child–related matters
6.Inform the policy formulation and implementation process in the Caribbean Region, through the published research findings;
7.Set up a website with information on all the annual conferences and highlights of the conference.
This outcome-based evaluation seeks to assess both the planned and unplanned outcomes of the annual Caribbean Child Research Conference. In particular the extent to which the adult research and the conference contributed to the policy dialogue and policy decisions in Jamaica; including the factors that aided or hindered this process.
An assessment of the results achieved to date will guide the Caribbean Child Research Conference (CCRC) Steering Committee in determining optimal strategies for continuation of the conference and providing direction regarding the efficiency of implementation, as well as for ensuring its continued relevance, sustainability and effectiveness.
Evaluation Scope and Objectives
The conduct of this evaluation is framed within the scope of the entire project from inception to completion. It takes into account the extent to which planned outcomes have been achieved so that recommended strategies can be made for sustainability of the project. The evaluation also takes into account the conference’s relevance, effectiveness, efficiency, sustainability, and recommendations.
The objectives of the evaluation are to:
1.Document the achievements and lessons learned in the execution of the Caribbean Child Research Conference and analyze the extent to which intended results were achieved.
2.Review and assess the relevance, effectiveness, efficiency and sustainability of the conference in light of its intended outcomes.
3.Recommend changes in strategy and process that should be incorporated to affect the desired long- term outcomes of the CCRC.
4.Assess the feasibility of expanding the conference to a more regional approach engaging multiple Caribbean countries in its planning and implementation.
5.Determine a possible role for the conference in the establishment of knowledge centres on children across the Caribbean and the development of a Regional Child Observatory.
A mixed method approach was employed in this evaluation using a number of data collection and analysis methods. Primary data was collected through focus groups, interviews, and a survey of participants in the October 2012 Conference. Secondary data came from a review of previous conference publications, evaluation reports and administrative documentation.
Findings and Conclusions:
The relevance of CCRC remains very high. The status of children continues to be on the forefront of intense daily discussions on the rights of children and more so on the violations of these rights. The CCRC is therefore both timely and highly relevant. It enhances local and international conventions and the prevailing needs of the targeted population in providing a forum for children and adult researchers to share empirical research for discussing and advocating solutions to child related issues.
Over the seven year period under review, the CCRC experienced a mixed level of efficiency. There was a high level of consistency in planning the conference, mixed level of efficiency in its implementation and a low level of efficiency in translating outputs into outcomes.
Overall the effectiveness of the conference cannot be comprehensively measured as the monitoring mechanism in place was not aligned to the set objectives of the conference. The commendations made from the yearly evaluations enables assumption that learning took place and many participants were extremely fulfilled by the day’s proceedings. However, the extent to which this fulfillment can be attributed to teachers creating child friendly classrooms or stakeholders work being enhanced can only be speculative.
The sustainability of the CCRC is intricately linked with the extent to which ownership of the overall objectives are achieved, the continued institutional support given to the project; the available institutional capacity to manage the CCRC; its continued relevance to participating stakeholders; available financial support; as well as the extent to which technology allows for innovative ways to implement the project with consistent overall governance.
There is need for a formal mechanism to include the wider Caribbean in the CCRC. The conference partners must decide whether the conference has the potential to impact regional policies, how this impact can be made and whether the conference should remain a Caribbean undertaking.
Develop effective monitoring indicators to sufficiently capture information related to CCRC objectives. In moving forward conference partners must review the overall aim and objectives of the CCRC to ensure achievable outcomes in a structured accountable and timely manner.
Institutional Capacity for Managing the CCRC
Create a Knowledge Centre for Dissemination of Data and Information at the CCRC workshops. The possibilities for mining and generating the rich data from the CCRC in a knowledge centre are endless and so too are the possibilities for impacting advocacy on child related issues.
Further Inclusion of the Ministry of Education
In moving forward it will be critical to institutionalize critical aspects of teacher and student participation in the CCRC within the Ministry of Education.
The output required should dictate the format in which the conference is organized. As the CCRC operates now the format does not lend itself to gaining required tangible results and informing government policy.
Dissemination of Research
The extent to which all conference outputs can benefit stakeholders will be dependent on the ability to receive the research information in a timely manner. Greater effort should be directed at improving dissemination particularly with the use of technology, if the CCRC is to impact policy and significantly contribute to compendium of knowledge on the situation of children in the region.
It is recommended that the logic model (as exemplified on pages 17 /49) be refined with input from all stakeholders to guide future implementation of the CCRC.
Participation: The CCRC objectives did not specifically outline child participation among its objectives, and this may be regarded as a major unintended results achieved. It is well recognized that this forum has provided an arena for children to express their views as well as receive and impart information among themselves and relevant policy-makers, in compliance with the Convention on the Rights of the Child.
Monitoring & Evaluation: A key lesson learned in the execution of the CCRC is the absence of a detailed implementation plan including a monitoring framework. Although conference objectives were set, these were not ‘SMART’ . Consequently the data collection carried out in annual assessments of the conferences did not fully reflect and measure the attainment of these objectives.
Sustainability: In the implementation of the project from year to year the secretariat has managed to secure additional funding support to supplement the budget. This has resulted in a reduction in UNICEF support over the period. The fact that the conference attendance was still high among the researchers in 2012 despite the introduction of a registration fee augurs well for both the relevance and sustainability of the conference. Never the less additional efforts at fund raising will have to be made to mitigate against the high risk of reduced donor funding.
In light of Jamaica’s current middle income status and the general reduction in the availability of grant funding, project managers and oversight committees should investigate strategies to compensate or incorporate measures in planning and implementation that would mitigate against this risk.
Full report in PDF
PDF files require Acrobat Reader.