2009 Jordan: Evaluation of the Better Parenting Program
Author: Dr. Suha Al-Hassan (The Hashemite University)
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
The BPP involves a network of 13 national partners who aspire to provide comprehensive care for young children and has so far been implemented in more than 200 centers nation-wide. More than 130,000 parents and caregivers acquired knowledge about proper child care through this program. The Program was evaluated in 2000 by an external evaluator. The evaluation commended the achievements of the Program, in particular the level of coordination between the different parties and the low cost of reaching caregivers and their children, which amounted to only 3 US Dollars per child. The Program was found to be a very suitable nucleus for a more comprehensive early childhood development (ECD) approach to childcare. An important recommendation was the need to expand the Program scope to a more holistic Early Childhood approach, including protection of children from abuse and neglect. These lessons learned were taken into consideration in the design of the ECD program, which started in 2003.
A national training team was formed and equipped with needed skills and knowledge to expand the ECD training throughout the Kingdom. UNICEF succeeded in establishing a cadre of ECD professionals (trainers, facilitators, and liaison officers) from the four ministries (Education, Social Development, Health, and Awqaf), UNRWA, and NGOs who gained higher skills and knowledge for supporting field-level implementation and reporting on activities.
The Program makes effective use of the infrastructure of the participating agencies that employ professionals who, through their regular work transfer knowledge and skills of early childhood care and development to the targeted families. Thus, the facilitators of the Program are social workers from MoSD and NGOs; health workers at MoH; rural guides at NGOs; and KG teachers from MoE and NGOs. They conduct parenting courses for interested parents comprising of 16 training hours on various topics of child development.
In 2007, UNICEF decided to undergo another evaluation of the BPP. The goal of the evaluation is to assess the Program’s efficiency and to explore ways that the Ministry of Education (MoE) and the Ministry of the Social Development (MoSD) can mainstream within their directorates and responsibilities, and duties of their employees. The evaluation is intended to help the decision makers of the different partners, technical directors, liaison officers, facilitators, UNICEF offices in other countries, and all local and international development agencies dealing with families and children.
The main objectives of the valuation are to assess the following:
1- The efficiency of the delivery of the Better Parenting Program and its effectiveness in reaching its objectives
2- The degree to which the sustainability factors of the Better Parenting Program were relevant and appropriate in partners
3- The degree of impact of the program on the family and child
First of all and based on international experiences, a desk review was conducted that discusses the rationale and objective of parenting programs, the delivery methods, and the component of parenting programs to compare the BPP in Jordan with other international experiences.
In order to achieve the objectives of the evaluation, qualitative and quantitative research methodology approaches were implemented. A steering committee was formed of experts in the early childhood field to approve the steps of the evaluation. The quantitative part consisted of two questionnaires (Pre and Post) which were developed based on the content and objectives of the BPP with suitable reliability and validity, to assess perceptions, attitudes, practices, and knowledge of a sample of 337 males and females of two groups participants (control and experimental groups) before and after attending the program all over the kingdom representing all partners and the three regions (north, middle, and south). Twelve males and females with different and related background and experiences were commissioned to collect data for the purpose of the evaluation. They were divided into two groups and each group attended a training session. During these sessions, the program, the purpose of the evaluation, the pre and post questionnaires, and the data collection procedure were explained to the data collectors as well as how to communicate with the participants when they ask them to fill in the questionnaire. In recruiting the data collectors, it was taken into consideration the place where they live (e.g., for the sessions in the south, people from the south were recruited to collect data). In order to have an accurate and reliable results, two data collectors were placed in each session.
The qualitative part consisted of three focus groups and interviews which were organized with the facilitators, the Liaison Officers, and the Technical Directors to assess the efficiency and effectiveness of program delivery and the appropriateness of the program design in achieving its objectives. In addition to these focus groups structured interviews were conducted with the decision makers at participating ministries (Ministry of Education, Ministry of Health, Ministry of Social Development, and Ministry of Awqaf and Islamic Affairs), UNRWA, and NGOs of the different partners to discuss the sustainability factors affecting the program. That was in addition to conducting a desk review of international experiences regarding parenting programs.
To answer the questions of this study the data was analyzed using descriptive statistics (frequencies and percentages); in addition, t-test for means was used to indicate whether a difference between the means of two groups is significant. On the other hand, the interviews and focus group data was analyzed qualitatively according to research questions, SWOT analysis was incorporated which focuses on identifying Strengths, Weaknesses, Opportunities, and Threats.
The main findings of the evaluation can be summarized as follows:
# The program is very effective as measured by the attendance, the interest of the community in the program, changes in the behavior of mothers and fathers with their kids through the stories these beneficiaries reported to the facilitators, and raising participants' awareness and knowledge in child rearing. All partners feel comfortable with the flexibility of the program and its ability to meet the needs of the partners and the families. When it comes to the general monitoring of the program, UNICEF is doing well in this regard. In addition, all partners agreed that the fund provided and allocated to the program is insufficient.
# All partners are committed to the program, and it is well institutionalized. In other words, the Program is embedded in local institutional structures and all partners are well- prepared for taking over, technically and managerially. However, some of the partners such as, MoE, MoSD, MoH, MoAIA, UNRWA, and JOHUD are capable of continuing the work after the Program ends with, adequate budget and equipments, according to each partner's capacities and priorities, while on the other hand, most of the NGOs reported that they have to look for other sources of funding in order to continue with the Program.
# The total cost of the Better Parenting Program for the years (2006-2008) is JD 342,703 (US$ 479,784). The total number of program beneficiaries in these three years is 66,151, including 17,642 participants in 2006, 19,204 participants in 2007, and 29,305 in 2008. Using the total cost and the total number of participants for the three years, it can be seen that a rough cost per participant is JD 5.18 (US$ 7.25). If the national average of four children per household was used, (knowing that four children is a modest average for the poorer communities served in this program), the estimated cost per child reached by this program is JD 1.29 (US$ 1.81).
# The impact of the program on family and child is as follows:
* The program is useful and effective in bringing out some positive changes in their knowledge and attitudes toward child rearing such as decrease in beating the child and increase in talking to the child and explain wrong doings.
* The program changed the participants' behaviors in positive ways that after the program they started to spend more time in various activities with their children comparing with their responses at the beginning of the program.
* The participants acquired and retained knowledge and information related to parenting and child rearing covered in the training sessions.
* Participants adopted more positive discipline methods than they did before attending the program.
* The program is useful and discuses many subjects that are important to participants.
* In general, the change in knowledge was more significant and obvious than changes in behaviours, attitudes, and perceptions.
Based on the findings, the study recommends the following in order to improve the Better Parenting Program:
1. Ensuring the sustainability of the program by taking serious steps to shift the management of the program from the UNICEF and handle it to a council formed from all partners in addition to a representative from the UNICEF and NCFA. This council should be formed in a formal way to ensure accountability.
2. Integrating the BPP with other related programs of each partners' institution whenever that is possible.
3. Giving the Ministry of Social Development and the Ministry of Health more fundamental roles in the BPP due to the nature of services they provide to community. This can be done by communicating with the decision makers at these two ministries to insure expansion of their role in this regard especially to allocate more funds to support the BPP.
4. Allocating more fund for the BPP program especially to increase the financial incentives for the facilitators.
5. Dividing the manual of BPP into 3 or 4 sub-manuals, according to the subject, to be covered in several training sessions instead of having one session only. This would give the facilitators an opportunity to cover the entire manual, in addition, to cover it more in depth. This would also give the interested participants an opportunity to attend more than one session regarding parenting.
6. Training the facilitators on effective TOT skills including communication skills and adult education, in addition to equip the facilitators with specialized training in the content of the manual in order to be delivered effectively, especially if the manual is divided as it was mentioned in the previous recommendation. Moreover, it is recommended to prepare a training manual for the facilitators including the skills they need to have.
7. Preparing booklets or brochures for participants to keep. These booklets/brochures should include a summary of the main points and ideas of the topics covered in the training. In addition to that, it would be helpful to provide the training facilities/facilitators with video tapes related to the topics of the program to display to the participants.
8. Continuously revising and developing the training manual. For example, it is essential to include a chapter on parenting skills targeting children with special needs. This kind of information is missing from the current manual, so there is a need to target this group of children as their parents critically need parenting skills.
9. Strengthening the monitoring and evaluation system to have clear performance indicators in order to be able to evaluate its effectiveness. In addition to that, it is recommended to simplify the forms currently used and focus more on the impact of the program on the participants.
10. Designing a system for continuous communication and follow up with the participants after the workshop is over. This can be done by creating a data base for the target groups and participants.
11. Promoting networks between partners to be active to insure full cooperation and coordination between partners. That could be done through periodic and frequent meetings between partners at all levels (TD, LO, and facilitators).
12. Activating the job description and role of each the LO, facilitators, and the technical directors because there are still some facilitators and LOs who thinks that the cut points of the job descriptions are not clear.
13. Organizing an annual conference under the patronage of her Majesty to give more value and attention to the program. Through this conference several success stories can be presented, shared, and evaluated.
14. Investigating reasons behind the low turn off of male participation. For example, if the timing of the sessions is the main obstacle, then they may offer afternoon sessions. In addition to that, it is recommended to provide participants with some incentives such as providing transportation and meal so that the partners can attract high numbers from both males and females. It is also recommended to plan in advance for the training sessions and organize these sessions at different times of the year.
Full report in PDF
PDF files require Acrobat Reader.