Each year the Evaluation Office commissions global thematic evaluations under the corporate Plan for Global Thematic Evaluation 2014-2017, which is developed in consultation with the Evaluation Committee and offices at headquarters and regional levels. Institutional Effectiveness Evaluations analyze key organizational strategies and activities and inform corporate decisions to improve institutional relevance, efficiency and effectiveness.
Evaluations completed under the Plan for Global Thematic Evaluation 2014-2017
Other major global thematic evaluations
Evaluability Assessment: UNFPA-UNICEF Global Programme to Accelerate Action to End Child Marriage
RFPS-USA-2016-502309 Evaluability Assessment-UNFPA-UNICEF Global Programme to Accelerate Action to End Child Marriage
Questions and Answers on the ToR and the RFPS-USA-2016-502309
[This section will be updated as questions are received. Bidders are advised to check this space periodically.]
30 August 2016
This evaluability assessment is not an evaluation and has been designed as a ‘light exercise’. The purpose of the Evaluability Assessment is to determine the extent to which progress towards the objectives of the UNFPA-UNICEF Global Programme to Accelerate Action to End Child Marriage can be readily and reliably measured, monitored and evaluated.
1. Coverage of the Evaluability Study
The Programme to Accelerate Action to End Child Marriage is designed in three phases, Initial phase: 2016-2020, Midterm phase and longer term phase (page 3, TOR August 2016). Is it the correct perception that the subject of the evaluability assessment will be the recently started phase I only and not the totality of the programme with phases I, II and III?
Answer: The assessment will look at the initial phase of the programme from 2016 – 2019.
2. Budget upper limit for the evaluability assessment
A. Is there a budget ceiling indicated for programme M&E, either as a percentage of the total programme budget, or as an amount in USD, in one of the project documents?
Answer: There is no budget ceiling for M&E included in the project documents.
B. Does a budget limit exist for this evaluability assessment?
Answer: We are unable to provide an indicative budget as that is outside the scope of the process we are required to follow. We encourage you to refer to the indicative estimates we have provided on expected level, time commitment and duration for the evaluability exercise.
3. Post-Employment Restrictions
Referring to the RFPS, D-9, page 14. Do the restrictions stated in this paragraph also apply for UN staff members who retired from service more than three years ago?
Answer: Since this will be an institutional contract this clause does not apply.
4. Governance and Management of the Evaluability Study
A. Who will assume the role of the Evaluation Manager?
Answer: The evaluability assessment will be managed by the UNICEF and UNFPA Evaluation Offices, independent of stakeholders involved in the programme at global, regional and national level. The joint evaluation management group (EMG) composed of staff from UNICEF and UNFPA Evaluation Offices will have overall responsibility for the management of the evaluability assessment.
B. Is it the correct perception that the Evaluation Team Leader will be the external, independent consultant nominated as TL in the Technical Proposal? If not, who else it would be?
Answer: The assumption is correct. The Evaluability Assessment Team Leader will be the lead external independent consultant from the firm that wins this bid.
C. Whereas several evaluation steering and oversight functions are mentioned in the ToR, the role and function of the Team of External Evaluability Assessment is not. What is the defined role for the external Evaluability Assessment Team?
Answer: The external evaluability assessment team is the team of consultants from the firm that wins this bid. The team’s task will be to conduct the evaluability assessment, as described in the terms of reference. Specifically, the team is responsible for the deliverables, outlined on pages 15-16 of the terms of reference. Additional information on the role of the evaluability assessment team is provided on page 16-17 of the terms of reference.
D. The Link between the evaluation team and, for example, the evaluation focal points in each of the 12 programme countries is not reflected in the organigram copied below. Most of the links are only pointing in one direction, not signaling a dialogue between the entities involved. Can you clarify the intended communication channels in course of this assignment?
Answer: The main point of contact of the external evaluability team will be the joint evaluation management group. As noted in the terms of reference, the evaluability assessment will be conducted in a participatory manner. Any interaction between focal points in the twelve programme countries and the external evaluability team will be facilitated by the joint evaluation management group. Although the arrows are one directional, there will be dialogue between all entities involved. Additional information on process can be found on pages 11-13 of the terms of reference. The inception phase briefing from the Joint UNICEF/UNFPA Evaluation Management Group will clarify the communication channels further.
5. Stakeholders to be interviewed
The ToR, page 12, III Data Collection and Analysis Phase, does not explicitly include National Stakeholders, for example Governmental Institutions, in the list of country level consultations. Is this because it is anticipated that the Inception Report will identify stakeholders at this level, based on an assessment of their roles in planning and implementation of the programme in UNICEF and UNFPA? Or is it intended in the Assessment design that National Stakeholders are excluded from the list of interviews at country level?
Answer: Because this is an evaluability assessment, and not a full evaluation, it is thought less consultation is needed and, as such, national stakeholders were intentionally excluded from the lists of interviews at country level. However, should the team deem it necessary to interview national stakeholders beyond what is listed in the TORs, the team can propose this in their bidding proposal.
6. Risk Management
The ToR states: Prior to launching the evaluability assessment the following elements will need to be in place:
- Clarity on evaluation fund allocation and what funds will cover the costs of the evaluation plan.
- Sufficient implementation data in view of being able to assess the systems in place.
- Clarity on management structures around the global joint programme at HQ, RO and CO levels.
- Clarity on programme budget allocation overall.
- Clarity on architecture of Programme in the 12 countries and regions. This will influence the oversight structure and data collection.
- Availability of finalized results frameworks. Availability of work plans are set and country-level strategies.
Which of the seven elements are, at present (24/31 August 2016), readily available/in place? What is the risk to the winning bidder of a delay in launching the work because such elements are not in place, and in particular of encountering a delay in the country visits?
Answer: There is no foreseen delay in launching the work. Overall, these elements are largely in place. However, the evaluability assessment will consider the adequacy of these arrangements.
7. Can bidders structure the team in a format that they believe is most effective?
8. Is there flexibility regarding the Level of Effort, or should bidders adhere to the 125 days listed in the RFP for the entire team?
Answer: The TOR states the combined qualification and general level of effort required by the evaluation team. Please bear in mind that the level of effort are estimates. The bidders can adjust the proposed 65 days for the Team Leader and 60 days for the team members. However, any adjustments/changes must be accompanied by justifications.
9. Is the evaluability assessment meant to address all phases of the global program, or just the initial phase (first four years)?
Answer: please see question one, above. The assessment will look at the initial phase of the programme from 2016 – 2019.
10. Is there an evaluation framework that is based off of the global theory of change listed on page 5 of the TOR?
Answer: The Global Programme has a global Results Framework associated and an evaluation plan has been developed. The results framework will be shared with the team that wins the bid.
11. Since the programme implementation began in 2016, has monitoring data already been collected?
Answer: Yes for certain Country Offices.
12. Are there Monitoring and Evaluation plans/systems in place for the global programme?
Answer: In progress.
13. Can UNFPA/UNICEF share the relevant background technical documents as mentioned on page 10 of the RFP?
Answer: Please refer to the selected bibliography p.23 of the Terms of Reference. A full set of programme specific documents will be shared with the consultant team during the inception phase.
14. Are the nine desk based country assessments during the desk review expected to result in nine country reports? Are there any deliverables tied to the desk review?
Answer: The nine desk based country assessments will result in nine Country Reports (deliverable 2, on page 16 of the terms of reference). Country reports (max 5 pages) for each of the programme countries. For the 3 countries visited the lengths can be extended to max 10 pages. Regarding the desk review, there is no specific deliverable but the desk review is expected to inform all the other reports.
15. Can UNICEF/UNFPA provide more details regarding the briefing from the UNICEF/UNFPA Evaluation Management Group and Reference Group during the Inception Phase? Specifically, will this be in-person and if so, for how many days? Will this be at the onset of the Inception Phase?
Answer: This meeting will take place in person at the beginning of the inception phase at UNFPA or UNICEF headquarters in New York. The inception phase provides an opportunity to meet a selection of UNFPA and UNICEF staff at headquarters in New York over a period of several days.
16. When will the three debriefing meetings during the Validation and Reporting Phase be held? Will they be held before, or after submission of the draft Report, and will the first two in-person meetings be held concurrently?
Answer: The three debriefing meetings will be held at UNFPA or UNICEF headquarters in New York in April 2017, after the completion of the draft evaluability assessment. The meetings will be held separately.
17. The TOR mentions Workshop Reports (as mentioned on page 16) as deliverables. Are these reports based off of the three workshops held during the Validation and Reporting Phase?
Answer: The Workshop reports refer to minutes and summaries from all meetings, including the workshops held during the inception, validation and reporting phase.
18. Can UNICEF and UNFPA provide bidders with the budget that has been allocated to this assessment?
Answer: No. Please see question 2b, above.
19. Given that the field visit countries are unknown, how should bidders reflect per diem costs affiliated with each country that will allow for comparability between financial proposals?
Answer: The field visit sites will be identified during the Inception Phase. Please note that the cost of travel is the only reimbursable item, based on actual expenditure. Daily Subsistence Allowance (DSA) rates for all countries and destinations can be found in the attached document.