We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2009 Ethiopia: Evaluation of EthioInfo Utilization in Ethiopia



Author: Adebayo A. Bamitale, Ph.D. Partners: Central Statistical Aency of Ethiopia (CSA), United Nation Country Team (UNCT)

Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:

EthioInfo is the Ethiopian in-country customization of DevInfo, an advanced database system providing functionalities for the organization, storage and presentation of data in a uniform format for facilitating data sharing at the country level across government ministries and UN agencies. DevInfo was adapted from the UNICEF Child Info database technology and has been globally adopted by the UN as the key toll for monitoring and evaluation of MDG indicators

EthioInfo is completely owned and managed by the Ethiopian Statistical AGency (CSA). The United National Country Team (UNCT), with the technical leadership of UNICEF, supported the Central Statistical Agency (CSA) to adopt DevInfo as EthioInfo in 2005 - 2006. EthioInfo was envisaged as a tool for strengthening the capacity of counterparts in generating and disseminating data for situation and outcome monitoring of national priorities within the Plan for Accelerated and Sustaining Development to End Poverty (PASDEP) and United Nations Development Assistance Framework (UNDAF).

EthioInfo was also planned as a tool for enhancing the dissemination and use of statistical information and contributiing to knowledge management in Ethiopia. The customization involved many efforts including the development and population of the database filling it with data from major national surveys such as Welfare Monitoring Survey (WMS) 2000 & 2004, Demographic Health survey (DHS) 2000, Household Income Comsumption Expenditure (HICE) 1996 & 2000. The first customized desktop version of the database system, called EthioInfo v1.1, was launched in February 2006. In order to further increase the accessibility of the database to the end-users, in 2008, an updated and web-based version, called EthioInfo v2.1, was deployed on the central statistical web server, and is available online at:www.csa.gov.et/wthioinfo/.

Purpose/Objective:

This study aims

1. To Evaluate the efficiency, effectiveness, relevance, sustainability and impact of EthioInfo with special emphasis on promotion of the use of Government Statistical information in policy development, planning and research in Ethiopia.

2. To obtain information in the following areas (a) EthioInfo database and system (b) users and use of the EthioInfo (c) the CSA and support for EthioInfo, (d) UN System and its support of the Government of Ethiopia on EthioInfo, and (e) the strengths, weaknesses, opportunities and threats (SWOT) analysis of the EthioInfo.

Methodology:

An analysis of these areas indicates that one method of data collection is incapable of satisfying the needs. For this reason, a multi-method approach was designed to handle the data collection for the system. Data was collected through desk review, interview, observation, and questionnaire. To achieve the broad and specific objectives of the study, the study used the following methodology, broken down into five major categories of: (a) Desk review, (b) system study, (c) observation, (d) survey, and (e) SWOT Analysis

Multi-method approach

1.Desk review of relevant documents and the system, this led to document analysis involving examination of existing EthioInfo documents, especially on the system, government statistical system, millennium development goals, indicators, as well as the place of EthioInfo in the government statistical system in Ethiopia;

2. System Study - To study the technological basis of the EthioInfo system as a way of understanding the implementation environment and also for possible optimization;

3. Observation - To observe actual users of system in operation, for in-depth understanding of users experiences;

4. Survey - Questionnaires and Interview - to gather information from users and implementers, (a) Questionnaire survey to elicit relevant information on the evaluation criteria on randomly selected users drawn from the domain specified in the Terms of Reference ; (b) key informant interview with high level government officials and senior planners and decision makers and renowned researchers and evaluators of development programmes for in-depth understanding of the utilization of EthioInfo;

5. SWOT Analysis - to review the strength, weaknesses, opportunity and threat (SWOT) of the EthioInfo implementation

Findings and Conclusions:

1. EthioInfo is underperforming as a statistical database system. There is little evidence that EthioInfo has had any significant impact on the dissemination and utilization of government information in Ethiopia.

- The data on the system still refer back to surveys of 2005, four years ago, although the updating was
completed in 2007;
- The online system is error-prone, not fully supported, and the system still has various missing weblinks

2. It is difficult to see EthioInfo as it currently is as the central data dissemination system of the government and the United Nations. Its various problems have not made it possible for EthioInfo to be the central tool for dissemination and utilization of statistical information that it could have been. It must have moved to the central stage at a point in time, but now the system is at a stand-still, because of the emergence of competing UN agencies supported products and the perception that the system was not easy to support and use.

3. There are ample evidences that CSA has played significant roles in the devolution of DEvInfo in Ethiopia the implementation of EthioInfo. The ownership of the EthioInfo system by the CSA has been a strong point of the implementation. However, CSA faces challenges which have affected the management and sustainability of the EthioInfo. EthioInfo is affected by;

a) Social and cultural data problems
b) Human resources issues - high turnover of staff and heavy workload of the remaining staff
c) Standardization of datasets and indicators
d) Data Analysis issues, especially for trends, Generally low level of awareness of role of statistics in decision making

Development partners need to work with the CSA to handle these challenges so that the agency can continue to deliver not only on Ethinfo but on other data dissemination platforms. Key problems with EthioInfo are extendable to other application whether owned and promoted by a UN agency or not. So, whatever tools are introduced to be used in the CSA, unless the identified problems are confronted and solved, there is little hope of success. The problems are not new; they have been identified by CSA itself and indicated in a report on database management systems derived from DevInfo, the support of the UN will be better articulated, coordinated and channeled if the UNCT focuses on the challenges of the CSA, helping the institution to combat the base problems.

Recommendations:

A: System Related Recommendations

As a way forward for the implementation, the following recommendations for system enhancement should be considered:
1. Both CD and Online versions of EthioInfo should continue to be made available because of the nature of internet connection available to the users
2. It is necessary to ensure complete configuration so that all databases could work as designed
3. Possible optimization strategies should be explored to speed up system response this may include technology such as caching, AJAX-based interfaces, and so on
4. The feedback and user support system should be made operational so that users can give feedback and request support. Doing this will ensure that the system could respond to the users’ demands;
5. The data should be updated with new surveys and other data including administrative and routine data; and
6. Considering the various advantages due to elimination of the issues with previous version, it may be necessary to upgrade to Version 6 of DevInfo.
7. EthioInfo Core Technical Issues that needs :

      a) EthioInfo is bandwidth hogging
      b) Interface simplification and possible Wizardification
      c) Database issues –Need for pooled index for easy global searching and appropriateness of MS Access as the database back-end should be re-examined
      d) Simplify and strengthen the data importing and exporting functionalities.

8. Enhance the capacity of EthioInfo for data analysis including extrapolation, interpolation, data gap filling, and other data analysis functionalities.
B: User Related Recommendation

To enhance user experience, and put the EthioInfo at the position it needs to be, the following user related recommendation are made:
1. Need for public enlightenment – this includes emphasizing the role of statistics in policy making and decision support and drawing attention to dissemination platforms;
2. Awareness creation and more training for users need to be done to enable the potential users to be aware of existence and capacity of EthioInfo
3. Creating a strong an effective user support and feedback system
C: CSA Related Recommendations

As the core institution for supporting the implementation of EthioInfo, the CSA needs to take some actions to ensure the effective implementation of the database system. Recommended actions include:
1. Dedicated team for EthioInfo at the CSA. This team should be charged with: (a) updating; (b) extended data and indicator sourcing; (c) support for all users – individual and corporate, (d) training of users, (e) consistent awareness creation for EthioInfo use; and (f) ensuring effectively maintained feedback system – so that the system could be demand driven.
2. Faster internet connection
3. Shorten the time required for data updating so that users may have access to up-to-date information.
4. Deal with human resources problem – attracting and retaining quality, skilled and experienced staff – especially, in IT domain.
5. Support for consistent EthioInfo monitoring and evaluation. There is need to ensure that the status and utilization metrics of the EthioInfo are monitored on a regular basis. This could be done through:

      a) Web-surveys
      b) Feedback Form
      c) 24/7 Online Support
      d) Web counters

6. Resolve data compatibility issues, by bringing in data interfacing skill-set into the team
7. Empower Data Producers for the EthioInfo should be empowered with capacity building for direct data importation by creating satellite databases
D: UN Related Recommendation

1. The UN must take the lead in harmonization of UN-based statistical systems
2. Awareness for UN system on EthioInfo.
3. The UNCT should relieve UNICEF of core-role in the leadership of the DevInfo to disconnect the system from its linkage to UNICE F and to enable other agencies to stop seeing the system as the sole baby of UNICEF.
4. UN Resident Coordination Office should play increasingly dominant role in the devolution of DevInfo or any other UN tool

Lessons Learned (Optional):

1. Enabling environment is available in Ethiopia in every perspective - CSA, Government, and UNICEF
2. Other dissemination tools promoted by the UN agencies are creating confusion in the CSA
3. UNICEF is perceived as trying to impose its standard on the UN and to thus dominate other UN agencies
4. CSA realization of the possibility of customizing other applications to local situation
5. CSA is investigating all the software provided by UN agency to know which of the software to stabilize on.
6. Data Conversion from CSPro to EthioInfo is Data compatibility has been huge problem with updating data on the EthioInfo
7. Engaging the partners and creating ownership of the EthioInfo at the CSA has been an outstanding achievement of the UN system in Ethiopia.
8. It would seem that no matter the strength of the core system, the customization and infrastructure available to the disseminating institution is very crucial to effective delivery



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search