2000 CHN : An External Evaluation Report on the Health Management Pilot County Project (1999-2000)
Author: Beijing University
MOH-UNICEF selected 40 poor counties in five provinces in the northwest and the southwest of China, and initiated a general project on primary health care and nourishment improvement. On the basis of the training in medical theory and clinical techniques of those grassroots health workers, the project plans to strengthen the effective management, supervision and guidance of the grassroots health organizations and to seek ways for the sustainable development of the project.
Purpose / Objective
The objectives of the external evaluation are:
1. To understand the operation and management of the grassroots 3-tier health care network in the project area, especially those of village health organizations and their "network bottom" facilities, management measures, meeting and reporting systems, human resources management, and payments and remuneration management;
2. To evaluate the efficiency, cost effectiveness, and cost-benefit ratio of the "overall" primary health services;
3. To evaluate the impact and efficiency of related project components;
4. To evaluate the method of assessment, operation procedures, results and problems of introducing profit motives in health care services at both xiang and village levels;
5. To evaluate the systems of supervision, operation, and method of improvement used by the central, provincial, county, xiang and village governments;
6. To assess the method of operation at each management level and the extent of government support and participation;
7. On the basis of findings of above and after learning the success stories, special characteristics and problems of the project, to make recommendations for the improvement of the design, management, and implementation of the project in the next project cycle;
8. To explore ways for promoting the sustainability of the project that are workable for China.
Each of the five provinces was sent an evaluation questionnaire. Within the provinces, 11 xiangs and 17 villages were selected. Interviews were held with 78 parents of children under age 5 and 67 mothers of children below age 1 (including pregnant women). Interviews were also conducted with government officials at all levels: national, provincial, and county. Xiang hospital workers, and village doctors and midwives were also spoken with. In each area, scheduled meetings were observed. Additionally, a desk review of project documents and materials was done.
Key Findings and Conclusions
According to the project, national experts would be assigned to designated provinces to participate in project formulation and to provide field supervision and guidance once every two months. In this evaluation process, the evaluation group has learned, through interviewing and reviewing of documentation, that most national experts have made relevant and workable recommendations that are helpful for the further improvement of the project.
The evaluation group has learned through the review of records of the xiang and village workers that evaluation exercises had been carried out periodically (once every 2-3 months at the village level, 3-4 months at the xiang level). On the basis of the evaluation results and following the rules of project financial management, reward was given to the xiang by the county and to the village by the xiang. The evaluation group, through its field interviews, has learned that the economic incentive mechanism has received general support at the local areas. There are, however, problems in the actual operation of the system.
As we find in this evaluation, the score and subsidy of the village health workers are determined only by the leader of the township hospital and, therefore, impartiality probably exists. In some townships, the village doctor and leaders in clinics at the above level can get twice the amount of what a woman in pregnancy can get, which is usually about 4-10 RMB subsidy per woman, and it is said that such had been the case from the very beginning of the project. In addition, in some provinces, the project outlay was distributed to personnel in other departments, which made the project work become accessorial, and the arrangement, schedule and effect of the project were watered down. Leading and technical groups at the province, county and township level didn't take any measures of intervention and control.
The irregularity of the subsidy provision, sometimes once in 3 months or 5 months, makes the health workers at the village level feel insecure. This is a prevalent problem. The funding for the townships and villages is often cut off by the offices at the county level. The supervision outlay is insufficient at the province and county level, especially in the mountain and minority area (translating the local language), as well as in the Provincial MCH Clinics. (They think that provincial experts' going to the countryside will affect their daily wok.)
Too many links in the examination and approval of funding; one trouble incurred in one link will lead to a chain reaction. People at the county level were not aware of their violation of the project regulation by cutting off the subsidy for the villages.
The project specifies that for those most basic and widely used drugs, there must be unified drug retail prices that are clearly labeled and publicly displayed. The rural doctors must keep proper records for every patient examined, stating the problems and prescription given (medicine name, dosage, usage and price). The evaluation group has learned that most village doctors have followed what they were asked to do.
For each village visited, the evaluation group has collected detailed information on drug dispensary of the village clinics using questionnaires, on-site observation and interviewing of village doctors. All these clinics have small dispensaries, completed prescription records, good purchase plans and well-kept purchase receipts. An examination of 10-20 prescription records in each village clinic shows that the drug charges are indeed relatively low.
Through field interviews and record reviews, the evaluation group has learned that the project has received strong support from the government at all levels. The project units have also sought actively the support and coordination from local governments. Some had allocated the counterpart funds and prepared specific project planning and programs.
In this evaluation, the methods of health education and its effect didn't meet the requirement of the project. For example, the times of health education is different from the information we got from the interview, which may be the result of falsification or misunderstanding of health education methods. Thus, it is necessary to adjust the health education pattern, to direct and train health education at township and village levels, and to establish an effective evaluation scheme.
We find that some local leaders pay more attention to the aiding fund. They often adopt one-way thinking and consider that the program means the money, yet neglecting to consider the idea, management pattern and sustainable development of the project. When asked about the practical work of the project, such as the characteristics, purpose, strategy, new management pattern, incentive mechanism, or future development, they know little.
Some managers didn't realize the important role they will play in the organization, operation and coordination of the project. Falsification of counterpart funding existed in some counties. (The financial voucher for counterpart funding is only made for check.) As a consequence, such project activities as commencement meeting, material printing, which entail money, couldn't be carried out on time or effectively.
In the evaluation, we find that some experts' supervision tended to be ostensible, with more case book and less feedback. People at the grassroots reported that the participation of national experts had little effect and that no training for the provincial experts existed.
To clarify the responsibility, right and interest of different administration levels in the project document, and give local governments room for inherent regulation
To constitute the responsibility contract with specialists and grassroots to ensure the seriousness and controllability of the project.
T o make clear who will be responsible for the interpretation of the project, so as to avoid misunderstanding at the grassroots through communication.
To provide fair and square feedback by applying tracing evaluation.
To attach feasibility evaluation to the project document.
We should begin to train the experts at the upper level. Moreover, when training, some local members should participate and check whether or not the experts at the upper level are eligible to communicate with the local experts. The national experts should be capable to guide the formation of the document of the province, county, township, and village; to control the execution and effect of different management levels; and to discover problems and solve them in time.
In general, this project dealing with key issues is a bold innovation in exploring new pattern of health system. It should be understood and realized that only by way of intensive management could health resources "drip irrigation" system be achieved. This is the first-rate management pattern in view of special status in China. Seeing that China's rural grassroots health service network and system are on the verge of collapse, we have no choice but to set "network mending" and "network fixing" as priority strategies.
Full report in PDF
PDF files require Acrobat Reader.