September 2008, Stuti Khemani
The push toward community monitoring of public service delivery in development projects is based on the notion that local oversight will raise the social accountability of public service providers and thereby improve the quality of services. Diverging results from evaluations of two community monitoring programs in health and education suggest that local monitoring does not guarantee better service delivery.
Two evaluations of ambitious efforts to improve public services show strikingly different impacts. Both projects attempted to mobilize communities to achieve this. One project on health in Uganda succeeded, while another on education in India had no impact on public schools even as a private initiative outside the public school system was able to improve the reading skills of students.
The push toward community monitoring of public services in development projects is based on the notion that local oversight raises the social accountability of public service providers. The diverging results from the two evaluations suggest that local monitoring does not guarantee better services.
In Uganda community monitoring led to improved quality of health delivery
In 25 randomly selected communities local NGOs organized meetings of residents and health service providers about the quality of care in public clinics. Quality of care was measured previously through user surveys about the quality of service received at the clinics and was compiled in “citizen report cards.” In another 25 randomly selected “control” communities, no such discussion took place.
In the communities where the meetings were held, absenteeism by providers decreased and the quality of service (measured by wait time and quality of care) improved. Ultimately, immunization rates rose and child mortality rates dropped in these communities, suggesting that mobilizing the community to monitor providers more actively can improve services. In Uganda, this oversight could be done through the beneficiary-control institutions called Health Users Management Committees (HUMCs).1
In India community monitoring did not lead to improved quality in public education
Reading test volunteers in Uttar Pradesh. Photo by Shekhar Shah.
Village volunteers prepared “report cards” on the reading ability of children; these were shared with teachers, local government representatives, and residents in village-wide meetings. An evaluation found no difference in community participation, teacher effort, or learning outcomes in public schools between the villages where the meetings took place and 85 randomly selected “control” villages where no meetings were held.
A large number of factors may affect the performance of community monitoring
The success of this private initiative outside the public school system compared with the failure to improve the public schools through the village meetings suggests that collective public action does not guarantee better public services. VECs in India and HUMCs in Uganda were largely inactive at baseline, and both had only indirect influence, mainly in the form of complaints from parents and users about non-performing teachers or health workers. Action against them would have to be taken by a senior bureaucrat.
So what accounts for the different outcomes in the two countries?
Variation in NGO activism in the interventions. In Uganda, the local NGOs seemed more active in pressuring the health care providers to improve performance than the education NGO in India. The facilitators in Uganda directly negotiated with the dispensary staff before involving the villagers, and the villagers who became involved were hand-picked by them. Because the community served by each health clinic is almost ten times larger than a village in India, the activist role for external facilitators may have been more scalable. In India the focus was on community-led engagement.
Differences between health and education services. Poor health services are more directly observed by users than poor teaching which can remain invisible to parents. As a result, the users of health clinics could be more easily spurred to demand better services they did not know they were entitled to. In contrast, young pupils need advocates. Monitoring the quality of teaching requires parental visits to the classroom to observe teacher effort and performance, and one parent’s complaint may go unheeded unless it sparks complaints from more parents.
Differences in country political economy. Teachers are often organized, politically powerful, and therefore resistant to social or bureaucratic reproach. Larger political obstacles can constrain local collective action. Mobilizing citizens in the Indian political economy context may be more successful by providing information for comparisons and benchmarking purposes across jurisdictions, thereby putting public pressure on politicians to improve the quality of services.4
The contrasting results in Uganda and India underscore the importance of continuing to experiment with information interventions in different socio-political contexts.
1. Martina Bjorkman and Jakob Svensson. Forthcoming. Power to the People: Evidence from a Randomized Field Experiment of Community-Based Monitoring in Uganda. Quarterly Journal of Economics.
2. Abhijit V. Banerjee, Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2008. Pitfalls of participatory programs: evidence from a randomized evaluation in education in India. World Bank Policy Research Working Paper 4584, April.
3. Abhijit V. Banerjee, Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2006. Can information campaigns spark local participation and improve outcomes ? A study of primary education in Uttar Pradesh, India. World Bank Policy Research Working Paper 3967, July.
4. Stuti Khemani. 2007. Can Information Campaigns Overcome Political Obstacles to Serving the Poor? In The Politics of Service Delivery in Democracies: Better Access for the Poor, ed. Shantayanan Devarajan and Ingrid Widlund. Expert Group on Development Issues, Ministry for Foreign Affairs, Sweden. (Download the report)