If you want to get an NGO on the defensive, just says the words “monitoring and evaluation”. Almost inevitably, these words will provoke spluttering about donors-in-the-driving-seat, the validity of qualitative approaches to impact evaluation, and the objectivity of participatory research. I’d argue that too much time is spent on these debates, and not enough about how civil society organisations can use M&E as instruments of change. This may sound rich coming from a funder, but I think it’s true to say that the words “M&E and accountability” are spoken in the same breath far more often than “M&E and transformation”. Yet civil society organisations should pioneer a growing understanding of how to change our society, based on what they do and what they see.
There are many ways to categorise non-government organisations in South Africa based, for example, on size, sector and formality. For the purpose of this discussion, I’d like to divide them into two overlapping categories, namely ‘programmatic’ and ‘transformational’. The first type of NGOs delivers services (and in so doing, improves lives). Typically, they are commissioned – either by a foreign donor in the case of large programmes, or through partial support from the Department of Social Development in the case of small community-based organisations – as agents of service delivery. The large programmes tend to report on outputs, in compliance with set donor templates. The community-based organisations report, in varying degrees of detail, to DSD and their other funders.
The second type – that I’ve labelled as ‘transformational’ – may also deliver services, but their mission is to effect change in the system or society through advocacy, innovation or behavioural change. Many advocacy organisations are event or breakthrough-focused, and their impact is easy to measure: The Treatment Action Campaign pushed for anti-retroviral drugs to be made available – and achieved it; children’s rights groups lobbied for the Children’s Act and got it, etc. Other NGOs are focused on what we might call the hard ‘soft’ aspects of social transformation – how to reduce society’s tolerance of risk, how to cultivate a culture of aspiration, how to promote reading as a means of improving literacy. Many of these NGOs operate at the interface of the tough structure and delicate fabric of society, and grapple with how to change the two-way causal relationships between the two. This is where NGOs can make a real contribution to global knowledge through smart monitoring and evaluation.
Let me give you two examples:
Activate! is a national network of over 700 young leaders equipped to drive change for the public good. It is premised on the theory, proposed by Douglas and Wildavsky and further developed by Slovic, Weber and others, that low social solidarity and perceptions of constrained choice result in a fatalism (that manifests in high rates of HIV, crime, domestic abuse and substance abuse in South Africa). Activate! aims to develop a constituency of innovators who are connected and sussed enough to change their own communities – and use their collective influence to change social norms and power differentials. In the words of the initiative’s evaluators, Malcolm Keswell and Justine Burns, “can we show that pro-social behaviour is malleable to intervention, and can (non-market) social interventions effect changes in economic fundamentals?” Through a combination of behavioural economic games, self-administered questionnaires and network analysis, they hope to detect changes in risk and time preferences, perceptions of identity, civic participation and social capital. In 2012 and 2013, they controlled for bias by matching participants with similar young people from other programmes. In 2014, they hope to undertake a full randomised controlled trial. We hope to be able to show both documented change in the Activators’ communities, as well as changes in the economic preferences that drive social behaviour.
FunDza is a mobile-based platform promoting reading among teenagers. In South Africa, low educational levels and poor access and availability of books, on the one hand, and low appreciation and motivation for reading on the other, means that the ‘opportunity cost’ for reading is very high. Put another way, teenagers would rather do something else. That is, until FunDza arrived on the scene two years ago. It now has over 50 000 regular and active readers on MXit, suggesting that it’s meeting a demand – especially as over 85% of the readers report fewer than 50 books in their homes. Of course, it may be that FunDza is merely attracting the already-active readers who value a new outlet for books. But encouragingly, the highest increase in reading intensity has occurred among the teens with fewest books in their own homes. And FunDza doesn’t have to just take their word for it; they can verify it using MXit’s detailed user statistics. Active readers are more literate. Greater literacy improves educational and economic outcomes. Ergo, a social intervention can change economic fundamentals.
These are the exciting frontiers that NGO’s should be pushing, where economic structure and social outcomes meet. Yes, the power differentials between donors and civil society organisations should be tackled – and brave NGOs that understand what’s best for their communities will put their feet down when externally-imposed M&E threatens the integrity of their work. But power does not only come through protest, but in changing the differential ‘value’ that the respective players bring to the table. Civil society organisations should reclaim the primacy of ‘monitoring and evaluation’ as instruments of change that they can help bring about – and secondarily as a means of accountability to Government and funders.
Download slides here.
