Impact Evaluation: Finding What Works

As the skills and knowledge of people in developing countries continue to increase, is there a role in developing countries for Western development workers? I was asked this question during a recent job interview. It’s an important question. Although I drew, in my answer, upon the (idealistic) cliché of development workers working with the objective of putting themselves out of work, there’s probably more to it than that.

Over the past few days though, I’ve been reminded of an even deeper existential crisis within the development field. Several casual conversations I’ve had with people I’ve met here in Ghana (we development folk know how to keep it light!), as well as recent high-profile critiques – such as that levelled at celebrity economist Jeffrey Sachs (author of End of Poverty, Bono’s best buddy, and aid champion) in Nina Munk’s The Idealist – reflect an endemic uncertainty among development professionals: does aid make a difference? Or in other words, do the hundreds of billions of dollars being spent each year contribute to positive change?

That I’m not the only one looking for the answer to this question might be more disconcerting than it is reassuring. It’s a question that marks a deep chasm between some of the most influential thinkers in international development. For skeptics such as William Easterly and Dambisa Moyo, aid is, at best, a waste of money, and, at worst, more harmful than helpful. For others, such as the vilified Sachs*, it’s simply a matter of needing to spend more.

For me, the key is learning about what works. It might look like a cop-out, like I’m avoiding having to take a stance. In reality, however, I believe there are projects that can produce sustainable results, and that by promoting accountability (in terms of demonstrating impact) and transparency, we can avoid some of the potential problems (such as dependence, corruption, and ineffectiveness) discussed by aid skeptics. In other words, by putting the onus on development NGOs and governments to credibly demonstrate the impact of their aid expenditure, we can improve their accountability to donors and to the beneficiaries themselves. Furthermore, it enables us to learn about what works and to make better decisions on future expenditure.

Learning about what works requires rigorous impact evaluation. In Poor Economics, for instance, Banerjee and Duflo present some remarkable findings from the application of randomized control trials (RCTs) (like those used to test the effects of medicine in clinical trials) to social projects. While there are other means of carrying out impact evaluations, and RCTs aren’t everyone’s cup of tea, this post has to end somewhere and I don’t have the space to get into that right now (though I’d love to hear the thoughts of others – I’m a fan!).

My point is this: while the philosophical questions are important, they have a tendency to generate a lot of greyness. Impact evaluation, on the other hand, when done properly, can allow us to look through the greyness, and find what works.

* As someone who spent three months researching the impacts of electrification at Sachs’ Millennium Villages Project in Rwanda, I’d like to say my findings pointed to positive changes in the livelihoods of those that accessed electricity. While the  Millennium Villages Project might not have had the macro-level impacts that Sachs had hoped for, there are surely lessons there to be learnt and some successes that can be replicated. At the very least, Jeffrey Sachs succeeded in drawing widespread mainstream attention to poverty and in defining sustainable development as a global concern.

One thought on “Impact Evaluation: Finding What Works”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s