Nobel Prize, Randomized Controlled Trials, and Poverty
There are two great things about the 2019 Nobel Prize for Economics that went to Abhijit Banerjee, Esther Duflo, and Michael Kremer. The first is the focus on poverty, which for some reason is rare for the Prize. The second is that one of the winners is a woman – Esther Duflo – which is only the second time that’s happened in the history of the prize.
Having said that, there are a few things that worry me about this prize. The three won for their work on adapting Randomized Controlled Trials (RCT) to development interventions. You know, the type of experiment scientists use to figure out which new drugs might treat diseases. You have a control group that gets a placebo, and the treatment group that gets the drug you’re testing, at least in its most classic form. Except in this case the control and treatment groups are communities, or schools, or anywhere poor people are found.
The method is a powerful one for trying out ideas and generating data on what works better. The prize winners have tried it with immunization rates, microenterprise programs, refugee livelihoods, and a stack of other problems related to poverty. I recently participated in an RCT in West Africa, measuring the effects of a radio program on levels of political tolerance. For that radio show, it’s hard to imagine another method that would have produced convincing results.
It works great for drugs. What could go wrong applying it to social problems? Turns out quite a bit. A lot of ink has been spilled over this issue, so let me focus on the most concerning ones.
Service Delivery – RCT works best for micro-interventions or service delivery, that is, interventions dealing with a single problem like how to increase immunization rates or use of bed nets. For these purposes the method is powerful. For anything more profound though, the method ignores the wider structural issues that create poverty and keep people in it. The trend in recent years has been to see the systems that people rely on, and that often fail them. Understanding these complex systems has led us to understand that local efforts must be accompanied by regional interventions and connection with national or international policy to really change things. With RCTs, there is a risk of “individualizing” the problem of poverty. The onus is on individual behavior. If you just use a bed net, or if you just get your teachers to show up at school, then your problems would be solved. This Victorian view of poverty, which sees the individual character as the cause of poverty, ignores the systems that are so important to keeping people poor.
As 15 leading economists (including two other Nobel Prize winners) recently pointed out in The Guardian,
Handing out performance bonuses to teachers, for example, is an inadequate response to education budgets that have been slashed in order to pay down onerous external debts.
Water purification tablets are too little in the face of droughts induced by climate change; what is at stake is an ecological emergency that demands coordinated public policy strategies. In agriculture, real progress requires putting an end to the excessive subsidies paid by rich nations to large producers, regulating food commodity derivatives markets, and ending the land grabs that dispossess the small-scale farmers who play vital roles in feeding the world.
I get it, though, not all interventions have to take on the whole mess at once. Which would be fine, if the “randomistas” (as the more fervent evangelists have come to be known) didn’t pose RCT as the gold standard, the only method that reveals the Truth. In her 2010 Ted talk, Esther Duflo in fact suggests that without RCTs “we are not any better than the Medieval doctors and their leeches.” Which would come as a surprise to all those astronomers, paleontologists, and other scientists who somehow find a way to do good science with RCTs.
Policy can’t be randomized – Because RCTs work best on discrete interventions, they really aren’t much help in policy change, which drives much of the change we work on. Poverty is a political phenomenon, and contested control over state policy and resources determines as much about people’s life chances as access to microloans or self-help workshops. Unfortunately, this contested terrain is a tough place to do RCTs, so they’re not much help there. There is stacks of great work these days on how to do and assess advocacy and policy change work that takes into account how social change really happens in a fundamental way. Getting better at doing RCTs should not deter us from pushing these approaches even farther.
Cost – RCTs work best when there is a lot of data, to cover over the inevitable differences across individuals and groups. Leaving aside the issue for a moment of whether you can really find “control groups” across large numbers of people, the result is that interventions tend to be large and expensive. Which would be fine if they actually solved the root causes of poverty in key areas, but they don’t. So the risk of donors pivoting scarce resources from the messy and long term work of building social power and bringing about policy change to the flashy results provided by RCTs is a concern. Given the trends to improving “aid effectiveness,” there is a danger that short term results will take precedence over long term effectiveness.
So congratulations to the three winners, and hats off to them for innovating in what is often a moribund space. Let’s keep in mind that RCTs are one tool among many, better suited for some purposes than others. And let’s keep figuring out how to bring about the fundamental social change that so many of us are working for, using these tools and the many others that we have in our tool kits.