That’s the name of this website – Social Programs That Work – run by the Coalition for Evidence-Based Policy. As they say, “U.S. social programs are often implemented with little regard to rigorous evidence, costing billions of dollars yet failing to address critical needs of our society — in areas such as education, crime and substance abuse, and poverty reduction. A key piece of the solution, we believe, is to provide policymakers and practitioners with clear, actionable information on what works, as demonstrated in scientifically-valid studies, that they can use to improve the lives of the people they serve.”
Thus, the site reports on “well-designed randomized controlled trials” across a range of important social issues. They also set out the criteria that they used for considering whether a study is worthy of inclusion on their site (they say only 40-50 studies meet these criteria). Partcularly important is their focus on outcomes:
-Reporting of the intervention’s effects on all outcomes that the study measured, not just those for which there are positive effects.
-For each claim of a positive effect, a reporting of (i) the size of the effect, and whether it is of policy or practical importance; and (ii) tests showing that the effect is statistically significant (i.e., unlikely to be due to chance). These tests should take into account key features of the study design, such as whether individuals or groups were randomized.
-If possible, corroboration of reported effects in more than one implementation site and/or population.
The site provides detail on each study by its theme. So in education, one example is SMART – Start Making a Reader Today; for crime there is Multisystemic Therapy for Juvenile Offenders; in substance abuse DARE – Drug Abuse Resistance Education is shown to be ineffective despite the program’s popularity. On the employment/welfare side there is Riverside’s Greater Avenues for Independence (GAIN), showing a “sizable increase in employment rates and job earnings, reduction in welfare dependency, and savings to the government, especially for single parents.”
I definitely support this sort of research, given the insight it provides into what works and what doesn’t. So it’s great to find a site gathering this information together. However, as an anthropologist, I might also add some caveats. First, there is an almost exclusive US focus, and what works here doesn’t necessarily work elsewhere. Second, the focus is on techniques and outcomes, and not on context, relationships, resources and other things that can also make an enormous social difference. Third, this sort of research is about the workings of specific programs, and not radical change – these programs don’t address the root causes of social inequality or the ideologies that support some in favor of others.
Finally, outcome studies are no substitute for creative thinking, program development, and innovative work. These are still very much needed, so for some ideas there, see some previous posts on Cellphones Save the World; CeaseFire: Violence Prevention and Why Gary Slutkin Is An Anthropologist and Successful Weight Loss.
If outcomes are not the ultimate criterion (preceding criteria being feasibility, ethics, etc.) then for who do we engage in social programs? If recipients are not helped, or even harmed, what argument could be made for a program?
Also:
“these programs don’t address the root causes of social inequality…”
Who has the omniscience to say what exactly are the root causes of social inequality, or what exactly is social inequality? I suspect there are widely divergent thoughts regarding this.
“or the ideologies that support some in favor of others.” Is there any ideology that doesn’t (except maybe…just maybe, nihilism)?
“Finally, outcome studies are no substitute for creative thinking, program development, and innovative work.”
No they are just different parts of the same process cycle.
Thanks for the comment. A story I’ve often heard in different forms comes to mind. Here’s a short version:
“If you come to a river and there are people drowning in the river, compassionate people pull them out, even if they keep coming, people keep pulling them out. You don’t worry about much insomuch as they’re in the river and can’t get out of it – that’s compassion. Eventually, when someone decides to go up river and find out who’s throwing them in the river – that’s justice.”
In this longer version below, the analogy to programs that work would be to figure out how to have better swimmers or more effective ways to rescue people. But that’s not the only way to see the problem, or try to address what to do:
One morning, a man was walking along the river, and he suddenly saw someone struggling to keep afloat in the fast flowing waters. Without a thought for his own safety, he jumped in, swam out to the woman, and pulled her back to the bank, lifted her out, and let her recover.
Gratitude was given, addresses exchanged, and after half an hour, he walked on. Then, unbelievably, he saw someone else in the river. ‘Oh well’ he thought, ‘here we go again.’ The second person was rescued. As both those he had saved were in a state of shock, he decided not to ask how they got in the river in first place.
By now he was pretty tired. He had to get to his destination by mid-day. He would have gone by another route, but the river path was the quickest. He dreaded continuing to look at the river. But look he did, and the screams of another drowning person were so loud, a number of other people had rushed to the river bank. He turned around to the quickly assembling group:
‘Will someone else please help that man? I’m exhausted after rescuing two others this morning. I’m off upstream.’
‘Why,’ said a bystander, ‘you are a strong swimmer, and it seems more people are ending up in the water, we need you here.’
‘That may be true, but I’m going to find out who is throwing them in the river in the first place.’