Evidence-based policy movement "unhelpful and unscientific"
29 May 2015
Dr Flora Cornish, Associate Professor in Qualitative Research Methodology at the London School of Economics, on why the aspirations of the evidence-based policy movement for bottom line answers are unscientific and unhelpful when evaluating the efficacy of community mobilisation in tackling HIV.
The holy grail of evidence has become all pervasive in the global HIV response. The drive for universalising knowledge on what works, derived from the emergence of ‘high quality’ evidence for biomedical prevention technologies, is now so entrenched in policy that sensitive appraisals of the way in which interventions are engaged with in different contexts are being overlooked.
Nowhere is this more apparent than in debates around community mobilisation. Two studies I conducted in collaboration with the International HIV/AIDS Alliance fundamentally contest the desire to want to evaluate community mobilisation as if it is a simple technical component that either works or does not.
Working across 14 countries, our first study – a reflection on current practices in Linking Organisations of the International HIV/AIDS Alliance – found community mobilisation to be many things. Indeed, its strength is in its flexibility and adaptability. In Uganda, the Community Alliance for Health (CHAU) employs people living with HIV to link other positive people to health services, act as their medical advocate and provide adherence support. In Indonesia, Rumah Cemara supports people who are homeless and use drugs to take part as a team in the Homeless World Cup. In Latin America and the Caribbean, REDTRASEX brings sex workers together to fight for their human rights. With representatives on Colombia’s and Costa Rica’s Global Fund Country Coordinating Mechanisms, and regular consultation with UNAIDS and governments, they are making a significant impact.
In their different ways, each of these organisations is building the kind of social networks that help people to be healthy. Moreover, being so entrenched in these communities allows, or indeed forces, these organisations to understand, and respond to, local contexts and needs.
The findings of our second study, a systematic review of the evidence on community mobilisation’s efficacy, were inconclusive. This was particularly true of our review of general population studies although we were more positive about studies on most at risk groups, particularly sex workers. This is not surprising perhaps, given the diverse nature of community mobilisation – but often, what is being evaluated is, at best, a weak version. For instance, some studies labelled support groups as community mobilisation, a hugely different intervention from the diverse social networks being built by the organisations in our first study.
Crucially, a systematic review of the evidence cannot answer the crude “what works” question. It cannot tell us whether, if sex workers form an action committee, local police will follow the rule of law and local officials will allow them to access welfare. This appears to be a good strategy in parts of India and Latin America but in South Africa it has proved almost impossible to forge a sense of collectivity among sex workers who compete desperately for needed custom, and none of whom feel entitled to state benefits.
In their book Evidence Based Policy, Nancy Cartwright and James Hardie, deplore the tendency to assume that because “it worked there” it will “work here”. They argue that recourse to evidence cannot be used as a substitute for the hard work of thinking through what is needed to make something work in a particular context. Similarly, in the British Medical Journal, Trisha Greenhalgh, a professor of Primary Healthcare, recently called for “real evidence-based medicine” which emphasises ethical treatment, individualised care and expert judgement rather than mechanical rule following. Analogies can easily be made for ‘real evidence based policy/practice’ for community health interventions with an emphasis on ethics, local responsiveness, and careful judgement of what might work and how, given the context.
Yet evidence of technologies’ effectiveness will not be sufficient to make a difference to the epidemic alone. Technologies are not just ‘rolled out’ – unfurled like a carpet to smoothly cover an affected area with a uniform surface. Technologies can be provided, or offered, but they have to be engaged with, appropriated, demanded, adopted, committed to, and made to work. A Randomised Control Trial can tell us that, under ideal conditions, PrEP can reduce HIV transmission by 50% or 90% but it can’t tell us how to achieve these ideal conditions so that people take their medicines consistently.
Interventions do not just work automatically, they have to be made to work – and people have responsibility for making them work. And herein lies the rub – for the question we should ask is not “what works?” but “what are we committed to making work?” To be able to answer that we must be prepared for the hard work of gathering evidence in context, and mobilising the personal and political will to make interventions work against the backdrop of adverse circumstances and complicated lives.
This blog is adapted from Flora’s presentation at a panel debate held at the London School of Economics in October 2014. Watch the podcast & video.