David Buck has written an excellent blog on The King’s Fund website about the problems of ROI in public health https://www.kingsfund.org.uk/blog/2018/04/return-investment-public-health
I had been having conversations with Greg Fell, about similar issues and challenges, many of which echo those David has expressed in his blog. The below is a brief synopsis of those concerns.
Thoughts on ROI in a changing public health paradigm:
Few would argue against the proposition that good quality evidence on cost-effectiveness is essential if those commissioning public health services are to make informed decisions. This has not changed since the earliest attempts at factoring economics into public health decision making, and is unlikely to change anytime soon. Similarly, the methodologies for assessing the cost-effectiveness of health care treatments and programmes have existed for several years, but were not designed in the realm of public health but rather the realm of pharmacoeconomics, and are firmly rooted in the narrow decision space of competing drug treatments.
In 2007, Drummond et al. published an excellent report on the challenges of applying standard methods of economic evaluation to public health interventions. More than ten years after this report was published, one could reasonably argue that all of the pertinent challenges identified in their paper remain today, though it is perhaps interesting to note that ROI analysis did not feature in their paper, in which a preference for CCA was established where CUA may not be possible or appropriate.
The trend has been for ROI to become synonymous with cashable savings, making the approach attractive to organisations that are being forced to find savings as a result of austerity measures. The technique also holds appeal for those business cases that are being proposed utilising alternative mechanisms of funding such as social investment bonds (SIBs). For local authorities, the key questions given the current financial situation is where does the I come from, and who accrues the R?
If an ROI model is focussed on the reduction of some natural unit, such as CVD events, and the return on investment is expressed as savings from reduced hospital admissions, length of stay, drug treatments, then the R is very much focussed on the NHS as the primary beneficiary. Preventing CVD events is a good thing, but as a health economist in a local government setting I want to know what is the R to not only the NHS and Health and Social Care services, but to the wider City economy. How does preventing CVD have knock-on impacts for multimorbidity? Mental health? Productivity? Inequalities in healthy life-expectancy across the city? How does preventing CVD contribute to making Sheffield a thriving city, and what is the monetary value we can put on that? QALYs only go a limited way towards answering these questions.
I think another reason ROI is problematic is that it creates an alignment problem between public health priorities and framing how the policies and interventions designed to deliver on those priorities are evaluated. Sheffield has a stated aim to be a Person Centred City, and part of that involves more person-centred care that is informed by a holistic and goal oriented approach. Improving aspects of this, such as Patient Activation, will have softer outcomes than £&p that are trickier to value in economic terms. The expectation of positive ROI at the very least, and in reality some implied delivery of cashable savings, can make building business cases difficult. The conversation moves away from “does this option offer greater value than we currently purchase, in which case how do we make the case for disinvestment and reallocation of funds” to “will this produce cashable savings?”. Of course, there are wider pressures of shrinking budgets and austerity looming large over this, but if the consequence of that is a narrowing of thinking to the space of interventions which produce savings, there will be significant opportunity costs (where value is the opportunity cost) for population health at the city scale.
And it is with those city scale policy aims that the alignment issue is most stark. ROI models can tell you why you should be using pedometers to tackle obesity, but they cannot (or don’t yet) tell you a great deal about the value of improving the city cycle lane infrastructure, green spaces, or architectural designs to illicit behavioural change. They can tell you about the value of methadone clinics, but not about the reduction in drug related harm from improved economic inclusivity in a city, or better housing. They can tell you that breastfeeding is a good “early years” intervention, but don’t provide an obvious compliment to a city approach to tackling adverse childhood experiences. These are all policy areas which aim to shift the paradigm away from operational level interventions to more systems thinking and systems (re)engineering.
The challenge for health economics in public health, if it is to help facilitate this paradigm shift, is to embrace this complexity and develop (and adapt from other disciplines) modelling approaches accordingly.