Thoughts on ROI in a changing Public Health paradigm

David Buck has written an excellent blog on The King’s Fund website about the problems of ROI in public health

I had been having conversations with Greg Fell, about similar issues and challenges, many of which echo those David has expressed in his blog. The below is a brief synopsis of those concerns. 

Thoughts on ROI in a changing public health paradigm:

Few would argue against the proposition that good quality evidence on cost-effectiveness is essential if those commissioning public health services are to make informed decisions. This has not changed since the earliest attempts at factoring economics into public health decision making, and is unlikely to change anytime soon. Similarly, the methodologies for assessing the cost-effectiveness of health care treatments and programmes have existed for several years, but were not designed in the realm of public health but rather the realm of pharmacoeconomics, and are firmly rooted in the narrow decision space of competing drug treatments.

In 2007, Drummond et al. published an excellent report on the challenges of applying standard methods of economic evaluation to public health interventions. More than ten years after this report was published, one could reasonably argue that all of the pertinent challenges identified in their paper remain today, though it is perhaps interesting to note that ROI analysis did not feature in their paper, in which a preference for CCA was established where CUA may not be possible or appropriate.

The trend has been for ROI to become synonymous with cashable savings, making the approach attractive to organisations that are being forced to find savings as a result of austerity measures. The technique also holds appeal for those business cases that are being proposed utilising alternative mechanisms of funding such as social investment bonds (SIBs). For local authorities, the key questions given the current financial situation is where does the I come from, and who accrues the R?

If an ROI model is focussed on the reduction of some natural unit, such as CVD events, and the return on investment is expressed as savings from reduced hospital admissions, length of stay, drug treatments, then the R is very much focussed on the NHS as the primary beneficiary. Preventing CVD events is a good thing, but as a health economist in a local government setting I want to know what is the R to not only the NHS and Health and Social Care services, but to the wider City economy. How does preventing CVD have knock-on impacts for multimorbidity? Mental health? Productivity? Inequalities in healthy life-expectancy across the city? How does preventing CVD contribute to making Sheffield a thriving city, and what is the monetary value we can put on that? QALYs only go a limited way towards answering these questions.

I think another reason ROI is problematic is that it creates an alignment problem between public health priorities and framing how the policies and interventions designed to deliver on those priorities are evaluated. Sheffield has a stated aim to be a Person Centred City, and part of that involves more person-centred care that is informed by a holistic and goal oriented approach. Improving aspects of this, such as Patient Activation, will have softer outcomes than £&p that are trickier to value in economic terms. The expectation of positive ROI at the very least, and in reality some implied delivery of cashable savings, can make building business cases difficult. The conversation moves away from “does this option offer greater value than we currently purchase, in which case how do we make the case for disinvestment and reallocation of funds” to “will this produce cashable savings?”. Of course, there are wider pressures of shrinking budgets and austerity looming large over this, but if the consequence of that is a narrowing of thinking to the space of interventions which produce savings, there will be significant opportunity costs (where value is the opportunity cost) for population health at the city scale.

And it is with those city scale policy aims that the alignment issue is most stark. ROI models can tell you why you should be using pedometers to tackle obesity, but they cannot (or don’t yet) tell you a great deal about the value of improving the city cycle lane infrastructure, green spaces, or architectural designs to illicit behavioural change. They can tell you about the value of methadone clinics, but not about the reduction in drug related harm from improved economic inclusivity in a city, or better housing. They can tell you that breastfeeding is a good “early years” intervention, but don’t provide an obvious compliment to a city approach to tackling adverse childhood experiences. These are all policy areas which aim to shift the paradigm away from operational level interventions to more systems thinking and systems (re)engineering.

The challenge for health economics in public health, if it is to help facilitate this paradigm shift,  is to embrace this complexity and develop (and adapt from other disciplines) modelling approaches accordingly.


Much of my working life has been taken up by figuring out if innovations are just that. As a health economist I’ve worked on NICE clinical guidelines, the uptake of NICE appraised medicines as part of the PPRS programme and the innovation, health and wealth agenda. I now work in as a health economist in a public health team in a local authority where the principals of economic evaluation remain the same, but the decision-making context is quite different and much closer to the coalface.

Over the years I have seen many things branded and hyped as innovations unravel as we learn more.  Take the NOACs. The heady claims about the reduction in INR monitoring requirements, which would free up cash and reduce bleeds have gone from shiny and beacon-like to murky and grey as the evidence has unravelled and questionable fiddling of the numbers has emerged.

Then there are surgical robots and laser-assisted cataract surgery – two “innovations” that cost a fortune; have the potential to do harm; and yet have been purchased by the NHS at enormous costs for no real evidence of benefit.

But all this time evaluating shiny new health technologies with questionable beneftis was brought into sharp focus when my dad met Neil.

In November 2016, my dad, at 72 years old came home from his weekly game of walking football feeling a bit tired. He’d been bricklaying with me at my house, and was finishing off the last of the autumn gardening tasks so put his fatigue down to that.

Dad was not your average 72yr old. His longstanding neighbour was fond of describing him as a “bull of a man, for as long as I’ve known him”. Dad had aged well – he was fit and strong.

When he passed out on the floor of the GPs office during a blood test, with a high fever and flu-like symptoms later that November everything changed. I got an email at work from my sister-in-law telling me that he had been taken to hospital, protesting that he was fine all the way to emergency in the back of the ambulance.

I went to see him in the acute admissions unit that evening.  He’d had some antibiotics and was still tired but feeling okay. He wanted to go home. The doctors were worried that his bloods showed some anomalies in his liver function tests (LFTs). He also had a swollen gland in his neck that was starting to look almost like mumps. His fever was tracking up and there were murmurings about sepsis.

Dad started to get leg pain that night. It was bad. Bad enough to need a whack of morphine in addition to the intravenous paracetamol he was on for his fever, which was still high.

He was moved to a hepatology ward; his LFTs were all over the place. He was visibly yellowing. His pain was getting worse.

CT scanned. More swollen lymph nodes in his chest and abdomen. Infection? Billiary sepsis? Biopsy of the lump in the neck followed. Dad was getting really poorly. Third of fourth round of broad-spectrum antibiotics. More scans. A vegetation on a wire on his pacemaker causing endocarditis? Stones in the bile duct? Ten days in hospital. We rallied around. “Now dad, come on mate….you can fight it off….”

The leg pain was now excruciating. Dad would writhe in bed at night. Pain relief came but wasn’t lasting. He was losing weight rapidly. Fighting back tears, Dad told me, in a hushed voice, that when the pain came at night there was a blackness in the corner of the room, a black gaping hole that he could feel coming for him, pulling him in.

He looked at me for a moment and then broke down and sobbed, for several minutes, while I held his hand. He didn’t have the strength to sit up for a hug, so I leant over and did the best I could to get an arm around him. I told him that my brother and I had spoken, and that we wouldn’t leave him alone at night. Visiting hours stopped at 10pm, but we would not be leaving. The nurses understood.

Two weeks later – crash.

Sodium down to 114 – hyponatraemia. I went in at 1am and Dad was on HDU, delirious. Slipping in and out of consciousness. He didn’t recognise me. Called me “Malcom” and said that under no circumstances should I trust my brother, and that a helicopter was coming to take him out of here to Derby.

Sometimes he was just unintelligible, mumbling incoherently. My brother, sister and I took shifts, one would stay until midnight then the other would stay overnight. We alternated our shifts, slipping into a different circadian rhythm than the world around us. We’d talk about how our shift with dad went, handover the details. There was never much progress to report. A hug and then see you later, then sitting in the chair by the bed, waiting. At least, we thought, his delirium meant that he was no longer being wracked with pain in the night. Mum was struggling to cope. We had to care for both parents now.

About ten days later and his sodium levels were tracking back up. It was nearly five weeks in hospital now, and for all that time Dad had been lying down. His belly was swollen with ascites, made worse by the fact that the rest of him had wasted away so rapidly.

Just as he was back on a ward and out of the high dependency unit (HDU), we got a call that the family should come in for a meeting. We were told that dad’s biopsy results had initially been indeterminate, and that they had sent them over to the labs in Sheffield for a second opinion. The news wasn’t good. Profuse B-cell lymphoma, EBV positive. At least we had a diagnosis. The kicker was dad was too ill to start the chemo. It would kill him, unless a week of steroids could get his health back up to a level at which he’d tolerate the first round.

The first two lots of chemo where given in hospital, dad was to be treated as an outpatient for the rest.

But dad was not in good shape. His legs, still painful, had wasted considerably. The combination of the infection, the lymphoma, and the subsequent chemotherapy caused extensive neuropathy. He couldn’t walk. We took him home with a wheelchair.

Over Christmas, including on Christmas day, there were more admissions. Lots of fever spikes, intravenous antibiotics and the like. More time lying down in bed. Some distressing moments when dad couldn’t get to the toilet in time.

One night, while I was changing his diaper and making sure he had the requisite number of cardboard flutes to pee in before I headed home, dad said to me that all he wanted was three things. He wanted to be well enough again to get up to the top of the garden and tend his vegetables. He wanted to be able to walk in the Yorkshire Dales one more time, and he really, really wanted to be able to pick up and play with his youngest grandson – my son who was 18 months old.

About early February, dad’s chemo regimen was changed as a couple of the drugs were not getting on with him. His condition improved on the fevers front, and he spent a good stretch as an outpatient. During that time, a member of the support and recovery team came out and gave dad regular physiotherapy and mobility exercises.

Neil (not his real name) visited dad two to three times a week and over the course of around seven weeks got him from a wheelchair to taking his first, tearful, joyful steps across the living room. Neil talked to dad about his goals and about the things he wanted to be able to achieve in the limited time they would have working together.

Neil sorted out hand rails in the house, got dad a better wheelchair that we could get him out and about in, and progressed him onto crutches gradually. There was pain, frustration, disappointment at the slow progress but there was also success and joy and determination. Neil supported dad through all of that in a way that we couldn’t. The relationship was different.

The sad thing was that every session had to be a bit rushed. Neil had other patients to see, and his service was being cut back – hard by the trust. As a result, Neil and his team were struggling to keep up with the demand and couldn’t give the time they really wanted to give to their patients.

Dad wanted more help – he could feel the difference it was making and we could all see the changes in him. Our dad was coming back to us, one day at a time, one more shuffled step with Neil holding onto his arm as they went from chair to window and back. The doctors and nurses and drugs had all helped and worked miracles to get him into remission, but it was Neil who was getting dad back to being dad again. It was Neil that was getting dad back up the garden to see his veg. Neil meant dad could walk to and cuddle my son, again. Neil that was giving dad a glimmer of hope that he might get out walking in his beloved Yorkshire Dales again.

The pressure on Neil and his colleagues is the price we pay for “innovation”. Do we want a health system that purchases Michaelangelo robots and FemtoSecond Lasers, which are shiny and new and high-tech but offer no real benefit  – and may even harm. The opportunity cost of spending our health service pounds on these shiny prestigious gadgets are Neil and his colleagues and the health they create.

But is this message getting through? I saw an interesting poster presentation at UCL the other week that highlighted the key characteristics of decisions to commission “innovations” in healthcare.



Given what we know about Duodopa, NOACs, FemtoSecond Lasers, and surgical robots and the decisions to purchase them, we can infer that cost-effectiveness remains poorly understood by decision makers.  People are not therefore thinking in terms of opportunity costs. There is urgent work needed here.

Why then are we prioritising spending the way we are? Ask dad what the most significant part of his treatment was, and he’ll tell you straight away that it was Neil’s visits to the house that made the difference. Without that, he would have been lost.

Innovation has many guises. Innovative ways of thinking about the hospital system. From the point of admission to the successful discharge and rehabilitation of people in a place of their preference then linking up that system seamlessly with health and social care system. We need to emphasise the importance that people doing Neil’s job have in linking that all together as a person centred, goal oriented approach to recovery and rehabilitation. That’s where the real value in innovation sits. It doesn’t fit neatly into a Markov model, or have fancy branding and the backing of a pharma company that’ll send you to Honolulu for a conference. But it is the kind of innovation that we should be judging against all the other “innovations” that do.




Thanks to Deborah Cohen for proofreading and very helpful edits.