The potential and the problems of public health economic modelling

Why model?

Public health activities often not only offer value for money, but can also save money, yet local authorities in England and their associated public health budgets are one of austerity’s losers. Shrinking budgets mean that choosing how to best spend finite resources to maximise population health is increasingly important.

At the same time, diet and physical activity remain two of the leading causes of disease in England, resulting in morbidity and premature mortality both directly from conditions such as diabetes and heart disease, and indirectly via risk factors such as high blood pressure and obesity.

My DPhil aims to help local and national policy makers in England decide where to put their limited resources for the greatest population health benefit.

Selecting an appropriate method

To estimate how cost effective a public health policy might be, a model or study needs to quantify the costs and savings of an intervention and its potential effects on health. However, there is little consensus about what structure public health economic models should adopt and what inputs and outputs should be included. This is in part due to some of the specific challenges faced by the field; for example, benefits may take many years to be realised and there may be an incomplete understanding of the complex systems in which the public health intervention operates.

Numerous public health economic models have been developed, each designed to address a different public health economic problem, and it is usually not possible to directly compare results between models. This is because different models use different data and have different structures which, independent of the public health policy being analysed, can affect the final cost-effectiveness estimate.

For example, baseline input population data and the parameters describing how an intervention affects a risk factor and disease outcomes may vary. Likewise, costs and morbidity scores assigned to different disease states often differ. Added to this there is nearly always scope to model additional complexity depending on the amount of time and data available. For example, adding time trends in diseases and risk factors, interaction terms and feedback loops, and more diseases and costs.

The resulting differences between models makes it very difficult for decision makers to choose how best to spend their money. Does a policy examined using one model really offer a better bang-for-buck than another policy investigated using another model, or are the different results simply due to the model used?

Overcoming comparability challenges  

I hope that PRIMEtime CE is a tool that can help solve this problem by directly comparing the cost-effectiveness of very different public health interventions. It was developed from a model called PRIMEtime which uses multiple lifetables to estimate how different diets affect the incidence of cardiovascular disease, cancers, and diabetes. As part of my DPhil, I added physical activity as a risk factor, NHS costs, social care costs, and utilities to the model, as well as some other tweaks to make it a functional cost-effectiveness model.

When PRIMEtime CE was developed, a guiding principle was to make sure that data used across the multiple diseases and risk factors included were comparable – using the same sources and methodology where possible. The result is that policies analysed using the model are comparable with one another and any biases and limitations are shared equally. Therefore, although decision makers will never know exactly how much money they will save or how many fewer people will have a disease following an intervention, they will at least be able to have some confidence that one intervention is likely to be more cost effective than another.

Next steps

Moving forward, significant work has already been done to try and validate PRIMEtime CE although this is not without its difficulties (for reasons I won’t go in to here). However, if I’ve learnt anything from the process it’s that the choices made about what model structure to use, what data to put into it, and what baseline settings to choose can have a major impact on the results.

Therefore, being explicit and transparent about these choices is crucial, particularly for the decision makers trying to make sense of it all.

In terms of the future of this work, PRIMEtime CE has been presented to and shared with a range of stakeholders, from the Department of Health to local authorities. Following feedback, future work will focus on adding in more risk factors and adapting the model to be better suited to local and regional analyses.

For me, the past three years have been a valuable insight into public health economic modelling, both in terms of its value and its limitations. However, more complexity does not always mean a model is better. To paraphrase Prof Chris Whitty when he stepped down as Chief Scientific Adviser to the UK Department for International Development (and to quote from one of our recent papers): “a model that is simple, timely, and lays bare its problems is far more useful to a policymaker than one that is more detailed, more complicated, possibly more accurate, but less interpretable and arrives after the policy decision is made.”

Having recently finished his DPhil, Adam Briggs has returned to the Oxford public health specialty registrar training scheme and is currently spending a year in the US as a Harkness Fellow in Health Care Policy and Practice funded by the Commonwealth Fund. Before public health training, Adam studied Natural Sciences at Cambridge University and then moved to Oxford University to study Medicine. He has an MSc in Global Health from the University of Oxford and worked in London and America before moving back to Oxford to pursue a career in Public Health.