The Chamberlain News

Dr Foster’s day out in the sun: the use and abuse of hospital mortality rates

It was an odd happenstance that Dr Foster – a gentleman best known for his rain-ruined, nursery rhyme expedition to Gloucester – should have his proverbial 15 minutes of contemporary news fame in the middle of last week’s heat wave.

The unfortunate doctor, you may recall, went to Gloucester in a shower of rain. Ignoring the truly excruciating rhyme ahead, he stepped in a puddle, right up to his middle, and never went there again.

The news to which this doggerel relates is, of course, allegedly failing hospital trusts, and specifically Hospital Standardised Mortality Ratios (HSMRs) – the widely used measures of hospital death rates developed, publicised, defended and refined by today’s equally fictional Dr Foster.

The doctor and his HSMRs took a beating in last week’s , National Medical Director for the NHS in England: “However tempting, it is clinically meaningless and academically reckless to use such statistical measures to quantify actual numbers of avoidable deaths”.

Strong words, because this is precisely what had happened the previous weekend, with numerous media claims that the report would be about 13,000 ‘needless deaths’ at the 14 NHS hospitals selected, because of their high mortality rates, for special investigation. It wasn’t. The report contained no such numbers, and instead provided detailed, focused recommendations to assist the improvement of the hospitals’ serious but not irremediable problems.

Sir Bruce’s report had been calculatedly hijacked, but who he held chiefly responsible – Ministers and their advisers, the media, even some collusive involvement of Dr F himself – was unclear. The outcome, sadly, was unmistakeably clear. Health Secretary Jeremy Hunt’s parliamentary presentation of the report became a shameful partisan blame-fest – so depressing for so important a topic that, as a completely non-expert observer but low-key Dr Foster fan, I was moved to attack my keyboard.

My own first encounter with Dr Foster was in January 2001. I was teaching a university course on policy research methods, and in, of all places, a two-part Sunday Times supplement, there appeared some perfect raw material for a student assignment: the first ever listing of standardised ‘death rates’ (HSMRs) for England’s or any other nation’s hospitals.

So what, my students discussed, did these ratios really measure? What did they include, and exclude? Who’d collected and analysed the data? How did they relate to other measures of a hospital’s care and performance? What was the range, and where were the highest and lowest ratios – that ‘Where?’ question providing an additional reason for my recalling that first Dr Foster’s Good Hospital Guide.

A hospital’s Standardised Mortality Ratio is usually presented as a percentage: the recorded deaths in hospital from most (but not all) diseases, as a percentage of the number that would normally be expected, after taking account of, or standardising for, a wide range of factors concerning the patients and their illnesses.

HSMRs’ other key feature, consistently misunderstood, is that they measure hospitals not against some objective clinical standard, but against each other. An HSMR of 100 is the national average; below 100 means fewer deaths than statistically expected; over 100 means more. Not needless or avoidable deaths, not deaths from incompetent care, simply more than expected. Even if all hospitals were good, half would still have ratios of 100+ and look ‘bad’ – and vice versa.

The Dr Foster Guides and website emphasise these points scrupulously. A high HSMR should be treated as a warning: a risk, but not proof, of failings in care, and reason for further investigation, with attention focusing mainly on ‘outliers’ – those outside, especially if repeatedly outside, the normal range. University Hospitals Birmingham NHS Foundation Trust’s HSMRs, though consistently over 100, are thus less immediately concerning than the 130+ ratios of Basildon & Thurrock (2005-09) and Mid Staffordshire (2005-07).

These literally health warnings, though, are quickly elbowed aside in the dash for political advantage or media headlines. So, although those first hospital ratios weren’t listed in league table format, they were quickly sorted into one and the range calculated.

It was wide and, although all mortality rates have fallen significantly in the past decade, it remains wide today. Then, University College London Hospitals had the lowest ratio of 68, but two of the three highest were in the West Midlands: Walsall Hospitals Trust with 119 and Sandwell with 117.

My recollection is that these hospitals and trusts had little advance notification of their figures. Certainly, there were widespread protests – by those assuming that, if this was a ‘Good Hospital Guide’, high-ratio hospitals must be ‘bad’. However, despite their susceptibility to such misinterpretation, HSMRs were here to stay. Which begged the obvious question: who was this pioneering but troublesome Dr Foster?

As already indicated, there is no actual Dr Foster. The name was the whimsical invention of two journalists involved in producing the 2001 Sunday Times supplements. But, if there were a real doctor, the only possible candidate is someone you may well have seen recently on your TV screens, Professor Sir Brian Jarman.

A distinguished Imperial College academic, he developed the ‘Jarman Index’ – a formula for distributing government funding to the nation’s hospitals – which gradually evolved into the HSMR, a formula for identifying a hospital’s share of responsibility for its death rates. It was a major statistical advance, but the then Health Secretary was nervous and refused Jarman permission to publish individual hospitals’ HSMRs.