Views And Reviews No Holds Barred

Margaret McCartney: Innovation without sufficient evidence is a disservice to all

BMJ 2017; 358 doi: https://doi.org/10.1136/bmj.j3980 (Published 05 September 2017) Cite this as: BMJ 2017;358:j3980

Could Babylon please supply evidence ?

It's a shame that Dr Butt does not use his response to put evidence supporting Babylon's technology forward.

It's no secret that I support the founding values of the NHS and my DOI has been publicly available for several years. I've even written a whole book about the State of Medicine - keeping the promise of the NHS. In it, I argue that non evidence based policy making, and serial IT innovations, have wasted money and provided poor value for patients.

I have been asking for details of the pilot in North London since March 2017. I have been given no details about the evaluation and how safety and accuracy will be assessed. I would be grateful if Dr Butt would supply this. Dr Butt says they have been publishing their results. Neither NHS London or Babylon could direct me towards any publication of any results and only this paper seems to have been published https://arxiv.org/pdf/1606.02041.pdf which is not a real life study of use. I have also been trying to find out what the financial arrangement is between the NHS and Babylon: no information has yet been forthcoming and I would be grateful if Dr Butt could supply these.

Additionally, Babylon are still saying on their website under 'is it safe?' that "An independent study tested the app’s symptom checker against nurses and junior doctors. It found that the app gave safe advice 100% of the time, whereas doctors gave safe advice 98% of the time, and nurses gave safe advice 97% of the time. It also found that the app is more accurate than doctors or nurses, sending patients to the most appropriate place for their more often than either doctors or nurses. This means that the app not only gives safe advice, but also saves patients from spending time in Accident and Emergency or at their GP’s surgery when this isn’t necessary."https://support.babylontech.co.uk/hc/en-us/articles/115000931729-Is-it-s...

This was not an independent study; it was not a real life study; it is not capable of examining real life harms.

New technology should be treated like any other medical intervention capable of benefit and harm: it should be tested in high quality trials capable of finding unintended harm as well as benefit. When one asks for evidence but is rebuffed with a supply of popularity scores I think I am right to be concerned.

Competing interests: I wrote the article, and I have and will campaign for the founding values of the NHS; DOI http://www.whopaysthisdoctor.org/doctor/6

08 September 2017
Margaret McCartney
GP
writer of article