New approaches to evaluating complex health and care systemsBMJ 2016; 352 doi: https://doi.org/10.1136/bmj.i154 (Published 01 February 2016) Cite this as: BMJ 2016;352:i154
All rapid responses
I find the idea of 'evaluation' of complex health systems interesting: presumably, in the real world this amounts to a comparative evaluation between two or more different health systems [or different 'health system models']. So you would end up with which is 'better', but not a description of 'the best possible system', from such an evaluation.
My instinct is to question the application of such understanding, even if the methodology of the evaluation is robust: because most health systems are much influenced by politicians, and politicians are not known for 'simply following the evidence', are they ?
The 'complexity' I keep coming across, and analysing, in my end-of-life 'campaign' is probably of a different type, and hinges on conflicts between the beliefs and objectives of the different 'players' surrounding the dying patient. Put simply, there is an enormous amount of 'complexity' because relatives, 999 paramedics, GPs, trust lawyers, angry bereaved relatives, etc, 'see different problems, and also weight the importance of those problems they all agree exist, very differently'. I'm not convinced that you need a sophisticated 'model' to identify these differences [I am not very sophisticated, and I can see these conflicts very clearly] - what you need, is a 'sophisticated solution' if the differences are not going to inevitably prevent the creation of balanced and joined-up behaviour.
In that sense I agree with JK Anand (this series of responses 8 February) - I think patients would be better helped by thinking harder about the behaviour closest to the patient. The further from the patient, the further from the service user's reality - as I wrote to a professor:
'I can assure you that arguing the toss around EoL issues with professors 'at leisure', is not the same as trying to get on the same wavelength with paramedics and police officers 'in stressful situations'.'
Competing interests: No competing interests
If the distinguished authors were to return from the excursions into these rarefied researches back to treating patients, I think we would be more grateful. As for Prof Bryony Soper's rapid response - I found it hard to understand.
The McKinseys (1974 reorganisation), the RAND Corporation and such like will, I am sure, be impressed.
Perhaps I am in a minority of one in expressing such heresies. I await counter responses with interest.
Competing interests: Patient. Like the nation's money to be spent on treating patients.
The recent paper by Lamont and colleagues on evaluating complex health and care systems starts with a clear call to action, “complexity needs to be embraced, not eliminated” - and the discussion on which it is based was clearly influenced by complexity thinking. Throughout the paper there are implicit references to key characteristics of complex systems, such as emergence (“planned service changes can change over the lifetime of a project”), temporal ordering (the paper talks about “which preconditions make certain outcomes more likely”), and interactions between agents as components of complex systems (for example the paper mentions the co-production of the objectives of an evaluation by researchers and participating sites). Complex adaptive theory is mentioned in passing.
Because this paper is very short it can do no more than set markers for further discussion, and this inevitably leaves much to be addressed. One key issue is the nature of the role that complexity science can and should play in evaluating healthcare systems. Here, I believe, there is an important dimension that this paper fails to mention.
The extent to which complexity thinking challenges existing methodologies and raises deeper issues (such as the nature of causality and the limitations of knowledge) should not be under-estimated. There is a huge literature on complexity science, much of it devoted to ever more sophisticated computational models. For all that - or perhaps just because of that - George Cowan (former president of the Santa Fe Institute that developed the concept of complex adaptive systems) has been quoted as saying: “It's very hard to do science on complex systems". Cowan was, of course, talking from a particular perspective - an approach that recognises the limitations of “Newton’s clockwork universe” but that nonetheless explores complexity within the traditions of science, i.e. science as the process of discovering and modelling the rules and laws that govern the behaviour of all phenomena , and draws on work on fractals and chaos theory. This work has been linked with systems thinking, such as general system theory , and with the increasing use of statistics to model social realities such as voting patterns . It is this understanding of complexity and the methodological tools that it employs that has had most influence on the evaluation of healthcare systems. ,
But others, while noting its usefulness, have also noted the conceptual and practical limitations of this approach - especially when applied to complex social systems - and have postulated a more radical understanding of complexity. Influenced by earlier thinking on complexity (including a particularly prescient paper by Warren Weaver ), this understanding distinguishes between ‘restricted complexity’ (the approach described above), and ‘general complexity’. The latter, it is argued, “is not merely a methodology; it involves a rethink of our fundamental definitions of what knowledge is. When dealing with complexity, the traditional method of analysis does not work”.
Among those who have written on general complexity Edgar Morin is, arguably, its foremost theoretical proponent. He writes in French and his work is only available to most of us in translation, but it is noteworthy for its breadth of vision and beauty of expression. On restricted complexity he writes:
“Restricted complexity made […] possible important advances in formalization, in the possibilities of modelling, which themselves favour interdisciplinary potentialities. But one still remains within the epistemology of classical science. When one searches for the “laws of complexity”, one still attaches complexity as a kind of wagon behind the truth locomotive, that which produces laws. A hybrid was formed between the principles of traditional science and the advances towards its hereafter. Actually, one avoids the fundamental problem of complexity which is epistemological, cognitive, paradigmatic. To some extent, one recognizes complexity, but by decomplexifying it.”13
The theoretical challenges are immense. However, the concern of Lamont and her colleagues in their paper is with practice. In these terms too the notion of general complexity has important implications, and these are beginning to be explored. Thus in a recent paper David Byrne covers much of the ground discussed by Lamont et al, and considers how the use of methodologies such as realist evaluation, process tracing, and systematic case comparison can help us, in his words, “get a handle on what works where (in what context), when (in what temporal context), and in what order” .
Given all the above, it is, I suggest, important that insights from general complexity are used to inform further debate on evaluating complex health and care systems as well as those from the more familiar territory of restricted complexity.
Competing interests: No competing interests