Effects of Sure Start local programmes on children and families: early findings from a quasi-experimental, cross sectional study

Dear Ms Godlee

As a Member of Parliament whose constituency contains significant
pockets of social deprivation (like many seaside and coastal towns) and
which now has three Sure Start projects (two of them estate-based)
operating as part of the Government’s strategy to combat this, I was
naturally concerned to see media reports that the report you published in
your June 16 issue of the British Medical Journal by Jay Belsky and his
colleagues was suggesting that Sure Start projects were making things
worse, not better, for some of those who used them.

I have therefore now read carefully the piece that you carried, which
seems to be very far from substantiating the way in which it has been
treated in the non-specialist media. Unfortunately it also seems to me
that the conclusions Jay Belsky and his colleagues draw from what they
themselves describe as ‘early findings from a quasi-experimental cross
sectional study’ have also been too sweeping and have contributed, however
unwittingly, to the distorted headlines and intros that appear in, for
example, the Daily Mail of June 16: ‘Flagship project to help deprived
children and families is making crime and truancy worse, not better, an
alarming investigation revealed.’

I accept as a layman that others may be better placed than I to
critique some of the details and protocols that this study used. But
apart from my constituency interest – which has given me close contact
with the Sure Start centres and meetings with staff, support workers and
parents using them – I have followed these issues closely as a member of
the Commons Education Select Committee and national vice-president of
Early Education. I also – in a previous existence – spent nearly five
years working as a public relations adviser on medical and related
subjects, followed by twelve years as a magazine editor, which gave me a
crash-course on the complexities and sensitivities of interpreting
research presentations to a non-specialist audience and the pitfalls that
lie in wait for the unwary.

In the case of the assertions in the Belsky report that ‘children
from relatively more socially deprived families…were adversely affected by
living in SSLP’ – the assertion leapt on by the Daily Mail and others –
there appears to be little substantial material elsewhere in the report to
back it up. We are told that children of teenage mothers, like those who
lived in workless or lone parent households, ‘scored lower on tested
verbal ability’ – hardly surprising given that such mothers often have
themselves had schooling and family difficulties. But the crucial
question – how does this compare with their skills before they entered the
Sure Start programme – does not appear to have been asked. Sure Start
programmes have come in successive waves – some of those interviewed may
have been in them for two to three years, some for much less. Again, no
allowance appears to have been made for this in the evidence presented.

It is surely a fundamental principle that in order to make valid or
worthwhile judgements about how well the Sure Start programmes have
performed – either overall or for particular groups – you have to look at
the participants before and after entering the programmes to measure the
progress – or non-progress – between A and B. The Belsky article does not
appear to do this. We are told that ‘mothers of children aged 36 months
(but not 9 months) living in SSLP areas rated their communities a little
less favourably than mothers in comparison areas.’ Again, this is hardly
surprising, given that pre- and continuing publicity for such Sure Start
projects – as here in Blackpool – would rightly have highlighted they were
being set up to help families in areas with elements of deprivation and
social exclusion. It is not an evidential basis on which to judge their
effectiveness.

I would add that the ability of the non-specialist – and possibly
many of the specialist – readers to penetrate and assess the Belsky paper
is not helped by linguistic monstrosities such as on page 2: ‘the findings
presented are based on multiple imputed data sets in which missing values
of all independent and dependent variables were estimated based on
standard multiple imputation procedures.’ I would hope that contributors
to the BMJ, as a long-established journal of record with an impressive
pedigree in communicating to a broader public as well as to the
specialist, could do better than that!

Jay Belsky and his colleagues have reported positively elsewhere in
their article on other aspects of the Sure Start programme and, to be
fair, issue their own health warning about the paper: ‘because this
evaluation was quasi-experimental, cross sectional, and evaluated the
impact of a programme that had been in place for only a few years, the
detected effects of SSLPs and the conclusions much be treated with
caution.’ Wise words, but the presentation of the paper’s conclusions did
not follow them – and the broadsheet coverage which exploited them
certainly did not.

This is not just a specialist issue about academics’ findings which
can cheerfully be discussed and argued over at learned symposia and
conferences. The impact on the morale and self-esteem of the people using
and involved with Sure Starts countrywide – including my constituents –
will be affected by the coverage of papers such as this. I do believe
that academics and researchers have a public responsibility to consider
how they present such material – and to be acutely aware of how it may be
misinterpreted or abused. It is hardly as if in taking on the Sure Start
project the authors of your paper would have been unaware of the high-
profile and public sensitivities of what they wrote. I hope they will
reflect on how their study has been presented and make some effort to
address the distortions and the issues I have raised here.

Yours sincerely

Gordon Marsden

Competing interests:
None declared