Intended for healthcare professionals


The UK’s Research Excellence Framework 2014

BMJ 2012; 345 doi: (Published 21 November 2012) Cite this as: BMJ 2012;345:e7797
  1. Graham Watt, professor
  1. 1Department of General Practice, Division of Community Based Sciences, University of Glasgow, Glasgow G12 9LX, UK
  1. gcmw1j{at}

A tool with many uses including, now, assessment of the usefulness of research

In 2014 the United Kingdom will conduct its seventh universities research assessment exercise in which it will assess, rank, and reward universities according to the excellence of their research between 2008 and 2013. As usual there are changes to previous processes, which include a new name (the Research Excellence Framework, or REF), an attempt to reduce costs, and an increased focus on the impact of research.1

As an exercise in accounting and rationing, the process enables the four higher education funding bodies in the UK to allocate nearly £2bn (€2.5bn; $3.2bn) of research funding—about 30% of all university research funding each year—on the basis of peer review of research outputs. The involvement of more than 140 of the UK’s most senior health researchers in six subpanels will lend legitimacy and authority to the task.

A researcher’s four best publications (described as “outputs”) during the period will be assessed for their originality, importance, and rigour. They will then be graded as world leading (4*), internationally excellent (3*), internationally recognised (2*), nationally recognised (1*), or unclassified. Only a small number of research studies can really be considered world leading or internationally excellent, especially in health services research, which is usually concerned with local issues. However, grades 4* and 3* are the ones that count in the REF. Anything below 2* will attract no funding and too many 2* papers will weaken a submission.

After the 2008 research assessment exercise, about 50% of the available research funding was allocated to the “top 10” universities.2 A great deal of effort was therefore expended to produce fine gradations between universities and to distribute the remaining funding. Mindful of the cost of the exercise, assessed at £47m in 2008 (about 0.5% of the value of public research funding), with little change from the previous exercise, REF 2014 will involve half the previous number of panels and subpanels.3

UK research compares well internationally and produces more publications and citations per British pound of public funding than any other G8 country. Because other countries do not carry out national research assessment exercises, such comparisons do not depend on REF data. The “reputational yardsticks” and “valuable benchmarking information” that the REF produces are therefore for local consumption and are increasingly used to monitor and manage research activity within institutions. Although at a national level the REF results are not concerned with individual researchers, within institutions that are seeking to concentrate and prioritise their resources the focus is very much on individuals.

An important new feature of REF 2014 is its focus on research impact, which will account for 20% of the “quality profile” to be awarded to each submission.2 After substantial piloting and consultation with universities and research users, it has been agreed that one example of research impact “outside the academic sector” should be described and assessed for every 10 included in each submission. Impacts must be linked to at least one 2* paper published since 1993.

With 20 years of research findings to choose from and a broad definition of research impact (“any effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia”), it should not be difficult for universities to find examples of research impact to satisfy the public purse. This exercise will force researchers and institutions to consider the relevance and usefulness of their work and will require them to put systems in place to maximise their research impact. The examples should also inform wider discussion in the public domain.

One effect of serial university research assessment exercises has been to concentrate resources in centres of excellence, and for these centres to concentrate on smaller numbers of research areas. As universities become knowledge factories, producing particular types of knowledge, the question arises of how well the public is being served.

Whatever the excellence of medical knowledge, professional skills, and health policies, these are often irrelevant to the needs of patients, incompletely applied, or deployed in ways that provide poor value for money. Health systems around the world are struggling to find ways of coping with ageing populations, endemic multimorbidity, service fragmentation, resource constraints, and widening inequality. We will increasingly need well researched local solutions to these internationally prevalent problems.4

The REF has many uses, and it will soon be possible to add another, the assessment of usefulness. Peer review will sort out false claims, but when REF 2014 has reported, a wider discussion can begin—on what society needs and gets from its health research.


Cite this as: BMJ 2012;345:e7797


  • Competing interests: The author has completed the ICMJE uniform disclosure form at (available on request from the corresponding author) and declares: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Provenance and peer review: Commissioned; not externally peer reviewed.


View Abstract

Log in

Log in through your institution


* For online subscription