FST JOURNAL

What will the 2028 Research Excellence Framework (REF) mean for UK Higher Education Institutions (HEIs)? As a major mechanism for assessing and rewarding research quality and excellence – to the tune of £2 billion a year in Quality-Related (QR) funding and devolved equivalents allocated to institutions across the UK – there are few levers as influential as REF in shaping the value system underpinning UK research. Revisions to the REF have ramifications for institutions in terms of both prestige and funding, prompting both excitement and apprehension in the sector.

It is no surprise therefore that the release of the initial plans for REF 2028 inspired rich discussion across two panel discussions at UCL on 5 July 2023, with representatives from across the sector. The excitement about REF’s potential to effect positive change was evident among the Research England representatives speaking at the event, Dame Jessica Corner, Executive Chair of Research England, and Dr Steven Hill, Director of Research. As Dame Jessica reflected on the evolution of research assessment over the decades, she deemed REF 2028 “a once in a generation moment when we have the opportunity to shift in direction”. Indeed, she sees potential for REF to tackle major challenges in the sector, from the ‘publish or perish’ culture and lack of research reproducibility to the need to promote diversity and collaboration and broaden what research excellence means.

Measuring culture

As described by Professor Geraint Rees, UCL Vice-Provost (Research, Innovation and Global Engagement) in his opening remarks to the event, one of the key shifts in REF is the reduction of the weighting of research outputs to 45-50%, alongside an increased focus on research culture, with a ‘People, Culture and Environment’ (PCE) element weighted at 25%. This is a notable shift – and a major incentive for universities to promote positive research cultures – that has been welcomed by some in the sector and challenged by others. A key question, noted Professor Rees, will be whether we have robust metrics to assess culture.

Dr Hill acknowledged that there is still a lot of work to do to develop indicators of research culture. Indeed, following the event, in October 2023, the funding bodies launched a tender for work to develop outcome-focussed indicators for the PCE assessment, alongside a consultation on the challenges and opportunities in this area. So how might one approach developing such robust measures?

At the event, Sir Peter Gluckman, Chair of the International Advisory Group to the Future Research Assessment Programme, was pragmatic in his evaluation of how to do this. He pointed out that we are trying to be objective about something that is inherently subjective, as research quality, excellence and impact mean different things to different people. The goal, Sir Peter argued, is to find the best proxy measures and consider how accurate they need to be. Given that the measurement involved in REF drives conformity, he also questioned how we can maintain diversity, innovation and intellectual thought.

Another factor to consider is where the balance should sit between quantitative and qualitative metrics. Dr Elizabeth Gadd, Vice-Chair of the Coalition for Advancing Research Assessment (CoARA), described how CoARA favours qualitative evaluation, supported by responsible use of quantitative indicators. She also highlighted the risk of unintended consequences from new approaches and metrics, and the importance of listening to community feedback on proposed changes.

Diversity

Diego Baptista, Wellcome’s Head of Research Funding and Equity, also cautioned against unintended consequences, in particular for staff with protected characteristics, in the context of efforts to promote diversity in REF. As FST Chair, Lord Willetts pointed out that REF has a clear commitment to equality and diversity, and the impact of REF on EDI is a live topic under discussion. Recognising the need to get this right, Dr Baptista urged an iterative approach to assessment in REF.

Diversity is an important consideration not just with respect to staff but also to institutions. Dr Hill assured the meeting that the funding bodies are cognisant of the need to strike a balance between culture metrics intended to make fair, consistent comparisons across institutions and the flexibility to recognise institutional diversity.

Institutional change

There is value in research culture indicators beyond REF, of course. Emma Todd, Director of Research Culture at UCL, described UCL’s approach to promoting a healthy, inclusive research culture, through a 10-year roadmap for both top-down and grassroots change, drawing on academic expertise in behaviour change. As she pointed out, meaningful indicators are needed to know that the work we are doing is making a difference. While she argued that REF should not dictate work on research culture, the emphasis on culture in REF 2028 does help to sharpen the focus on how institutions are creating an enabling environment for research.

The potential impact of REF on individual institutions was evident from the story of how REF 2021 was used as a driver for change at Northumbria University. Professor Louise Bracken, Northumbria’s Pro Vice-Chancellor (Research and Knowledge Exchange), explained how the university significantly increased the number of staff it submitted to REF 2021 and enjoyed success in its REF outcomes. With regard to the upcoming REF exercise, Professor Bracken acknowledged a mix of excitement and nervousness about the changes, including the decoupling of outputs from individuals.

Decoupling

This ‘decoupling’ means there will no longer be minimum and maximum requirements on individual staff to submit research outputs. This is part of a broader move away from focussing on individuals and towards institutions and ‘team research’. This principle was welcomed by Professor Simon Hettrick, Chair of the Hidden REF, a competition to recognise the breadth of contributions to research. Hettrick sees decoupling as a good policy but cautioned that its impact will depend on how it is implemented by HEIs. Decoupling has been criticised by some and Dr Hill recognised the risk that universities respond by focussing resources on a small portion of their research to maximise their REF scores.

As initial principles for REF 2028 are developed into more detailed plans, it is clear that many in the sector are deeply invested in the impacts they will have. As Professor Rees emphasised in his closing remarks, REF 2028 will need to straddle the numerous tensions that inevitably result from measuring something as complex as research excellence. There are tensions between qualitative and quantitative approaches, between the technical work that needs to be done and the limited time available, and between seeking the intended behaviour change without leading to ‘game playing’.

Opposing imperatives?

As Professor James Wilsdon, Director of the Research on Research Institute, recognised, REF sits alongside many other imperatives and drivers in the sector, some of which may act as obstacles to the REF’s agenda.

While there will be no perfect way to master these challenges, as REF takes shape, we must ask ourselves the final questions posed by Professor Wilsdon: How should we judge the success of REF? What would look different in the sector if REF ‘works’? Whatever one’s particular view on what that picture should look like, there is certainly potential for REF 2028 to bring positive change to university research cultures.