Open research and research assessment reform at the University of Reading

Adrian Bell (Associate Pro-Vice-Chancellor Research), Robert Darby (Research Data Manager), Wanda Tejada (Research Excellence Framework and Research Planning Manager)

See Implementation guide section: Strategy and planning

About the University of Reading

The University of Reading is a broad-based research-intensive university, with over 19,000 students and almost 4,000 members of staff. See the University of Reading’s OR4 project profile.

What has changed?

In the period 2017-21 members of the University Library and a core group of senior research stakeholders began to develop a shared understanding of drivers related to responsible research assessment and open research and started to articulate strategic objectives. During this time the University published statements on the responsible use of metrics (2018) and open research (2019), established a Committee on Open research and Research Integrity (CORRI, 2019), and sought to develop the conversation about open research within our research community by various means, including conference-style events and open research award competitions. At this stage strategic direction was still maturing, and actions did not necessarily entail significant operational change. There was evidence of continuing uncritical use of metrics such as the Journal Impact Factor and the H-index in promotion and recruitment activities. Open research culture was still in a very early stage of evolution.

From 2021 the University instigated more organised strategic activity under the sponsorship of CORRI to develop open research culture and practice and to align the University to the principles of responsible research assessment. Open Research Action Plans (ORAP) were created and adopted for the periods 2021-23 and 2024-2029, and an Implementation Group was formed to undertake delivery. (The current ORAP is published on the University’s Open Research web pages.)

The first ORAP identified implementation of recognition and reward for open research as an objective. To address both this requirement and the need for broader research assessment reform, CORRI established a Research Assessment Working Group (2021). The Working Group led the University to become an early member of the Coalition for Advancing Research Assessment (CoARA) and has set out a five-year plan of work in its CoARA Action Plan. It has built stakeholder engagement, drafted a Research Assessment Policy, and laid the foundations for policy adoption and implementation over the period to 2028. One of the key principles of the proposed policy is that ‘Assessment should recognise openness and reproducibility in research’.

Why was the change undertaken?

We recognise that the integration of open research into research practice requires long-term cultural and behavioural change. Many researchers do not currently use relevant open research methods, even where they are required by University policy, as in the case of data sharing. We believe that researcher assessment policies can be powerful enablers of desirable behaviours and can contribute to changes in practice and culture over the long term.

We have developed and championed this view in the University with reference to discussions in the sector, including the EU’s Open Science Career Assessment Matrix (OS-CAM, 2017), the LERU white paper, Open Science and its role in universities: a roadmap for cultural change (2018), and the UNESCO Recommendation on Open Science (2021). The last of these states: ‘Assessment of scientific contribution and career progression rewarding good open science practices is needed for operationalization of open science’.

The gathering momentum for research assessment reform, marked by publication of the Agreement on Reforming Research Assessment and the formation of CoARA in 2022, provided a timely stimulus. The Agreement highlights in its ‘Diversity, inclusiveness and collaboration’ principle the importance of considering open research activities and outputs within the framework of responsible research assessment.

The REF has also focused institutional attention. Principles of responsible research assessment and recognition of the growing importance of open research informed REF 2021; guidance on the REF 2029 People, Culture and Environment element indicates that institutional support for open research and responsible research assessment will feature more strongly still. This provides a compelling narrative within which stakeholders can understand and support the work being undertaken.

How was the change carried out?

The Research Assessment Working Group reports to CORRI, which itself reports to University Senate. It is chaired by Associate Pro-Vice-Chancellor for Research Professor Adrian Bell, who also leads the ORAP Implementation Group. Several individuals are members of both groups. This ensures both groups are co-ordinated and that integrating recognition and reward for open research is a central objective of research assessment reform.

The Working Group includes representation from the academic community at different levels (a Head of School, a Research Division Lead, a senior academic, and an early career researcher) and professional services. Members of the Library’s Research Engagement team provide expertise in open research and responsible research assessment; the REF Planning Manager ensures alignment with REF preparation; an HR Partner provides valuable operational insight and influence. Academic members have experience of recruitment, promotion and academic probation, and ensure that the interests of researchers at different career stages and in different disciplines are represented.

The Working Group has drafted a Research Assessment Policy and created a policy co-development framework informed by the SCOPE principles. Members of the group provide paths for engagement with key stakeholders and wider communities: throughout Spring 2025 different members will be leading co-development workshops or meetings with different stakeholder groups. In addition to engagements with specific stakeholders, workshops open to the research community will enable participation from across disciplines, career stages and levels of responsibility. A self-assessment workshop using the OR4 project’s maturity framework is planned for later in the year as an additional co-development element focused on recognition and reward for open research.

We are aiming for adoption of a University Research Assessment Policy later in 2025. Strategic activities and communications linked to phases of policy implementation will grow awareness in the research community of responsible research assessment and the role of open research. Guidance and training will be developed to support the practical use of open research criteria within researcher assessments, and implementation will be monitored and evaluated on an ongoing basis. Activities are co-ordinated with the wider ORAP project to build open research capability across the institution. The initial implementation phase will continue until 2028 and will be following by ongoing consolidation, as practices are progressively integrated into business as usual.

Challenges and lessons learnt

Lack of models for integrating open research into research assessment: While a number of UK universities have policies on the responsible use of publication metrics, there are currently few policies that articulate a broader framework of research assessment encompassing open research activities and outputs. We found a small number of universities that specified some open research criteria in their promotions policies, but these were quite minimal and we could find no evidence of how the criteria were used in practice. We found a few isolated examples of job descriptions that mentioned Open Research, but no evidence of an institutional policy-based approach to including open research criteria in research-based job specifications.

Practical application of recognition and reward for open research is therefore largely uncharted territory. We hope OR4 can help by sharing emerging practice; as may initiatives to build practical models that institutions can use, such as the OPUS project, which is developing and piloting Open Science indicators.

Limited awareness and understanding of the role of open research in research assessment: Awareness, understanding and acceptance of the place of open research in research assessment is very low across the institution, and indeed within the sector as a whole. Education is needed at all levels: for those in institutions who must lead in developing and implementing policy; for those undertaking assessment activities, such as members of promotion panels, so that they can use open research criteria appropriately; and for researchers, so that they can effectively develop and apply open research practices in their own work and evidence relevant activities and outputs in an assessment context. Policy adoption will only be the start: it will take many years for practice to mature and for the impacts to work themselves out.

Engagement with the OR4 project has enabled a core group of stakeholders to understand and champion the place of open research in responsible research assessment. We hope to progressively disseminate understanding and practical knowledge across the institution as we implement policy and the associated communications, guidance, training, and operational supports.

Recommendations

Identify strategic levers: Recognition and reward for open research can intersect with a number of different strategic activities within an institution, such as to develop open research culture and practice, to undertake research assessment reform, and to prepare the institution for the REF. These can be useful strategic levers when seeking to develop the case or gain the buy-in of key stakeholders who may provide leadership or play an enabling role. It is important to ensure that objectives related to open research are clearly identified in the scope of related activities.

Co-develop with your stakeholders: It is essential to ensure that all key stakeholder groups are identified and representative stakeholders are engaged. We created a co-development framework informed by the SCOPE principles to ensure that policy is developed and implemented inclusively and that it reflects our shared values as a research community. This framework recognises that effective implementation of policy requires co-ownership and an understanding of its impacts in a variety of processes and contexts. Our CoARA Action Plan includes a table of key stakeholders by role in an Appendix.

Resources

University of Reading Open Research Action Plan 2024-29. Download from https://www.reading.ac.uk/research/research-environment/open-research

University of Reading. (2024). Reforming research assessment at the University of Reading: CoARA Action Plan 2024-2028. Zenodo. https://doi.org/10.5281/zenodo.14501207

© University of Reading. Licensed under CC BY.