University of Surrey Open Research logo

Responsible use of research and innovation metrics

Understand the responsible use of research and innovation metrics, the University’s commitment to DORA, and the actions we are taking to support this.

Contact the Open Research team

Defining the responsible use of metrics

The term “research and innovation metrics” refers to the quantitative indicators which can be used to evidence research activity, quality and characteristics.

There is a wealth of data on research and innovation outputs and impact which can evidence the quality of research and innovation activity. Such metrics encompass publication data (bibliometrics such as citation data and journal impact factors), indicators of social and economic impact (such as those incorporated into the knowledge exchange framework), indicators supporting contributions to the research culture and environment (such as supervision, collaboration, and open research practices) and related measures such as funding income, and measures of esteem (such as honours, awards, and invitations to present).

The ‘responsible use of metrics’ is defined as “framing appropriate uses of quantitative indicators in the governance, management and assessment of research" (The Metric Tide, 2005), with the Universities UK Forum for the Responsible Use of Metrics defining responsible use by five key principles:

  1. Robustness: Base metrics on the best possible data in terms of accuracy and scope
  2. Humility: Recognise that quantitative evaluation should support, but not supplant, qualitative, expert assessment
  3. Transparency: That those being evaluated can test and verify the results
  4. Diversity: Account for variation by research field, and use a range of indicators to reflect and support a plurality of research and researcher career paths across the system
  5. Reflexivity: Recognise and anticipate the systemic and potential effects of indicators, and updating them in response.

Research funders, government, institutions, and researchers all place importance on the responsible use of metrics.

Our commitment to responsible metrics

The University published its first institutional statement on the responsible use of metrics in 2017 and, following a commitment in our 2020 Open Research Position Statement to further develop and promote the responsible and fair use of metrics, an institution-wide Responsible Use of Metrics Implementation Plan (2021) set out how the University would achieve this. In 2022 the University signed the San Francisco Declaration on Research Assessment (commonly known as DORA).

Our approach to research metrics forms part of broader efforts to enhance research culture and practice and reflects:

  • Our ambition, as set out in the University’s strategy to be a global leader in research and innovation and a destination of choice for researchers in the UK and internationally for the practise of excellence in research and innovation
  • Our values of inclusion, inspiration, innovation, integrity
  • Our culture of open and collaborative research
  • Our aspirations, through interdisciplinary collaboration and industry engagement, to bring together diverse groups of researchers to address global challenges.

DORA: The San Francisco Declaration on Research Assessment

In 2022 the University of Surrey signed DORA, and as a signatory the University commits to supporting the adoption of the following practices in research assessment:

  • An overarching general recommendation of DORA states “Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.”

Two further recommendations of DORA form the institutional commitment:

  • The first of these advises that institutions “be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published”

  • The second recommendation considers that “for the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.”

Responsible metrics implementation plan

In 2021 the University of Surrey Open Research Working Group developed a responsible use of metrics implementation plan to support the practical application of responsible metrics across the University, and to promote the benefits of this. The implementation plan comprised ten key steps including the creation of resources, the development of evaluation, and recognition of the value of all relevant research and innovation outputs.

In addition to drawing upon DORA and the UK Forum for Responsible Use, the implementation plan also draws upon the Leiden Manifesto, INFORM’s Scope Model, and further sector statements, as well as our local context, culture and experience at Surrey.

The implementation plan (PDF) was approved by the University Research and Innovation Committee in March 2021.

Best practice guidance

In 2017, the Open Research team developed the following suite of best-practice tips to consider when approaching publication data (bibliometrics) to help elicit the richness, diversity, and complexity of research and researchers.

Resources: Sector statements