The preparation of the Action Plan stems from MU’s membership in the international Coalition for Advancing Research Assessment (CoARA). Masaryk University joined the Coalition in 2022 and signed up to ten commitments, including combating inappropriate assessment practices, especially the overuse of journal- and publication-based metrics (quantitative analysis of publications), and promoting qualitative assessment methods (especially peer review) to reward diverse results that lead to improved research quality and societal impact. “Responsible research evaluation is one of the important institutional values of Masaryk University. Joining the coalition is one of the biggest scientific events for us in 2023,” said Šárka Pospíšilová, Masaryk University’s Vice-Rector for Research and Doctoral Studies. CoARA has around 600 members, ten of which are from the Czech Republic, including four universities.
The Action Plan, which has been viewed by around a hundred users from other institutions involved in CoARA since it was published in November 2023, was prepared by the Centre for Scientometric Support and Evaluation (CSSE) at the MU Rector’s Office. “A strong motivation for the creation of the Action Plan was the fact that CoARA came at a time when we were gradually introducing tools at the university, completing the triad of evaluation activities with a specific purpose and tools – research assessment (Internal Evaluation of Research and Doctoral Studies – IHVD), funding (performance component of the budget) and monitoring (bibliometric analysis). In all steps of preparing the Internal Evaluation of Research and Doctoral Studies, the University kept the concept of responsible evaluation in mind. It drew inspiration from existing systems, initiatives and case studies (DORA, Leiden Manifesto, SEP, REF and similar). As a result, MU is already implementing some of the CoARA commitments to a large extent,” explains CSSE Head Michal Petr.
The Action Plan itself is divided into three areas, which correspond in content to the individual CoARA commitments: the intention is to analyse evaluation activities and their impact on the MU at central level and at the level of faculties and institutes, including disciplinary quality expectations and criteria relevant to the evaluation of research and individuals. Based on the analysis, the Action Plan will then be updated with specific objectives adapted to the identified gaps and situations, aiming for responsible evaluation in a motivating environment. This area envisages the establishment of an institutional policy for all evaluation activities at MU (“Responsible Research Evaluation at MU”), the continuation of the current usage of bibliometrics solely for strategic, monitoring, and support purposes, and the complete disregard of global university rankings in research evaluation and subsequent funding at MU. The third area is the Platform for CoARA Implementation and Experience Sharing, which aims to inform, involve the broader academic community, and increase collaboration between the departments of the Rector's Office dealing with research evaluation and the faculties and institutes of MU, as well as those outside the university (e.g. the CZARMA association).
Challenges related to setting criteria for research evaluation were the focus of the Science for Society conference held at MU on 31 May. The keynote speaker was James Wilsdon, Professor of Science Policy at University College London and Director of the interdisciplinary international consortium Research on Research Institute. “We need to have a dialogue and create space in the evaluation process to allow the evaluation system to work with the experience of a particular discipline, a particular institution, particular researchers,” he said in Brno, stressing the need to simplify existing systems and also to take ownership of them across the research community. “Research is a complex, messy but beautiful thing that takes many forms. And if we apply narrow, brittle and reductive measures to this complex and beautiful thing, we will suffocate it. Getting these things right really matters,” said James Wilsdon.