The evaluation took place at the level of individual organisational units (institutes, departments and research teams) and involved almost a hundred renowned international experts from many different disciplines, who visited Brno in 2022 to meet with scientists, doctoral students and their supervisors. This resulted in recommendations on how to further improve research and doctoral study programmes at Masaryk University.
The first-ever university-wide evaluation of the quality of research and doctoral studies was carried out at Masaryk University. An assessment of this scale was motivated by the lack of a unified mechanism providing direct independent feedback and strategic support to organisational units below the level of faculties. So far, only some university subunits and departments have carried out their own peer review evaluations conducted by respected international or Czech experts in the field.
“This is a significant step forward for Masaryk University because the quality of education and research is one of our key priorities. This evaluation is a conceptual and comprehensive solution that should move the entire university forward. It underlines our responsible approach to research assessment that is enshrined in the Agreement on Reforming Research Assessment, which we signed last year,” said Šárka Pospíšilová, MU Vice-Rector for Research and Doctoral Studies.
Three pillars of research and doctoral quality evaluation
The assessment system’s first activity at Masaryk University consists of the annual evaluation interviews between the faculties and the university management. The second annual activity involves the specifications of performance indicators used to allocate national research funding within the university budget. The indicators reflect desirable trends with regard to publishing scientific results and securing prestigious, mostly international grants. The last and most recent activity is the Internal Evaluation of Research and Doctoral Studies which first took place last year and will be conducted on a five-year basis, without being directly linked to funding.
“The decoupling of the evaluation results from funding is purposeful. We want to look the truth in the eye and find out how we are really doing, without having to worry that a negative result would immediately impact our budget. This has allowed us to select respected and strict evaluators and avoid tactical sugar-coating when writing the self-evaluation reports. We believe that unvarnished and accurate feedback will help us improve our individual departments and doctoral studies as a whole,” explains Stanislav Balík, Dean of the MU Faculty of Social Studies.
Dutch and British inspiration for internal evaluation
The design and process of the internal assessment were inspired by the Czech Methodology 2017+, the Dutch SEP and the UK REF. It follows the international best practice principles in research assessment and bibliometrics (DORA, the Leiden Manifesto, Agreement on Reforming Research Assessment and the SCOPE framework). It focuses on the criteria of research quality and societal impact down to the level of individual research topics, the quality of doctoral studies, and the strategies and research environments at departments and individual teams. The evaluation concerned a 5-year period from 2017 to 2021. It was mainly carried out using self-evaluation reports prepared separately for each department and doctoral programme. Supplementary sources included bibliometric analyses of the individual departments and analytical data for doctoral programmes (numbers of students, supervisors, thesis defences, dissertation topics, etc.) and served to provide the evaluators with the full context of doctoral studies at Masaryk University.
Nearly one hundred international experts participated in the internal evaluation
Experts from various countries (the US, UK, Germany, Austria, France, etc.) played a key part in the internal evaluation. Faculties and institutes had the opportunity to structure the units under evaluation according to usual practices in the individual disciplines and specific relevance to their internal environment. Based on uniform university-wide rules, they then invited almost 100 international experts to Brno to participate in discipline-specific panels or interdisciplinary International Scientific Advisory Boards (ISAB) at the individual faculties. The panels met with faculty leaders, heads of departments and doctoral supervisors for in-depth discussions, exchange of experience and sharing of best practices and useful suggestions. The process also included private interviews between the evaluators and doctoral students without the lecturers and faculty representatives being present.
“The evaluation mainly yielded recommendations for improving the quality, strategy and further development of research and doctoral studies. Based on these recommendations, the faculties and institutes of Masaryk University will create development plans for the next five-year period, which will be continuously supervised by the university management,” added Michal Bulant, MU Vice-Rector for Studies and Quality.
The evaluation reports were produced during the calendar year 2022, in each case within 2 months after a personal visit to the evaluated units. The evaluation’s important strategic result is connecting stakeholders and significantly improving communication between all those involved in the assessment and quality control of research, doctoral programmes and studies at Masaryk University in general. Some organizational units have also retained their international evaluation panels as permanent advisory bodies that will meet regularly in one- or two-year intervals.