Big fish and small ponds: why the departmental h-index should not be used to rank universities

• Published in: Scientometrics Vol. 127; no. 6; pp. 3279 - 3292
• Main Authors: Mryglod, O., Holovatch, Yu, Kenna, R.
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 01.06.2022; Springer Nature B.V
• Subjects: Computer Science; Hirsch index; Information Storage and Retrieval; Institutions; Library Science; Quality assessment; Quality control; Ratings; Research facilities; Research institutions; Scientometrics; Size effects; Group h-index; Hirsch index
• ISSN: 0138-9130; 1588-2861
• DOI: 10.1007/s11192-022-04373-y

The size-dependent nature of the so-called group or departmental h-index is reconsidered in this paper. While the influence of unit size on such collective measures was already demonstrated a decade ago, institutional ratings based on this metric can still be found and still impact on the reputations and funding of many research institutions. The aim of this paper is to demonstrate the fallacy of this approach to collective research-quality assessment in a simple way, focusing on the h-index in its original form. We show that randomly reshuffling real scientometric data (varying numbers of citations) amongst institutions of varying size, while maintaining the volume of their research outputs, has little effect on their departmental h-index . This suggests that the relative position in ratings based on the collective h-index is determined not only by quality (impact) of particular research outputs but by their volume. Therefore, the application of the collective h-index in its original form is disputable as a basement for comparison at aggregated levels such as to research groups, institutions or journals. We suggest a possible remedy for this failing which is implementable in a manner that is as simple and understandable as the h-index itself.