Evaluating the quality of research publications: A pilot study of school librarianship

• Published in: Journal of the American Society for Information Science and Technology Vol. 55; no. 13; pp. 1119 - 1130
• Main Author: Clyde, Laurel A.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.11.2004; Wiley; Wiley Periodicals Inc
• Subjects: Abstracting; Academic Language; Citation analysis; Citation Indexes; Citations; Cluster analysis; Collaboration; College Faculty; Evaluation; Evaluators; Evidence Based Practice; Exact sciences and technology; Factor Analysis; Impact factors; Indexing; Information and communication sciences; Information dissemination (open archives, open access,...) and document delivery; Information publishing, dissemination and reproduction; Information Science Education; Information science. Documentation; Libraries; Library and information science; Library Schools; Literature reviews; Multivariate Analysis; Perceptions; Periodicals; Pilot projects; Quality; Research design; Research Reports; School libraries; Science; Sciences and techniques of general use; Statistical analysis; Studies; Iceland; Human; Evaluation; School library; Quality; Scientific research; Published document; Scientific communication; Librarianship; Child
• ISSN: 1532-2882; 2330-1635; 1532-2890; 2330-1643
• DOI: 10.1002/asi.20066

Evaluation of research quality is becoming more important in the field of library and information science, as in other fields. This pilot study is a preliminary attempt to address issues associated with determining the quality of the published research in one area of library and information science, specifically school librarianship. The main aims were (1) to test the extent to which experienced evaluators agreed in their rankings of published research articles on the basis of quality and (2) to investigate the approaches to evaluation used by the experienced evaluators. A qualitative, naturalistic research design was used. On the basis of a comprehensive literature review, four approaches to evaluation were identified; they were generally supported through an analysis of the responses of the experienced evaluators. However, although the majority of the evaluators agreed on the article ranked lowest, basic statistical analyses showed less agreement about the other articles. Although subject knowledge (of the field of school librarianship in this case) may have some influence on the evaluations, cluster analysis suggests that there may be differences in the value perceptions of the evaluators that also carry weight. More research would be needed to gain a better understanding of these value perceptions and their relationship, if any, to the four approaches to evaluation that were identified through the literature.