Developing Reviewer Profiles Using Analysis of Prior Authorship

Peer review is a cornerstone of academic research dissemination. It is a fundamental prerequisite for “good” research, even though the process of selecting reviewers is largely shrouded in mystery. Under ideal conditions, reviewers are experts on a paper’s topic, but the process of identifying that...

Full description

Saved in:
Bibliographic Details
Published inAssociation for Engineering Education - Engineering Library Division Papers
Main Author Verleger, Matthew A
Format Conference Proceeding
LanguageEnglish
Published Atlanta American Society for Engineering Education-ASEE 23.06.2018
Subjects
Algorithms
Authoring
Data mining
Identification methods
Methodology
Peer review
Online AccessGet full text

Cover

More Information
Summary:Peer review is a cornerstone of academic research dissemination. It is a fundamental prerequisite for “good” research, even though the process of selecting reviewers is largely shrouded in mystery. Under ideal conditions, reviewers are experts on a paper’s topic, but the process of identifying that expertise varies widely from publication venue to venue. The purpose of this paper is to (1) identify some of the methods currently used by conference and journal editors, and (2) present an alternative methodology used by the author to identify and assign reviewers for a division of ASEE during the summer of 2017. The author, as program chair of a large division, developed software that data-mined the prior ASEE conference papers of any potential reviewer to identify the words and phrases frequently used in their prior ASEE papers. Using this data, a Reviewer Profile was developed that identified areas of expertise. Using a similar process, papers being reviewed were profiled to the types of expertise necessary – a Paper Need Profile. For each paper, a list of potential reviewers was selected based on alignment between the Reviewer Profiles and Paper Need Profile for that paper. The list was further filtered to remove (co-)authors and reviewers who had already been assigned at least two papers. The remaining list was then used to select the final reviewers. This paper will present details of the Reviewer Profile algorithm, the Paper Need Profile algorithm, the matching algorithm, and an analysis of the effectiveness of the overall assignment methodology. The conclusion will explore how this work may translate into classroom peer review as well as a discussion of how this methodology could be applied across ASEE and in other professional publication venues.