A New Method of Subjective Evaluation Using Visual Analog Scale for Small Sample Data Analysis

• Published in: Journal of Information Processing Vol. 29; pp. 424 - 433
• Main Authors: Shirahama, Naruki, Watanabe, Satoshi, Moriya, Kenji, Koshi, Kazuhiro, Matsumoto, Keiji
• Format: Journal Article
• Language: English
• Published: Information Processing Society of Japan 2021
• Subjects: small sample; subjective evaluation; talking robot; visual analog scale
• ISSN: 1882-6652; 1882-6652
• DOI: 10.2197/ipsjjip.29.424

The Likert Scale (LS) is more commonly used in the psychological and affective engineering field. However, LS has some problems, such as the fact that it can only be used for non-parametric analysis if it cannot be treated as an interval measure and is susceptible to biases such as the central tendency and the halo effect. In this study, we propose an analysis method using the Visual Analog Scale (VAS) in which a point marked on a straight line is the evaluation value instead of a five or seven-point scale. The VAS allows us to identify trends in the distribution of data, even in small samples. We visualize the VAS experimental results by overlaying box plots and beeswarm plots to visually grasp the data's distributional trends, even for small samples. We experimented with 30 subjects on conversations with a talking toy robot. We investigated the user's emotions from the conversation with a robot and whether it relates to the conversation's smoothness. The number of questions was 10, and two cases of smooth and non-smooth conversations with the talking robot were evaluated using the VAS method, respectively. The hierarchical clustering results showed that a group of questions expected to show a similar trend was classified into the same cluster. Parametric tests were also performed on data groups following a normal distribution.