Techniques for Assessing Safety Symbol Comprehension: Web-Based vs. In-Person Questionnaire Administration

• Published in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting Vol. 50; no. 19; pp. 2207 - 2211
• Main Authors: Wisniewski, Elaine C., Isaacson, Judith J., Hall, Steven M.
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.10.2006
• ISSN: 1541-9312; 1071-1813; 2169-5067
• DOI: 10.1177/154193120605001904

This study compares methods for testing comprehension of a set of three safety symbols – in-person questionnaire administration and three variations of a web-based questionnaire using similar protocols. Both the medium and the particular questions asked were varied. The in-person method included follow-up and probing questions and used questions about the individual symbols only if needed to elicit complete answers. The web-based questionnaires omitted follow-up and probing questions, and always included questions about the individual symbols. The second variation of the web-based questionnaire added an example of a “good/poor answer” as shown in ANSI Z535.3 Annex B. The third included the example answer and an additional question regarding consequences. The in-person method measured 90% correct comprehension with 2% critical confusions. Comprehension measured by each of the three web-based methods did not significantly differ from the in-person method. The comparison of different web-based questionnaires indicates that for web-based testing, asking about individual symbols within a symbol set, providing an example of a good/poor answer, and/or including an additional question regarding consequences can improve the method's ability to capture comprehension data. These results suggest that under some circumstances, a web-based questionnaire could substitute for in-person comprehension testing.