A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study

• Published in: Journal of medical Internet research Vol. 24; no. 1; p. e28368
• Main Authors: Leong, Victoria, Raheel, Kausar, Sim, Jia Yi, Kacker, Kriti, Karlaftis, Vasilis M, Vassiliu, Chrysoula, Kalaivanan, Kastoori, Chen, S H Annabel, Robbins, Trevor W, Sahakian, Barbara J, Kourtzi, Zoe
• Format: Journal Article
• Language: English
• Published: Canada Journal of Medical Internet Research 06.01.2022; Gunther Eysenbach MD MPH, Associate Professor; JMIR Publications
• Subjects: Analysis; Automation; Cognition & reasoning; Cognitive tasks; Coronaviruses; COVID-19; Data collection; Data quality; Disease transmission; Epidemics; Executive function; Flexibility; High risk; Humans; Information management; Intelligence; Internet; Laboratories; Learning; Methods; Minority & ethnic groups; Neuropsychological assessment; Neuropsychological Tests; Neuropsychology; Online education; Original Paper; Pandemics; Quality control; Reaction time; Reproducibility; SARS-CoV-2; Social aspects; Social distancing; Social networks; Standardized tests; Task performance; Telemedicine; United Kingdom; Verbal intelligence; Young Adult; Young adults; United Kingdom; COVID-19; neurocognitive assessment; learning; executive functions; web-based testing
• ISSN: 1438-8871; 1439-4456; 1438-8871
• DOI: 10.2196/28368

The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies. This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT). A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures. The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication. These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality-particularly for studies involving high-risk or rare cohorts-and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.