VF-Lens: Enhancing Visual Perception of Visually Impaired Users in VR via Adversarial Learning with Visual Field Attention

• Published in: Proceedings (IEEE Conference on Virtual Reality and 3D User Interfaces. Online) pp. 420 - 430
• Main Authors: Chen, Xiaoming, Han, Dehao, Qu, Qiang, Shen, Yiran
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08.03.2025
• Subjects: Adaptation models; Adversarial machine learning; Generators; Human computer interaction; Human-centered computing-Human computer interaction (HCI); Human-centered computing-Virtual Reality; Three-dimensional displays; Training; Virtual reality; Visual impairment; Visual perception; Visualization
• ISSN: 2642-5254
• DOI: 10.1109/VR59515.2025.00066

This research aims to enhance the image perception of visually impaired users in VR environments. We propose VF-Lens, a model that adaptively compensates for light sensitivity based on the user's visual field impairment, acting as a virtual lens between the visually impaired users and the VR world. VF-Lens is designed as a tailored generative adversarial learning model with a generator and discriminator, offering applicability to various types of visual impairments while bypassing engineering complexities. The generator creates a "hyperimage" tailored to the user's visual field impairment, which then undergoes a particular regression process to predict and replicate the real perception of the visually impaired user. The discriminator then evaluates the similarity between the replicated perception and the original image. Through adversarial training, the generator can produce hyperimages that adapt to the user's visual field parameters, enabling them to perceive the image more similarly to normal-vision users. We further improve VF-Lens by proposing new "visual field attention" mechanisms that prioritize and refine visual information in the user's visual field. Extensive evaluation, encompassing both visually impaired participants and simulations, has been conducted to demonstrate the effectiveness of VF-Lens in improving visual perception for visually impaired users. Moreover, we establish a standardized evaluation process involving tailored metrics as well as objective and subjective evaluations to promote reusability and comparability for future research in this field.