Alignment- and orientation-based task assistance in an AR environment

• Main Authors: Price, Robert R, Wu, Hsuan-Ting
• Format: Patent
• Language: English
• Published: 13.09.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FORADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORYOR FORECASTING PURPOSES; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE,COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTINGPURPOSES, NOT OTHERWISE PROVIDED FOR

Embodiments described herein provide a system for facilitating dynamic assistance to a user in an augmented reality (AR) environment of an AR device. During operation, the system detects a first element of an object using an object detector, wherein the object is associated with a task and the first element is associated with a step of the task. The system then determines an orientation and an alignment of the first element in the physical world of the user, and an overlay for the first element. The overlay can distinctly highlight one or more regions of the first element and indicate how the first element fits in the object. The system then applies the overlay to the one or more regions of the first element at the determined orientation in the AR environment.