Real-Time Tracking of Facial Features in Unconstrained Video

• Main Authors: Rogers, Mike, Walker, Kevin, Moore, Carl, Oakes, Matthew, Taylor, Mike, Drouard, Vincent
• Format: Patent
• Language: English
• Published: 27.08.2020
• Subjects: CALCULATING; COMPUTING; COUNTING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

A method for locating and tracking facial features in an unconstrained video sequence includes: in a face-detecting process, delineating, with region-bounding coordinates, the face of the subject within an image selected from the sequence; detecting, in the selected image, a small set of landmarks, corresponding to the face of the subject, using a convolutional neural network, trained to take as input an image region corresponding to the face of the-subject and to return a set of coordinates at computational speeds approximating real time; projectively fitting a three-dimensional character model to the detected landmarks, and using the fitted model to estimate physical locations of additional landmarks, so as to provide a complete hypothesized set of facial landmarks; and in a feature tracker process, updating the hypothesized set of facial landmarks to improve convergence between predicted feature locations and their actual physical locations based on data sampled from the selected image.