From aging to early-stage Alzheimer's: Uncovering handwriting multimodal behaviors by semi-supervised learning and sequential representation learning

• Published in: Pattern recognition Vol. 86; pp. 112 - 133
• Main Authors: El-Yacoubi, Mounîm A., Garcia-Salicetti, Sonia, Kahindo, Christian, Rigaud, Anne-Sophie, Cristancho-Lacroix, Victoria
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.02.2019; Elsevier
• Subjects: Aging; Alzheimer’s; Computer Science; Computer Vision and Pattern Recognition; Document and Text Processing; Geriatry and gerontology; Human health and pathology; Life Sciences; Mild Cognitive Impairment; Online handwriting; Signal and Image Processing; Temporal representation learning; Unsupervised & semi-supervised learning; Aging; Online handwriting; Unsupervised & semi-supervised learning; Temporal representation learning; Mild Cognitive Impairment; Alzheimer’s; Unsupervised & Mild cognitive impairment; semi-supervised learning
• DOI: 10.1016/j.patcog.2018.07.029
• ISSN: 0031-3203

•We propose a paradigm unveiling handwriting changes due to aging and Alzheimer‘s.•Our new semi-supervised learning and sequential representation learning are key.•Semi-supervised learning brings to light handwriting multimodal behavioral trends.•Our sequential representation learning uncovers temporal feature representations.•Classification based on temporal representations outperforms the state of the art. We present, in this paper, a novel paradigm for assessing Alzheimer’s disease and aging by analyzing impairment of handwriting (HW) on tablets, a challenging problem that is still in its infancy. The state of the art is dominated by methods that assume a unique behavioral trend for each cognitive profile or age group, and that extract global kinematic parameters, assessed by standard statistical tests or classification models, for discriminating the neuropathological disorders (Alzheimer’s (AD), Mild Cognitive Impairment (MCI)) from Healthy Controls (HC), or HC age groups from each other. Our work tackles these two major limitations as follows. First, instead of considering a unique behavioral pattern for each cognitive profile or age group, we relax this heavy constraint by allowing the emergence of multimodal behavioral patterns. We achieve this by performing semi or unsupervised learning to uncover homogeneous clusters of subjects, and then we analyze how much information these clusters carry on the cognitive profiles (or age groups). Second, instead of relying on global kinematic parameters, mostly consisting of their average, we refine the encoding either by a semi-global parameterization, or by modeling the full dynamics of each parameter, harnessing thereby the rich temporal information inherently characterizing online HW. To illustrate the power of our paradigm, we present three studies, one regarding age, and two regarding Alzheimer’s. Thanks to our modeling, we obtain new findings that are the first of their kind on this research field. On aging, unlike previous works reporting only one pattern of HW change with age, our study, based on a semiglobal parametrization scheme, uncovers three major aging HW styles, one specific to aged subjects and two shared with other age groups. On Alzheimer’s, a striking finding is revealed: two major clusters are unveiled, one dominated by HC and MCI subjects, and one by MCI and ES-AD, thus revealing that MCI patients have fine motor skills leaning towards either HC’s or ES-AD’s. Our paper introduces also a new temporal representation learning from HW trajectories that uncovers a rich set of features simultaneously like the full velocity profile, size and slant, fluidity, and shakiness, and reveals, in a naturally explainable way, how these HW features conjointly characterize, with fine and subtle details, the cognitive profiles.