Hierarchical Feature Alignment for Transfer Learning on Neural Decoding Tasks

• Published in: Proceedings / Annual IEEE International Symposium on Bioinformatics and Bioengineering (BIBE) pp. 249 - 254
• Main Authors: Eryol, Erkin, Vural, Fatos T. Yarman
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.11.2022
• Subjects: Brain decoding; Buildings; Correlation; Data visualization; Decoding; Eigenvalues and eigenfunctions; fMRI; Functional magnetic resonance imaging; Multi-view learning; Principal component analysis; Recording; Transfer learning
• ISSN: 2471-7819
• DOI: 10.1109/BIBE55377.2022.00059

In this work, we propose two novel methods to align the feature representation of fMRI data to improve the transfer learning performance on brain decoding problem under a limited data regime. Transfer learning on brain decoding is most viable when the source and target datasets share the same feature space. In such a condition, the feature alignment method should prioritize dominant variance directions of data. Following this requirement, in our first method, we modify the standard formulation of maximum variance generalized canonical correlation analysis (maxvar-GCCA) to work on linear projectors, scaled by eigenvalues. Our method suppresses the low variance dimensions of the feature space, which, otherwise, can spoil the feature alignment process. We estimate a series of linear transformations, using principal component analysis to align hierarchically organized data at session, subject, and dataset levels. In the second method, we incorporate label information of fMRI recordings for better transferable features in a limited data regime. We extend our first method to utilize correlation between labels and data representations. Our solution imposes label dependence by maximizing the Hilbert Schmidt Independence Criterion as an intermediate step in our hierarchical feature alignment procedure. Both methods are applicable to heterogeneously labeled time samples, which do not impose the same label at each time instant of different sessions or require an equal number of time points across the sessions. We compare our methods with the state-of- the-art solution in transfer learning methods for neural decoding that follows the method of maxvar-GCCA. We evaluate our transfer learning methods using four non-intersecting stop-signal paradigm datasets. We observe that the suggested Hierarchical Feature Alignment method and its label-guided version improve the state-of-the-art transfer learning method for brain decoding consistently on all transfer learning cases.