MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter

Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose Mo...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Liu, Zhiyuan, Li, Sihang, Luo, Yanchen, Hao Fei, Cao, Yixin, Kawaguchi, Kenji, Wang, Xiang, Tat-Seng Chua
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 18.01.2024
Subjects
Adapters
Coders
Graphical representations
Human performance
Modelling
Online AccessGet full text

Cover

More Information
Summary:Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at https://github.com/acharkq/MolCA.
ISSN:2331-8422