Multimodal Brain Imaging for Brain Disorder Classification: A Review of Graph Neural Networks and Transformer-Based Methods

Abstract

With the rapid advancement of neuroimaging technologies and deep learning methods, multimodal brain imaging analysis has emerged as an important research direction for the early diagnosis and progression assessment of neurological disorders. Different neuroimaging modalities, such as structural magnetic resonance imaging (sMRI), functional magnetic resonance imaging (fMRI), and diffusion tensor imaging (DTI), provide complementary information from anatomical, functional, and structural connectivity perspectives. Compared with single-modality analysis, multimodal learning can offer a more comprehensive characterization of pathological alterations in the brain. At the same time, the increasing use of connectome-based representations has introduced graph-structured data into neuroimaging analysis, making graph neural networks (GNNs) particularly suitable for this field. More recently, Transformer-based architectures have further enhanced the ability of deep models to capture long-range dependencies and complex interactions across temporal windows, graph nodes, and heterogeneous modalities. As a result, the integration of multimodal learning, graph neural networks, and Transformer models has become a promising paradigm for intelligent brain disorder classification. This review focuses on these three key aspects. First, the importance and major strategies of multimodal brain imaging fusion are discussed. Second, the development and applications of graph neural networks in connectome analysis and brain disease classification are summarized. Third, the role of Transformer-based models in multimodal fusion and spatiotemporal brain network modeling is analyzed. Finally, the current challenges and future research directions are discussed. This review aims to provide a structured and theoretically grounded overview of recent methodological progress in multimodal neuroimaging-based diagnosis.