MS-VSSM: Multiscale Enhanced Visual State Space Model for Facial Expression Recognition

Abstract

Facial Expression Recognition (FER) is a key technology in fields such as human-computer interaction and mental health assessment. However, its performance is constrained by the subtleties and dynamics of expressions, as well as the complexity arising from individual differences and environmental variations. To overcome the limitations of traditional Convolutional Neural Networks (CNNs), such as their restricted receptive fields, and the high computational complexity of Transformers, this paper proposes a Multiscale Enhanced Visual State Space Model (MS-VSSM) based on the Visual State Space Model (VSSM), aiming to improve the accuracy and robustness of FER. The model introduces three core improvements upon VSSM: (1) integrating a Path-aware Channel Attention mechanism (SE-SS2D) into the SS2D module to enhance the targeted capture of critical local facial features; (2) embedding a Dense Spatial Pyramid Pooling module (DSPP) at the beginning of each network stage to achieve multi-scale contextual information fusion; and (3) employing a Layer Scale mechanism to finely adjust the scale of deep features, thereby improving training stability and representational flexibility. In a four-category emotion recognition task constructed on the DEAP dataset, MS-VSSM achieved an accuracy of 97.31% and a weighted average F1-score of 97.56%, significantly outperforming the original VSSM and various mainstream visual backbone networks. These results validate the effectiveness and advancement of the proposed method. This study provides a new solution for efficient and accurate fine-grained facial expression recognition.