A Feature Selection Method for High-Dimensional Medical Data Based on Adaptive Redundancy Penalty

Abstract

High-dimensional medical data are often characterized by extremely high feature dimensionality, limited sample sizes, substantial redundant information, and strong noise interference. To address these challenges, this paper proposes a filter-based feature selection method, termed ARMR (Adaptive Redundancy-based Minimum Redundancy Maximum Relevance Feature Selection). The proposed method first employs mutual information to select initial features from the original feature set. It then uses joint mutual information to characterize the joint discriminative capability of candidate features and selected features with respect to class labels. Furthermore, an adaptive penalty factor, constructed from conditional mutual information and three-way interaction information, is introduced to dynamically weight the redundancy term, thereby enabling a more flexible balance between feature complementarity and redundancy suppression. To evaluate the effectiveness of the proposed method, experiments were conducted on six benchmark datasets and four medical datasets, with comparisons against several representative and recently developed methods, including CFR, DCSF, JMI, MRMD, and mRMR. Two classifiers, namely SVM and NB, were employed for performance assessment. The results show that the proposed method achieves superior overall performance. In particular, it attains the highest average classification accuracy of 84.58% with SVM and 76.36% with NB. Further analysis based on classification accuracy curves under different feature subset sizes, together with boxplots of F1-score and AUC, demonstrates that the proposed method exhibits strong performance in terms of classification accuracy, stability, and classifier adaptability.