Information Sciences 2024 | MFCA: 基于多模态模糊特征融合的脑龄协同预测算法

发布时间：2024-08-21 阅读次数：36 分享至：

MFCA: Collaborative prediction algorithm of brain age based on multimodal fuzzy feature fusion

Weiping Ding , Jing Wang , Jiashuang Huang , Chun Cheng*, Shu Jiang

MOTIVATION

Despite the success of unimodal neuroimaging data in predicting brain age, there are significant limitations. For example, sMRI data provide detailed information about the brain’s gray and white matter structure but lack functional activity information. Conversely, fMRI reflects the brain’s functional activity during different tasks and resting states but lacks structural details. As a result, unimodal data often can only provide specific aspects of information, unable to reflect the multifaceted characteristics of the brain globally, and its generalization ability is also limited.
Choosing an appropriate method to fuse information from these modalities can enhance prediction accuracy. However, current multimodal data fusion methods often use simple concatenation or weighted fusion, which fail to fully exploit the rich information between different modalities. Simple concatenation may lead to information redundancy and increased model complexity without necessarily enhancing performance.

INNOVATION

This paper introduces fuzzy theory in the feature fusion stage, designing a fuzzy fusion module. A fuzzy measure matrix is first constructed to assess correlations between multimodal spatial features. Based on the Choquet integral, features are then weighted and fused according to the fuzzy measure matrix. This process effectively captures the correlations between different modalities, providing a more detailed and accurate feature fusion compared to traditional methods.
In the context of multimodal fusion, this paper proposes a collaborative convolutional fusion layer. This layer enhances complementary information between different modalities using specially designed convolutional kernels, dilated convolution parameters, batch normalization, and ELU activation functions. Unlike simple concatenation or weighted fusion, this design further extracts complementary information, improving the fusion of multimodal data.
The paper improves the performance of the brain age prediction model by employing an optimized sorting contrast loss. This loss function integrates characteristics of different loss functions and enhances model prediction accuracy through weighted fusion of multiple loss functions during training and validation.

METHOD

Fig. 1 presents the framework of this work. This paper proposes a multimodal fuzzy feature fusion collaborative algorithm for brain age prediction (MFCA). This algorithm includes two novel fusion modules: a fuzzy fusion module (FFM) and a multimodal collaborative convolutional module (MCCM). The fuzzy fusion module utilizes fuzzy theory, particularly the fuzzy Choquet integral, to fuse data from

different modalities. The multimodal collaborative convolutional module uses specific convolutional layers to enhance complementary information, effectively mining and utilizing the complementarity of multimodal data to improve brain age prediction accuracy and stability. Additionally, an optimized sorting contrast loss is used to improve the prediction accuracy during training and validation.

Feature Extraction Module: Using convolutional neural networks to extract features from multimodal brain imaging data (fALFF, ReHo, and T1w).
Fuzzy Fusion Module: Correlation information between different modalities is fused using the fuzzy Choquet integral, which assigns different weights to different modalities based on their importance.
Multimodal Collaborative Convolutional Module: This module processes the fused features using convolution operations that enhance complementary information between modalities.
Brain Age Prediction Module: The fused features are passed through linear regression layers to predict brain age, further enhanced with the optimized sorting contrast loss function.