Abstract: As a core research task within the fields of artificial intelligence and multimodal understanding, Visual Question Answering (VQA) requires models to jointly comprehend the visual content of images and natural language questions in order to generate accurate answers. Addressing the limitations of existing VQA methods—which typically lack large-scale pre-training and suffer from interference by weakly correlated background elements and redundant regions during multimodal feature modeling—this paper proposes a multi-module collaborative VQA method based on multimodal filtering and spatial-channel attention. Specifically, the proposed method first introduces a Multimodal Filtering Module. This module extracts question feature vectors from specific positions to serve as a global semantic representation; it then performs a Hadamard product interaction with aligned visual features and utilizes a perceptron combined with an activation function to generate normalized filtering gate weights. By element-wise scaling the image features prior to deep-level interaction, this process explicitly suppresses redundant background information, thereby providing a "cleaner" visual input for subsequent deep interactions. Subsequently, a Spatial-Channel Self-Attention Unit is introduced to replace the traditional self-attention architecture. Within a six-layer stacked encoder-decoder framework, this unit reshapes features into 2D maps along the spatial dimension; by combining multi-scale convolutional kernels with global pooling, it generates a spatial attention distribution that effectively captures spatial dependencies between different image regions. Simultaneously, along the channel dimension, it employs pooling and fully connected networks to dynamically adjust the semantic importance of individual channels. Finally, a gating mechanism is utilized to perform a weighted fusion of the spatial-channel-modulated local features with the global context captured by multi-head self-attention, thereby effectively enhancing the discriminative power of the visual features. Building upon this foundation, a Dynamic Modality Fusion Module is constructed to move beyond static fusion mechanisms. After multiple layers of interaction, the model generates attention maps for both visual and textual features, concatenates them, and feeds them into a fully connected classification network (incorporating dropout) to adaptively learn 2D dynamic fusion weight factors. This process ultimately achieves a dynamic weighted integration of multimodal features, with the overall model parameters optimized using a binary cross-entropy loss function. Extensive quantitative experiments conducted on the VQA-v2 and GQA datasets demonstrate that the proposed method achieves significant performance improvements, even without relying on massive-scale cross-modal pre-training data. Specifically, on the VQA-v2 test-dev split, the proposed method achieved an overall accuracy of 71.85%, representing a significant improvement of 1.22% and 2.33% over comparable baseline models—MCAN and BAN—respectively. Notably, the accuracy rates for binary "Yes/No" questions and "Other" question types reached as high as 88.03% and 62.39%, respectively. On the GQA test set—which emphasizes compositional reasoning and relational understanding—the model achieved an overall accuracy of 57.92%, with binary "Yes/No" questions attaining an accuracy of 76.68%. Furthermore, rigorous ablation studies quantitatively validated the efficacy of each core module; the introduction of the multi-modal filtering, spatial-channel attention, and dynamic modality fusion modules yielded absolute performance gains of 0.63%, 0.69%, and 0.52%, respectively. Additionally, an analysis of the impact of model depth confirmed that feature interactions are most comprehensive—and overall model performance reaches its peak—when both the encoder and decoder consist of six stacked layers. In summary, by employing fine-grained feature modeling and dynamic interactions, the proposed method successfully mitigates issues such as severe noise interference, isolated spatial-channel processing, and the poor adaptability of static fusion mechanisms, thereby offering an improved solution for visual-language multi-modal understanding and deep feature representation within specific vertical domains characterized by limited computational resources.

摘要： 视觉问答作为人工智能与多模态理解领域的一项核心研究任务，要求模型联合理解图像的视觉内容与自然语言问题并生成精准回答。针对现有非大规模预训练视觉问答方法在多模态特征建模时易受弱关联背景及冗余区域干扰，传统自注意力未能充分捕捉空间位置依赖及通道间深层语义关联，且多采用静态融合策略难以自适应复杂问题场景等缺陷，本文提出一种基于多模态过滤与空间通道注意力的多模块协同视觉问答方法。在具体实现上，该方法首先设计多模态过滤模块，通过提取特定位置的问题特征向量作为全局语义表示，与对齐后的视觉特征进行哈达玛积交互，并利用感知机与激活函数生成归一化过滤门控权重，从而在深层交互前逐元素缩放图像特征，显式抑制背景冗余信息，为后续深度交互提供纯净的视觉输入。随后，引入空间通道自注意力单元以替代传统自注意力架构。在六层堆叠的编解码器基础架构中，该单元于空间维度将特征重塑为二维图，通过多尺度卷积核结合全局池化生成空间注意力分布，充分捕捉图像区域间的空间依赖关系；同时在通道维度利用池化与全连接网络动态调整语义通道重要性，进而采用门控机制将空间通道调制后的局部特征与多头自注意力捕获的全局上下文进行加权融合，有效增强视觉特征的判别性。在此基础上，构建动态模态融合模块以摒弃静态融合机制。经过多层交互后，模型生成视觉与文本特征的注意力映射并进行拼接，将其输入至包含随机失活的全连接分类网络中，自适应学习二维动态融合权重因子，最终完成多模态特征的动态加权整合，并利用二元交叉熵损失函数进行整体参数的优化训练。基于VQA-v2和GQA数据集的广泛定量实验结果表明，在不依赖超大规模跨模态预训练数据的条件下，所提方法取得了一定的性能优化。具体而言，在VQA-v2的test-dev测试集上，所提方法的总体准确率达到71.85%，较同类基线模型MCAN和BAN分别显著提升了1.22%和2.33%，其中二元是非判断类和其它类问题的准确率分别高达88.03%和62.39%。在强调组合推理与关系理解的GQA测试集上，该模型的总体准确率达到57.92%，其中二元是非判断类问题取得了76.68%的准确率。此外，严格的消融实验定量证实了各核心模块的有效性，多模态过滤、空间通道注意力与动态模态融合模块的引入分别带来了0.63%、0.69%和0.52%的绝对性能提升，且模型层数影响分析进一步证实编解码器各堆叠六层时特征交互最为充分，模型整体性能达到最高峰值。综上所述，本文方法通过细粒度特征建模与动态交互，成功缓解了噪声干扰重、空间通道孤立与静态融合适应性差等问题，为计算资源受限等特定垂直领域下的视觉语言多模态理解与深度特征表达提供了一种改进方案。