Author Login Chief Editor Login Reviewer Login Editor Login Remote Office

Computer Engineering

   

Depth Calibration and Attribute-aware Network for User-needs Saliency

  

  • Published:2026-07-13

用户需求显著性的深度校准与属性感知网络

Abstract: Existing salient object detection (SOD) methods generally follow the principle of passive visual stimulation. These methods rely on low-level features such as color, texture, and contrast to determine salient regions. They treat the object with the strongest visual features in the scene as the focus of user attention. However, they neglect the decisive role of active user demands in determining salient targets. In real scenarios such as human-computer interaction and robot inspection, target objects are often submerged in complex backgrounds. In these cases, user attention has a strong intent orientation. Recently, the user-demand-driven salient object detection (UserSOD) task is introduced. It shifts the perception paradigm from passive visual response to active intent matching. This task also provides corresponding benchmark datasets and baseline models. Existing methods lack deep fusion and dynamic calibration between visual features and user demand semantics. Consequently, models only capture shallow semantic associations of keywords. Moreover, the downsampling process in hierarchical backbone networks easily loses spatial details. Therefore, models fail to accurately locate targets that match abstract demands in complex backgrounds. The detection performance remains heavily limited. To address these issues, this paper proposes a depth calibration and attribute-aware network (DCA-Net) within the UserSOD task framework. This network adopts the Swin-Transformer model as the visual backbone. It combines a pre-trained Contrastive Language-Image Pre-training (CLIP) text encoder as the text branch. The network designs a cascaded semantic recalibration encoder (CSRE) as its core. The CSRE contains four independent semantic recalibration modules (SRM). Each SRM utilizes a cross-modal cross-attention mechanism to achieve step-by-step semantic alignment between visual features and user intent at the source of feature extraction. Meanwhile, each SRM introduces a gated feature flow mechanism (ScGate) based on a multilayer perceptron structure. The ScGate uses full-connection transformation and ReLU-Tanh activation to generate adaptive weights. These weights dynamically adjust the intensity of semantic flows in each channel. This mechanism accurately locks the target region in the early stage of the feature flow. Meanwhile, the network constructs a convolutional cross-scale interaction module (CCIM) to achieve cross-level feature compensation. The CCIM aggregates global features using a pyramid pooling aggregation operator and a 1×1 convolution. It captures multi-scale context information through three parallel 3×3 depthwise separable convolution branches with different dilation rates. Then, the module applies weights by combining serial channel and spatial attention modules. Finally, it uses bilinear interpolation upsampling to restore the feature size. This module significantly enhances the perception ability of the model for spatial details of multi-scale targets.Furthermore, the network introduces a fine-grained attribute-aware decoder (AGD) to provide fine-grained constraints. The AGD adopts a top-down path to step-by-step upsample and merge high-level features. It utilizes the CLIP text encoder to initialize four types of preset attribute prompts, including category, color, appearance, and functional requirements. This initialization provides semantic priors and avoids the random convergence of attribute queries. Next, the AGD extracts explicit semantic constraint vectors of the four attribute dimensions from user instructions via multi-head cross-attention. This operation decouples abstract intent into four independent fine-grained attribute constraints. It avoids semantic confusion caused by multi-attribute demands. Then, the decoder feeds visual features into four parallel attribute-specific feature branches to perceive the visual patterns of corresponding attributes. These features then fuse with the constraint vectors. Finally, global text features dynamically predict the attribute existence probability weights. The decoder adaptively adjusts the contribution intensity of each branch based on the attribute of the four categories. This process ensures the precise correspondence between fine-grained constraints and user intent. It guides the model to generate detection masks with clear boundaries and consistent semantics.Experimental results on the UserSOD benchmark dataset demonstrate the effectiveness of the model. Compared with the state-of-the-art methods, DCA-Net improves the S-measure, F-measure, and E-measure by 4.5%, 6.8%, and 3.8%, respectively. It also reduces the mean absolute error (MAE) to 0.028. These results effectively validate the superiority and strong robustness of the proposed architecture in complex background interference, multi-scale target capture, and abstract intent alignment.

摘要: 现有显著性目标检测(SOD)方法普遍遵循被动视觉刺激原理,依赖颜色、纹理、对比度等底层特征判定显著区域,以场景中视觉特征最强的物体作为用户关注焦点,忽视了用户主动需求对显著目标判断的决定性作用。然而在人机交互、机器人巡检等真实场景中,待检测对象常淹没于复杂背景之中,用户注意力具有强烈的意图导向。近年来,用户需求驱动的显著性目标检测(UserSOD)任务被提出,将感知范式从被动视觉响应转向主动意图匹配,并构建了相应的基准数据集与基线模型。由于现有方法在视觉特征与用户需求语义之间缺乏深层融合与动态校准,模型仅能捕捉关键词的浅层语义关联,且在分层主干网络下采样过程中易丢失空间细节,导致模型在复杂背景下难以精准定位与抽象需求匹配的目标,检测性能仍受较大限制。针对上述问题,在UserSOD任务框架下提出一种深度校准与属性感知网络(DCA-Net)。该网络以Swin-Transformer模型为视觉主干,结合预训练对比语言-图像预训练(CLIP)文本编码器作为文本分支。网络核心设计了级联语义重校准编码器(CSRE),包含四个独立的语义重校准模块(SRM),每个SRM模块内部通过跨模态交叉注意力机制在特征提取源头实现视觉特征与用户意图的逐级语义对齐 ,同时在其内部引入基于多层感知机结构的门控特征流机制(ScGate),利用全连接变换与ReLU-Tanh激活生成自适应权重以动态调节各通道语义流强度,确保在特征流早期精准锁定目标区域。同时,构建卷积跨尺度交互模块(CCIM)以实现跨层级特征补偿;CCIM利用金字塔池化聚合算子与1×1卷积聚合全局特征,通过三路并行且具备不同膨胀率的3×3深度可分离卷积分支捕获多尺度上下文信息,结合串联的通道与空间注意力模块实施加权,并利用双线性插值上采样恢复尺寸,显著增强模型对多尺度目标空间细节的感知能力。引入细粒度属性感知解码器(AGD)提供细粒度约束;AGD采用自顶向下的路径将高层特征逐级上采样合并,利用CLIP文本编码器对类别、颜色、外观和功能需求四类预设属性提示词进行初始化以提供语义先验,有效避免属性查询从随机状态收敛的弊端;随后通过多头交叉注意力从用户指令中分别抽取四类属性维度的显式语义约束向量,将抽象意图解耦为四个独立的细粒度属性约束以避免多属性需求的语义混淆;将视觉特征输入四个并行的属性专用特征分支以感知对应属性的视觉模式,并与约束向量融合;最后由全局文本特征动态预测属性存在概率权重,对四类属性进行自适应调节各分支贡献强度,确保细粒度约束与用户意图精准对应,引导模型生成边界清晰、语义一致的检测掩码。在UserSOD基准数据集上的实验结果表明,DCA-Net在S-measure、F-measure、E-measure指标上相比现有最优方法分别提升4.5%、6.8%和3.8%,MAE降低至0.028,有效验证了所提架构在复杂背景干扰、变尺度目标捕获以及抽象意图对齐方面的优越性与强鲁棒性。