原型对齐和域感知的零样本哈希

doi:10.19678/j.issn.1000-3428.0067994

摘要/Abstract

摘要： 为了实现对未见类别图像的有效检索,零样本哈希(ZSH)方法通常将类别属性中的监督知识从已见类转移到未见类。然而,获取类别属性需要花费额外的计算资源,并且视觉特征和类别属性之间存在跨模态的异构鸿沟。此外,现有方法忽视了强偏差问题,导致模型错误地将已见类样本识别为未见类,从而降低了检索精度。与此同时,ZSH在保持哈希码和原始数据语义一致性以及实现哈希码的离散优化等方面也面临着挑战。为此,提出一种原型对齐和域感知的ZSH方法,其不依赖类别属性等特殊监督知识,能够节省注释属性的花销同时避免跨模态异构鸿沟的影响。首先计算各类样本在海明空间中的原型,然后通过对齐哈希码和类原型来学习语义一致的哈希码。为了避免松弛策略造成的量化误差,提出一种离散优化算法来求解哈希码的离散约束,并且实现线性的计算复杂度。同时,设计一个域感知策略用于分离源域和目标域样本,以缓解强偏差问题的影响。在aPY、AWA2和ImageNet数据集上的实验结果表明,该方法的检索精度相较对比方法中的最优值分别提升了2.6、9.4和14.9个百分点,训练时间也远低于大部分对比方法。

关键词: 哈希, 图像检索, 零样本学习, 原型对齐, 域感知

Abstract: Zero-Shot Hashing (ZSH) methods typically rely on category attributes to transfer supervised knowledge from seen to unseen classes, and efficiently retrieve unseen category images. However, obtaining category attributes requires additional computing resources, and a heterogeneous gap exists between visual features and category attributes. Additionally, existing methods ignore the strong bias problem, leading the model to mistakenly identify seen class samples as unseen classes, thereby reducing retrieval accuracy. Moreover, ZSH encounters challenges in preserving semantic consistency between hash codes and the original data, as well as in achieving discrete optimization of hash codes. To address these challenges, a prototype-aligned and domain-aware ZSH method is proposed. It eliminates the dependence on special supervision knowledge, such as category attributes, thus reducing the cost of attribute annotation and avoiding the impact of cross-modal heterogeneous gaps. Specifically, the method first calculates prototypes for each class in the Hamming space and thereafter aligns the hash codes with the class prototypes to learn semantically consistent hash codes. To mitigate the quantization errors caused by relaxation strategies, a discrete optimization algorithm is proposed to address the discrete constraints of hash codes and achieve linear computational complexity. Moreover, a domain-aware strategy is used to separate the samples from the source and target domains, thereby alleviating the strong bias problem. The experimental results on the aPY, AWA2, and ImageNet datasets demonstrate that the retrieval accuracy of this method is improved by 2.6, 9.4, and 14.9 percentage points compared to the optimal values in the comparison method, respectively, and the training time is significantly reduced compared to most baseline methods.

Key words: hashing, image retrieval, Zero-Shot Learning(ZSL), prototype alignment, domain awareness

中图分类号:

TP391

董峰, 王永欣, 马玉玲, 王奎奎. 原型对齐和域感知的零样本哈希[J]. 计算机工程, 2024, 50(5): 260-271.

DONG Feng, WANG Yongxin, MA Yuling, WANG Kuikui. Prototype-Aligned and Domain-Aware Zero-Shot Hashing[J]. Computer Engineering, 2024, 50(5): 260-271.

https://www.ecice06.com/CN/Y2024/V50/I5/260

参考文献

[1] WANG J D, ZHANG T, SONG J K, et al. A survey on learning to hash[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):769-790.
[2] 代亚兰, 何朗, 黄樟灿. 基于稀疏自编码的无监督图像哈希算法[J]. 计算机工程, 2019, 45(5):222-225, 236. DAI Y L, HE L, HUANG Z C. Unsupervised image hashing algorithm based on sparse-autoencoder[J]. Computer Engineering, 2019, 45(5):222-225, 236.(in Chinese)
[3] 张仕, 赖会霞, 肖如良, 等. 开放环境多分布特性的局部敏感哈希检索方法[J]. 软件学报, 2022, 33(4):1200-1217. ZHANG S, LAI H X, XIAO R L, et al. Open environmental locality-sensitive hashing retrieval for multiple distributed characteristics[J]. Journal of Software, 2022, 33(4):1200-1217.(in Chinese)
[4] WEISS Y, TORRALBA A, FERGUS R. Spectral hashing[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York,USA:ACM Press,2008:1753-1760.
[5] LIU W, MU C, KUMAR S, et al. Discrete graph hashing[C]//Proceedings of Annual Conference on Neural Information Processing Systems. New York,USA:ACM Press,2014:3419-3427.
[6] LAI Z H, CHEN Y D, WU J, et al. Jointly sparse hashing for image retrieval[J]. IEEE Transactions on Image Processing, 2018, 27(12):6147-6158.
[7] JIN S, YAO H X, ZHOU Q, et al. Unsupervised discrete hashing with affinity similarity[J]. IEEE Transactions on Image Processing, 2021, 30:6130-6141.
[8] SHEN F M, SHEN C H, LIU W, et al. Supervised discrete hashing[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2015:37-45.
[9] KANG W C, LI W J, ZHOU Z H. Column sampling based discrete supervised hashing[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence.[S.l.]:AAAI Press,2016:1230-1236.
[10] CHEN Y, TIAN Z, ZHANG H, et al. Strongly constrained discrete hashing[J]. IEEE Transactions on Image Processing, 2020, 29(1):3596-3611.
[11] ZHANG Z, ZHU X, LU G, et al. Probability ordinal-preserving semantic hashing for large-scale image retrieval[J]. ACM Transactions on Knowledge Discovery from Data, 2021, 15(3):1-22.
[12] LAMPERT C H, NICKISCH H, HARMELING S. Learning to detect unseen object classes by between-class attribute transfer[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2009:951-958.
[13] XIAN Y Q, LAMPERT C H, SCHIELE B, et al. Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(9):2251-2265.
[14] YANG Y, LUO Y, CHEN W, et al. Zero-shot hashing via transferring supervised knowledge[C]//Proceedings of ACM International Conference on Multimedia. New York,USA:ACM Press,2016:1286-1295.
[15] XU Y H, YANG Y, SHEN F M, et al. Attribute hashing for zero-shot image retrieval[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Washington D.C.,USA:IEEE Press,2017:133-138.
[16] ZHANG H F, LONG Y, SHAO L. Zero-shot hashing with orthogonal projection for image retrieval[J]. Pattern Recognition Letters, 2019, 117:201-209.
[17] GUO Y, DING G, HAN J, et al. SitNet:discrete similarity transfer network for zero-shot hashing[C]//Proceedings of International Joint Conference on Artificial Intelligence. New York,USA:ACM Press,2017:1767-1773.
[18] SHI Y, NIE X, LIU X, et al. Zero-shot hashing via asymmetric ratio similarity matrix[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(5):5426-5437.
[19] WU J, ZHANG T, ZHA Z, et al. Self-supervised domain-aware generative network for generalized zero-shot learning[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2020:12764-12773.
[20] ROMERA-PAREDES B, TORR P. An embarrassingly simple approach to zero-shot learning[C]//Proceedings of International Conference on Machine Learning. New York,USA:ACM Press,2015:2152-2161.
[21] AKATA Z, PERRONNIN F, HARCHAOUI Z, et al. Label-embedding for image classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(7):1425-1438.
[22] MISHRA A, REDDY S K, MITTAL A, et al. A generative model for zero shot learning using conditional variational autoencoders[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2018:2188-2196.
[23] XIAN Y Q, SHARMA S, SCHIELE B, et al. F-VAEGAN-D2:a feature generating framework for any-shot learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2019:10275-10284.
[24] LIN M B, JI R R, LIU H, et al. Hadamard matrix guided online hashing[J]. International Journal of Computer Vision, 2020, 128(8):2279-2306.
[25] RUDIN W. Principles of mathematical analysis[M]. 3rd ed. New York,USA:McGraw-Hill, 1976.
[26] FARHADI A, ENDRES I, HOIEM D, et al. Describing objects by their attributes[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2009:1778-1785.
[27] RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3):211-252.
[28] CHANGPINYO S, CHAO W L, GONG B Q, et al. Synthesized classifiers for zero-shot learning[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C.,USA:IEEE Press,2016:5327-5336.
[29] JIN Z M, LI C, LIN Y, et al. Density sensitive hashing[J]. IEEE Transactions on Cybernetics, 2014, 44(8):1362-1371.
[30] GUI J, LIU T L, SUN Z N, et al. Fast supervised discrete hashing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(2):490-496.
[31] SHI X S, XING F Y, ZHANG Z Z, et al. A scalable optimization mechanism for pairwise based discrete hashing[J]. IEEE Transactions on Image Processing, 2021, 30:1130-1142.

选择文件类型/文献管理软件名称

选择包含的内容