领域自适应研究综述

doi:10.19678/j.issn.1000-3428.0060659

摘要/Abstract

摘要： 经典机器学习算法假设训练数据和测试数据具有相同的输入特征空间和数据分布，但在很多现实应用中这一假设通常并不成立，导致经典机器学习算法失效。领域自适应是一种新的机器学习策略，其关键技术在于通过学习新的特征表达来对齐源域和目标域的数据分布，使得在有标签源域中训练的模型可以直接迁移到没有标签的目标域上，且不会引起模型性能的明显下降。介绍领域自适应的定义、分类和代表性算法，讨论基于度量学习和基于对抗学习的两类领域自适应算法。在此基础上，分析领域自适应的典型应用和现存挑战，并对其发展趋势及未来研究方向进行展望。

关键词: 领域自适应, 迁移学习, 距离度量, 对抗学习, 单源域适应

Abstract: Classical machine learning algorithms assume that the training and testing instances share the same input feature space and data distribution.In many real-world applications, however, this assumption cannot be satisfied, resulting in the failure of the algorithms.Domain adaptation is a novel learning paradigm which aligns the data distribution of the source domain and the target domain by learning new feature representations, so that the models trained on the labeled source domain can be transferred to the unlabeled target domain without significant loss of model performance.This paper introduces the definition, classification and representative algorithms of domain adaptation, focusing on the metric learning-based algorithms and the adversarial learning-based algorithms.On this basis, the typical applications and existing challenges of domain adaptation are discussed.The paper also proposes the development trends and possible future research directions.

Key words: domain adaptation, transfer learning, distance metric, adversarial learning, single-source domain adaptation

中图分类号:

TP181

李晶晶, 孟利超, 张可, 鲁珂, 申恒涛. 领域自适应研究综述[J]. 计算机工程, 2021, 47(6): 1-13.

LI Jingjing, MENG Lichao, ZHANG Ke, LU Ke, SHEN Hengtao. Review of Studies on Domain Adaptation[J]. Computer Engineering, 2021, 47(6): 1-13.

https://www.ecice06.com/CN/Y2021/V47/I6/1

图/表 16

20210618110830

20210618110835

20210618110839

20210618110843

20210618110906

20210618110910

20210618110914

20210618110918

20210618110922

20210618110926

20210618110951

20210618110955

20210618110958

20210618111003

20210618111006

20210618111011

参考文献

[1] WANG M,DENG W.Deep visual domain adaptation:a survey[J].Neurocomputing,2018,312:135-153.
[2] TAN Chuanqi,SUN Fuchun,KONG Tao,et al.A survey on deep transfer learning[C]//Proceedings of International Conference on Artificial Neural Networks.Berlin,Germany:Springer,2018:270-279.
[3] ZHUANG Fuzhen,LUO Ping,HE Qing,et al.Research progress of transfer learning[J].Journal of Software,2015,26(1):26-39.(in Chinese)庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报,2015,26(1):26-39.
[4] LIU Jianwei,SUN Zhengkang,LUO Xionglin.Research progress of domain adaptation learning[J].Acta Automatica Sinica,2014,41(8):1576-1600.(in Chinese)刘建伟,孙正康,罗雄麟.域自适应学习研究进展[J].自动化学报,2014,41(8):1576-1600.
[5] ZHAO S,YUE X,ZHANG S,et al.A review of single-source deep unsupervised visual domain adaptation[EB/OL].[2020-12-20].https://arxiv.org/pdf/2009.00155.pdf.
[6] XU Hao,LI Zongyin,GUO Weibin.Multi-level and step-by-step domain adaptation in image classification[J].Journal of Chinese Computer Systems,2019,40(9):1921-1925.(in Chinese)许浩,李宗印,郭卫斌.多层面的分步领域适应图像分类算法[J].小型微型计算机系统,2019,40(9):1921-1925.
[7] BI Chaoyang.Research on domain adaptation method based on feature selection[D].Guangzhou:South China University of Technology,2019.(in Chinese)毕朝阳.基于特征选择的领域自适应方法研究[D].广州:华南理工大学,2019.
[8] WANG Gege,GUO Tao,YU You,et al.Unsupervised domain adaptation classification model based on generative confrontation network[J].Chinese Journal of Electronics,2019,48(6):1190-1197.(in Chinese)王格格,郭涛,余游,等.基于生成对抗网络的无监督域适应分类模型[J].电子学报,2019,48(6):1190-1197.
[9] MAO Xiaofeng.Research on deep vision domain adaptation method based on adversarial learning[D].Harbin:Harbin Engineering University,2019.(in Chinese)毛潇锋.基于对抗学习的深度视觉域适应方法研究[D].哈尔滨:哈尔滨工程大学,2019.
[10] CHEN Di.Research on domain adaptation algorithm based on confrontation and regularization method[D].Chendu:University of Electronic Science and Technology of China,2020.(in Chinese)陈迪.基于对抗和正则化方法的域适应算法研究[D].成都:电子科技大学,2020.
[11] FAN Cangning,LIU Peng,XIAO Ting,et al.A review of depth domain adaptation:general and complex situations[J].Acta Automatica Sinica,2020,47(3):515-548.(in Chinese)范苍宁,刘鹏,肖婷,等.深度域适应综述:一般情况与复杂情况[J].自动化学报,2020,47(3):515-548.
[12] CSURKA G.Domain adaptation for visual applications:a comprehensive survey[EB/OL].[2020-12-20].https://arxiv.org/pdf/1702.05374.pdf.
[13] WILSON G,COOK D J.A survey of unsupervised deep domain adaptation[J].ACM Transactions on Intelligent Systems and Technology,2020,11(5):40-46.
[14] PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering,2009,22(10):1345-1359.
[15] SAITO K,KIM D,SCLAROFF S,et al.Semi-supervised domain adaptation via minimax entropy[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2019:8050-8058.
[16] KIM T,KIM C.Attract,perturb,and explore:learning a feature alignment network for semi-supervised domain adaptation[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2020:591-607.
[17] LIU F,LU J,HAN B,et al.Butterfly:a panacea for all difficulties in wildly unsupervised domain adaptation[EB/OL].[2020-12-20].https://arxiv.org/pdf/1905.07720v2.pdf.
[18] SHU Y,CAO Z,LONG M,et al.Transferable curriculum for weakly-supervised domain adaptation[C]//Proceedings of AAAI Conference on Artificial Intelligence.New York,USA:ACM Press,2019:4951-4958.
[19] MANSOUR Y,MOHRI M,ROSTAMIZADEH A.Domain adaptation with multiple sources[J].Advances in Neural Information Processing Systems,2008,21:1041-1048.
[20] XU R,CHEN Z,ZUO W,et al.Deep cocktail network:multi-source unsupervised domain adaptation with category shift[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:3964-3973.
[21] PENG X,BAI Q,XIA X,et al.Moment matching for multi-source domain adaptation[C]//Proceedings of IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2019:1406-1415.
[22] YU H,HU M,CHEN S.Multi-target unsupervised domain adaptation without exactly shared categories[EB/OL].[2020-12-20].https://arxiv.org/pdf/1809.00852.pdf.
[23] GHOLAMI B,SAHU P,RUDOVIC O,et al.Unsupervised multi-target domain adaptation:an information theoretic approach[J].IEEE Transactions on Image Processing,2020,29:3993-4002.
[24] LI J,LU K,HUANG Z,et al.Heterogeneous domain adaptation through progressive alignment[J].IEEE Transactions on Neural Networks and Learning Systems,2018,30(5):1381-1391.
[25] MATSUURA T,HARADA T.Domain generalization using a mixture of multiple latent domains[C]//Proceedings of AAAI Conference on Artificial Intelligence.New York,USA:ACM Press,2020:11749-11756.
[26] ZHUANG F,CHENG X,LUO P,et al.Supervised representation learning:transfer learning with deep autoencoders[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2015:15-28.
[27] PAN Y,YAO T,LI Y,et al.Transferrable prototypical networks for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:2239-2247.
[28] PAN S J,TSANG I W,KWOK J T,et al.Domain adaptation via transfer component analysis[J].IEEE Transactions on Neural Networks,2010,22(2):199-210.
[29] LI J,JING M,LU K,et al.Locality preserving joint transfer for domain adaptation[J].IEEE Transactions on Image Processing,2019,28(12):6103-6115.
[30] LONG M,CAO Y,WANG J,et al.Learning transferable features with deep adaptation networks[EB/OL].[2020-12-20].http://ise.thss.tsinghua.edu.cn/~mlong/doc/deep-adaptation-networks-icml15.pdf.
[31] LONG M,ZHU H,WANG J,et al.Deep transfer learning with joint adaptation networks[C]//Proceedings of the 34th International Conference on Machine Learning.Washington D.C.,USA:IEEE Press,2017:2208-2217.
[32] LI J,LU K,HUANG Z,et al.Transfer independently together:a generalized framework for domain adaptation[J].IEEE Transactions on Cybernetics,2018,49(6):2144-2155.
[33] LI J,ZHAO J,LU K.Joint feature selection and structure preservation for domain adaptation[C]//Proceedings of International Joint Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2016:1697-1703.
[34] XU P,GURRAM P,WHIPPS G,et al.Wasserstein distance based domain adaptation for object detection[EB/OL].[2020-12-20].https://arxiv.org/pdf/1909.08675.pdf.
[35] SHEN J,QU Y,ZHANG W,et al.Wasserstein distance guided representation learning for domain adaptation[EB/OL].[2020-12-20].https://arxiv.org/pdf/1707.01217v2.pdf.
[36] LEE C Y,BATRA T,BAIG M H,et al.Sliced wasserstein discrepancy for unsupervised domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:10285-10295.
[37] LI J,CHEN E,DING Z,et al.Maximum density divergence for domain adaptation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,35(3):12-35.
[38] ZELLINGER W,GRUBINGER T,LUGHOFER E,et al.Central moment discrepancy for domain-invariant representation learning[EB/OL].[2020-12-20].https://arxiv.org/pdf/1702.08811.pdf.
[39] CHEN C,FU Z,CHEN Z,et al.HoMM:higher-order moment matching for unsupervised domain adaptation[J].Order,2020,1(10):20-21.
[40] TZENG E,HOFFMAN J,SAENKO K,et al.Adversarial discriminative domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:7167-7176.
[41] GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].The Journal of Machine Learning Research,2016,17(1):2096-2030.
[42] LONG M,CAO Z,WANG J,et al.Conditional adversarial domain adaptation[EB/OL].[2020-12-20].https://arxiv.org/pdf/1705.10667.pdf.
[43] HOFFMAN J,TZENG E,PARK T,et al.Cycada:cycle-consistent adversarial domain adaptation[C]//Proceedings of International Conference on Machine Learning.Washington D.C.,USA:IEEE Press,2018:1989-1998.
[44] TZENG E,BURNS K,SAENKO K,et al.SPLAT:semantic pixel-level adaptation transforms for detection[EB/OL].[2020-12-20].https://www.researchgate.net/publication/329387982_SPLAT_Semantic_Pixel-Level_Adaptation_Transforms_for_Detection.
[45] LI J,CHEN E,DING Z,et al.Cycle-consistent conditional adversarial transfer networks[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York,USA:ACM Press,2019:747-755.
[46] GHIFARY M,KLEIJN W B,ZHANG M,et al.Deep reconstruction-classification networks for unsupervised domain adaptation[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2016:597-613.
[47] DAI W,YANG Q,XUE G R,et al.Boosting for transfer learning[C]//Proceedings of the 24th International Conference on Machine learning.Washington D.C.,USA:IEEE Press,2007:193-200.
[48] ZHANG J,DING Z,LI W,et al.Importance weighted adversarial nets for partial domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:8156-8164.
[49] CAO Z,LONG M,WANG J,et al.Partial transfer learning with selective adversarial networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2724-2732.
[50] LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[51] NETZER Y,WANG T,COATES A,et al.Reading Digits in natural images with unsupervised feature learning[EB/OL].[2020-12-20].https://www.researchgate.net/publication/266031774_Reading_Digits_in_Natural_Images_with_Unsupervised_Feature_Learning.
[52] SAENKO K,KULIS B,FRITZ M,et al.Adapting visual category models to new domains[C]//Proceedings of European Conference on Computer Vision.Berlin,Germany:Springer,2010:213-226.
[53] ZHANG L.Transfer adaptation learning:a decade survey[EB/OL].[2020-12-20].https://arxiv.org/pdf/1903.04687.pdf.
[54] HOFFMAN J,GUADARRAMA S,TZENG E S,et al.LSDA:large scale detection through adaptation[J].Advances in Neural Information Processing Systems,2014,27:3536-3544.
[55] WANG Ling.Research and application of domain adaptation algorithm[D].Nanjing:Nanjing University of Aeronautics and Astronautics,2019.(in Chinese)王翎.域适应算法研究及应用[D].南京:南京航空航天大学,2019.
[56] CHEN Y,LI W,SAKARIDIS C,et al.Domain adaptive Faster R-CNN for object detection in the wild[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:3339-3348.
[57] HOFFMAN J,WANG D,YU F,et al.FCNs in the wild:pixel-level adversarial and constraint-based adaptation[EB/OL].[2020-12-20].https://arxiv.org/pdf/1612.02649.pdf.
[58] INOUE N,FURUTA R,YAMASAKI T,et al.Cross-domain weakly-supervised object detection through progressive domain adaptation[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:5001-5009.
[59] ZHANG Y,BARZILAY R,JAAKKOLA T.Aspect-augmented adversarial networks for domain adaptation[J].Transactions of the Association for Computational Linguistics,2017,5:515-528.
[60] LIU P,QIU X,HUANG X.Adversarial multi-task learning for text classification[EB/OL].[2020-12-20].https://arxiv.org/pdf/1704.05742.pdf.
[61] FU L,NGUYEN T H,MIN B,et al.Domain adaptation for relation extraction with domain adversarial neural network[C]//Proceedings of the 8th International Joint Conference on Natural Language Processing.New York,USA:ACM Press,2017:425-429.
[62] YANG Z,HU J,SALAKHUTDINOV R,et al.Semi-supervised QA with generative domain-adaptive nets[EB/OL].[2020-12-20].https://arxiv.org/pdf/1702.02206.pdf.
[63] BRITZ D,LE Q,PRYZANT R.Effective domain mixing for neural machine translation[C]//Proceedings of the 2nd Conference on Machine Translation.Washington D.C.,USA:IEEE Press,2017:118-126.
[64] PAN W,XIANG E W,YANG Q.Transfer learning in collaborative filtering with uncertain ratings[C]//Proceedings of AAAI Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2012:662-668.
[65] PAN W,XIANG E W,LIU N N,et al.Transfer learning in collaborative filtering for sparsity reduction[C]//Proceedings of AAAI Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2010:230-235.
[66] LI J,LU K,HUANG Z,et al.On both cold-start and long-tail recommendation with social data[J].IEEE Transactions on Knowledge and Data Engineering,2019,33(1):194-208.
[67] LI J,LU K,HUANG Z,et al.Two birds one stone:on both cold-start and long-tail recommendation[C]//Proceedings of the 25th ACM International Conference on Multimedia.New York,USA:ACM Press,2017:898-906.
[68] ZHUANG F,ZHENG J,CHEN J,et al.Transfer collaborative filtering from multiple sources via consensus regularization[J].Neural Networks,2018,108:287-295.
[69] LI J,JING M,ZHU L,et al.Learning modality-invariant latent representations for generalized zero-shot learning[C]//Proceedings of the 28th ACM International Conference on Multimedia.New York,USA:ACM Press,2020:1348-1356.
[70] LI J,JING M,LU K,et al.Leveraging the invariant side of generative zero-shot learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2019:7402-7411.
[71] LI J,JING M,LU K,et al.From zero-shot learning to cold-start recommendation[C]//Proceedings of AAAI Conference on Artificial Intelligence.Washington D.C.,USA:IEEE Press,2019:4189-4196.

选择文件类型/文献管理软件名称

选择包含的内容