一种半监督对抗鲁棒模型无关元学习方法

doi:10.19678/j.issn.1000-3428.0063365

摘要/Abstract

摘要： 元学习期望训练所得的元模型在学习到的“元知识”基础上利用来自新任务的少量标注样本，仅通过较少的梯度下降步骤微调模型就能够快速适应该任务。但是，由于缺乏训练样本，元学习算法在元训练期间对现有任务过度训练时所得的分类器决策边界不够准确，不合理的决策边界使得元模型更容易受到微小对抗扰动的影响，导致元模型在新任务上的鲁棒性能降低。提出一种半监督对抗鲁棒模型无关元学习（semi-ARMAML）方法，在目标函数中分别引入半监督的对抗鲁棒正则项和基于信息熵的任务无偏正则项，以此优化决策边界，其中对抗鲁棒正则项的计算允许未标注样本包含未见过类样本，从而使得元模型能更好地适应真实应用场景，降低对输入扰动的敏感性，提高对抗鲁棒性。实验结果表明，相比ADML、R-MAML-TRADES等当下主流的对抗元学习方法，semi-ARMAML方法在干净样本上准确率较高，在MiniImageNet数据集的5-way 1-shot与5-way 5-shot任务上对抗鲁棒性能分别约提升1.8%和2.7%，在CIFAR-FS数据集上分别约提升5.2%和8.1%。

关键词: 少样本学习, 元学习, 对抗训练, 半监督学习, 对抗鲁棒性

Abstract: The meta model developed by meta learning is expected to have the ability of "learning to learn". Therefore, it can quickly adapt to new tasks based on the learned "meta knowledge", with a small number of gradient descent steps to fine-tune the model using a few labeled training data from new tasks.However, owing to the scarcity of training sampling data, when the meta-learning algorithm overtrains the existing tasks during the meta-training phase, the decision boundary trained by the meta learner is not sufficiently accurate.The unreasonable decision boundary makes the meta model more vulnerable to adversarial perturbation, which may lead to poor robustness performance of the meta model on new tasks.Therefore, a semi-supervised Adversarial Robust Model-Agnostic Meta-Learning(semi-ARMAML) method is proposed.A semi-supervised adversarial robust regularizer and a task-agnostic regularizer based on information entropy are integrated into the objective function to optimize the decision boundary.In particular, the calculation of the adversarial robust regularizer allows the unlabeled dataset to contain the classes unseen in the labeled dataset.The developed meta model performs better on new tasks of real application scenes and is more robust on input disturbances.The experimental results indicate that the scheme has a higher accuracy rate on clean samples.Compared with current mainstream adversarial meta-learning schemes such as ADML and R-MAML-TRADES, the adversarial robustness performance of the semi-ARMAML method improved by approximately 1.8% and 2.7% on the five-way one-shot and five-way five-shot tasks of the MiniImageNet dataset, respectively, and by approximately 5.2% and 8.1% on the five-way one-shot and five-way five-shot tasks of the CIFAR-FS dataset, respectively.

Key words: Few-Shot Learning(FSL), meta-learning, adversarial training, semi-supervised learning, adversarial robustness

中图分类号:

TP181

胡彬, 王晓军, 张雷. 一种半监督对抗鲁棒模型无关元学习方法[J]. 计算机工程, 2022, 48(12): 112-118.

HU Bin, WANG Xiaojun, ZHANG Lei. A Semi-Supervised Adversarial Robust Model-Agnostic Meta-Learning Method[J]. Computer Engineering, 2022, 48(12): 112-118.

http://www.ecice06.com/CN/Y2022/V48/I12/112

图/表 8

20230112183108

20230112183112

20230112183116

20230112183120

20230112183123

20230112183126

20230112183130

20230112183134

参考文献

[1] 代磊超, 冯林, 杨玉亭, 等.一种鲁棒性的少样本学习方法[J].小型微型计算机系统, 2021, 42(2):340-347. DAI L C, FENG L, YANG Y T, et al.Robust few-shot learning method[J].Journal of Chinese Computer Systems, 2021, 42(2):340-347.(in Chinese)
[2] RAVI S, LAROCHELLE H.Optimization as a model for few-shot learning[EB/OL].[2021-10-05].https://openreview.net/pdf?id=rJY0-Kcll.
[3] KINGMA D, BA J.Adam:a method for stochastic optimization[EB/OL].[2021-10-05].https://arxiv.org/pdf/1412.6980.pdf.
[4] DUCHI J, HAZAN E, SINGER Y.Adaptive subgradient methods for online learning and stochastic optimization[EB/OL].[2021-10-05].https://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf.
[5] 赵凯琳, 靳小龙, 王元卓.小样本学习研究综述[J].软件学报, 2021, 32(2):349-369. ZHAO K L, JIN X L, WANG Y Z.Survey on few-shot learning[J].Journal of Software, 2021, 32(2):349-369.(in Chinese)
[6] HOWARD J, RUDER S.Universal language model fine-tuning for text classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg, USA:Association for Computational Linguistics, 2018:328-339.
[7] GAO R, HOU X S, QIN J, et al.Zero-VAE-GAN:generating unseen features for generalized and transductive zero-shot learning[J].IEEE Transactions on Image Processing, 2020, 29:3665-3680.
[8] LIU L L, ZHANG H J, XU X F, et al.Collocating clothes with generative adversarial networks cosupervised by categories and attributes:a multidiscriminator framework[J].IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(9):3540-3554.
[9] WEI Y Y, ZHANG Z, WANG Y, et al.DerainCycleGAN:rain attentive CycleGAN for single image deraining and rainmaking[J].IEEE Transactions on Image Processing, 2021, 30:4788-4801.
[10] WANG Y X, HEBERT M.Learning to learn:model regression networks for easy small sample learning[C]//Proceedings of European Conference on Computer Vision.Berlin, Germany:Springer, 2016:616-634.
[11] 汪荣贵, 汤明空, 杨娟, 等.语义匹配网络的小样本学习[J].计算机工程, 2021, 47(5):244-250, 259. WANG R G, TANG M K, YANG J, et al.Semantic matching network for few-shot learning[J].Computer Engineering, 2021, 47(5):244-250, 259.(in Chinese)
[12] GIDARIS S, KOMODAKIS N.Generating classification weights with GNN denoising autoencoders for few-shot learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:21-30.
[13] THRUN S, PRATT L.Learning to learn:introduction and overview[M].Berlin, Germany:Springer, 1998.
[14] FINN C, ABBEEL P, LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of International Conference on Machine Learning.Washington D.C., USA:IEEE Press, 2017:1126-1135.
[15] GOLDBLUM M, FOWL L, GOLDSTEIN T.Adversarially robust few-shot learning:a meta-learning approach[J].Advances in Neural Information Processing Systems, 2020, 33:12-36.
[16] SHIN H, KIM D, KWON Y, et al.Illusion and dazzle:adversarial optical channel exploits against lidars for automotive applications[C]//Proceedings of International Conference on Cryptographic Hardware and Embedded Systems.Berlin, Germany:Springer, 2017:445-467.
[17] SHAN S, WENGER E, ZHANG J, et al.Fawkes:protecting privacy against unauthorized deep learning models[EB/OL].[2021-10-05].https://arxiv.org/pdf/2002.08327.pdf.
[18] YIN C, TANG J, XU Z, et al.Adversarial meta-learning[EB/OL].[2021-10-05].https://arxiv.org/pdf/1806.03316.pdf.
[19] GOODFELLOW I J, SHLENS J, SZEGEDY C.Explaining and harnessing adversarial examples[EB/OL].[2021-10-05].https://arxiv.org/pdf/1412.6572.pdf.
[20] MADRY A, MAKELOV A, SCHMIDT L, et al.Towards deep learning models resistant to adversarial attacks[EB/OL].[2021-10-05].https://arxiv.org/abs/1706.06083.
[21] WANG R, XU K, LIU S, et al.On fast adversarial robustness adaptation in model-agnostic meta-learning[EB/OL].[2021-10-05].https://arxiv.org/pdf/2102.10454v1.pdf.
[22] REN M, TRIANTAFILLOU E, RAVI S, et al.Meta-learning for semi-supervised few-shot classification[EB/OL].[2021-10-05].https://arxiv.org/pdf/1803.00676.pdf.
[23] HUANG S X, ZENG X P, WU S, et al.Behavior regularized prototypical networks for semi-supervised few-shot image classification[J].Pattern Recognition, 2021, 112:107765.
[24] MIYATO T, MAEDA S I, KOYAMA M, et al.Virtual adversarial training:a regularization method for supervised and semi-supervised learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8):1979-1993.
[25] JAMAL M A, QI G J.Task agnostic meta-learning for few-shot learning[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition.Washington D.C., USA:IEEE Press, 2019:11719-11727.

选择文件类型/文献管理软件名称

选择包含的内容