摘要：

近年来, 众多研究工作聚焦于类增量学习(CIL)领域, 其中一种常见的策略是在模型的初始化训练阶段结束后, 对特征提取器的参数进行冻结处理, 以此来维持已习得知识的稳定性。这种策略依赖于在初始训练阶段构建一个鲁棒的特征提取器, 然而在面对初始阶段数据量有限的场景时, 这种方法可能无法充分发挥其潜力, 导致增量学习效果欠佳。为了优化增量学习的初始训练环节, 应对初始阶段数据量有限的挑战, 设计了一套两步骤训练方法用于初始阶段的特征提取器训练, 称为基于类边界样本生成与微调的类增量学习初始阶段训练方法。该方法包括生成对抗训练和微调训练两个步骤。生成对抗训练采用生成对抗网络(GAN)来生成类边界附近的样本, 使用对比学习迫使类中心远离这些生成样本, 从而增强不同类别间的特征的区分度并提升模型的泛化表现。微调训练阶段结合交叉熵损失和自监督损失微调模型, 提升对初始任务基类的分类准确率。经过训练后的特征提取器用作其他增量学习方法的起点, 该方法在CIFAR-100数据集上50个基类5个增量任务设置下将FeCAM的平均准确率提升了3.8百分点, 将FeTrIL的平均准确率提升了2.2百分点。

关键词: 持续学习, 类增量学习, 对比学习, 生成对抗网络, 微调

Abstract:

Several recent studies have focused on the field of Class Incremental Learning (CIL). A common strategy involves freezing the parameters of the feature extractor after the initial training phase of the model for maintaining the stability of the learned knowledge. This strategy relies on building a robust feature extractor during the initial training phase. However, this method may not reach its full potential in scenarios with limited data during the initial phase, resulting in poor incremental learning. To optimize the initial training in incremental learning and cope with the challenge of limited data at the initial stage, a two-step training method that is the initial phase training method of CIL based on class boundary sample generation and fine-tuning, is proposed for the initial stage of feature extractor training. The method comprises two steps: generative adversarial training and fine-tuning training. Generative adversarial training uses a generative adversarial network to generate samples near the boundary of each class and uses contrastive learning to force the center of the class away from these generated samples for enhancing the discrimination of features between different classes and improving the generalization performance of the model. In the fine-tuning stage, cross-entropy loss and self-supervised loss are combined to fine-tune the model and improve the classification accuracy on the initial task. The resulting trained feature extractor is used as the starting point for other incremental learning methods. The method improves the average accuracy of FeCAM by 3.8 percentage points and FeTrIL by 2.2 percentage points on the CIFAR-100 dataset with 50 base classes and five incremental tasks.

Key words: continual learning, Class-Incremental Learning (CIL), contrastive learning, Generative Adversarial Network (GAN), fine-tuning