Abstract: As a deep learning paradigm which can train in continuous data streams, continuous learning is suitable for increasingly open and complex intelligent application scenarios. The main challenge for incremental learning is catastrophic forgetting, which refers to the precipitous drop in performance on previously learned tasks after learning a new one. The state of art works on continuous learning ignore the impact of uncertainty on model training. In addition, existing works mainly focus on mitigating forgetting in phases after the initial one while the role of the initial phase is largely neglected. Motivated by this, we propose a trusted continuous learning method based on uncertainty correction, which constrains the uncertainty of the model at the initial stage. Thus, this constraint can alleviate errors caused by model parameter drift and the catastrophic forgetting can be relieved. Our method can be combined with other continuous learning methods, so it is pretty universal, for example, we improve three classic methods in continuous learning by our method, and the experimental results show that improved models outperform the original ones: the average accuracy is improved by 1.2% to 19.1% on two datasets. Moreover, we use expected calibration error to evaluate the reliability of the models. Experimental results show that the models improved by our method have lower expected calibration error, which proves our method can improve the reliability of original ones.

摘要： 连续学习是一种能在连续的数据流中训练模型的深度学习范式，适合日益开放复杂的智能应用场景。连续学习最大的挑战是“灾难性遗忘”，即模型在学习新知识后会遗忘过去学习的知识。现有的连续学习忽视了不确定性对模型训练的影响，且大多数方法都聚焦于连续学习后续阶段的改进，对模型初始阶段研究较少。本文提出一种基于不确定性校正的可信连续学习方法，通过在初始阶段约束模型输出的不确定性来弥补模型参数漂移带来的分类误差，从而缓解“灾难性遗忘”。本文方法可以与其它连续学习方法相结合并形成改进模型，具有较强的通用性。本文对三种经典的连续学习方法进行改进，实验结果表明均能有效提高原始模型的性能：在两个数据集上平均准确率提升1.2%-19.1%。除此之外，本文引入期望校准误差评判连续学习模型的可靠性，实验表明，对比原始模型，基于本文方法的改进模型具有更低的期望校准误差，这证明基于本文方法的改进模型更加可信。