Author Login Editor-in-Chief Peer Review Editor Work Office Work

Computer Engineering ›› 2021, Vol. 47 ›› Issue (7): 67-73,80. doi: 10.19678/j.issn.1000-3428.0058173

• Artificial Intelligence and Pattern Recognition • Previous Articles     Next Articles

Research on Online Sequence Extreme Learning Machine Based on Multi-Modal

LI Qi1, XIE Jun1, ZHANG Zhe2, DONG Junjie1, XU Xinying2   

  1. 1. College of Information and Computer, Taiyuan University of Technology, Jinzhong, Shanxi 030600, China;
    2. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, China
  • Received:2020-04-26 Revised:2020-06-12 Published:2020-07-01

基于多模态的在线序列极限学习机研究

李琦1, 谢珺1, 张喆2, 董俊杰1, 续欣莹2   

  1. 1. 太原理工大学 信息与计算机学院, 山西 晋中 030600;
    2. 太原理工大学 电气与动力工程学院, 太原 030024
  • 作者简介:李琦(1994-),女,硕士研究生,主研方向为计算机视觉、智能信息处理;谢珺,副教授、博士;张喆,讲师、博士;董俊杰,硕士研究生;续欣莹,教授、博士。
  • 基金资助:
    国家自然科学基金(61503271,61603267);山西省自然科学基金(201801D121144,201801D221190)。

Abstract: The object information that a single modality contains is limited,degrading the performance in object material recognition and classification.At the same time,the sample training of the traditional multi-modal fusion methods require all data to participate.To address the problem,a multi-modal online sequence extreme learning machine method with multi-scale Local Receptive Fields(LRF) is proposed.The method employs an improved feature to extract the framework of different modality samples of the objects,and then uses multi-scale local receptive fields to perceive sample information and extract the features.Different modality features are fused through the Online Sequence Extreme Learning Machine(OSELM) for training and learning.The online sequence extreme learning machine can be trained with incrementally input samples during the training process,and does not need to retrain all the data every time there is new data to be trained.The method is verified on the TUM tactile texture database.The experimental results show that the classification accuracy of fused multi-modal is higher than that of the single modality,and the improved feature extraction framework can significantly improve the classification performance.

Key words: multi-modal, RGB color three channels, Local Receptive Field(LRF), Online Sequence Extreme Learning Machine(OSELM), surface material classification

摘要: 单一模态包含的物体信息有限,导致在物体材质识别分类中表现不佳,而传统多模态融合方法在样本训练过程中需要输入所有数据。提出一种多模态的多尺度局部感受野在线序列极限学习机方法。对物体不同模态样本运用改进的特征提取框架,利用多尺度局部感受野感知样本信息提取特征,并将不同模态特征融合后通过在线序列极限学习机进行训练学习。在线序列极限学习机在训练过程中增量式地输入样本进行训练,当有新数据需要训练时无需对所有数据重新训练。在TUM触觉纹理数据库上进行验证,实验结果表明,多模态融合的分类精度高于单模态的分类精度,且改进的特征提取框架可以显著提升分类性能。

关键词: 多模态, RGB颜色三通道, 局部感受野, 在线序列极限学习机, 物体材质分类

CLC Number: