Author Login Editor-in-Chief Peer Review Editor Work Office Work

Computer Engineering ›› 2020, Vol. 46 ›› Issue (8): 243-249,257. doi: 10.19678/j.issn.1000-3428.0055034

• Graphics and Image Processing • Previous Articles     Next Articles

Infrared and Visible Light Image Fusion Method Based on Improved Fully Convolutional Neural Network

FENG Yufang1,2, YIN Hong1, LU Houqing1, CHENG Kai1, CAO Lin1, LIU Man1   

  1. 1. Graduate School, Army Engineering University, Nanjing 210007, China;
    2. 71375 Troops of the People's Liberation Army of China, Harbin 150038, China
  • Received:2019-05-27 Revised:2019-08-17 Published:2019-08-28

基于改进全卷积神经网络的红外与可见光图像融合方法

冯玉芳1,2, 殷宏1, 卢厚清1, 程恺1, 曹林1, 刘满1   

  1. 1. 陆军工程大学 研究生院, 南京 210007;
    2. 中国人民解放军 71375部队, 哈尔滨 150038
  • 作者简介:冯玉芳(1981-),女,博士研究生,主研方向为图像处理、模式识别;殷宏、卢厚清,教授、博士、博士生导师;程恺,副教授、博士;曹林,工程师、博士研究生;刘满,博士研究生。
  • 基金资助:
    国家自然科学基金(61806221)。

Abstract: Image fusion technology based on deep learning is easy to lose the shallow feature information of the network and difficult to recognize the image accurately.For this reason,this paper proposes an infrared and visible light image fusion method that uses improved Fully Convolutional Neural Network(FCN).The Non-Subsampled Shearlet Transform(NSST) is used to decompose the source image in a multi-scale and multi-directional way to generate high-frequency and low-frequency sub-band images.Then the high-frequency sub-band is input into the FCN model to extract multi-scale features,and the high-frequency sub-band feature mapping graph is generated.The maximum weighted average algorithm is used to complete the fusion of high-frequency sub-band.At the same time,the local energy and fusion strategy are used to fuse the low-frequency sub-band,and the final fusion image is obtained by implementing NSST inverse transform on the fused high frequency sub-band and low frequency sub-band.Experimental results show that compared with GFF,WLS,IFE and other methods,the fusion method provides better visual effects of fused images and evaluation results of indexes.

Key words: Convolutional Neural Network(CNN), Non-Subsampled Shearlet Transform(NSST), Fully Convolutional Neural Network(FCN) model, multi-scale feature, feature mapping graph

摘要: 基于深度学习的图像融合技术易丢失网络浅层特征信息,难以实现图像的精准识别。提出一种利用全卷积神经网络(FCN)提取特征的红外与可见光图像融合方法。采用非下采样剪切波变换(NSST)对源图像进行多尺度和多方向分解,生成高频子带和低频子带图像,将高频子带输入FCN模型提取多尺度特征,并生成高频子带特征映射图,使用最大加权平均算法完成高频子带的融合,同时采用区域能量和融合策略融合低频子带,对融合后的高频子带和低频子带进行NSST逆变换,得到最终的融合图像。实验结果表明,与GFF、WLS和IFE等方法相比,该方法融合图像的主观视觉效果和客观评价指标更好。

关键词: 卷积神经网络, 非下采样剪切波变换, 全卷积神经网络模型, 多尺度特征, 特征映射图

CLC Number: