基于改进ConvNeXt的皮肤镜图像分类方法

doi:10.19678/j.issn.1000-3428.0066050

计算机工程 ›› 2023, Vol. 49 ›› Issue (10): 239-246, 254. doi: 10.19678/j.issn.1000-3428.0066050

基于改进ConvNeXt的皮肤镜图像分类方法

李建威¹, 吕晓琪¹^,²^,*, 谷宇¹

1. 内蒙古科技大学信息工程学院内蒙古自治区模式识别与智能图像处理重点实验室, 内蒙古自治区包头 014010
2. 内蒙古工业大学信息工程学院, 呼和浩特 010051

收稿日期:2022-10-20 出版日期:2023-10-15 发布日期:2023-01-03
通讯作者: 吕晓琪
作者简介:
李建威(1997-), 男, 硕士研究生, 主研方向为医学图像处理、深度学习
谷宇, 副教授、博士
基金资助:
国家自然科学基金(62001255); 国家自然科学基金(61841204); 国家自然科学基金(61771266); 中央引导地方科技发展资金(2021ZY0004); 内蒙古自治区自然科学基金(2019MS06003); 内蒙古自治区自然科学基金(2015MS0604); 内蒙古自治区高等学校青年科技英才计划项目(NJYT23057); 教育部“春晖计划”合作科研项目(教外司留[2019]1383号); 内蒙古科技大学基本科研业务费专项资金优秀青年基金(2022042)

Dermoscopy Image Classification Method Based on Improved ConvNeXt

Jianwei LI¹, Xiaoqi LÜ¹^,²^,*, Yu GU¹

1. Inner Mongolia Key Laboratory of Pattern Recognition and Intelligent Image Progressing, School of Information Engineering, Inner Mongolia University of Science and Technology, Baotou 014010, Inner Mongolia Autonomous Region, China
2. School of Information Engineering, Inner Mongolia University of Technology, Hohhot 010051, China

Received:2022-10-20 Online:2023-10-15 Published:2023-01-03
Contact: Xiaoqi LÜ

摘要/Abstract

摘要：

皮肤癌是最致命的癌症之一，对皮肤镜图像进行精确分类尤为关键，然而现有的皮肤镜图像存在形态复杂、样本数量较少的问题，导致现有的自动分类方法难以提取图像特征信息，误判率较高。提出一种改进ConvNeXt的方法，并构建SE-SimAM-ConvNeXt模型。以ConvNeXt为基础网络，加入SimAM无参注意力模块，提升网络的特征提取能力，并在基础网络中引入通道注意力机制，增强ConvNeXt对潜在关键特征的挖掘能力。在训练初始时加入预热机制Cosine Warmup，在该过程中使用余弦函数值进行学习率的衰减，进一步加速ConvNeXt的收敛，提高ConvNeXt模型的分类能力。在HAM10000皮肤数据集上的实验结果表明，该模型的分类准确率、精确度、召回率、特异性分别为92.9%、85.3%、78.0%、97.5%，具有较好的皮肤镜图像分类能力，对皮肤癌病变的辅助诊断有一定程度的应用价值，可帮助皮肤科医生对皮肤癌做进一步的诊断。

关键词: 皮肤镜图像分类, ConvNeXt网络, 通道注意力机制, SimAM无参注意力, 预热机制

Abstract:

Skin cancer is one of the deadliest cancers, and it is particularly critical to accurately classify dermoscopy images. However, the existing dermoscopy images have complex shapes and a small number of samples, which makes it difficult for the existing automatic classification methods to extract image feature information; these methods also have a high error rate. To solve this problem, this paper proposes an improved ConvNeXt method and build, SE-SimAM-ConvNeXt model. First, with ConvNeXt as the basic network, the SimAM nonparametric attention module is added to improve the network's feature extraction capability. Second, channel attention is added to the basic network to enhance the mining ability of ConvNeXt for potential key features. Finally, the Cosine Warmup mechanism is added at the beginning of training, and the cosine function value is used to attenuate the learning rate during the process, further accelerating the convergence of ConvNeXt and improving the classification ability of the ConvNeXt model. The experimental results on the HAM10000 skin dataset show that the classification accuracy, precision, recall, and specificity of the model reach 92.9%, 85.3%, 78.0%, and 97.5%, respectively, and is demonstrated effective classification capability for dermoscopy images. This bears significant potential in aiding the auxiliary diagnosis of skin cancer lesions, providing valuable assistance to dermatologists in making accurate diagnoses of skin cancer.

Key words: dermoscopy image classification, ConvNeXt network, channel attention mechanism, SimAM without reference attention, warmup mechanism

李建威, 吕晓琪, 谷宇. 基于改进ConvNeXt的皮肤镜图像分类方法[J]. 计算机工程, 2023, 49(10): 239-246, 254.

Jianwei LI, Xiaoqi LÜ, Yu GU. Dermoscopy Image Classification Method Based on Improved ConvNeXt[J]. Computer Engineering, 2023, 49(10): 239-246, 254.

https://www.ecice06.com/CN/Y2023/V49/I10/239

图/表 22

图1 ConvNeXt模型结构

Fig.1 Model structure of ConvNeXt

图2 ConvNeXt模型结构的修改

Fig.2 Modification of ConvNeXt model structure

图3 Swin Transformer模块与ConvNeXt模块的对比

Fig.3 Comparison of Swin Transformer module and ConvNeXt module

图4 SimAM模块的结构

Fig.4 Structure of SimAM module

图5 SE模块的结构

Fig.5 Structure of SE block

图6 使用Cosine Warmup的学习率系数变化

Fig.6 Change in learning rate coefficient using Cosine Warmup

图7 皮肤癌示例

Fig.7 Examples of skin cancer

图8 数据增强示例

Fig.8 Examples of data augmentation

图9 多分类混淆矩阵示例

Fig.9 Examples of multi category confusion matrix

图10 Cosine Warmup对损失值的影响（无预训练）

Fig.10 Influence of Cosine Warmup on loss value (without pre-training)

图11 Cosine Warmup对损失值的影响（有预训练）

Fig.11 Influence of Cosine Warmup on loss value (with pre-training)

图12 ConvNeXt模型结构

Fig.12 Structure of ConvNeXt model

图13 不同插入位置的SimAM模块

Fig.13 SimAM modules with different insertion positions

图14 不同插入位置的SE模块

Fig.14 SE modules with different insertion positions

图15 同时加入SimAM模块和SE模块后的ConvNeXt模型结构

Fig.15 ConvNeXt model structure after adding both SimAM and SE modules

参考文献 30

1	JANA G K , GUPTA A , DAS A , et al. Herbal treatment to skin diseases: a global approach. Drug Invention Today, 2010, 2 (8): 54- 67.
2	唐超颖. 基于可见光图像估计黑色素分布的光学快捷算法. 计算机工程, 2016, 42 (8): 227- 232. doi: 10.3969/j.issn.1000-3428.2016.08.040
	TANG C Y . Fast optical algorithm for melanin distribution estimation based on visible image. Computer Engineering, 2016, 42 (8): 227- 232. doi: 10.3969/j.issn.1000-3428.2016.08.040
3	PATHAN S , PRABHU K G , SIDDALINGASWAMY P C . Techniques and algorithms for computer aided diagnosis of pigmented skin lesions-a review. Biomedical Signal Processing and Control, 2018, 39, 237- 262. doi: 10.1016/j.bspc.2017.07.010
4	LU X , GU Y , YANG L , et al. Multi-level 3D densenets for false-positive reduction in lung nodule detection based on chest computed tomography. Current Medical Imaging, 2020, 16 (8): 1004- 1021. doi: 10.2174/1573405615666191113122840
5	赵文慧, 杨霄, 孟丽洁. 充分利用多尺度特征改进UNet实现皮肤病变分割. 电子测量技术, 2022, 45 (2): 110- 116. doi: 10.19651/j.cnki.emt.2108213
	ZHAO W H , YANG X , MENG L J . Improved UNet for skin lesion segmentation by leveraging multi-scale features. Electronic Measurement Technology, 2022, 45 (2): 110- 116. doi: 10.19651/j.cnki.emt.2108213
6	HUANG H W , HSU B W Y , LEE C H , et al. Development of a light-weight deep learning model for cloud applications and remote diagnosis of skin cancers. The Journal of Dermatology, 2021, 48 (3): 310- 316. doi: 10.1111/1346-8138.15683
7	THURNHOFER-HEMSI K , DOMÍNGUEZ E . A convolutional neural network framework for accurate skin cancer detection. Neural Processing Letters, 2020, 53 (5): 1- 21.
8	KHAN M A , MUHAMMAD K , SHARIF M , et al. Intelligent fusion-assisted skin lesion localization and classification for smart healthcare. Neural Computing and Applications, 2021, 43 (11): 1- 16.
9	KHAN M A , ZHANG Y D , SHARIF M , et al. Pixels to classes: intelligent learning framework for multiclass skin lesion localization and classification. Computers & Electrical Engineering, 2021, 90, 106956.
10	SAE-LIM W, WETTAYAPRASIT W, AIYARAK P. Convolutional neural networks using MobileNet for skin lesion classification[C]//Proceedings of the 16th International Joint Conference on Computer Science and Software Engineering. Washington D.C., USA: IEEE Press, 2019: 242-247.
11	AL-MASNI M A , KIM D H , KIM T S . Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Computer Methods and Programs in Biomedicine, 2020, 190, 105351. doi: 10.1016/j.cmpb.2020.105351
12	GONG A , YAO X J , LIN W . Classification for dermoscopy images using convolutional neural networks based on the ensemble of individual advantage and group decision. IEEE Access, 2020, 8, 155337- 155351. doi: 10.1109/ACCESS.2020.3019210
13	罗清, 周维, 马梓钧, 等. 基于FL-ResNet50的皮肤镜图像分类方法. 激光与光电子学进展, 2020, 57 (18): 181022. URL
	LUO Q , ZHOU W , MA Z J , et al. Dermatoscope image classification method based on FL-ResNet50. Laser & Optoelectronics Progress, 2020, 57 (18): 181022. URL
14	GU Y , CHI J Q , LIU J Q , et al. A survey of computer-aided diagnosis of lung nodules from CT scans using deep learning. Computers in Biology and Medicine, 2021, 137, 104806.
15	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2022: 9992-10002.
16	LIU Z, MAO H Z, WU C Y, et al. A ConvNet for the 2020s[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2022: 11966-11976.
17	XIE S N, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 5987-5995.
18	SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1801.04381.
19	GU Y , LU X Q , ZHANG B H , et al. Automatic lung nodule detection using multi-scale dot nodule-enhancement filter and weighted support vector machines in chest computed tomography. PLoS One, 2019, 14 (1): e0210551.
20	YANG L, ZHANG R Y, LI L, et al. Simam: a simple, parameter-free attention module for convolutional neural networks[C]//Proceedings of International Conference on Machine Learning. Washington D.C., USA: IEEE Press, 2021: 11863-11874.
21	WEBB B S , DHRUV N T , SOLOMON S G , et al. Early and late mechanisms of surround suppression in striate cortex of macaque. The Journal of Neuroscience, 2005, 25 (50): 11666- 11675.
22	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1709.01507.
23	GU Y , LU X Q , YANG L D , et al. Automatic lung nodule detection using a 3D deep convolutional neural network combined with a multi-scale prediction strategy in chest CTs. Computers in Biology and Medicine, 2018, 103, 220- 231.
24	GOYAL P, DOLLÁR P, GIRSHICK R, et al. Accurate, large minibatch SGD: training ImageNet in 1 hour[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1706.02677.
25	HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 558-567.
26	CODELLA N, ROTEMBERG V, TSCHANDL P, et al. Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1902.03368.
27	TSCHANDL P , ROSENDAHL C , KITTLER H . The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 2018, 5, 180161.
28	谷宇. 基于深度卷积神经网络的CT影像肺结节检测技术研究[D]. 上海: 上海大学, 2019.
	GU Y. Research on detection technology of lung nodules in CT images based on deep convolution neural network[D]. Shanghai: Shanghai University, 2019. (in Chinese)
29	TOUVRON H, CORD M, SABLAYROLLES A, et al. Going deeper with image transformers[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2022: 32-42.
30	MATSOUKAS C, HASLUM J F, SÖDERBERG M, et al. Is it time to replace CNNs with transformers for medical images?[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2108.09038.

[1]	杨硕, 王一丁. 基于改进薄板样条运动模型的人脸动画算法[J]. 计算机工程, 2024, 50(6): 255-265.
[2]	白杰, 张赛, 李艳萍. 基于改进交错组卷积的眼底硬性渗出物自动分割[J]. 计算机工程, 2022, 48(7): 307-314.
[3]	张云帆, 易尧华, 汤梓伟, 王新宇. 基于通道注意力机制的文本生成图像方法[J]. 计算机工程, 2022, 48(4): 206-212,222.
[4]	宋佳, 陈程立诏. 基于多流网络一致性的视频显著性检测[J]. 计算机工程, 2022, 48(2): 215-223.
[5]	武茜, 贾世杰. 基于多通道注意力机制的人脸替换鉴别[J]. 计算机工程, 2022, 48(2): 180-185,193.
[6]	张海涛, 张梦. 引入通道注意力机制的SSD目标检测算法[J]. 计算机工程, 2020, 46(8): 264-270.

选择文件类型/文献管理软件名称

选择包含的内容

基于改进ConvNeXt的皮肤镜图像分类方法

Dermoscopy Image Classification Method Based on Improved ConvNeXt

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 22

参考文献 30

相关文章 6

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于改进ConvNeXt的皮肤镜图像分类方法

Dermoscopy Image Classification Method Based on Improved ConvNeXt

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 22

参考文献 30

相关文章 6

编辑推荐

Metrics

本文评价