Dermoscopy Image Classification Method Based on Improved ConvNeXt

doi:10.19678/j.issn.1000-3428.0066050

Abstract

Abstract:

Skin cancer is one of the deadliest cancers, and it is particularly critical to accurately classify dermoscopy images. However, the existing dermoscopy images have complex shapes and a small number of samples, which makes it difficult for the existing automatic classification methods to extract image feature information; these methods also have a high error rate. To solve this problem, this paper proposes an improved ConvNeXt method and build, SE-SimAM-ConvNeXt model. First, with ConvNeXt as the basic network, the SimAM nonparametric attention module is added to improve the network's feature extraction capability. Second, channel attention is added to the basic network to enhance the mining ability of ConvNeXt for potential key features. Finally, the Cosine Warmup mechanism is added at the beginning of training, and the cosine function value is used to attenuate the learning rate during the process, further accelerating the convergence of ConvNeXt and improving the classification ability of the ConvNeXt model. The experimental results on the HAM10000 skin dataset show that the classification accuracy, precision, recall, and specificity of the model reach 92.9%, 85.3%, 78.0%, and 97.5%, respectively, and is demonstrated effective classification capability for dermoscopy images. This bears significant potential in aiding the auxiliary diagnosis of skin cancer lesions, providing valuable assistance to dermatologists in making accurate diagnoses of skin cancer.

Key words: dermoscopy image classification, ConvNeXt network, channel attention mechanism, SimAM without reference attention, warmup mechanism

摘要：

皮肤癌是最致命的癌症之一，对皮肤镜图像进行精确分类尤为关键，然而现有的皮肤镜图像存在形态复杂、样本数量较少的问题，导致现有的自动分类方法难以提取图像特征信息，误判率较高。提出一种改进ConvNeXt的方法，并构建SE-SimAM-ConvNeXt模型。以ConvNeXt为基础网络，加入SimAM无参注意力模块，提升网络的特征提取能力，并在基础网络中引入通道注意力机制，增强ConvNeXt对潜在关键特征的挖掘能力。在训练初始时加入预热机制Cosine Warmup，在该过程中使用余弦函数值进行学习率的衰减，进一步加速ConvNeXt的收敛，提高ConvNeXt模型的分类能力。在HAM10000皮肤数据集上的实验结果表明，该模型的分类准确率、精确度、召回率、特异性分别为92.9%、85.3%、78.0%、97.5%，具有较好的皮肤镜图像分类能力，对皮肤癌病变的辅助诊断有一定程度的应用价值，可帮助皮肤科医生对皮肤癌做进一步的诊断。

关键词: 皮肤镜图像分类, ConvNeXt网络, 通道注意力机制, SimAM无参注意力, 预热机制

Jianwei LI, Xiaoqi LÜ, Yu GU. Dermoscopy Image Classification Method Based on Improved ConvNeXt[J]. Computer Engineering, 2023, 49(10): 239-246, 254.

李建威, 吕晓琪, 谷宇. 基于改进ConvNeXt的皮肤镜图像分类方法[J]. 计算机工程, 2023, 49(10): 239-246, 254.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0066050

http://www.ecice06.com/EN/Y2023/V49/I10/239

Figures/Tables 22

Fig.1 Model structure of ConvNeXt

Fig.2 Modification of ConvNeXt model structure

Fig.3 Comparison of Swin Transformer module and ConvNeXt module

Fig.4 Structure of SimAM module

Fig.5 Structure of SE block

Fig.6 Change in learning rate coefficient using Cosine Warmup

Fig.7 Examples of skin cancer

Fig.8 Examples of data augmentation

Fig.9 Examples of multi category confusion matrix

Fig.10 Influence of Cosine Warmup on loss value (without pre-training)

Fig.11 Influence of Cosine Warmup on loss value (with pre-training)

Fig.12 Structure of ConvNeXt model

Fig.13 SimAM modules with different insertion positions

Fig.14 SE modules with different insertion positions

Fig.15 ConvNeXt model structure after adding both SimAM and SE modules

References 30

1	JANA G K , GUPTA A , DAS A , et al. Herbal treatment to skin diseases: a global approach. Drug Invention Today, 2010, 2 (8): 54- 67.
2	唐超颖. 基于可见光图像估计黑色素分布的光学快捷算法. 计算机工程, 2016, 42 (8): 227- 232. doi: 10.3969/j.issn.1000-3428.2016.08.040
	TANG C Y . Fast optical algorithm for melanin distribution estimation based on visible image. Computer Engineering, 2016, 42 (8): 227- 232. doi: 10.3969/j.issn.1000-3428.2016.08.040
3	PATHAN S , PRABHU K G , SIDDALINGASWAMY P C . Techniques and algorithms for computer aided diagnosis of pigmented skin lesions-a review. Biomedical Signal Processing and Control, 2018, 39, 237- 262. doi: 10.1016/j.bspc.2017.07.010
4	LU X , GU Y , YANG L , et al. Multi-level 3D densenets for false-positive reduction in lung nodule detection based on chest computed tomography. Current Medical Imaging, 2020, 16 (8): 1004- 1021. doi: 10.2174/1573405615666191113122840
5	赵文慧, 杨霄, 孟丽洁. 充分利用多尺度特征改进UNet实现皮肤病变分割. 电子测量技术, 2022, 45 (2): 110- 116. doi: 10.19651/j.cnki.emt.2108213
	ZHAO W H , YANG X , MENG L J . Improved UNet for skin lesion segmentation by leveraging multi-scale features. Electronic Measurement Technology, 2022, 45 (2): 110- 116. doi: 10.19651/j.cnki.emt.2108213
6	HUANG H W , HSU B W Y , LEE C H , et al. Development of a light-weight deep learning model for cloud applications and remote diagnosis of skin cancers. The Journal of Dermatology, 2021, 48 (3): 310- 316. doi: 10.1111/1346-8138.15683
7	THURNHOFER-HEMSI K , DOMÍNGUEZ E . A convolutional neural network framework for accurate skin cancer detection. Neural Processing Letters, 2020, 53 (5): 1- 21.
8	KHAN M A , MUHAMMAD K , SHARIF M , et al. Intelligent fusion-assisted skin lesion localization and classification for smart healthcare. Neural Computing and Applications, 2021, 43 (11): 1- 16.
9	KHAN M A , ZHANG Y D , SHARIF M , et al. Pixels to classes: intelligent learning framework for multiclass skin lesion localization and classification. Computers & Electrical Engineering, 2021, 90, 106956.
10	SAE-LIM W, WETTAYAPRASIT W, AIYARAK P. Convolutional neural networks using MobileNet for skin lesion classification[C]//Proceedings of the 16th International Joint Conference on Computer Science and Software Engineering. Washington D.C., USA: IEEE Press, 2019: 242-247.
11	AL-MASNI M A , KIM D H , KIM T S . Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification. Computer Methods and Programs in Biomedicine, 2020, 190, 105351. doi: 10.1016/j.cmpb.2020.105351
12	GONG A , YAO X J , LIN W . Classification for dermoscopy images using convolutional neural networks based on the ensemble of individual advantage and group decision. IEEE Access, 2020, 8, 155337- 155351. doi: 10.1109/ACCESS.2020.3019210
13	罗清, 周维, 马梓钧, 等. 基于FL-ResNet50的皮肤镜图像分类方法. 激光与光电子学进展, 2020, 57 (18): 181022. URL
	LUO Q , ZHOU W , MA Z J , et al. Dermatoscope image classification method based on FL-ResNet50. Laser & Optoelectronics Progress, 2020, 57 (18): 181022. URL
14	GU Y , CHI J Q , LIU J Q , et al. A survey of computer-aided diagnosis of lung nodules from CT scans using deep learning. Computers in Biology and Medicine, 2021, 137, 104806.
15	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2022: 9992-10002.
16	LIU Z, MAO H Z, WU C Y, et al. A ConvNet for the 2020s[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2022: 11966-11976.
17	XIE S N, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 5987-5995.
18	SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1801.04381.
19	GU Y , LU X Q , ZHANG B H , et al. Automatic lung nodule detection using multi-scale dot nodule-enhancement filter and weighted support vector machines in chest computed tomography. PLoS One, 2019, 14 (1): e0210551.
20	YANG L, ZHANG R Y, LI L, et al. Simam: a simple, parameter-free attention module for convolutional neural networks[C]//Proceedings of International Conference on Machine Learning. Washington D.C., USA: IEEE Press, 2021: 11863-11874.
21	WEBB B S , DHRUV N T , SOLOMON S G , et al. Early and late mechanisms of surround suppression in striate cortex of macaque. The Journal of Neuroscience, 2005, 25 (50): 11666- 11675.
22	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1709.01507.
23	GU Y , LU X Q , YANG L D , et al. Automatic lung nodule detection using a 3D deep convolutional neural network combined with a multi-scale prediction strategy in chest CTs. Computers in Biology and Medicine, 2018, 103, 220- 231.
24	GOYAL P, DOLLÁR P, GIRSHICK R, et al. Accurate, large minibatch SGD: training ImageNet in 1 hour[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1706.02677.
25	HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 558-567.
26	CODELLA N, ROTEMBERG V, TSCHANDL P, et al. Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration[EB/OL]. [2022-09-10]. https://arxiv.org/abs/1902.03368.
27	TSCHANDL P , ROSENDAHL C , KITTLER H . The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 2018, 5, 180161.
28	谷宇. 基于深度卷积神经网络的CT影像肺结节检测技术研究[D]. 上海: 上海大学, 2019.
	GU Y. Research on detection technology of lung nodules in CT images based on deep convolution neural network[D]. Shanghai: Shanghai University, 2019. (in Chinese)
29	TOUVRON H, CORD M, SABLAYROLLES A, et al. Going deeper with image transformers[C]//Proceedings of IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2022: 32-42.
30	MATSOUKAS C, HASLUM J F, SÖDERBERG M, et al. Is it time to replace CNNs with transformers for medical images?[EB/OL]. [2022-09-10]. https://arxiv.org/abs/2108.09038.

[1]	BAI Jie, ZHANG Sai, LI Yanping. Automatic Segmentation for Fundus Hard Exudate Based on Improved Interleaved Group Convolution [J]. Computer Engineering, 2022, 48(7): 307-314.
[2]	ZHANG Yunfan, YI Yaohua, TANG Ziwei, WANG Xinyu. Text-to-Image Synthesis Method Based on Channel Attention Mechanism [J]. Computer Engineering, 2022, 48(4): 206-212,222.
[3]	WU Qian, JIA Shijie. Face Swapping Detection Based on Multi-Channel Attention Mechanism [J]. Computer Engineering, 2022, 48(2): 180-185,193.
[4]	SONG Jia, CHEN Chenglizhao. Video Saliency Detection Based on Multi-Stream Network Consistency [J]. Computer Engineering, 2022, 48(2): 215-223.
[5]	ZHANG Haitao, ZHANG Meng. SSD Target Detection Algorithm with Channel Attention Mechanism [J]. Computer Engineering, 2020, 46(8): 264-270.

Please choose a citation manager

Content to export