GroupMNL: A Strong Robustness Convolutional Neural Network Design Method for Internet of Things Devices

doi:10.19678/j.issn.1000-3428.0252317

Abstract

Abstract: With the widespread application of Internet of Things (IoT) devices, the efficient deployment of robust Convolutional Neural Networks (CNNs) on resource-constrained IoT devices has become a significant challenge. Existing methods that rely on cloud servers to assist CNN training reduce parameter transmission between the cloud and devices but neither reduce the inference computation nor exhibit sufficient robustness. To address this issue, a group-based method for generating filters using multiple nonlinear transformation functions, called GroupMNL, is proposed. First, in each convolutional layer, a small number of standard filters are randomly generated as seed filters. These seed filters are then grouped, and different types of nonlinear transformation functions are applied to each group to generate diverse filters on demand. The generated parameters are unlearned, which effectively reduces the number of trainable parameters in the CNN. Second, the seed filters and generated filters are concatenated to form a complete convolutional layer, and group convolution is introduced during the convolution operation to reduce the computational cost. Finally, to further enhance the robustness of the CNN model, Group Normalization (GN) is introduced and combined with the regularization effect of multiple nonlinear transformations on the model, thereby improving its robustness. The experimental results show that the ResNet101 model based on the GroupMNL method, compared to the standard model, reduces the number of learnable parameters by 87%, decreases the computational cost by 71%, and improves the robustness of the model by 6.09 percentage points.

Key words: Internet of Things (IoT) devices, Convolutional Neural Network (CNN), nonlinear transformation function, robustness, learnable parameters

摘要： 随着物联网（IoT）设备的广泛应用，如何在资源受限的物联网设备上高效部署强鲁棒性的卷积神经网络(CNN)成为一项重要挑战。现有的云服务器辅助物联网设备训练CNN模型的方法虽然可以降低云和设备间的参数传输量，但并未降低推理所需计算量且鲁棒性有限。为了解决上述问题，提出了一种分组的由多个非线性变换函数生成滤波器的方法GroupMNL。首先，在每层卷积层中随机生成少量的标准滤波器作为种子滤波器。将种子滤波器分组并对每组的种子滤波器使用不同类型的非线性变换函数按需生成多样性的滤波器，且参数是不可学习的，从而降低了CNN模型的可学习参数量。其次，将种子滤波器和生成的滤波器进行拼接生成完整卷积层，并在卷积操作中引入分组卷积机制以降低模型的计算量。最后，为了进一步增强CNN模型的鲁棒性，引入组归一化（GN）技术并结合多个非线性变换函数对模型的正则化作用，从而提升模型的鲁棒性。实验结果表明，基于GroupMNL的ResNet101模型相比标准模型，减少了87%的可学习参数量，降低了71%的计算量，并将模型鲁棒性提高了6.09百分点。

关键词: 物联网设备, 卷积神经网络, 非线性变换函数, 鲁棒性, 可学习参数

CLC Number:

TP183

GUO Hui, DING Chuntao, ZHANG Junna. GroupMNL: A Strong Robustness Convolutional Neural Network Design Method for Internet of Things Devices[J]. Computer Engineering, 2026, 52(11): 142-152.

郭慧, 丁春涛, 张俊娜. GroupMNL：面向物联网设备的强鲁棒性卷积神经网络设计方法[J]. 计算机工程, 2026, 52(11): 142-152.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0252317

https://www.ecice06.com/EN/Y2026/V52/I11/142

References

[1] SATYAJIT S. State of IoT summer 2024[EB/OL]. [2025-03-05]. https://iot-analytics.com/number-connected-iot-devices.
[2] ALTOWAIJRI A H, ALFAIFI M S, ALSHAWI T A, et al. A privacy-preserving IoT-based fire detector[J]. IEEE Access, 2021, 9: 51393-51402.
[3] ARKIN E, YADIKAR N, XU X B, et al. A survey: object detection methods from CNN to transformer[J]. Multimedia Tools and Applications, 2023, 82(14): 21353-21383.
[4] 杨红菊, 吉昌. 学习驱动的图像压缩算法研究[J]. 计算机工程, 2025, 51(1): 190-197.YANG H J, JI C. Research on learning-driven image compression algorithm[J]. Computer Engineering, 2025, 51(1): 190-197. (in Chinese)
[5] YANG X, XU Z K, QI Q, et al. PICO: pipeline inference framework for versatile CNNs on diverse mobile devices[J]. IEEE Transactions on Mobile Computing, 2024, 23(4): 2712-2730.
[6] 陈晨, 王宇航. 面向资源受限设备的轻量化深度学习模型研究进展[J]. 电子学报, 2022, 50(8): 1450-1462. CHEN C, WANG Y H. Research progress on lightweight deep learning models for resource-constrained devices [J]. Acta Electronica Sinica, 2022, 50(8): 1450-1462. (in Chinese)
[7] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2016: 770-778.
[8] LU Z C, DING C T, JUEFEI-XU F, et al. TFormer: a transmission-friendly ViT model for IoT devices[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(2): 598-610.
[9] KHANI M, HAMADANIAN P, NASR-ESFAHANY A, et al. Real-time video inference on edge devices via adaptive model streaming[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2022: 4552-4562.
[10] DING C T, LU Z C, JUEFEI-XU F, et al. Towards transmission-friendly and robust CNN models over cloud and device[J]. IEEE Transactions on Mobile Computing, 2023, 22(10): 6176-6189.
[11] JUEFEI-XU F, BODDETI V N, SAVVIDES M. Local binary convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2017: 4284-4293.
[12] ESHRATIFAR A E, ABRISHAMI M S, PEDRAM M. JointDNN: an efficient training and inference engine for intelligent mobile cloud computing services[J]. IEEE Transactions on Mobile Computing, 2021, 20(2): 565-576.
[13] LI Y R, LIU Z, KOU Z, et al. Real-time adaptive partition and resource allocation for multi-user end-cloud inference collaboration in mobile environment[J]. IEEE Transactions on Mobile Computing, 2024, 23(12): 13076-13094.
[14] LASKARIDIS S, VENIERIS S I, ALMEIDA M, et al. SPINN: synergistic progressive inference of neural networks over device and cloud[C]//Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. New York, USA: ACM Press, 2020: 1-15.
[15] DING C T, XIE J H, ZHANG J N, et al. GroupNL: low-resource and robust CNN design over cloud and device[EB/OL]. [2025-03-05]. https://arxiv.org/abs/2506.12335.
[16] LU Z C, DING C T, WANG S G, et al. Seed feature maps-based CNN models for LEO satellite remote sensing services[C]//Proceedings of the IEEE International Conference on Web Services (ICWS). Washington D.C., USA: IEEE Press, 2023: 415-425.
[17] YU J H, DING C T, LI H F, et al. Localized knowledge distillation helps IoT devices provide high-performance visual services[C]//Proceedings of the IEEE International Conference on Web Services (ICWS). Washington D.C., USA: IEEE Press, 2023: 170-178.
[18] 王月昊, 周若华. 低资源环境下的语音唤醒研究综述[J]. 计算机工程, 2025, 51(2): 35-53.WANG Y H, ZHOU R H. Review of research on keyword spotting in low-resource environments[J]. Computer Engineering, 2025, 51(2): 35-53. (in Chinese)
[19] 王楠, 王淇, 欧阳丹彤. 基于知识蒸馏与动态调整机制的多模态情感分析模型[J]. 计算机学报, 2025, 48(8): 1923-1942.WANG N, WANG Q, OUYANG D T. Multimodal sentiment analysis model based on knowledge distillation and dynamic adjustment mechanism[J]. Chinese Journal of Computers, 2025, 48(8): 1923-1942. (in Chinese)
[20] HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. [2025-03-05]. https://arxiv.org/abs/1704.04861.
[21] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 4510-4520.
[22] HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2019: 1314-1324.
[23] ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 6848-6856.
[24] MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 122-138.
[25] AZULAY A, WEISS Y. Why do deep convolutional networks generalize so poorly to small image transformations? [EB/OL]. [2025-03-05]. https://arxiv.org/abs/1805.12177.
[26] YUAN Y G, XU B B, HOU L, et al. TEA: test-time energy adaptation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2024: 23901-23911.
[27] HENDRYCKS D, DIETTERICH T. Benchmarking neural network robustness to common corruptions and perturbations[EB/OL]. [2025-03-05]. https://arxiv.org/abs/1903.12261.
[28] LIANG B, LI H C, SU M Q, et al. Detecting adversarial image examples in deep neural networks with adaptive noise reduction[J]. IEEE Transactions on Dependable and Secure Computing, 2021, 18(1): 72-85.
[29] CHOI S H, SHIN J, CHOI Y H. PIHA: detection method using perceptual image hashing against query-based adversarial attacks[J]. Future Generation Computer Systems, 2023, 145: 563-577.
[30] WANG Z K, PANG T Y, DU C, et al. Better diffusion models further improve adversarial training[EB/OL]. [2025-03-05]. https://arxiv.org/abs/2302.04638.
[31] HENDRYCKS D, MU N, CUBUK E D, et al. AugMix: a simple data processing method to improve robustness and uncertainty[EB/OL]. [2025-03-05]. https://arxiv.org/abs/1912.02781.
[32] HENDRYCKS D, BASART S, MU N, et al. The many faces of robustness: a critical analysis of out-of-distribution generalization[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Washington D.C., USA: IEEE Press, 2022: 8320-8329.
[33] WU Y X, HE K M. Group normalization[C]//Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2018: 3-19.
[34] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[EB/OL]. [2025-03-05]. https://arxiv.org/abs/1502.03167.
[35] KRIZHEVSKY A, HINTON G. Learning multiple layers of features from tiny images[EB/OL]. [2025-03-05]. https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf.
[36] JUEFEI-XU F, BODDETI V N, SAVVIDES M. Perturbative neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 3310-3318.
[37] HAN K, WANG Y H, TIAN Q, et al. GhostNet: more features from cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 1577-1586.

Please choose a citation manager

Content to export