Application of Bionic Design Algorithms Based on Latent Spatial Multi-Scale Fusion of Cross-Domain Images

doi:10.19678/j.issn.1000-3428.0069684

Abstract

Abstract:

Biomimetic design draws inspiration from nature and skillfully integrates biological characteristics into product design. Traditional biomimetic design methods often present limited scope for innovation, particularly when effectively integrating abstract biological inspiration with concrete product forms. To address these challenges, this paper proposes a novel algorithm called BioFusion for the multi-scale fusion of cross-domain images, which is aimed at facilitating the high-quality amalgamation of products and biological features. BioFusion involves a warm-start optimal inversion method to map real images into the latent space of a Generative Adversarial Network (GAN) generator. The latent space of the pre-trained model is originally derived from a product dataset. Through domain expansion via few-shot fine tuning, this latent space can be extended to a fusion space containing biological characteristics. Subsequently, a cross-domain multi-scale Latent Interpolation Style Mixing (LISM) method is proposed to effectively integrate the semantic features of the product image and biological image domains. The model is trained on a self-constructed product dataset and compared with methods such as DGBID and Smooth Diffusion in terms of the inversion quality and cross-domain image fusion effect. The experimental results show that BioFusion is capable of generating realistic and morphologically aware fused images, and it shows excellent performance in terms of the Fréchet Inception Distance (FID), image Interpolation Standard Deviation (ISTD) and Blended Image Quality Index (BIQI), with values reaching 34.65, 18.37, and 1.11, respectively. Additionally, BioFusion performs well in multi-scale bionic fusion and can generate fused images containing different dimensions of semantic information, providing designers with rich bionic design inspirations and references.

Key words: bionic design, BioFusion algorithm, cross-domain images, multi-scale interpolated fusion, Generative Adversarial Networks (GAN)

摘要：

在工业设计领域, 仿生设计是一种从自然界中汲取灵感并将生物特征与产品设计巧妙结合的方法。然而, 传统仿生设计方法往往存在创新性不足的问题, 难以有效融合抽象生物灵感与具象产品形态。为了解决上述问题, 提出一种跨域图像多尺度仿生融合算法BioFusion, 旨在实现产品与生物特征的高质量融合。首先采用热启动优化反演方法, 将图像映射至生成对抗网络(GAN)的生成器潜在空间, 然后通过基于少样本微调的生成模型域扩展, 将基于产品数据集训练的潜在空间扩展至包含生物特征的融合空间, 之后提出一种跨域多尺度插值融合方法LISM, 有效整合产品图像域和生物图像域的语义特征。在自建的产品数据集上训练该算法模型, 并在反演质量及跨域图像融合效果方面将其与DGBID、Smooth Diffusion等方法进行对比, 实验结果表明, BioFusion能够生成逼真且富有形态感知的融合图像, 在弗雷谢特距离(FID)、图像插值标准差(ISTD)和融合图像质量(BIQI)上表现较好, 分别达到34.65、18.37和1.11。此外, BioFusion在多尺度仿生融合方面表现良好, 能够生成包含不同维度语义信息的融合图像, 从而为设计者提供丰富的仿生设计灵感和参考。

关键词: 仿生设计, BioFusion算法, 跨域图像, 多尺度插值融合, 生成对抗网络

ZHANG Yimin, HUANG Xiaoying, HUANG Zhengyang, YANG Chaoxiang, WAN Yongjing, JIANG Cuiling. Application of Bionic Design Algorithms Based on Latent Spatial Multi-Scale Fusion of Cross-Domain Images[J]. Computer Engineering, 2025, 51(5): 266-278.

章艺敏, 黄晓英, 黄正洋, 杨超翔, 万永菁, 蒋翠玲. 基于跨域图像潜在空间多尺度融合的仿生设计算法应用[J]. 计算机工程, 2025, 51(5): 266-278.

/ Recommend / Download Citations

URL: https://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0069684

https://www.ecice06.com/EN/Y2025/V51/I5/266

Figures/Tables 16

Fig.1 BioFusion algorithm structure

Fig.2 Schematic diagram of domain expansion of Few-shot synthetic network

Fig.3 Schematic diagram of cross-domain fusion based on LISM

Fig.4 Sample display of self built dataset

Fig.5 Comparison of inversion and reconstruction image results

Fig.6 Linear interpolation results based on BioFusion

Fig.7 Relationship between interpolation coefficient and image fusion quality

Fig.8 Comparison of bionic image fusion algorithms based on image interpolation fusion

Fig.9 Comparison of multi-scale bionic fusion results based on LISM

Fig.10 Integration scheme set of bionic design

Fig.11 Bionic design sketch based on BioFusion

References 37

1	YANG Y , ZHU Q X , WANG W , et al. Structure bionic design method oriented to integration of biological advantages. Structural and Multidisciplinary Optimization, 2021, 64 (3): 1017- 1039. doi: 10.1007/s00158-021-02912-4
2	罗仕鉴, 边泽, 张宇飞, 等. 基于形态匹配的产品仿生设计融合. 计算机集成制造系统, 2020, 26 (10): 2633- 2641. doi: 10.13196/j.cims.2020.10.003
	LUO S J , BIAN Z , ZHANG Y F , et al. Product bionic fusion design based on shape matching. Computer Integrated Manufacturing Systems, 2020, 26 (10): 2633- 2641. doi: 10.13196/j.cims.2020.10.003
3	SOMEPALLI G, SINGLA V, GOLDBLUM M, et al. Diffusion art or digital forgery? investigating data replication in diffusion models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2023: 6048-6058.
4	BUTTERICK M. Stable diffusion litigation[EB/OL]. [2023-10-05]. http://www.stablediffusionfrivolous.com/.
5	LIU P , WANG Y Y , DU A G , et al. Disentangling latent space better for few-shot image-to-image translation. International Journal of Machine Learning and Cybernetics, 2023, 14 (2): 419- 427. doi: 10.1007/s13042-022-01552-4
6	SHEN Y J, GU J J, TANG X O, et al. Interpreting the latent space of GANs for semantic face editing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 9243-52.
7	罗嗣卿, 陈慧. 基于生成对抗网络的图像场景转换. 计算机工程, 2023, 49 (4): 217- 225. doi: 10.19678/j.issn.1000-3428.0066077
	LUO S Q , CHEN H . Image-scene transformation based on generative adversarial networks. Computer Engineering, 2023, 49 (4): 217- 225. doi: 10.19678/j.issn.1000-3428.0066077
8	CHEN Y C, LIN H J, SHU M, et al. Facelet-bank for fast portrait manipulation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 3541-3549.
9	BERTHELOT D, RAFFEL C, ROY A, et al. Understanding and improving interpolation in autoencoders via an adversarial regularizer[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1807.07543.
10	ABDAL R, QIN Y P, WONKA P. Image2StyleGAN: how to embed images into the StyleGAN latent space?[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Washington D.C., USA: IEEE Press, 2019: 4432-4441.
11	KARRAS T, LAINE S, AILA T M. A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2019: 4401-4410.
12	KARRAS T, LAINE S, AITTALA M, et al. Analyzing and improving the image quality of StyleGAN[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2020: 8110-8119.
13	KARRAS T, AITTALA M, LAINE S, et al. Alias-free generative adversarial networks[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. New York, USA: ACM Press, 2021: 852-863.
14	TOV O , ALALUF Y , NITZAN Y , et al. Designing an encoder for StyleGAN image manipulation. ACM Transactions on Graphics, 2021, 40 (4): 1- 14. URL
15	XIA W , ZHANG Y , YANG Y , et al. Gan inversion: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45 (3): 3121- 3138. URL
16	BROCK A, DONAHUE J, SIMONYAN K. Large scale GAN training for high fidelity natural image synthesis[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1809.11096.
17	KARRAS T, AILA T, LAINE S, et al. Progressive growing of GANs for improved quality, stability, and variation[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1710.10196.
18	JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1603.08155.
19	KARRAS T, AITTALA M, HELLSTEN J, et al. Training generative adversarial networks with limited data[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2006.06676.
20	PAN X , ZHAN X , DAI B , et al. Exploiting deep generative prior for versatile image restoration and manipulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44 (11): 7474- 7489.
21	ROICH D , MOKADY R , BERMANO A H , et al. Pivotal tuning for latent-based editing of real images. ACM Transactions on Graphics, 2022, 42 (1): 1- 13. URL
22	NITZAN Y, GHARBI M, ZHANG R, et al. Domain expansion of image generators[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2023: 15933-15942.
23	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2018: 586-595.
24	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2017: 4700-4708.
25	KIRKPATRICK J, PASCANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1612.00796.
26	RICHARDSON E, ALALUF Y, PATASHNIK O, et al. Encoding in style: a StyleGAN encoder for image-to-image translation[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2021: 2287-2296.
27	YANG L J, LUO P, LOY C C, et al. A large-scale car dataset for fine-grained categorization and verification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D.C., USA: IEEE Press, 2015: 3973-3981.
28	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2023-10-05]. https://arxiv.org/abs/1412.6980.
29	SARA U , AKTER M , UDDIN M S . Image quality assessment through FSIM, SSIM, MSE and PSNR—a comparative study. Journal of Computer and Communications, 2019, 7 (3): 8- 18. URL
30	SETIADI D R I M . PSNR vs SSIM: imperceptibility quality assessment for image steganography. Multimedia Tools and Applications, 2021, 80 (6): 8423- 8444. doi: 10.1007/s11042-020-10035-z
31	WANG Z , BOVIK A C , SHEIKH H R , et al. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612. URL
32	KYNKääNNIEMI T, KARRAS T, AITTALA M, et al. The role of imagenet classes in Fréchet inception distance[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2203.06026.
33	GUO J Y, XU X Q, PU Y F, et al. Smooth Diffusion: crafting smooth latent spaces in diffusion models[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2312.04410.
34	DENG Z G , LV J , LIU X , et al. Bionic design model for co-creative product innovation based on deep generative and BID. International Journal of Computational Intelligence Systems, 2023, 16 (1): 8. doi: 10.1007/s44196-023-00187-9
35	RAMESH A, DHARIWAL P, NICHOL A, et al. Hierarchical text-conditional image generation with CLIP latents[EB/OL]. [2023-10-05]. https://arxiv.org/abs/2204.06125.
36	ABHINAY N S. UnCLIP image interpolation demo[EB/OL]. [2023-10-05]. https://huggingface.co/spaces/NagaSaiAbhinay/UnCLIP_Image_Interpolation_Demo.
37	SCHARR H. Optimal operators in digital image processing[EB/OL]. [2023-10-05]. https://www.semanticscholar.org/paper/Optimal-operators-in-digital-image-processing-Scharr/aa06f150c3a4ffc98f9359de4df5e8d81f403e0a.

Please choose a citation manager

Content to export