摘要：

生成扩散模型是一种能够学习数据生成过程的模型，它可以根据输入的高斯噪声，逐步去噪生成新的数据样本，因此被广泛应用于图像生成领域。近期，有研究发现扩散模型性能取决于网络复杂度而非U-Net的归纳偏置，因此可以利用Transformer作为骨干网络，使得扩散模型能够继承对Transformer的最新研究成果。然而，Transformer的引入又会导致模型体积增大、训练速度减慢。针对使用Transformer骨干网络的扩散模型训练速度慢、生成的图像细节信息不佳的问题，提出一种基于邻域注意力架构的扩散模型，该模型引入带有邻域注意力的Transformer骨干网络，利用邻域注意力机制的稀疏全局注意力模式，指数级扩展了扩散模型对图像的感知范围，使模型用较小的代价关注到了全局信息。通过注意力扩展层的渐进式膨胀变化，在模型训练阶段捕获到更多的视觉信息，使得模型生成的图像具有更好的全局性。实验结果表明，所提模型生成的图像有较优的全局细节，生成效果优于当前的SOTA模型。

关键词: 扩散模型, 图像生成, 生成式模型, 邻域注意力, Transformer骨干网络

Abstract:

Generative diffusion models can learn to generate data. They progressively denoise and generate new data samples based on input Gaussian noise; therefore, they are widely applied in the field of image generation. Recently, the inductive bias provided by the U-Net backbone used in diffusion models has been revealed to be non-critical, and the Transformer can be adopted as the backbone network to inherit the latest advancements from other domains. However, introducing the Transformer increases the model size and slows the training. To address the issues of slow training and inadequate image detail associated with diffusion models utilizing the Transformer backbone, this paper introduces a diffusion model based on a neighborhood attention architecture. This model incorporates a Transformer backbone network with neighborhood attention, utilizes the sparse global attention pattern of the neighborhood attention mechanism, which exponentially expands the model′s perception range of images, and focuses on global information at a lower cost. By employing progressive expansion in the attention expansion layer, more visual information is captured during model training, resulting in images with better global aspects. Experimental results demonstrate that this design provides better global consistency, yields superior global details in the generated images, and outperforms current State-Of-The-Art (SOTA) models.

Key words: diffusion model, image generation, generative model, neighborhood attention, Transformer backbone network