摘要：

近年来某国产深度学习加速器发展迅速, 硬件资源持续变化并引入一系列张量核心指令, 使得开发者在该加速器上进行卷积算子的手工适配和优化面临巨大挑战。为此, 本文提出了一种面向该国产加速器的卷积代码生成器, 以简化卷积算子的适配与优化过程。该生成器提供配置参数作为对外接口, 用户仅需配置参数即可生成特定的卷积算子。生成器本身由三层架构组成: 指令层封装底层指令, 并根据硬件架构进行区分; 组件层根据预设的硬件架构信息组织相应指令, 从线程块和线程束角度提供高度抽象且可复用的功能组件; 算子构建层则通过隐式卷积算法拼接功能组件, 最终生成卷积算子。为保证卷积算子的计算性能, 生成器从两方面进行优化: 使用向量化算法和线程划分算法优化全局访存性能; 使用转置算法转化乘累加指令的线程结构以优化写回性能。测试结果表明: 该生成器的优化算法可显著提升算子性能; 在两种硬件版本下, NHWC存储布局的卷积算子性能分别达到官方算子库中卷积算子性能的95%与90%。该生成器为国产加速器的卷积算子适配优化提供了一种全新的解决方案。

关键词: 国产加速器, 隐式卷积算法, 自动代码生成, 张量核心, 向量化访存

Abstract:

Domestic deep-learning accelerators have developed rapidly in recent years.Hardware resources have been changing continuously, and a series of tensor core instructions have been introduced. Therefore, manually adapting and optimizing the convolution operator in the accelerator is a considerable challenge for developers. To this end, this study proposes a convolution code generator for a domestic accelerator to simplify the adaptation and optimization process of the convolution operator. The generator provides configuration parameters as an external interface, and users only need to configure these parameters to generate specific convolution operators. The generator itself consists of a three-layer architecture: the instruction layer encapsulates the underlying instructions and categorizes them according to the hardware architecture; the component layer organizes the corresponding instructions according to the preset hardware architecture information and provides highly abstract and reusable functional components from the perspective of thread blocks and thread bundles; and the operator construction layer splices the functional components according to an implicit convolution algorithm and generates a convolution operator. To ensure the computing performance of the convolution operator, the generator is optimized from two aspects: using the vectorization and thread partitioning algorithms to optimize the global memory access performance and using the transposition algorithm to transform the thread structure of the multiply-accumulate instruction to optimize the write-back performance. The test results show that the optimization algorithm of the generator can significantly improve the operator performance; under two hardware versions, the convolution operator performance of the NHWC storage layout reaches 95% and 90% of the official operator performance. The generator provides a new solution for adaptation and optimization of the convolution operators of domestic accelerators.

Key words: domestic accelerator, implicit convolution algorithm, automatic code generation, tensor core, vectorized memory access