Abstract: In Industrial Internet of Things (IIoT) scenarios, Time-Sensitive Networking (TSN) is required to guarantee high reliability, determinism, and low latency for data transmission. However, the traditional Cyclic Queuing and Forwarding (CQF) model faces challenges such as severe resource preemption, load imbalance, and insufficient network resource utilization when processing mixed traffic. To address these issues, this paper proposes A Deep Noisy Q-Network-Based Multi-CQF Scheduling Algorithm (DNQN-MCQF). Initially, the algorithm constructs a four-channel Multi-Cyclic Queuing and Forwarding (Multi-CQF) architecture, establishing a dedicated queue to guarantee the deterministic transmission of high-priority traffic, and utilizing weights determined by Particle Swarm Optimization (PSO) to calculate dynamic sorting scores for optimizing the transmission sequence. Subsequently, a hybrid feature extraction framework is constructed, employing a Convolutional Neural Network-Gated Recurrent Unit (CNN-GRU) to capture spatiotemporal dynamic features of slot utilization and queue states, combined with a Graph Convolutional Network (GCN) to aggregate static global topological information, generating state embeddings via an Attention Fusion Mechanism. Ultimately, in the scheduling decision phase, the Deep Noisy Q-Network is integrated to inject parameter space noise, achieving adaptive policy exploration and slot optimization. Simulation results show that under different network topologies and time slot conditions, DNQN-MCQF improves the average scheduling success rate and load balance by 14.7% and 19.2%, respectively, compared with baseline algorithms.

摘要： 在工业物联网（IIoT）场景下，时间敏感网络（Time-Sensitive Networking，TSN）需保障数据传输的高可靠性、确定性及低时延。然而，传统循环排队转发（Cyclic Queuing and Forwarding，CQF）模型在处理多种优先级混合的流量时，面临着资源抢占、负载失衡以及网络资源利用率不足等挑战。针对上述问题，本文提出一种基于深度噪声Q网络的多队列调度算法（Deep Noisy Q-Network Based Multi-CQF Scheduling Algorithm）。该算法首先构建四通道多重循环排队转发（Multi-Cyclic Queuing and Forwarding，Multi-CQF）架构，设置专用队列保障高优先级流量的确定性传输，并利用粒子群优化（Particle Swarm Optimization，PSO）确定的权重计算动态排序分数，优化发送序列。其次构建混合特征提取框架，利用卷积-门控循环单元网络（Convolutional Neural Network-Gated Recurrent Unit，CNN-GRU）捕获时隙利用率和队列状态的时空动态特征，结合图卷积网络（Graph Convolutional Network，GCN）聚合静态全局拓扑信息，经由注意力融合机制（Attention Fusion Mechanism）生成状态嵌入。最后在调度决策阶段，结合深度噪声Q网络（Deep Noisy Q-Network），注入参数空间噪声实现策略自适应探索与时隙优化。计算机仿真结果表明，在不同网络拓扑和时隙条件下，DNQN-MCQF较基线算法平均调度成功率和负载均衡分别提升14.7%和19.2%。