Abstract: Stochastic Configuration Networks (SCNs) incorporate randomized learning mechanisms into neural network training to enhance modeling efficiency and employ a data-dependent supervisory mechanism to ensure the universal approximation capability of the model. However, during the incremental construction process, the computation of hidden layer output weights after each newly added hidden node relies on repeated calculation of the pseudoinverse of the hidden layer output matrix, which limits the training efficiency to some extent. In addition, while randomized learning improves modeling efficiency, it inevitably introduces potential redundant hidden nodes and parameters. To address these issues, this paper proposes a group-sparse learning method for Incremental Regularized Stochastic Configuration Networks (GSL-IRSCN). First, to improve the training efficiency of regularized SCNs in the incremental modeling process, an incremental output-weight updating strategy for L2-regularized SCNs is developed based on the Woodbury matrix identity, thereby avoiding repeated computation of the inverse of the regularized normal matrix and effectively reducing the computational cost of the model. Then, to mitigate redundancy in hidden nodes induced by the randomized learning mechanism, group L1/2 regularization is introduced and optimized via the Alternating Direction Method of Multipliers (ADMM), achieving sparsity constraints on redundant nodes and simplifying the network architecture. Experimental results on four benchmark datasets from UCI and KEEL demonstrate that the proposed GSL-IRSCN outperforms existing comparative methods in terms of both training efficiency and model compactness.

摘要： 随机配置网络(Stochastic configuration networks, SCNs)在神经网络训练过程中引入随机化学习机制以提升建模效率，并提出一种数据驱动的监督机制保证模型的通用逼近能力。然而，其增量构建过程中，每次新增隐藏层节点后对隐藏层输出权重的计算依赖于隐藏层输出矩阵伪逆的重复求解，这在一定程度上制约了模型的训练效率。此外，随机化学习方法在提升建模效率的同时，不可避免地会引入潜在冗余的隐藏层节点。为此，本文提出一种增量正则化随机配置网络的组稀疏学习方法(GSL-IRSCN）。首先，为提升正则化SCNs在增量建模过程中的训练效率，基于Woodbury分块矩阵求逆公式提出了带L2正则化项SCNs的输出权重增量更新策略，从而避免了对正则化正规矩阵逆的重复计算，有效降低了模型的计算开销。然后，针对随机化学习机制导致的冗余隐藏层节点问题，引入具有更强稀疏效果的组L1/2正则化并结合交替方向乘子法(Alternating Direction Method of Multipliers, ADMM)进行优化，实现了对模型中冗余节点的高效稀疏，简化了模型的网络结构。在4个UCI和KEEL数据集的实验结果表明，提出的GSL-IRSCN在训练效率和模型紧凑性方面均取得了优于现有对比方法的性能。