Ma Jun, Hou Xuena, Zhao Jingge, Li Xiaobing, Gao Yan, Zhou Qingwen, Tie Yun
Accepted: 2026-10-16
To address the problem that symbolic music generation models often fail to adequately capture harmonic relationships, rhythmic organization, and music-theoretical constraints among different instrument tracks in multi-track arrangement tasks, this paper proposes a multi-track music generation model integrating aesthetic computing and hybrid learning, namely HL-MTMG. Existing symbolic music generation methods mainly focus on note sequence modeling or general controllable generation. Although these methods can learn the distribution of music data to a certain extent, they still tend to produce loose harmonic relationships, unstable rhythmic structures, excessive repeated notes, and insufficient pitch control when generating multi-track music involving piano, guitar, and bass. To improve the coordination and artistic expressiveness of generated multi-track music, HL-MTMG combines cross-track collaborative modeling with music-theoretical rule constraints from the perspective of music aesthetic computing, enabling the model to learn both the internal sequential patterns of individual tracks and the interactions among different instrument tracks. Technically, HL-MTMG consists of a hybrid learning module and a reward feedback network. The hybrid learning module introduces both masked self-attention and cross-track attention mechanisms. The masked self-attention mechanism is used to learn temporal dependencies within each single track, while the cross-track attention mechanism calculates the correlations among piano, guitar, and bass tracks, thereby enhancing the model’s ability to represent multi-track collaborative relationships. The reward feedback network is constructed based on the idea of generative adversarial learning. It converts melodic pitch-range constraints and note repetition suppression rules into reward signals, which are combined with the cross-entropy loss during training. In this way, the generator is guided to learn the data distribution while maintaining better melodic regularity and harmonic rationality. Experiments are conducted on a three-track symbolic music subset constructed from the Lakh MIDI dataset. During data preprocessing, pop MIDI files containing piano, guitar, and bass tracks are selected, followed by track mapping, beat alignment, abnormal sample removal, and four-bar segmentation. Finally, 11,840 three-track music segments are obtained. To avoid data leakage, the dataset is divided at the original MIDI-file level rather than at the segment level, and the training, validation, and test sets are constructed at a ratio of 8:1:1. All comparison models and ablation models adopt the same data split, input representation, generation length, and evaluation procedure. To further evaluate the generalization ability of the proposed model, an External-MIDI test set containing 300 external three-track music segments is also constructed to examine its adaptability to non-training data sources. HL-MTMG is compared with three representative baseline models, namely MuseGAN, HRNN, and XMusic, as well as three ablation variants. Objective evaluation results show that HL-MTMG achieves a chord matching degree of 0.61 and a harmonious degree of 0.72 in the main experiment, outperforming the selected comparison models and ablation variants overall. In terms of note-level features, the ratio of three consecutive notes to the total number of notes (NCTN) of HL-MTMG is 0.22, which is lower than that of the other models. Meanwhile, the model achieves 0.47, 0.72, and 0.64 on qualified rhythm (QR), used pitch classes (UPC), and polyphonicity (PP), respectively, indicating better rhythmic organization, pitch richness, and multi-track polyphonic performance. On the External-MIDI test set, HL-MTMG obtains 0.56, 0.67, 0.25, 0.45, 0.68, and 0.60 in chord matching degree, harmonious degree, NCTN, QR, UPC, and PP, respectively, showing that the model can still maintain relatively stable generation quality on external data sources. For subjective evaluation, 10 professional composers and 90 music enthusiasts are invited to rate the generated results. The evaluation by music enthusiasts shows that HL-MTMG obtains scores of 7.87, 8.17, and 7.47 in melody, music form, and audibility, respectively. The professional composers’ evaluation shows that HL-MTMG achieves scores of 6.6, 7.6, 7.8, 7.4, and 9.1 in chord, interval, harmony, pitch, and music style, respectively. The Kendall’s coefficient of concordance test further shows that the Kendall’s W values for rhythm, melody, integrity, music form, audibility, and fluency are 0.74, 0.81, 0.76, 0.79, 0.72, and 0.75, respectively, all reaching a significant level, indicating a high degree of consistency in the subjective ratings. The experimental results demonstrate that the cross-track attention mechanism can enhance the model’s ability to capture collaborative relationships among piano, guitar, and bass, while the music-theoretical reward feedback helps improve melodic pitch control and repeated-note suppression. In the constructed three-track symbolic music generation task, HL-MTMG improves harmonic coordination, rhythmic organization, and several subjective evaluation dimensions of the generated results, providing a feasible approach for introducing aesthetic computing and music-theoretical rules into multi-track music generation. The current experiments are mainly conducted on fixed-length symbolic music segments with three fixed tracks. Future work will further verify the adaptability of the model in more instrument combinations, music styles, and long-structure music generation tasks.