Abstract: To address character confusion, discontinuous cross-sentence emotional progression, and unstable transitions between narration and dialogue in long-text multi-role audiobook generation, this paper proposes an agent-based decision-planning method for generating emotional multi-role audiobooks. The proposed method employs a scene chain as a discourse-level intermediate representation and integrates long-form text parsing, character modeling, emotion control, and rhythm planning into a structured front-end decision process without modifying the architecture of the underlying acoustic model. First, a script annotation agent identifies scene boundaries according to temporal changes, spatial transitions, variations in the character set, and shifts in event topics. It then performs narration/dialogue recognition, speaker annotation, and character-set extraction within each scene, thereby constructing a scene chain containing scene types, scene texts, character identities, and discourse-style information. This representation provides unified and traceable contextual information for subsequent modules. Reference voices are then assigned to characters according to attributes such as gender, age, identity, and personality, and the same reference voice is maintained for each character throughout the generation process to reduce cross-scene timbre drift. Second, an emotion-planning module consisting of an emotion vectorization agent and an emotion constraint agent is constructed. The emotion vectorization agent maps textual semantics into an eight-dimensional emotion vector comprising happiness, anger, sadness, fear, disgust, depression, surprise, and neutrality. The emotion constraint agent further refines the initial vector by incorporating character traits, scene-level emotional tone, and historical emotional states. Cross-turn emotional discontinuities are detected using cosine similarity. Plot-triggered emotional changes are preserved, whereas unsupported emotional jumps are suppressed. Third, a contextual rhythm-planning mechanism is designed. It establishes intra-segment pauses and discourse-style transition pauses according to scene type, and dynamically adjusts inter-segment pauses by considering character alternation, narration length, and changes between adjacent emotion vectors. This mechanism improves rhythmic continuity in long narration, consecutive utterances by the same character, character switching, and transitions between narration and dialogue. The experiments use 40 Chinese novel excerpts covering eight representative scene types, with five excerpts for each type. Each excerpt contains three to four speakers, including one narrator and two to three story characters. Reference voices from 24 speakers are selected from publicly available Chinese speech datasets. To ensure a fair comparison, all methods use the same texts, reference recordings, and predefined character-to-voice mappings. Experimental results show that, when the emotional-transition threshold is set to 0.65, the proposed method achieves subjective scores of 4.09, 4.21, 3.92, and 3.96 for naturalness, emotion accuracy, character consistency, and rhythm comfort, respectively. Compared with the second-best baseline, emotion accuracy and rhythm comfort improve by 24.2% and 29.8%, respectively, while the blind-listening preference rate reaches 72.1%. In the objective evaluation, the proposed method obtains a word error rate of 7.55%, a speaker similarity score of 0.89, a speaker assignment error rate of 0.13%, and an intra-speaker timbre consistency score of 0.027 under cross-scene and cross-emotion conditions. These results indicate that the method improves character identity stability while preserving speech intelligibility and reference-speaker characteristics. Comparative and ablation experiments further demonstrate that the scene-chain representation, multi-level emotion constraints, and contextual rhythm planning make complementary contributions to emotional continuity, character consistency, and rhythm naturalness. In particular, removing the character and scene constraints causes the most evident degradation in character consistency, whereas removing the rhythm-control module leads to the largest reduction in rhythm comfort. The results demonstrate that structured scene modeling and staged agent-based planning can effectively coordinate character, emotion, and rhythm control in long-form speech generation, providing an interpretable and extensible front-end control framework for discourse-level multi-role audiobook synthesis without requiring modifications to the acoustic model.

摘要： 针对长文本多角色有声书生成中角色边界易混淆、跨句情感演进不连续以及旁白与对话衔接节奏不稳定等问题，本文提出一种基于智能体决策规划的多角色情感有声书生成方法。该方法以场景链作为篇章级中间表示，在不修改声学模型结构的前提下，将长篇小说解析、角色建模、情感控制与节奏规划统一到结构化前端决策流程中。首先，剧本标注智能体依据时间变化、空间转移、人物集合变化和事件主题转移识别场景边界，并在场景内部完成旁白/对话识别、说话人标注及角色集合提取，形成包含场景类型、场景文本、角色身份和语体信息的场景链，为后续模块提供统一且可追踪的上下文。随后，根据角色性别、年龄、身份和性格等属性完成角色—音色绑定，并在生成过程中保持同一角色参考音色不变，以降低跨场景角色漂移。其次，构建由情感向量化智能体和情感约束智能体组成的情感规划模块。情感向量化智能体将文本语义映射为快乐、愤怒、悲伤、恐惧、厌恶、低落、惊讶和平静八维情感向量；情感约束智能体进一步结合角色特征、场景基调和历史情感状态进行分层修正，并通过余弦相似度检测跨轮次非逻辑情感突变，在保留剧情触发型合理变化的同时抑制无依据的情绪跳变。再次，设计上下文节奏规划机制，根据场景类型设置语段内部停顿和语体切换停顿，并结合角色轮替、旁白长度及相邻情感向量变化动态调整句段间停顿，从而改善长旁白、角色连续发言及旁白—对话切换位置的节奏衔接。实验选取40个中文小说片段，覆盖8类典型场景，每类包含5个片段，每个片段包含1名旁白和2～3名剧情角色，共3～4个说话人，并采用公开中文语音数据集中的24名说话人音频作为参考音色。为保证比较公平，所有方法均使用相同文本、相同参考音频和统一的角色—音色映射。实验结果表明，当情感突变阈值设为0.65时，本文方法在自然度、情感准确度、角色贴合度和节奏舒适度上的主观评分分别达到4.09、4.21、3.92和3.96，其中情感准确度和节奏舒适度较次优基线分别提高24.2%和29.8%，盲听偏好率达到72.1%。在客观评价中，本文方法的词错误率为7.55%，音色相似度为0.89，角色混淆率为0.13%，跨场景、跨情感条件下的ISTC为0.027，在保持语音可懂度和参考音色特征的同时，提高了角色身份稳定性。对比实验与消融实验进一步表明，场景链结构、多层情感约束和上下文节奏规划分别对情感连续性、角色贴合度和节奏自然性产生互补作用，其中移除角色/场景约束后角色贴合度下降最明显，移除节奏模块后节奏舒适度下降最明显。研究结果说明，面向长文本的结构化场景建模与分阶段智能体规划能够有效协调角色、情感和节奏控制，为篇章级多角色有声书生成提供一种可解释、可扩展且无需改动声学模型的前端控制方法。