摘要： 使用无监督的方法,将口语文本中的非正规维吾尔语词项归一化到正规文本中意思相近的正规词,基于神经网络,利用大规模语料将维吾尔语单词映射到低维向量空间,对向量空间的非正规词进行聚类。引入一个贪心解码器对非正规词做归一化处理,并进行重采样迭代,从而将之前未能成功归一化的非正规词归一化。实验结果表明,使用该方法对维汉机器翻译的待翻译口语文本进行前编辑后,生成的译文质量有显著提高。该方法给维汉口语文本机器翻译系统提供一个前处理的流程,在缺乏双语口语平行语料的情况下也能有效提高机器翻译系统性能。

关键词: 维吾尔语口语文本, 非正规词, 归一化, 神经网络, 重采样

Abstract: A unsupervised approach to normalize the irregular Uygur words in the spoken text to normal words in the formal text.Based on neural network,Uygur words are mapped to a low dimensional vector space by using a large corpus.The irregular words in vector space are clustered.A greedy decoder is introduced to normalize the unformal words and to resample iterations,so as to normalize the unformal words which have not been successfully normalized before.Experiment results show that using this approach to pre-edit the text to be translated by Uyghur-Chinese machine translation,the quality of the generated translation is significantly improved.This method provides a pretreatment process to spoken text and machine translation system,which can effectively improve the system performance of machine translation in the absence of bilingual parallel corpus of spoken.

Key words: Uyghur spoken text, unformal word, normalization, neural network, resample

中图分类号: 

• TP391