Author Login Chief Editor Login Reviewer Login Editor Login Remote Office

Computer Engineering ›› 2008, Vol. 34 ›› Issue (7): 41-43,4.

• Degree Paper • Previous Articles     Next Articles

Construction Approach of Large-scale Corpus Based on Web

LI Pei-feng, ZHU Qiao-ming, QIAN Pei-de   

  1. (School of Computer Science and Technology, Soochow University, Suzhou 215006)
  • Received:1900-01-01 Revised:1900-01-01 Online:2008-04-05 Published:2008-04-05

基于Web的大规模语料库构建方法

李培峰,朱巧明,钱培德   

  1. (苏州大学计算机科学和技术学院,苏州 215006)

Abstract: Nowadays, it’s hard to build a large-scale corpus with low cost and short period. A new approach is provided to build that on Web. It mainly focuses on how to build a large-scale corpus on Web and then how to correct the mistakes in the corpus. The language materials are collected and processed based on grid and Wiki. The untrustworthy language materials in the corpus are picked out to be checked manually on Wiki according to their trustworthiness. After the check finishes, the approach calculates the trustworthiness of each checked result and selects ones with highest trustworthiness as the correct result.

Key words: large scale corpus, grid, trustworthiness

摘要: 低成本、短周期构建大规模语料库是目前研究工作的难点之一。该文提出一种建设大规模语料库的新方法,主要解决如何基于Web构建大规模的语料库及对语料库纠错,从而提高其质量。该方法利用网格技术的大规模计算能力与Wiki的开放编辑环境去收集和处理语料,根据可信度模型挑选出不可信的语料并由人工进行校对,计算校对后结果的可信度,选择出最可信的结果作为正确语料存储到语料库中。

关键词: 大规模语料库, 网格, 可信度

CLC Number: