作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2008, Vol. 34 ›› Issue (7): 41-43,4. doi: 10.3969/j.issn.1000-3428.2008.07.014

• 博士论文 • 上一篇    下一篇

基于Web的大规模语料库构建方法

李培峰,朱巧明,钱培德   

  1. (苏州大学计算机科学和技术学院,苏州 215006)
  • 收稿日期:1900-01-01 修回日期:1900-01-01 出版日期:2008-04-05 发布日期:2008-04-05

Construction Approach of Large-scale Corpus Based on Web

LI Pei-feng, ZHU Qiao-ming, QIAN Pei-de   

  1. (School of Computer Science and Technology, Soochow University, Suzhou 215006)
  • Received:1900-01-01 Revised:1900-01-01 Online:2008-04-05 Published:2008-04-05

摘要: 低成本、短周期构建大规模语料库是目前研究工作的难点之一。该文提出一种建设大规模语料库的新方法,主要解决如何基于Web构建大规模的语料库及对语料库纠错,从而提高其质量。该方法利用网格技术的大规模计算能力与Wiki的开放编辑环境去收集和处理语料,根据可信度模型挑选出不可信的语料并由人工进行校对,计算校对后结果的可信度,选择出最可信的结果作为正确语料存储到语料库中。

关键词: 大规模语料库, 网格, 可信度

Abstract: Nowadays, it’s hard to build a large-scale corpus with low cost and short period. A new approach is provided to build that on Web. It mainly focuses on how to build a large-scale corpus on Web and then how to correct the mistakes in the corpus. The language materials are collected and processed based on grid and Wiki. The untrustworthy language materials in the corpus are picked out to be checked manually on Wiki according to their trustworthiness. After the check finishes, the approach calculates the trustworthiness of each checked result and selects ones with highest trustworthiness as the correct result.

Key words: large scale corpus, grid, trustworthiness

中图分类号: