Abstract:

Because most test difficulty prediction schemes are labor-intensive, time-consuming, and prone to leakage, or to some extent subjective, they seriously affect the progress of intelligent education evaluation systems. Therefore, the use of neural networks to predict question difficulty automatically is of great significance. For this purpose, this study proposes a Multi-feature Attention-based Bidirectional Recurrent Neural Network model (M-ABRNN). First, the model retrieves computer-related knowledge to enrich the question stem information based on multi-feature task learning methods. Second, it mines the logical relationships in the objective question text data, extracts statement representations through a bidirectional recurrent neural network, and uses attention mechanisms to measure the importance of associated statements to the question. Finally, the obtained features are input into the model for training, and after training, the difficulty of each new question can be predicted automatically. On a university computer fundamentals course dataset, the proposed model significantly improves the Pearson Correlation Coefficient (PCC) and Degree of Agreement (DOA). These findings show that the model can effectively predict the difficulty of objective questions and evaluate question difficulty automatically.

Key words: big data for education, multitasking features, objective question difficulty prediction, bidirectional recurrent neural network, attention mechanism

摘要：

由于大多数试题难度预测方案是劳动密集型的, 耗时且容易泄漏, 或者在某种程度上是主观的, 严重影响智能化教育评价体系的进步发展, 因此, 利用神经网络实现试题难度自动预测具有重要意义。提出一种基于多特征注意力的双向循环神经网络模型(M-ABRNN)。该模型首先基于多特征任务学习方法, 通过检索计算机关联知识以丰富题干信息; 其次通过双向循环神经网络挖掘客观题文本数据的逻辑关系并提取语句表征, 并利用注意力机制度量关联语句对试题的重要程度; 最后将获取的特征输入到模型中进行训练, 训练完后模型可以自动预测每个新试题的难度。在大学计算机基础课程数据集上的实验结果表明, 所提模型的皮尔逊相关系数(PCC)和一致性(DOA)均有显著提升, 可见该模型能够有效地对客观题难度进行预测, 实现题目难度的自动化评测。

关键词: 教育大数据, 多任务特征, 客观题难度预测, 双向循环神经网络, 注意力机制