作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2023, Vol. 49 ›› Issue (7): 251-258. doi: 10.19678/j.issn.1000-3428.0065369

• 开发研究与工程应用 • 上一篇    下一篇

基于深度学习的学生课堂行为识别方法

闫兴亚1, 匡娅茜2, 白光睿2, 李月2   

  1. 1. 西安邮电大学 数字艺术学院, 西安 710121
    2. 西安邮电大学 计算机学院, 西安 710121
  • 收稿日期:2022-07-27 出版日期:2023-07-15 发布日期:2022-10-12
  • 作者简介:

    闫兴亚(1974—),男,教授,主研方向为计算机视觉、增强现实

    匡娅茜,硕士研究生

    白光睿,硕士研究生

    李月,硕士研究生

  • 基金资助:
    陕西省哲学社会科学重大理论与现实问题研究项目(2021HZ0980); 2019年第二批产学合作协同育人项目(201902001030); 软科学研究计划一般项目(2020KRM005)

Student Classroom Behavior Recognition Method Based on Deep Learning

Xingya YAN1, Yaxi KUANG2, Guangrui BAI2, Yue LI2   

  1. 1. School of Digital Arts, Xi'an University of Posts and Telecommunications, Xi'an 710121, China
    2. School of Computer, Xi'an University of Posts and Telecommunications, Xi'an 710121, China
  • Received:2022-07-27 Online:2023-07-15 Published:2022-10-12

摘要:

学生课堂行为动作能够直接反映课堂质量,通过人工智能和大数据对课堂行为进行分析和评估,有助于提高教学质量。传统的学生课堂行为识别方法通过老师直接观察学生状态,或者是课后通过监控视频进行分析,该课堂行为识别方法耗时耗力且识别率较低,难以实时反映课堂以及考试中存在的问题。提出基于深度学习的姿态识别方法BetaPose。采用数据增强技术提高后续检测模型的鲁棒性,通过改进的YOLOv5目标检测算法得到人体检测框,并基于MobileNetV3模型设计轻量级姿态识别模型,提高在拥挤场景下的姿态识别准确率,将得到的人体关键点图输入到线性分类器中,获得最终行为结果,有效提高模型的建模和表达能力。实验结果表明,所提轻量级姿态识别方法BetaPose对人体各个部位的平均识别准确率最高可达82.6%,在简易和拥挤场景下对各种行为的识别率分别达到91%和85%以上,能够有效识别课堂的多种行为。

关键词: 计算机视觉, 行为识别, 目标检测, 姿态识别, 深度学习

Abstract:

Student classroom behaviors can directly reflect the quality of the class, whereby the analysis and evaluation of classroom behaviors through artificial intelligence and big data can help improve the quality of teaching. Traditional student classroom behavior recognition methods rely on teachers' direct observation of students or an analysis of student surveillance videos after class. This method is time-consuming, labor-intensive, and has low recognition rate, making it difficult to follow problems in the classroom and during exams in real time. This study proposes a posture recognition method based on deep learning BetaPose. The data enhancement technology is used to improve the robustness of the subsequent detection model. The improved YOLOv5 target detection algorithm is used to obtain the human detection frame. Based on the MobileNetV3 model, the lightweight posture recognition model is designed to improve the accuracy of posture recognition in crowded scenes. The keypoints of the human body thus obtained are input into the linear classifier with improved modeling and expression ability to determine the final behavior results. The experimental show that the proposed lightweight posture recognition model BetaPose had the highest average recognition accuracy of 82.6% for various parts of the human body, and the recognition rates for various behaviors in simple and crowded scenes are above 91% and 85%, respectively. Therefore, the proposed model can be effectively recognize multiple behaviors in the classroom.

Key words: computer vision, behavior recognition, object detection, posture recognition, deep learning