作者投稿和查稿 主编审稿 专家审稿 编委审稿 远程编辑

计算机工程 ›› 2019, Vol. 45 ›› Issue (6): 89-95. doi: 10.19678/j.issn.1000-3428.0050835

• 体系结构与软件技术 • 上一篇    下一篇

基于文字识别与页面布局的APP控件识别算法

丁世举1,2,顾乃杰1,2,黄章进1,2,侯津1,2   

  1. 1.中国科学技术大学 计算机科学与技术学院 合肥 230027; 2.安徽省计算与通信软件重点实验室 合肥 230027
  • 收稿日期:2018-03-19 出版日期:2019-06-15 发布日期:2019-06-15
  • 作者简介:丁世举(1992—),男,硕士,主研方向为软件脆弱性分析、软件自动化测试;顾乃杰(通信作者),教授、博士生导师;黄章进,副教授、博士;侯津,硕士。

APP control recognition algorithm based on text recognition and page layout

DING Shiju 1,2,GU Naijie 1,2,HUANG Zhangjin 1,2,HOU Jin 1,2   

  1. 1.School of Computer Science and Technology,University of Science and Technology of China,Hefei 230027,China;2.Anhui Province Key Laboratory of Computing and Communication Software,Hefei 230027,China
  • Received:2018-03-19 Online:2019-06-15 Published:2019-06-15

摘要: 在移动应用的开发周期中,使用图形用户界面(GUI)自动化测试技术可提高APP的开发测试效率。针对基于录制回放的GUI自动化测试方法存在不适用黑盒测试场景或识别正确率较低等不足,提出基于图像文字识别和应用页面布局的控件识别算法。根据移动应用的页面布局结构和手势操作坐标精确识别控件,采用控件树遍历的方式解决遍历过程中的页面布局重叠问题。分别在Android和iOS 2个平台进行实验,结果表明,与基于坐标和源代码的控件识别算法相比,该控件识别算法具有更高的正确率。

关键词: 图形用户界面, 自动化测试, 控件识别, 文字识别, 页面布局

Abstract: In the development cycle of mobile applications,the use of Graphical User Interface(GUI) automated testing technology can improve the efficiency of the development and test of APP.Still,the GUI automated testing method based on recording playback is not applicable to black box test scenarios,and its recognition accuracy rate is not high.To deal with the advantages,this paper proposes a control recognition algorithm based on text recognition in images and application page layout.Based on the page layout structure of mobile applications and the accurate recognition control for the coordinates of gesture operation,the overlapping problem of page layout in the process of traversing is solved by traversing the control tree.Experiments are carried out on Android and iOS platforms respectively,and results show that the control recognition algorithm has higher accuracy than the control recognition algorithm based on coordinates and source codes.

Key words: Graphical User Interface(GUI), automated testing, control recognition, text recognition, page layout

中图分类号: