Research on Test Adequacy of Command Information System Based on Mission Profile

doi:10.19678/j.issn.1000-3428.0065480

Abstract

Abstract:

Existing software test adequacy evaluation methods based on code and function coverages can meet the test adequacy evaluation requirements of embedded software or small-scale information system software.However, because of the large scale, multiple users, diverse tasks, and high complexity of the command information system, effectively measuring and comprehensively evaluating test adequacy from the perspective of code and function coverages can be difficult.In this study, a method for evaluating the sufficiency of command information system testing based on the mission profile is proposed. The mission profile is constructed by analyzing the use scenarios and business requirements of the command information system.Markov chain theory is used to study the relevant characteristics of the mission profile and comb the temporal interaction between task states. A test simulation model based on the mission profile and system characteristic state is established and the fuzzy comprehensive evaluation method is used to quantitatively evaluate its test adequacy. With message transmission and distribution taken as an example, the results show that the 13 test sequences with the highest test path probability product can achieve the test adequacy as a very sufficient evaluation result.Compared with the 89 test sequences identified using the test simulation model, the scale of the test sequence executed decreases by more than 85%, realizing a balance between test efficiency and test adequacy.

Key words: mission profile, command information system, test adequacy, Markov chain, fuzzy comprehensive evaluation

摘要：

现有的基于代码覆盖和功能覆盖的软件测试充分性评价方法，能够满足嵌入式软件或小规模信息系统软件的测试充分性评价要求，但由于指挥信息系统具有规模大、用户多、任务多样且复杂性高等特点，从代码覆盖和功能覆盖的角度难以做到对测试充分性的有效衡量和全面评价。提出一种基于任务剖面的指挥信息系统测试充分性评价方法，通过分析指挥信息系统的使用场景和业务需求构建任务剖面，根据马尔可夫链理论研究任务剖面相关特性和梳理任务状态间的时序交互关系，建立基于任务剖面和系统特征状态的测试仿真模型，通过构建基于系统特征状态、迁移边和变量空间等维度的测试充分性评价指标体系，采用模糊综合评价方法实现测试仿真模型的测试充分性量化评价。以文电传输下发为对象进行实例验证，结果表明，测试路径概率乘积最高的13条测试序列即可达到测试充分性为非常充分的评价结果，相较于测试仿真模型识别出的89条测试序列，执行的测试序列规模下降85%以上，实现了测试效率和测试充分性的平衡。

关键词: 任务剖面, 指挥信息系统, 测试充分性, 马尔可夫链, 模糊综合评价

Ping YANG, Zhenyu WU, Gang LONG, Huahui YANG, Junduo YANG, Penghui LIU, Yingbei NIU. Research on Test Adequacy of Command Information System Based on Mission Profile[J]. Computer Engineering, 2023, 49(7): 295-304.

杨萍, 吴振宇, 龙刚, 杨华晖, 杨竣铎, 刘鹏辉, 牛颖蓓. 基于任务剖面的指挥信息系统测试充分性研究[J]. 计算机工程, 2023, 49(7): 295-304.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0065480

http://www.ecice06.com/EN/Y2023/V49/I7/295

Figures/Tables 10

References 27

1	王文普, 刘光耀, 杨慧, 等. 指挥信息系统验证理论和验证方法. 指挥控制与仿真, 2014, 36 (6): 88-91, 95 URL
	WANG W P, LIU G Y, YANG H, et al. Theory and process of validation for C³ I. Command Control & Simulation, 2014, 36 (6): 88-91, 95 URL
2	刘军. 指挥自动化信息系统改进类软件测试实施研究. 电子技术与软件工程, 2022, (5): 84- 88. URL
	LIU J. Research on implementation of improved software testing for command automation information system. Electronic Technology & Software Engineering, 2022, (5): 84- 88. URL
3	HEMMATI H. How effective are code coverage criteria? [C]//Proceedings of IEEE International Conference on Software Quality, Reliability and Security. Washington D. C., USA: IEEE Press, 2015: 151-156.
4	DA H L , CHANG I H , PHAM H . Software reliability. New York, USA: John Wiley & Sons, Inc, 2014.
5	ABRAN A . Software metrics and software metrology. New York, USA: John Wiley & Sons, Inc, 2010.
6	王一彻, 高建华. 基于运行剖面的测试用例分配与选择方法. 计算机工程, 2020, 46 (6): 216- 220. URL
	WANG Y C, GAO J H. Allocation and selection method of test cases based on operational profile. Computer Engineering, 2020, 46 (6): 216- 220. URL
7	WURSTER C . Remote monitoring proven to help prolong life in patients with pacemakers available. S. 1.: Marshall Cavendish Editions, 2014.
8	DALAL S, LYU M, MALLOWS C. Software reliability[EB/OL]. [2022-07-01]. https://www.scirp.org/reference/referencespapers.aspx?referenceid=230484.
9	CARTE O A L. Reliability, on-going reliability test, HASS[C]//Proceedings of SMTA International Conference. Washington D. C., USA: IEEE Press, 2016: 111-123.
10	朱鸿, 金凌紫. 软件质量保障与测试. 北京: 科学出版社, 1997.
	ZHU H , JIN L Z . Software quality assurance and testing. Beijing: Science Press, 1997.
11	张兆晨. 指挥信息系统技术发展趋势分析[C]//第六届中国指挥控制大会会议论文集. 北京: [出版者不详], 2018: 165-176.
	ZHANG Z C. Command information system technology development trend analysis[C]//Proceedings of the 6th China Command and Control Conference. Beijing: [s. n. ], 2018: 165-176. (in Chinese)
12	陈峰, 赵玉林, 梅发国, 等. 基于软件定义的指挥信息系统设计思考. 中国电子科学研究院学报, 2021, 16 (3): 232- 238. doi: 10.3969/j.issn.1673-5692.2021.03.004
	CHEN F, ZHAO Y L, MEI F G, et al. Thinking of command information system design based on software defined technology. Journal of China Academy of Electronics and Information Technology, 2021, 16 (3): 232- 238. doi: 10.3969/j.issn.1673-5692.2021.03.004
13	赵萌, 石志强. 基于体系结构的指挥信息系统任务可靠性建模. 电子技术与软件工程, 2018, (16): 162- 163. URL
	ZHAO M, SHI Z Q. Task reliability modeling of command information system based on architecture. Electronic Technology & Software Engineering, 2018, (16): 162- 163. URL
14	冯静, 赵蔚婷. 军用信息系统软件可靠性评价指标研究. 军事运筹与系统工程, 2013, 27 (4): 50- 54. URL
	FENG J, ZHAO W T. Research on reliability evaluation index of military information system software. Military Operations Research and Systems Engineering, 2013, 27 (4): 50- 54. URL
15	吴东海, 谭齐, 王青松, 等. 基于复杂网络理论的军用通信系统任务剖面研究. 通信技术, 2017, 50 (11): 2527- 2534. URL
	WU D H, TAN Q, WANG Q S, et al. Connotation and frameworks of mission profile for military communication systems based on complex networks theory. Communications Technology, 2017, 50 (11): 2527- 2534. URL
16	MUSA J D. Operational profiles in software-reliability engineering. IEEE Software, 1993, 10 (2): 14- 32.
17	PRIVAULT N . Discrete-time Markov chains. Berlin, Germany: Springer, 2018: 89- 113.
18	IWANKIEWICZ R. Integro-differential Chapman-Kolmogorov equation for continuous-jump Markov processes and its use in problems of multi-component renewal impulse process excitations. Probabilistic Engineering Mechanics, 2011, 26 (1): 16- 25.
19	GLASS R L, COLLARD R, BERTOLINO A, et al. Software testing and industry needs. IEEE Software, 2006, 23 (4): 55- 57.
20	DE FREITAS JUNIOR M, FANTINATO M, SUN V. Improvements to the function point analysis method: a systematic literature review. IEEE Transactions on Engineering Management, 2015, 62 (4): 495- 506.
21	JONES J A, HARROLD M J. Test-suite reduction and prioritization for modified condition/decision coverage. IEEE Transactions on Software Engineering, 2003, 29 (3): 195- 209.
22	SCHWARTZ A, HETZEL M. The impact of fault type on the relationship between code coverage and fault detection[C]//Proceedings of the 11th International Workshop on Automation of Software Test. New York, USA: ACM Press, 2016: 29-35.
23	AMMANN P, DELAMARO M E, OFFUTT J. Establishing theoretical minimal sets of mutants[C]//Proceedings of the 7th IEEE International Conference on Software Testing, Verification and Validation. Washington D. C., USA: IEEE Press, 2014: 21-30.
24	孙全洲, 杨玲萍, 张素芬. 软件测试充分性模糊决策方法研究及应用. 北京化工大学学报(自然科学版), 2007, 34 (S1): 34- 38. URL
	SUN Q Z, YANG L P, ZHANG S F. Research and application of fuzzy decision-making for measurement of software testing sufficiency. Journal of Beijing University of Chemical Technology (Natural Science Edition), 2007, 34 (S1): 34- 38. URL
25	刘猛, 潘仁前, 葛建超. 联合作战指挥信息系统软件可靠性设计. 指挥信息系统与技术, 2021, 12 (6): 85- 93. URL
	LIU M, PAN R Q, GE J C. Software reliability design for joint operation command information system. Command Information System and Technology, 2021, 12 (6): 85- 93. URL
26	王海宁, 施展, 贺建平. 舰载指挥信息系统特点规律及保持先进性对策. 指挥信息系统与技术, 2022, 13 (2): 14- 18. URL
	WANG H N, SHI Z, HE J P. Characteristic disciplinarian of ship-borne command information systems and countermeasure for keeping advanced. Command Information System and Technology, 2022, 13 (2): 14- 18. URL
27	张文娟, 聂卫国, 张蓉, 等. 海军作战仿真文电系统的研究与实现. 信息技术, 2012, 36 (11): 148-151, 154 URL
	ZHANG W J, NIE W G, ZHANG R, et al. Research and implementation of message handling system in operation simulation for navy arms. Information Technology, 2012, 36 (11): 148-151, 154 URL

序号	标记	状态名称	状态空间
1	V_S	初始	—
2	V₀₁	文电新建	文电对象
3	V₀₂	模板选择	当前文电模板、文电模板列表
4	V₀₃	复用选择	附件列表、草稿列表、发文列表、收文列表、查询结果列表、文电对象
5	V₀₄	解码处理	文电对象、解码结果
6	V₀₅	解码切换	当前解码方式、解码方式列表
7	V₀₆	文电编辑	文电类型、收文方列表、文电正文、文电密级、报文生存时间(h)、文电优先级
8	V₀₇	代码查询	指挥代码字典
9	V₀₈	权限检查	权限控制表、权限检查结果
10	V₀₉	编码处理	文电对象、编码结果
11	V₁₀	编码切换	当前编码方式、编码方式列表
12	V₁₁	状态监控	送达状态、送达监控表、发文列表
13	V₁₂	网络提示	网络状态、送达状态、送达监控表
14	V_E	终止	—

序号	标记	状态名称	状态空间
1	V_S	初始	—
2	V₀₁	文电新建	文电对象
3	V₀₂	模板选择	当前文电模板、文电模板列表
4	V₀₃	复用选择	附件列表、草稿列表、发文列表、收文列表、查询结果列表、文电对象
5	V₀₄	解码处理	文电对象、解码结果
6	V₀₅	解码切换	当前解码方式、解码方式列表
7	V₀₆	文电编辑	文电类型、收文方列表、文电正文、文电密级、报文生存时间(h)、文电优先级
8	V₀₇	代码查询	指挥代码字典
9	V₀₈	权限检查	权限控制表、权限检查结果
10	V₀₉	编码处理	文电对象、编码结果
11	V₁₀	编码切换	当前编码方式、编码方式列表
12	V₁₁	状态监控	送达状态、送达监控表、发文列表
13	V₁₂	网络提示	网络状态、送达状态、送达监控表
14	V_E	终止	—

测试充分性指标项	总数
系统特征状态	14
系统特征状态迁移边	31
系统特征状态空间	27
关键系统特征状态	6
关键系统特征状态迁移边	11
关键系统特征状态空间	18

测试充分性指标项	总数
系统特征状态	14
系统特征状态迁移边	31
系统特征状态空间	27
关键系统特征状态	6
关键系统特征状态迁移边	11
关键系统特征状态空间	18

测试充分性指标项	总数/条	激活数/条	覆盖率/%
系统特征状态	14	9	64.29
系统特征状态迁移边	31	11	35.48
系统特征状态空间	27	20	74.07
关键系统特征状态	6	6	100.00
关键系统特征状态迁移边	11	9	81.82
关键系统特征状态空间	18	15	83.33

Please choose a citation manager

Content to export