基于虚拟网关的交通基础设施监测点位数据接入方法

doi:10.19678/j.issn.1000-3428.0069225

计算机工程 ›› 2024, Vol. 50 ›› Issue (4): 50-59. doi: 10.19678/j.issn.1000-3428.0069225

基于虚拟网关的交通基础设施监测点位数据接入方法

黄亮¹^,², 邹鹏³, 曹菁菁¹^,⁴^,*(), 胡健⁵, 颜泽锌³, 黄小蝶³

1. 武汉理工大学水路交通控制全国重点实验室, 湖北武汉 430063
2. 武汉理工大学国家水运安全工程技术研究中心, 湖北武汉 430063
3. 武汉理工大学智能交通系统研究中心, 湖北武汉 430063
4. 武汉理工大学交通与物流工程学院, 湖北武汉 430063
5. 北京四维图新科技股份有限公司, 北京 100093

收稿日期:2024-01-15 出版日期:2024-04-15 发布日期:2024-04-22
通讯作者: 曹菁菁
基金资助:
国家重点研发计划(2021YFB2600300)

Access Method for Traffic Infrastructure Monitoring Point Data Based on Virtual Gateway

Liang HUANG¹^,², Peng ZOU³, Jingjing CAO¹^,⁴^,*(), Jian HU⁵, Zexin YAN³, Xiaodie HUANG³

1. State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, Hubei, China
2. National Engineering Research Center for Water Transport Safety, Wuhan University of Technology, Wuhan 430063, Hubei, China
3. Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan 430063, Hubei, China
4. School of Transportation and Logistics Engineering, Wuhan University of Technology, Wuhan 430063, Hubei, China
5. NavInfo Co., Ltd., Beijing 100093, China

Received:2024-01-15 Online:2024-04-15 Published:2024-04-22
Contact: Jingjing CAO

摘要/Abstract

摘要：

针对交通基础设施点位监测数据来源多样、结构各异、协议不一的特点, 分析具体接入需求, 提出一种基于Netty架构的交通基础设施监测点位数据接入虚拟网关。阐述网关接入监测点位的配置方法及集群分配策略, 定义超文本传输协议(HTTP)、传输控制协议(TCP)、用户数据报协议(UDP)下的数据传输格式, 设计一种数据传输消息的编码和校验机制。在真实监测数据样本增强的基础上, 利用分布式消息模拟工具对虚拟网关性能进行测试, 结果表明, 该虚拟网关实现了多点位多协议的交通基础设施监测数据统一接入, 每亿条数据接入时间、存储时间分别达到8.14 s和9.75 s, 平均数据溯源时间为2.96 s, 具有亿级规模点位监测数据的接入能力, 可为交通基础设施数字化监测的研究和应用提供理论支撑。

关键词: Netty框架, Kafka消息队列, 数据接入, 平台化, 物联网数据

Abstract:

To address the challenges posed by the diversity of sources, structures, and protocols in transportation infrastructure monitoring point data, this study analyzes the requirements for data access and proposes a virtual gateway for the access of monitoring point data within a transportation infrastructure based on the Netty architecture. The study describe in detail the configuration methods and cluster allocation strategies for integrating monitoring points through the gateway, and it defines the data transmission formats under protocols such as the HyperText Transfer Protocol (HTTP), Transmission Control Protocol (TCP), and User Datagram Protocol (UDP). The study also designs an encoding and verification mechanism for data transmission messages. Through enhanced real monitoring data samples, the performance of the virtual gateway is evaluated using distributed message simulation tools. The findings reveal that the virtual gateway facilitates unified access to multipoint and multiprotocol transportation infrastructure monitoring data, achieving data access and storage time of 8.14 s and 9.75 s per hundred million records, respectively. The average time for data traceability is 2.96 s, thus showing the gateway's capacity to handle monitoring data of points at the billion-scale level. This capability significantly supports the research and application of digital monitoring and transportation infrastructure analysis.

Key words: Netty framework, Kafka message queue, data access, platformization, Internet of Things(IoT) data

黄亮, 邹鹏, 曹菁菁, 胡健, 颜泽锌, 黄小蝶. 基于虚拟网关的交通基础设施监测点位数据接入方法[J]. 计算机工程, 2024, 50(4): 50-59.

Liang HUANG, Peng ZOU, Jingjing CAO, Jian HU, Zexin YAN, Xiaodie HUANG. Access Method for Traffic Infrastructure Monitoring Point Data Based on Virtual Gateway[J]. Computer Engineering, 2024, 50(4): 50-59.

https://www.ecice06.com/CN/Y2024/V50/I4/50

图/表 17

图1 交通基础设施数据接入智能虚拟网关结构

Fig.1 Traffic infrastructure data access intelligent virtual gateway structure

图2 虚拟网关数据接入配置页面

Fig.2 Virtual gateway data access configuration page

图3 数据接入流程

Fig.3 Data access process

图4 TCP协议下的报文格式参考

Fig.4 Message format reference under TCP protocol

图5 报文格式

Fig.5 Message format

图6 UDP数据接入框架

Fig.6 UDP data access frame

图7 粘包与半粘包验证流程

Fig.7 Packet concatenation and half-packet verification process

图8 测试流程和数据传输实图

Fig.8 Testing process and real data transmission diagram

图9 数据存储测试结果

Fig.9 Data storage test results

图10 溯源场景4的执行结果

Fig.10 Execution results of traceability scenario 4

图11 灯浮监测数据可视化

Fig.11 Buoy light monitoring data visualization

参考文献 26

1	CHEUNG W F, LIN T H, LIN Y C. A real-time construction safety monitoring system for hazardous gas integrating wireless sensor network and building information modeling technologies. Sensors, 2018, 18 (2): 436. doi: 10.3390/s18020436
2	李琦, 姚龙. 基于REST架构的湖泊环境监测物联网平台. 计算机工程, 2016, 42 (11): 27-31, 37. doi: 10.3969/j.issn.1000-3428.2016.11.005
	LI Q, YAO L. Internet of Things platform for lake environmental monitoring based on REST architecture. Computer Engineering, 2016, 42 (11): 27-31, 37. doi: 10.3969/j.issn.1000-3428.2016.11.005
3	王庆贺, 刘瑞鑫, 孙立晔, 等. BIM与云、物联网技术在桥梁全生命周期中的研究及应用. 铁道标准设计, 2022, 66 (12): 78- 83. URL
	WANG Q H, LIU R X, SUN L Y, et al. Applications of BIM, cloud computing and Internet of Things technology in full life cycle of bridges. Railway Standard Design, 2022, 66 (12): 78- 83. URL
4	王建伟, 高超, 董是, 等. 道路基础设施数字化研究进展与展望. 中国公路学报, 2020, 33 (11): 101- 124. doi: 10.3969/j.issn.1001-7372.2020.11.011
	WANG J W, GAO C, DONG S, et al. Current status and future prospects of existing research on digitalization of highway infrastructure. China Journal of Highway and Transport, 2020, 33 (11): 101- 124. doi: 10.3969/j.issn.1001-7372.2020.11.011
5	夏烨, 龚丰宗, 尚志强, 等. 基于监测数据衰减响应模式提取的斜拉桥阻尼比特性分析. 中国公路学报, 2023, 36 (7): 90- 101. URL
	XIA Y, GONG F Z, SHANG Z Q, et al. Damping ratio analysis of cable-stayed bridges based on attenuation response pattern extraction from monitoring data. China Journal of Highway and Transport, 2023, 36 (7): 90- 101. URL
6	SHANG Z Q, XIA Y, CHEN L, et al. Damping ratio identification using attenuation responses extracted by time series semantic segmentation. Mechanical Systems and Signal Processing, 2022, 180, 109287. doi: 10.1016/j.ymssp.2022.109287
7	HOU Y, LI Q H, ZHANG C, et al. The state-of-the-art review on applications of intrusive sensing, image processing techniques, and machine learning methods in pavement monitoring and analysis. Engineering, 2021, 7 (6): 845- 856. doi: 10.1016/j.eng.2020.07.030
8	米春, 李思颖, 牟佳祎, 等. 基于多传感器的城市隧道环境监测数据协同融合方法研究. 现代隧道技术, 2023, 60 (5): 177- 185. URL
	MI C, LI S Y, MOU J Y, et al. Study on collaborative fusion method for urban tunnel environmental monitoring data based on multi-sensor. Modern Tunnelling Technology, 2023, 60 (5): 177- 185. URL
9	FAN X P, LIU Y F. Time-variant reliability prediction of bridge system based on BDGCM and SHM data. Structural Control and Health Monitoring, 2018, 25 (7): e2185. doi: 10.1002/stc.2185
10	黄观文, 陈孜, 徐永福. 膨胀土边坡GNSS实时监测技术. 测绘学报, 2023, 52 (11): 1873- 1882. URL
	HUANG G W, CHEN Z, XU Y F. GNSS real-time monitoring technology of expansive soil slope. Acta Geodaetica et Cartographica Sinica, 2023, 52 (11): 1873- 1882. URL
11	钟玉琪, 张红, 张舸, 等. 基于组合荷载响应特征融合的桥梁结构智能损伤识别方法. 实验力学, 2023, 38 (2): 151- 164. URL
	ZHONG Y Q, ZHANG H, ZHANG G, et al. An intelligent damage identification method based on response feature fusion under combined loadings for bridge structures. Journal of Experimental Mechanics, 2023, 38 (2): 151- 164. URL
12	龚小飞, 高志华, 郭颜斌. 基于熵权模型的引航风险评价研究. 中国水运, 2023, (11): 120- 123. URL
	GONG X F, GAO Z H, GUO Y B. Research on pilotage risk assessment based on entropy weight model. China Water Transport, 2023, (11): 120- 123. URL
13	周涛, 邹进贵, 郭际明. 隧道变形监测与智能预警方法研究. 测绘通报, 2022, (S2): 91- 94. URL
	ZHOU T, ZOU J G, GUO J M. Research on tunnel deformation monitoring and intelligent early warning method. Bulletin of Surveying and Mapping, 2022, (S2): 91- 94. URL
14	王代君, 傅俊敏, 周天图, 等. 区域交通基础设施群监测系统设计及应用关键技术研究. 公路交通科技, 2022, 39 (6): 124- 135. URL
	WANG D J, FU J M, ZHOU T T, et al. Study on key technology for design and application of regional transport infrastructure group monitoring system. Journal of Highway and Transportation Research and Development, 2022, 39 (6): 124- 135. URL
15	LAN L N, SHI R S, WANG B, et al. An IoT unified access platform for heterogeneity sensing devices based on edge computing. IEEE Access, 2019, 7, 44199- 44211. doi: 10.1109/ACCESS.2019.2908684
16	顾振德, 刘子辰, 龙隆, 等. 基于Netty的IoT终端通信服务系统设计. 计算机应用与软件, 2019, 36 (4): 135- 139. URL
	GU Z D, LIU Z C, LONG L, et al. Design of IoT terminal communication service system based on Netty. Computer Applications and Software, 2019, 36 (4): 135- 139. URL
17	林俊晖. 物联网通用数据接入平台的设计与实现[D]. 重庆: 重庆邮电大学, 2018.
	LIN J H. Design and implementation of a general data access platform for Internet of Things[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2018. (in Chinese)
18	甄凯成. 面向物联网应用的数据接入和存储系统的设计与实现[D]. 合肥: 中国科学技术大学, 2019.
	ZHEN K C. Design and implementation of data ingestion and storage system for IoT applications[D]. Hefei: University of Science and Technology of China, 2019. (in Chinese)
19	王圣博, 王泽昊, 高梓桐, 等. 一种物联网设备的数据接入方法. 物联网技术, 2023, 13 (3): 64- 66. URL
	WANG S B, WANG Z H, GAO Z T, et al. Data access method for Internet of Things equipment. Internet of Things Technologies, 2023, 13 (3): 64- 66. URL
20	赵润发, 娄渊胜, 叶枫, 等. 基于Flink的工业大数据平台研究与应用. 计算机工程与设计, 2022, 43 (3): 886- 894. URL
	ZHAO R F, LOU Y S, YE F, et al. Research and application of industrial big data platform based on Flink. Computer Engineering and Design, 2022, 43 (3): 886- 894. URL
21	甄凯成, 黄河, 宋良图. 基于Netty和Kafka的物联网数据接入系统. 计算机工程与应用, 2020, 56 (5): 135- 140. URL
	ZHEN K C, HUANG H, SONG L T. Internet of Things data injection system based on Netty and Kafka. Computer Engineering and Applications, 2020, 56 (5): 135- 140. URL
22	THEIN K M M. Apache Kafka: next generation distributed messaging system. International Journal of Scientific Engineering and Technology Research, 2014, 47 (3): 9478- 9483. URL
23	刘美佳, 张箐. 基于分布式集群架构的遥感数据传输机制. 计算机工程, 2021, 47 (10): 180- 185. doi: 10.19678/j.issn.1000-3428.0059668
	LIU M J, ZHANG Q. Remote sensing data transmission mechanism based on distributed cluster architecture. Computer Engineering, 2021, 47 (10): 180- 185. doi: 10.19678/j.issn.1000-3428.0059668
24	SOBIN C C. A survey on architecture, protocols and challenges in IoT. Wireless Personal Communications, 2020, 112 (3): 1383- 1429. URL
25	BENIWAL G, SINGHROVA A. A systematic literature review on IoT gateways. Journal of King Saud University-Computer and Information Sciences, 2022, 34 (10): 9541- 9563.
26	VAN D G, VAN D P D. Influencing factors in the scalability of distributed stream processing jobs. IEEE Access, 2021, 9, 109413- 109431. doi: 10.1109/ACCESS.2021.3102645

[1]	黄亮, 邹鹏, 曹菁菁, 胡健, 颜泽锌, 黄小蝶. 基于虚拟网关的交通基础设施监测点位数据接入方法[J]. 计算机工程, 2024, 50(4): 50-59.
[2]	刘美佳, 张箐. 基于分布式集群架构的遥感数据传输机制[J]. 计算机工程, 2021, 47(10): 180-185.

选择文件类型/文献管理软件名称

选择包含的内容

基于虚拟网关的交通基础设施监测点位数据接入方法

Access Method for Traffic Infrastructure Monitoring Point Data Based on Virtual Gateway

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 26

相关文章 2

编辑推荐

Metrics

本文评价

模态框（Modal）标题

选择文件类型/文献管理软件名称

选择包含的内容

基于虚拟网关的交通基础设施监测点位数据接入方法

Access Method for Traffic Infrastructure Monitoring Point Data Based on Virtual Gateway

RichHTML

PDF

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 17

参考文献 26

相关文章 2

编辑推荐

Metrics

本文评价