基于内容自适应融合的海上航标监控视频流质量增强方法

doi:10.19678/j.issn.1000-3428.0260594

摘要/Abstract

摘要：

海上多功能航标监控视频流具有静态海面背景占比高、动态船舶目标稀疏的鲜明场景特性, 同时受标载通信网络低带宽限制, 压缩后存在噪声叠加、块效应、振铃效应、帧模糊及帧间运动估计偏差等质量退化问题。现有通用视频质量增强方法难以兼顾单帧空域细节恢复与多帧时序信息利用, 无法满足上述场景需求。为此, 提出一种基于内容自适应融合的海上多功能航标监控视频流质量增强(CAF-VSMM)方法, 构建双分支融合增强模型。基于多尺度特征与时空网络的单帧增强分支, 通过虚拟帧挖掘单帧潜在时序信息, 结合像素重排与残差网络实现多尺度空域特征融合, 有效抑制压缩伪影与海上环境引发的空域质量退化; 基于动态移位窗口与帧间运动补偿的多帧增强分支, 高效建模长距离时空依赖, 利用分层偏移估计与可变形卷积实现多船场景帧间精准对齐, 缓解时域模糊与运动错位; 基于动态特征感知的内容自适应融合(DFP-CAF)模块, 依据监控视频流内容动态调整两分支特征权重, 实现静态背景重细节、动态目标重时序的自适应增强。实验结果表明: 在自建港口群航道监控视频数据集(PC-WMVD)上, 该方法的峰值信噪比(PSNR)、结构相似性(SSIM)、视频多方法评估融合(VMAF)指标分别达42.71 dB、99.33%、77.14%, 较次优对比方法提升3.90 dB、1.19百分点、6.50百分点; 在公开数据集JCT-VC上分别达28.66 dB、88.60%、55.40%, 较次优对比方法提升1.63 dB、5.42百分点、3.19百分点; 模型参数量为37.52×10⁶、浮点运算量(FLOPs)为51.74×10⁹, 处理帧率为41.52帧/s, 满足航道监控实时性要求。该方法已在某港口群航道智能监控系统中示范应用, 可显著提升船舶、航道标识等关键目标辨识度, 有效支撑船舶目标跟踪等下游任务, 验证了工程实用性与应用价值。

关键词: 视频质量增强, 多功能航标, 时空网络, 帧间运动补偿, 单帧多帧融合, 内容自适应融合

Abstract:

The video stream of maritime multifunctional navigation mark monitoring presents the distinctive scene characteristics of a dominant static sea surface background and sparse dynamic ship targets. Constrained by the low-bandwidth transmission of navigation mark-borne communication networks, compressed video streams experience severe quality degradation, including noise superposition, blocking artifacts, ringing artifacts, frame blurring, and inter-frame motion estimation bias. Existing general-purpose video quality enhancement methods are unable to jointly balance single-frame spatial detail recovery and multi-frame temporal information utilization, and therefore fall short of the requirements of the aforementioned scenario. To address this issue, a Content Adaptive Fusion-based Video Stream Quality Enhancement of Maritime Multifunctional Navigation Mark Monitoring (CAF-VSMM) method is proposed, in which a dual-branch fusion enhancement model is constructed. The single-frame enhancement branch, built on multiscale features and a spatio-temporal network, exploits the latent temporal information of a single-frame via virtual frames and integrates the pixel shuffle with a residual network to achieve multiscale spatial feature fusion, effectively suppressing compression artifacts and spatial degradation induced by the maritime environment. The multi-frame enhancement branch, based on a dynamic shifted window and inter-frame motion compensation, efficiently models long-range spatio-temporal dependencies and employs hierarchical offset estimation together with deformable convolution to achieve accurate inter-frame alignment in multiship scenes, thereby alleviating temporal blurring and motion misalignment. The Dynamic Feature Perception-based Content-Adaptive Fusion (DFP-CAF) module, based on dynamic feature perception, dynamically adjusts the feature weights of the two branches according to the video content, realizing an adaptive enhancement strategy that emphasizes detailed recovery for static backgrounds and temporal consistency for dynamic targets. Experimental results show that, on the self-built Port Cluster Waterway Monitoring Video Dataset (PC-WMVD), the Peak Signal to Noise Ratio (PSNR), Structural Similarity (SSIM), and Video Multimethod Evaluation Fusion (VMAF) of the proposed method reach 42.71 dB, 99.33%, and 77.14%, outperforming the second-best method by 3.90 dB, 1.19 percentage points, and 6.50 percentage points, respectively; on the public dataset JCT-VC, they reach 28.66 dB, 88.60%, and 55.40%, exceeding the second-best method by 1.63 dB, 5.42 percentage points, and 3.19 percentage points, respectively. The model has 37.52×10⁶ parameters and 51.74×10⁹ Floating-Point Operations Per Second (FLOPs) and runs at 41.52 frames per second, meeting the real-time requirements of waterway monitoring. The proposed method is tested in an intelligent monitoring system for the waterways of port clusters, where it significantly improves the identifiability of key targets, such as ships and navigation marks, and effectively supports downstream tasks, such as ship tracking, verifying its engineering practicability and application value.

Key words: video quality enhancement, multi-functional navigation mark, spatio-temporal network, inter-frame motion compensation, single-frame and multi-frame fusion, content-adaptive fusion

孙宇, 高曙, 张炎鑫. 基于内容自适应融合的海上航标监控视频流质量增强方法[J]. 计算机工程, 2026, 52(8): 71-83.

SUN Yu, GAO Shu, ZHANG Yanxin. Video Stream Quality Enhancement Method for Maritime Navigation Mark Monitoring Based on Content Adaptive Fusion[J]. Computer Engineering, 2026, 52(8): 71-83.

https://www.ecice06.com/CN/Y2026/V52/I8/71

图/表 15

图1 CAF-VSMM模型架构

Fig.1 CAF-VSMM model architecture

图2 双阶段递进融合结构

Fig.2 Two-stage progressive fusion structure

图3 DCS结构

Fig.3 DCS structure

图4 海上多功能航标工作现场

Fig.4 Marine multi-functional navigation mark work site

图5 PC-WMVD中不同时段与气象条件下的示例图

Fig.5 Example diagrams of PC-WMVD under different time periods and meteorological conditions

图6 PC-WMVD部分视频帧

Fig.6 Part of the video frames of PC-WMVD

图7 CAF-VSMM与次优模型UVE-Net在PC-WMVD和JCT-VC上可视化比较结果

Fig.7 Visual comparison results of CAF-VSMM and UVE-Net suboptimal model on PC-WMVD and JCT-VC

图8 视频帧处理速度示例图

Fig.8 Example diagrams of video frame processing speed

图9 监控视频流质量增强前后对比结果

Fig.9 Comparison results of surveillance video stream quality before and after enhancement

图10 监控视频流质量增强前后的船舶目标识别可视化比较结果

Fig.10 Visual comparison results of ship target identification before and after enhancing the quality of surveillance video streams

图11 失败案例

Fig.11 A failure case

参考文献 34

1	张胜男. 基于深度学习的海上船舶目标识别与跟踪[D]. 大连: 大连交通大学, 2025.
	ZHANG S N. Recognition and tracking of ship targets at sea based on deep learning[D]. Dalian: Dalian Jiaotong University, 2025. (in Chinese)
2	郑海林. 面向智慧海事监管的船舶异常行为检测研究[D]. 上海: 上海海事大学, 2025.
	ZHENG H L. Research on ship abnormal behavior detection for intelligent maritime supervision[D]. Shanghai: Shanghai Maritime University, 2025. (in Chinese)
3	KAU L J, TSENG C K, LEE M X. Perception-based H.264/AVC video coding for resource-constrained and low-bit-rate applications. Sensors, 2025, 25(14): 4259. doi: 10.3390/s25144259
4	LI F Y, ZHAI H J, LIU T, et al. Learning compressed artifact for JPEG manipulation localization using wide-receptive-field network. ACM Transactions on Multimedia Computing, Communications, and Applications, 2024, 20(10): 1- 23.
5	IBRAHEEM M K, DVORKOVICH A V. Enhancing versatile video coding efficiency via post-processing of decoded frames using residual network integration in deep convolutional neural networks[C]//Proceedings of the 26th International Conference on Digital Signal Processing and Its Applications (DSPA). Washington D.C., USA: IEEE Press, 2024: 1-9.
6	WANG T T, HU Z H, GUAN Y R. An efficient lightweight network for image denoising using progressive residual and convolutional attention feature fusion. Scientific Reports, 2024, 14, 9554. doi: 10.1038/s41598-024-60139-x
7	陈圣杰. 面向压缩视频质量增强的多帧对齐方法研究[D]. 成都: 电子科技大学, 2023.
	CHEN S J. Research on multi-frame alignment method for compressed video quality enhancement[D]. Chengdu: University of Electronic Science and Technology of China, 2023. (in Chinese)
8	罗登晏. 基于深度学习的多帧压缩视频质量增强方法研究[D]. 成都: 电子科技大学, 2023.
	LUO D Y. Deep learning methods for multi-frame quality enhancement on compressed video[D]. Chengdu: University of Electronic Science and Technology of China, 2023. (in Chinese)
9	DENG J N, WANG L, PU S L, et al. Spatio-temporal deformable convolution for compressed video quality enhancement[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2020: 10696-10703.
10	ZHANG X J, YANG S, LUO W Y, et al. Video compression artifact reduction by fusing motion compensation and global context in a swin-CNN based parallel architecture[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, USA: AAAI Press, 2023: 3489-3497.
11	LIU J H, ZHOU M C, XIAO M. Deformable convolution dense network for compressed video quality enhancement[C]// Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D.C., USA: IEEE Press, 2022: 1930-1934.
12	魏柳. 多尺度压缩视频增强算法研究与应用[D]. 成都: 电子科技大学, 2024.
	WEI L. Research and application of multi-scale compressed video enhancement algorithm[D]. Chengdu: University of Electronic Science and Technology of China, 2024. (in Chinese)
13	ZHAO M Y, XU Y, ZHOU S G. Recursive fusion and deformable spatiotemporal attention for video compression artifact reduction[C]//Proceedings of the 29th ACM International Conference on Multimedia. New York, USA: ACM Press, 2021: 5646-5654.
14	LI H, HE X H, XIONG S H, et al. A compressed video quality enhancement algorithm based on CNN and transformer hybrid network. The Journal of Supercomputing, 2024, 81(1): 144. doi: 10.1007/s11227-024-06654-0
15	XIE C H, TAN M X, GONG B Q, et al. Adversarial examples improve image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2020: 816-825.
16	LI J, YU Z T, HE Z Q, et al. PGD-Imp: rethinking and unleashing potential of classic PGD with dual strategies for imperceptible adversarial attacks[C]//Proceedings of the 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Washington D.C., USA: IEEE Press, 2025: 1-5.
17	MAENG J B, KWON H. Time-constrained adversarial attacks for video recognition models: temporally sparse but effective perturbations. Machine Vision and Applications, 2026, 37(2): 22. doi: 10.1007/s00138-025-01779-5
18	HUANG Z J, SUN J, GUO X P. FastCNN: towards fast and accurate spatiotemporal network for HEVC compressed video enhancement. ACM Transactions on Multimedia Computing, Communications, and Applications, 2023, 19(3): 1- 22. doi: 10.1145/3569583
19	LIU Z, HU H, LIN Y T, et al. Swin Transformer V2: scaling up capacity and resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2022: 11999-12009.
20	ZHANG X, CAI N, ZHANG H, et al. AFD-former: a hybrid transformer with asymmetric flow division for synthesized view quality enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8): 3786- 3798. doi: 10.1109/TCSVT.2023.3241920
21	AGNOLUCCI L, GALTERI L, BERTINI M, et al. Perceptual quality improvement in videoconferencing using keyframes-based GAN. IEEE Transactions on Multimedia, 2024, 26, 339- 352. doi: 10.1109/TMM.2023.3264882
22	PARK S W, JUNG S H, SIM C B. NeXtSRGAN: enhancing super-resolution GAN with ConvNeXt discriminator for superior realism. The Visual Computer, 2025, 41, 7141- 7167. doi: 10.1007/s00371-024-03797-2
23	XIE Y F, KONG L W, CHEN K, et al. UVEB: a large-scale benchmark and baseline towards real-world underwater video enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2024: 22358-22367.
24	YUE J, YE M, JI L P, et al. A survey of deep-learning-based compressed video quality enhancement. IEEE Transactions on Broadcasting, 2025, 71(4): 977- 992. doi: 10.1109/TBC.2025.3570871
25	YUAN Y, LI E Y, ZHANG J W, et al. PSTAN: a JND-aware pairwise spatio-temporal alignment network for compressed videos quality enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 2026, 36(6): 8491- 8505. doi: 10.1109/TCSVT.2026.3658352
26	XING Q L, ZHENG C, XU M, et al. Breaking the multi-enhancement bottleneck: domain-consistent quality enhancement for compressed images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, 48(4): 4692- 4707. doi: 10.1109/TPAMI.2025.3646223
27	HUANG Z Y, CHAN Y L, KWONG N W, et al. Long short-term fusion by multi-scale distillation for screen content video quality enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35(8): 7762- 7777. doi: 10.1109/TCSVT.2025.3544314
28	ZHU Q, HAO J H, DING Y K, et al. CPGA: coding priors-guided aggregation network for compressed video quality enhancement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Washington D.C., USA: IEEE Press, 2024: 2964-2974.
29	WANG W Z, DAI E Z. Multi-view video quality enhancement method based on multi-scale fusion convolutional neural network and visual saliency. IEEE Access, 2024, 12, 33100- 33108. doi: 10.1109/ACCESS.2024.3369036
30	SHALIN L A, KANSAL V, NIKITHA M, et al. Hybrid quantum-classical convolutional squeeze excitation neural network based HEVC video quality enhancement[C]//Proceedings of the 2nd International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS). Washington D.C., USA: IEEE Press, 2024: 100-106.
31	PEI S F, REN G M, ZHAO T S, et al. STDF-DSCS: a lightweight scheme for compressed video quality enhancement[C]//Proceedings of the 16th International Conference on Wireless Communications and Signal Processing (WCSP). Washington D.C., USA: IEEE Press, 2025: 1324-1329.
32	GEHLOT S, SU G M. LViCAR: diffusion models for perceptual quality enhancement in video compression artifact reduction[C]//Proceedings of the 3rd International Workshop on Deep Multimodal Generation and Retrieval. New York, USA: ACM Press, 2025: 1-10.
33	DESHPANDE S. Standards-based neural-network post-filters for improved video quality[C]//Proceedings of the 3rd Mile-High Video Conference. New York, USA: ACM Press, 2024: 75-81.
34	CHOI W, YOON J. Real-time enhancement of low-quality video for constrained camera systems[C]//Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. Washington D.C., USA: IEEE Press, 2024: 1659-1661.

选择文件类型/文献管理软件名称

选择包含的内容