Study on Adaptive Bitrate Algorithm in Decision Tree Based on Imitation Learning

doi:10.19678/j.issn.1000-3428.0065251

Abstract

Abstract: Adaptive Bitrate（ABR） algorithm is an effective method to improve the quality of streaming media services，mainly divided into heuristic and learning-based algorithms.The traditional heuristic algorithm is based on fixed rules，making it difficult to handle changing network conditions，whereas algorithms based on deep reinforcement learning have strong mapping expression capabilities but lack robustness and interpretability.To address these issues，a decision tree ABR algorithm ABRTree based on imitation learning is proposed.An effective expert ABR algorithm is designed for the frame-level live broadcast transmission system，and the temporal empirical data of the expert algorithm is discretized.The classification regression tree is used as the basic model of rate decisions.The DAgger algorithm is used to train the decision tree based on the example data provided by the expert algorithm.Pruning the sample set reduces its size，thus improving the generalizability of the decision tree model.The experimental results demonstrate that ABRTree can guarantee image quality in various video scenarios and achieve lower end-to-end delay and less stalling.Compared with BBA，HYSA，and FrameMPC algorithms，the ABRTree algorithm can improve the Quality of Experience（QoE） performance by 1.0% to 29.1%，and the decision tree model can intuitively express the relationship between input characteristics and rate decisions，with better interpretability and mapping expression ability.

Key words: HTTP Adaptive Streaming（HAS）, Adaptive Bitrate（ABR） algorithm, decision tree, imitation learning, live streaming

摘要： 码率自适应（ABR）算法是提升流媒体服务质量的有效方法，主要分为启发式算法和基于学习的算法两类。传统的启发式算法基于固定的规则，难以应对多变的网络环境，基于深度强化学习的算法映射表达能力较好，但其鲁棒性不佳且可解释性较差。针对上述问题，提出一种基于模仿学习的决策树码率自适应算法ABRTree。针对帧级别直播传输系统设计有效的专家ABR算法，并对专家算法的时序经验数据进行离散化处理。采用分类回归树作为码率决策的基础模型，基于专家算法给出的示例数据，采用DAgger算法进行决策树的训练。在此基础上，通过剪枝操作剔除出现较少的样本，从而提升决策树模型的泛化性。实验结果表明，ABRTree在多种视频场景下均能保证画面质量，同时取得较低的端到端延时和较少的卡顿，相比BBA、HYSA和FrameMPC算法，ABRTree算法的QoE性能可以提升1.0%~29.1%，且决策树模型能够直观表达输入特征与码率决策之间的关系，具有较好的可解释性和映射表达能力。

关键词: HTTP自适应流媒体, 码率自适应算法, 决策树, 模仿学习, 流媒体直播

CLC Number:

TP37

WANG Bo, ZHANG Yuan, YANG Yongbei. Study on Adaptive Bitrate Algorithm in Decision Tree Based on Imitation Learning[J]. Computer Engineering, 2023, 49(5): 206-214.

王博, 张远, 杨咏蓓. 基于模仿学习的决策树码率自适应算法研究[J]. 计算机工程, 2023, 49(5): 206-214.

/ / Recommend / Download Citations

URL: http://www.ecice06.com/EN/10.19678/j.issn.1000-3428.0065251

http://www.ecice06.com/EN/Y2023/V49/I5/206

Figures/Tables 19

References

[1] Sandvine.The mobile Internet phenomena report[EB/OL].[2022-06-05].https://www.sandvine.com/2019-mobile-internet-phenomena-report.
[2] SEUFERT M,EGGER S,SLANINA M,et al.A survey on quality of experience of HTTP adaptive streaming[J].IEEE Communications Surveys & Tutorials,2015,17(1):469-492.
[3] HTTP live streaming 2nd edition[EB/OL].[2022-06-05].https://www.semanticscholar.org/paper/HTTP-Live-Streaming-2nd-Edition-Pantos/b47873b8a5e62730c16784376d7f8bd00e4902ff.
[4] Introducing low-latency HLS[EB/OL].[2022-06-05].https://developer.apple.com/videos/play/wwdc2019/502/.
[5] SODAGAR I.The MPEG-DASH standard for multimedia streaming over the Internet[J].IEEE MultiMedia,2011,18(4):62-67.
[6] DASH Industry Forum.Low-latency modes for DASH[EB/OL].[2022-06-05].https://dashif.org/docs/CR-Low-Latency-Live-r8.pdf.
[7] BENTALEB A,TAANI B,BEGEN A C,et al.A survey on bitrate adaptation schemes for streaming media over HTTP[J].IEEE Communications Surveys & Tutorials,2018,21(1):562-585.
[8] DASH reference player[EB/OL].[2022-06-05].http://mediapm.edgesuite.net/dash/public/nightly/samples/dash-if-reference-player/index.html.
[9] HUANG T Y,JOHARI R,MCKEOWN N,et al.A buffer-based approach to rate adaptation:evidence from a large video streaming service[C]//Proceedings of 2014 ACM Conference on SIGCOMM.New York,USA:ACM Press,2014:187-198.
[10] SPITERI K,URGAONKAR R,SITARAMAN R K.BOLA:near-optimal bitrate adaptation for online videos[J].IEEE/ACM Transactions on Networking,2020,28(4):1698-1711.
[11] YIN X Q,JINDAL A,SEKAR V,et al.A control-theoretic approach for dynamic adaptive video streaming over HTTP[C]//Proceedings of 2015 ACM Conference on Special Interest Group on Data Communication.New York,USA:ACM Press,2015:325-338.
[12] PENG H,ZHANG Y,YANG Y B,et al.A hybrid control scheme for adaptive live streaming[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York,USA:ACM Press,2019:2627-2631.
[13] MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529-533.
[14] MAO H Z,NETRAVALI R,ALIZADEH M.Neural adaptive video streaming with pensieve[C]//Proceedings of ACM Special Interest Group on Data Communication.New York,USA:ACM Press,2017:197-210.
[15] MNIH V,BADIA A P,MIRZA M,et al.Asynchronous methods for deep reinforcement learning[C]//Proceedings of the 33rd International Conference on Machine Learning.New York,USA:ACM Press,2016:1928-1937.
[16] AKHTAR Z,NAM Y S,GOVINDAN R,et al.Oboe:auto-tuning video ABR algorithms to network conditions[C]//Proceedings of ACM Special Interest Group on Data Communication.New York,USA:ACM Press,2018:44-58.
[17] HUANG T C,ZHOU C,ZHANG R X,et al.Comyco:quality-aware adaptive video streaming via imitation learning[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York,USA:ACM Press,2019:429-437.
[18] OSA T,PAJARINEN J,NEUMANN G,et al.An algorithmic perspective on imitation learning[J].Foundations and Trends in Robotics,2018,7(1/2):1-179.
[19] HUANG T C,SUN L F.Deepmpc:a mixture ABR approach via deep learning and MPC[C]//Proceedings of IEEE International Conference on Image Processing.Washington D.C.,USA:IEEE Press,2020:1231-1235.
[20] SUTTON R S,MCALLESTER D,SINGH S,et al.Policy gradient methods for reinforcement learning with function approximation[C]//Proceedings of the 12th International Conference on Neural Information Processing Systems.New York,USA:ACM Press,1999:1057-1063.
[21] YI G,YANG D,WANG M W,et al.L3VTP:a low-latency live video transmission platform[C]//Proceedings of ACM SIGCOMM'19.New York,USA:ACM Press,2019:138-140.
[22] ROSS S,GORDON G,BAGNELL D.A reduction of imitation learning and structured prediction to no-regret online learning[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics.New York,USA:ACM Press,2011:627-635.
[23] LEWIS R J.An introduction to Classification and Regression Tree(CART) analysis[EB/OL].[2022-06-05].https://web.archive.org/web/20201105123752/https://cdn.mdedge.com/files/s3fs-public/issues/articles/wray0627_0318e.pdf.
[24] YI G,YANG D,BENTALEB A,et al.The ACM multimedia 2019 live video streaming grand challenge[C]//Proceedings of the 27th ACM International Conference on Multimedia.New York,USA:ACM Press,2019:2622-2626.
[25] GOLUBCHIK L,LUI J C S,MUNTZ R.Reducing I/O demand in video-on-demand storage servers[C]//Proceedings of 1995 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems.New York,USA:ACM Press,1995:25-36.

Please choose a citation manager

Content to export