Friday 5 August 2022

Reinforcement Learning-Based Routing for Real-Time Multimedia Traffic Transmission

 Recently published paper 



A Reinforcement Learning-Based Routing for Real-Time Multimedia Traffic Transmission over Software-Defined Networking

1
Department of Computing, University of Northampton, Northampton NN1 5PH, UK
2
Department of Computer Network, College of Information Technology, University of Babylon, Babil 51001, Iraq
3
School of Computing, Canterbury Christ Church University, Canterbury CT1 1QU, UK
4
Department of Technical and Applied Computing, University of Gloucestershire, The Park, Cheltenham GL50 2RH, UK
*
Electronics 202211(15), 2441; https://doi.org/10.3390/electronics11152441 Published: 5 August 2022

Full Text version(HTML) available at: 
https://www.mdpi.com/2079-9292/11/15/2441/htm




Recently, video streaming services consumption has grown massively and is foreseen to increase even more in the future. The tremendous traffic usage has negatively impacted the network’s quality of service due to network congestion and end-to-end customers’ satisfaction represented by the quality of experience, especially during evening peak hours. This paper introduces an intelligent multimedia framework that aims to optimise the network’s quality of service and users’ quality of experience by taking into account the integration of Software-Defined Networking and Reinforcement Learning, which enables exploring, learning, and exploiting potential paths for video streaming flows. Moreover, an objective study was conducted to assess video streaming for various realistic network environments and under low and high traffic loads to obtain two quality of experience metrics; video multimethod assessment fusion and structural similarity index measure. The experimental results validate the effectiveness of the proposed solution strategy, which demonstrated better viewing quality by achieving better customers’ quality of experience, higher throughput and lower data loss compared with the currently existing solutions.

Al Jameel, M.; Kanakis, T.; Turner, S.; Al-Sherbaz, A.; Bhaya, W.S. A Reinforcement Learning-Based Routing for Real-Time Multimedia Traffic Transmission over Software-Defined Networking. Electronics 202211, 2441. https://doi.org/10.3390/electronics11152441

Full Text version(HTML) available at: 
https://www.mdpi.com/2079-9292/11/15/2441/htm




References

Sandvine. The Global Internet Phenomena Report COVID-19 Spotlight; Sandvine: Waterloo, ON, Canada, 2020.

Trestian, R.; Comsa, I.S.; Tuysuz, M.F. Seamless multimedia delivery within a heterogeneous wireless networks environment: Are we there yet? IEEE Commun. Surv. Tutor. 2018, 20, 945-977.
https://doi.org/10.1109/COMST.2018.2789722

Doumanoglou, A.; Zioulis, N.; Griffin, D.; Serrano, J.; Phan, T.K.; Jiménez, D.; Zarpalas, D.; Alvarez, F.; Rio, M.; Daras, P. A system architecture for live immersive 3D-media transcoding over 5G networks. In Proceedings of the 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Valencia, Spain, 6-8 June 2018; pp. 11-15.
https://doi.org/10.1109/BMSB.2018.8436942

Jawad, N.; Salih, M.; Ali, K.; Meunier, B.; Zhang, Y.; Zhang, X.; Zetik, R.; Zarakovitis, C.; Koumaras, H.; Kourtis, M.A.; et al. Smart television services using NFV/SDN network management. IEEE Trans. Broadcast. 2019, 65, 404-413.
https://doi.org/10.1109/TBC.2019.2898159

Barakabitze, A.A. QoE-Centric Control and Management of Multimedia Services in Software Defined and Virtualized Networks. Ph.D. Thesis, University of Plymouth, Plymouth, UK, 2020.

Martin, A.; Egaña, J.; Flórez, J.; Montalban, J.; Olaizola, I.G.; Quartulli, M.; Viola, R.; Zorrilla, M. Network resource allocation system for QoE-aware delivery of media services in 5G networks. IEEE Trans. Broadcast. 2018, 64, 561-574.
https://doi.org/10.1109/TBC.2018.2828608

Comşa, I.S.; Muntean, G.M.; Trestian, R. An innovative machine-learning-based scheduling solution for improving live UHD video streaming quality in highly dynamic network environments. IEEE Trans. Broadcast. 2020, 67, 212-224.
https://doi.org/10.1109/TBC.2020.2983298

Huang, X.; Yuan, T.; Qiao, G.; Ren, Y. Deep reinforcement learning for multimedia traffic control in software defined networking. IEEE Netw. 2018, 32, 35-41.
https://doi.org/10.1109/MNET.2018.1800097

Grigoriou, E. Quality of Experience Monitoring and Management Strategies for Future Smart Networks. 2020. Available online: https://iris.unica.it/handle/11584/284401 (accessed on 16 February 2022).

Ullah, Z.; Al-Turjman, F.; Mostarda, L.; Gagliardi, R. Applications of artificial intelligence and machine learning in smart cities. Comput. Commun. 2020, 154, 313-323.
https://doi.org/10.1016/j.comcom.2020.02.069

Lekharu, A.; Moulii, K.; Sur, A.; Sarkar, A. Deep learning based prediction model for adaptive video streaming. In Proceedings of the 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS), Bangalore, India, 7-11 January 2020; pp. 152-159.
https://doi.org/10.1109/COMSNETS48256.2020.9027383

Anand, D.; Togou, M.A.; Muntean, G.M. A Machine Learning Solution for Automatic Network Selection to Enhance Quality of Service for Video Delivery. In Proceedings of the 2021 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Chengdu, China, 4-6 August 2021; pp. 1-5.
https://doi.org/10.1109/BMSB53066.2021.9547176

Kattadige, C.; Raman, A.; Thilakarathna, K.; Lutu, A.; Perino, D. 360NorVic: 360-degree video classification from mobile encrypted video traffic. In Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, Istanbul, Turkey, 28 September-1 October 2021; pp. 58-65.
https://doi.org/10.1145/3458306.3460998

Anerousis, N.; Chemouil, P.; Lazar, A.A.; Mihai, N.; Weinstein, S.B. The Origin and Evolution of Open Programmable Networks and SDN. IEEE Commun. Surv. Tutor. 2021, 23, 1956-1971.
https://doi.org/10.1109/COMST.2021.3060582

Egilmez, H.E.; Civanlar, S.; Tekalp, A.M. An optimization framework for QoS-enabled adaptive video streaming over OpenFlow networks. IEEE Trans. Multimed. 2012, 15, 710-715.
https://doi.org/10.1109/TMM.2012.2232645

Juttner, A.; Szviatovski, B.; Mécs, I.; Rajkó, Z. Lagrange relaxation based method for the QoS routing problem. In Proceedings of the Conference on Computer Communications-Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No. 01CH37213), Anchorage, AK, USA, 22-26 April 2001; Volume 2, pp. 859-868.

Yu, T.F.; Wang, K.; Hsu, Y.H. Adaptive routing for video streaming with QoS support over SDN networks. In Proceedings of the 2015 International Conference on Information Networking (ICOIN), Siem Reap, Cambodia, 12-14 January 2015; pp. 318-323.

Ongaro, F.; Cerqueira, E.; Foschini, L.; Corradi, A.; Gerla, M. Enhancing the quality level support for real-time multimedia applications in software-defined networks. In Proceedings of the 2015 International Conference on Computing, Networking and Communications (ICNC), Garden Grove, CA, USA, 16-19 February 2015; pp. 505-509.
https://doi.org/10.1109/ICCNC.2015.7069395

Rego, A.; Sendra, S.; Jimenez, J.M.; Lloret, J. OSPF routing protocol performance in Software Defined Networks. In Proceedings of the 2017 Fourth International Conference on Software Defined Systems (SDS), Valencia, Spain, 8-11 May 2017; pp. 131-136. [Google Scholar]
https://doi.org/10.1109/SDS.2017.7939153

Rego, A.; Sendra, S.; Jimenez, J.M.; Lloret, J. Dynamic metric OSPF-based routing protocol for software defined networks. Clust. Comput. 2019, 22, 705-720.
https://doi.org/10.1007/s10586-018-2875-7

Elbasheer, M.O.; Aldegheishem, A.; Lloret, J.; Alrajeh, N. A QoS-Based routing algorithm over software defined networks. J. Netw. Comput. Appl. 2021, 194, 103215.
https://doi.org/10.1016/j.jnca.2021.103215

Uzakgider, T.; Cetinkaya, C.; Sayit, M. Learning-based approach for layered adaptive video streaming over SDN. Comput. Netw. 2015, 92, 357-368. [Google Scholar] [CrossRef]
https://doi.org/10.1016/j.comnet.2015.09.027

Sendra, S.; Rego, A.; Lloret, J.; Jimenez, J.M.; Romero, O. Including artificial intelligence in a routing protocol using software defined networks. In Proceedings of the 2017 IEEE International Conference on Communications Workshops (ICC Workshops), Paris, France, 21-23 May 2017; pp. 670-674.
https://doi.org/10.1109/ICCW.2017.7962735

Al-Jawad, A.; Shah, P.; Gemikonakli, O.; Trestian, R. LearnQoS: A learning approach for optimizing QoS over multimedia-based SDNs. In Proceedings of the 2018 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Valencia, Spain, 6-8 June 2018; pp. 1-6.
https://doi.org/10.1109/BMSB.2018.8436781

Hossain, M.B.; Wei, J. Reinforcement learning-driven QoS-aware intelligent routing for software-defined networks. In Proceedings of the 2019 IEEE global conference on signal and information processing (GlobalSIP), Ottawa, ON, Canada, 11-14 November 2019; pp. 1-5.
https://doi.org/10.1109/GlobalSIP45357.2019.8969320

Godfrey, D.; Kim, B.S.; Miao, H.; Shah, B.; Hayat, B.; Khan, I.; Sung, T.E.; Kim, K.I. Q-learning based routing protocol for congestion avoidance. Comput. Mater. Contin. 2021, 68, 3671.
https://doi.org/10.32604/cmc.2021.017475

Al-Jawad, A.; Comşa, I.S.; Shah, P.; Gemikonakli, O.; Trestian, R. An innovative reinforcement learning-based framework for quality of service provisioning over multimedia-based sdn environments. IEEE Trans. Broadcast. 2021, 67, 851-867.
https://doi.org/10.1109/TBC.2021.3099728

Al-Jawad, A.; Comşa, I.-S.; Shah, P.; Gemikonakli, O.; Trestian, R. REDO: A reinforcement learning-based dynamic routing algorithm selection method for SDN. In Proceedings of the 2021 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), Online, 9-11 November 2021; pp. 54-59.
https://doi.org/10.1109/NFV-SDN53031.2021.9665140

Guo, Y.; Wang, W.; Zhang, H.; Guo, W.; Wang, Z.; Tian, Y.; Yin, X.; Wu, J. Traffic Engineering in hybrid Software Defined Network via Reinforcement Learning. J. Netw. Comput. Appl. 2021, 189, 103116.
https://doi.org/10.1016/j.jnca.2021.103116

Liu, W.x.; Cai, J.; Chen, Q.C.; Wang, Y. DRL-R: Deep reinforcement learning approach for intelligent routing in software-defined data-center networks. J. Netw. Comput. Appl. 2021, 177, 102865.
https://doi.org/10.1016/j.jnca.2020.102865

Gueant, V. iPerf-The Ultimate Speed Test Tool for TCP, UDP and SCTPTEST the Limits of Your Network + Internet Neutrality Test. Available online: https://iperf.fr/ (accessed on 10 December 2021).

Asadollahi, S.; Goswami, B.; Sameer, M. Ryu controller's scalability experiment on software defined networks. In Proceedings of the 2018 IEEE international conference on current trends in advanced computing (ICCTAC), Bangalore, India, 1-2 February 2018; pp. 1-5.
https://doi.org/10.1109/ICCTAC.2018.8370397

Vega, M.T.; Perra, C.; Liotta, A. Resilience of video streaming services to network impairments. IEEE Trans. Broadcast. 2018, 64, 220-234. [Google Scholar] [CrossRef]
https://doi.org/10.1109/TBC.2017.2781125

Kim, H.J.; Yun, D.G.; Kim, H.S.; Cho, K.S.; Choi, S.G. QoE assessment model for video streaming service using QoS parameters in wired-wireless network. In Proceedings of the 2012 14th International Conference on Advanced Communication Technology (ICACT), Pyeongchang, Korea, 19-22 February 2012; pp. 459-464.

Chen, Y.; Wu, K.; Zhang, Q. From QoS to QoE: A tutorial on video quality assessment. IEEE Commun. Surv. Tutor. 2014, 17, 1126-1165.
https://doi.org/10.1109/COMST.2014.2363139

Oginni, O.; Bull, P.; Wang, Y. Constraint-aware software-defined network for routing real-time multimedia. ACM SIGBED Rev. 2018, 15, 37-42.
https://doi.org/10.1145/3267419.3267425

Benmir, A.; Korichi, A.; Bourouis, A.; Alreshoodi, M.; Al-Jobouri, L. GeoQoE-Vanet: QoE-aware geographic routing protocol for video streaming over vehicular ad-hoc networks. Computers 2020, 9, 45.
https://doi.org/10.3390/computers9020045

Mammeri, Z. Reinforcement learning based routing in networks: Review and classification of approaches. IEEE Access 2019, 7, 55916-55950.
https://doi.org/10.1109/ACCESS.2019.2913776

Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [

Juluri, P.; Tamarapalli, V.; Medhi, D. Measurement of quality of experience of video-on-demand services: A survey. IEEE Commun. Surv. Tutor. 2015, 18, 401-418.
https://doi.org/10.1109/COMST.2015.2401424

Al Shalabi, L.; Shaaban, Z. Normalization as a preprocessing engine for data mining and the approach of preference matrix. In Proceedings of the 2006 International Conference on Dependability of Computer Systems, Szklarska Poręba, Poland, 25-27 May 2006; pp. 207-214.
https://doi.org/10.1109/DEPCOS-RELCOMEX.2006.38

de Oliveira, R.L.S.; Schweitzer, C.M.; Shinoda, A.A.; Prete, L.R. Using Mininet for emulation and prototyping Software-Defined Networks. In Proceedings of the 2014 IEEE Colombian Conference on Communications and Computing (COLCOM), Bogota, Colombia, 4-6 June 2014; pp. 1-6.
https://doi.org/10.1109/ColComCon.2014.6860404

Henni, D.E.; Ghomari, A.; Hadjadj-Aoul, Y. A consistent QoS routing strategy for video streaming services in SDN networks. Int. J. Commun. Syst. 2020, 33, e4177.
https://doi.org/10.1002/dac.4177

Kirstein, P.T. European International Academic Networking: A 20 Year Perspective. In Proceedings of the TERENA Networking Conference, Rhodes, Greece, 7-10 June 2004.

Liu, Y. Current situation and prospect of CERNET. In China's e-Science Blue Book 2020; Springer: Berlin/Heidelberg, Germany, 2021; pp. 327-334.
https://doi.org/10.1007/978-981-15-8342-1_19

Lahoulou, A.; Larabi, M.C.; Beghdadi, A.; Viennet, E.; Bouridane, A. Knowledge-based taxonomic scheme for full-reference objective image quality measurement models. J. Imaging Sci. Technol. 2016, 60, 60406-1.
https://doi.org/10.2352/J.ImagingSci.Technol.2016.60.6.060406

Li, Z.; Bampis, C.; Novak, J.; Aaron, A.; Swanson, K.; Moorthy, A.; Cock, J. Netflix Technology Blog-VMAF: The Journey Continues. 2018. Available online: http://mcl.usc.edu/wp-content/uploads/2018/10/2018-10-25-Netflix-Wor... (accessed on 25 March 2022).

Sara, U.; Akter, M.; Uddin, M.S. Image quality assessment through FSIM, SSIM, MSE and PSNR-A comparative study. J. Comput. Commun. 2019, 7, 8-18.
https://doi.org/10.4236/jcc.2019.73002

Big Buck Bunny. Available online: https://peach.blender.org/ (accessed on 18 January 2022).



No comments:

Post a Comment

Trustworthy Insights: A Novel Multi-Tier Explainable framework for ambient assisted living

  Trustworthy Insights: A Novel Multi-Tier Explainable framework for ambient assisted living Kasirajan, M., Azhar, H. and Turner, S. 2023.  ...