Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
MAIVS: Machine Learning Based Adaptive UHD 360-Degree Immersive Video Streaming
RISE Research Institutes of Sweden, Digital Systems, Data Science. Indian Institute of Technology, India.
Indian Institute of Technology, India.
Indian Institute of Technology, India.
Indian Institute of Technology, India.
2023 (English)Conference paper, Published paper (Refereed)
Abstract [en]

The 360-degree video transmission offers an immersive experience to viewers and is an integral part of several applications such as Metaverse. Ultra-High Definition (UHD) or greater resolutions for such content requires a substantially higher bitrate for transmission even when encoded using the latest codecs. In this work, we propose a machine learning based adaptive UHD 360° immersive video streaming solution, MAIVS, that reduces the data rate requirement to stream the high resolution 360-degree immersive videos. We divide the videos spatially into motion constrained tiles (MCTS), encode (using HEVC), and package them into mp4 containers at different quality levels. We train a Deep Neural Network (DNN) model for each segment of the video to upscale (at client) it to a higher resolution. We use the DASH (dynamic adaptive streaming over HTTP) framework for streaming the video tiles and the model parameters in a progressive manner. The tiles directly in the viewers Field of View (FoV) are streamed at the highest possible quality while a lower resolution is used for the other tiles. We use video quality parameter (PSNR), buffer conditions, and available network bandwidth, as feedback to train the Deep Q-network (DQN) and selectively pack the bitrate quality segments accordingly. Overall, by using reinforcement learning in our proposed MAIVS framework, we improve the client-side PSNR while reducing the bitrate requirement for streaming high resolution (UHD and higher) 360° videos over the internet. 

Place, publisher, year, edition, pages
IEEE, 2023. p. 1082-1087
Keywords [en]
Deep neural networks; Encoding (symbols); HTTP; Image coding; Image communication systems; Learning systems; Reinforcement learning; Signal encoding; 360-degree video streaming; Adaptive video encoding; Bit rates; High definition; High resolution; Immersive; Motion constrained tile; Ultra-high; Video quality; Video-streaming; Video streaming
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:ri:diva-69292DOI: 10.1109/ICCWorkshops57953.2023.10283691Scopus ID: 2-s2.0-85177888189OAI: oai:DiVA.org:ri-69292DiVA, id: diva2:1826167
Conference
2023 IEEE International Conference on Communications Workshops, ICC Workshops 2023. Rome, Italy. 28 May 2023 through 1 June 2023
Available from: 2024-01-11 Created: 2024-01-11 Last updated: 2025-09-23Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus
By organisation
Data Science
Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 38 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf