Subjective evaluation of visual quality and simulator sickness of short 360 videos: ITU-T Rec. P.919 Show others and affiliations
2022 (English) In: IEEE transactions on multimedia, ISSN 1520-9210, E-ISSN 1941-0077, Vol. 24, p. 3087-3100Article in journal (Refereed) Published
Abstract [en]
Recently an impressive development in immersive technologies, such as Augmented Reality (AR), Virtual Reality (VR) and 360 video, has been witnessed. However, methods for quality assessment have not been keeping up. This paper studies quality assessment of 360 video from the cross-lab tests (involving ten laboratories and more than 300 participants) carried out by the Immersive Media Group (IMG) of the Video Quality Experts Group (VQEG). These tests were addressed to assess and validate subjective evaluation methodologies for 360 video. Audiovisual quality, simulator sickness symptoms, and exploration behavior were evaluated with short (from 10 seconds to 30 seconds) 360 sequences. The following factors' influences were also analyzed: assessment methodology, sequence duration, Head-Mounted Display (HMD) device, uniform and non-uniform coding degradations, and simulator sickness assessment methods. The obtained results have demonstrated the validity of Absolute Category Rating (ACR) and Degradation Category Rating (DCR) for subjective tests with 360 videos, the possibility of using 10-second videos (with or without audio) when addressing quality evaluation of coding artifacts, as well as any commercial HMD (satisfying minimum requirements). Also, more efficient methods than the long Simulator Sickness Questionnaire (SSQ) have been proposed to evaluate related symptoms with 360 videos. These results have been instrumental for the development of the ITU-T Recommendation P.919. Finally, the annotated dataset from the tests is made publicly available for the research community.
Place, publisher, year, edition, pages 2022. Vol. 24, p. 3087-3100
Keywords [en]
Quality of experience, Quality assessment, Media, Video recording, Resists, ITU, Encoding
National Category
Telecommunications
Identifiers URN: urn:nbn:se:ri:diva-55409 DOI: 10.1109/TMM.2021.3093717 Scopus ID: 2-s2.0-85112629131 OAI: oai:DiVA.org:ri-55409 DiVA, id: diva2:1578747
2021-07-072021-07-072025-09-23 Bibliographically approved