Change search
Refine search result
12 1 - 50 of 70
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Allison, R.S.
    et al.
    York University, Canada.
    Brunnström, Kjell
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Chandler, D.M.
    Shizuoka University, Japan.
    Colett, H.
    Intel Corp, USA.
    Corriveau, P.
    Intel Corp, USA.
    Daly, S.
    Dolby Laboratories Inc, USA.
    Goel, J.
    Qualcomm Technologies, USA.
    Long, J.Y.
    Intel Corp, USA.
    Wilcox, L.M.
    York University, Canada.
    Yaacob, Y.
    Shizuoka University, Japan.
    Yang, S. N.
    Pacific University, USA.
    Zhang, Y.
    Xi’an Jiaotong University, China.
    Perspectives on the definition of visually lossless quality for mobile and large format displays2018In: Journal of Electronic Imaging, ISSN 10179909, Vol. 27, no 5, article id 053035Article in journal (Refereed)
    Abstract [en]

    Advances in imaging and display engineering have given rise to new and improved image and videoapplications that aim to maximize visual quality under given resource constraints (e.g., power, bandwidth).Because the human visual system is an imperfect sensor, the images/videos can be represented in a mathematicallylossy fashion but with enough fidelity that the losses are visually imperceptible—commonly termed“visually lossless.” Although a great deal of research has focused on gaining a better understanding ofthe limits of human vision when viewing natural images/video, a universally or even largely accepted definitionof visually lossless remains elusive. Differences in testing methodologies, research objectives, and targetapplications have led to multiple ad-hoc definitions that are often difficult to compare to or otherwise employ inother settings. We present a compendium of technical experiments relating to both vision science and visualquality testing that together explore the research and business perspectives of visually lossless image quality,as well as review recent scientific advances. Together, the studies presented in this paper suggest that a singledefinition of visually lossless quality might not be appropriate; rather, a better goal would be to establish varyinglevels of visually lossless quality that can be quantified in terms of the testing paradigm.

    Download full text (pdf)
    fulltext
  • 2.
    Andrén, Börje
    et al.
    RISE, Swedish ICT, Acreo.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Wang, Kun
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Readability of displays in bright outdoor surroundings SID Symposium Digest of Technical Papers2014In: SID Display Week 2014, 2014, p. 1100-1103Conference paper (Refereed)
    Abstract [en]

    Readability of displays in bright outdoor surroundings is a problem for many types of visual displays. The wide range of lighting conditions that these displays should be able to operate in, makes the design of the test set-up very challenging. Mobile displays of today usually have LCDs with LED backlight or OLEDs, which gives the possibility to control the light levels from very low luminance or even zero luminance to very high luminance levels with good contrast and color rendering. There are many displays that can show high luminance (>1000 cd/m2) and with that are considered or claimed to have daylight or sunlight readability. However, these statements are rarely supported by scientific tests with test subjects. This paper will discuss and analyze the conditions for performing such experiments. Experimental results that the visual acuity is negatively affected by strong illuminance are presented.

  • 3.
    Andrén, Börje
    et al.
    RISE, Swedish ICT, Acreo, Broadband Technology.
    Wang, K
    RISE, Swedish ICT, Acreo, Broadband Technology.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo, Broadband Technology.
    Rydell, N
    Fuller, S
    A comparison of visual ergonomic measurements between active and passive 3DTV2011In: EURODISPLAY 2011, 2011Conference paper (Refereed)
  • 4.
    Andrén, Börje
    et al.
    RISE, Swedish ICT, Acreo.
    Wang, Kun
    RISE, Swedish ICT, Acreo.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Characterizations of 3D TV: Active vs passive2012Conference paper (Refereed)
  • 5.
    Arndt, Sebastian
    et al.
    Technical University of Berlin, Germany.
    Brunnström, Kjell
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Cheng, Eva
    RMIT University, Australia.
    Engelke, Ulrich
    CSIRO Commonwealth Scientific and Industrial Research Organisation, Australia.
    Moller, Sebastian
    Technical University of Berlin, Germany.
    Antons, Jan-Niklas
    Technical University of Berlin, Germany.
    Review on using physiology in quality of experience2016In: Electronic Imaging: Human Vision and Electronic Imaging 2016, 2016, p. 197-205Conference paper (Refereed)
    Abstract [en]

    In the area of Quality of Experience (QoE), one challenge is to design test methodologies in order to evaluate the perceived quality of multimedia content delivered through technical sys-tems. Traditionally, this evaluation is done using subjective opinion tests. However, sometimes it is difficult for observers to communicate the experienced quality through the given scale. Fur-thermore. those tests do not give insights into how the user is reacting on an internal physiological level. To overcome these issues, one approach is to use physiological measures, in order to derive a direct non-verbal response of the recipient. In this paper, we review studies that have been performed in the domain of QoE using physiological measures and we look into current activities in standardization bodies. We present challenges this research faces. and give an overview on what researchers should be aware of when they want to start working in this research area.

  • 6.
    Barkowsky, Marcus
    et al.
    Universite de Nantes, France.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Ebrahimi, Touradj
    EPFL, Switzerland.
    Karam, Lina J.
    Arizona State University, USA.
    Lebreton, Pierre
    TU Berlin, Germany.
    Le Callet, Patrick
    Universite de Nantes, France.
    Perkis, Andrews
    NTNU Norwegian University of Science and Technology, Norway.
    Raake, Alexander
    Intel Corporation, USA; Arizona State University, USA.
    Subedar, Mahesh
    Intel Corporation, USA; Arizona State University, USA.
    Wang, Kun
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Xing, Liyuan
    NTNU Norwegian University of Science and Technology, Norway.
    You, Junyong
    NTNU Norwegian University of Science and Technology, Norway.
    Subjective and objective visual quality assessment in the context of stereoscopic 3D-TV2013In: 3D-TV System with Depth-Image-Based Rendering: Architectures, Techniques and Challenges, Springer New York , 2013, Vol. 9781441999641, p. 413-437Chapter in book (Other academic)
    Abstract [en]

    Subjective and objective visual quality assessment in the context of stereoscopic three-dimensional TV (3D-TV) is still in the nascent stage and needs to consider the effect of the added depth dimension. As a matter of fact, quality assessment of 3D-TV cannot be considered as a trivial extension of two-dimensional (2D) cases. Furthermore, it may also introduce negative effects not experienced in 2D, e.g., discomfort or nausea. Based on efforts initiated within the cost action ICT 1003 QUALINET, this chapter discusses current challenges in relation to subjective and objective visual quality assessment for stereo-based 3D-TV. Two case studies are presented to illustrate the current state of the art and some of the remaining challenges.

  • 7.
    Barkowsky, Marcus
    et al.
    Universite de Nantes, France.
    Li, Jing
    Universite de Nantes, France.
    Han, Taehwan
    Yonsei University, South Korea.
    Youn, Sungwook
    Yonsei University, South Korea.
    Ok, Jiheon
    Yonsei University, South Korea.
    Lee, Chulhee
    Mid Sweden University, Sweden.
    Hedberg, Christer
    RISE, Swedish ICT, Acreo.
    Ananth, Indirajith Vijai
    RISE, Swedish ICT, Acreo.
    Wang, Kun
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Le Callet, Patrick
    Universite de Nantes, France.
    Towards standardized 3DTV QoE assessment: Cross-lab study on display technology and viewing environment parameters2013In: Proceedings of SPIE - The International Society for Optical Engineering, 2013, Vol. 8648, article id 864809Conference paper (Refereed)
    Abstract [en]

    Subjective assessment of Quality of Experience in stereoscopic 3D requires new guidelines for the environmental setup as existing standards such as ITU-R BT.500 may no longer be appropriate. A first step is to perform cross-lab experiments in different viewing conditions on the same video sequences. Three international labs performed Absolute Category Rating studies on a freely available video database containing degradations that are mainly related to video quality degradations. Different conditions have been used in the labs: Passive polarized displays, active shutter displays, differences in viewing distance, the number of parallel viewers, and the voting device. Implicit variations were introduced due to the three different languages in Sweden, South Korea, and France. Although the obtained Mean Opinion Scores are comparable, slight differences occur in function of the video degradations and the viewing distance. An analysis on the statistical differences obtained between the MOS of the video sequences revealed that obtaining an equivalent number of differences may require more observers in some viewing conditions. It was also seen that the alignment of the meaning of the attributes used in Absolute Category Rating in different languages may be beneficial. Statistical analysis was performed showing influence of the viewing distance on votes and MOS results.

  • 8.
    Barkowsky, Marcus
    et al.
    University of Nantes, France.
    Masala, Enrico
    Polytechnic University of Turin, Italy.
    Van Wallendael, Glenn
    Ghent University, Belgium.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Staelens, Nicolas
    Ghent University, Belgium.
    Le Callet, Patrick
    University of Nantes, France.
    Objective Video Quality Assessment - Towards large scale video database enhanced model development2015In: IEICE transactions on communications, ISSN 0916-8516, E-ISSN 1745-1345, Vol. E98.B, no 1, p. 2-11Article in journal (Refereed)
    Abstract [en]

    The current development of video quality assessment algorithms suffers from the lack of available video sequences for training, verification and validation to determine and enhance the algorithm’s application scope. The Joint Effort Group of the Video Quality Experts Group (VQEG-JEG) is currently driving efforts towards the creation of large scale, reproducible, and easy to use databases. These databases will contain bitstreams of recent video encoders (H.264, H.265), packet loss impairment patterns and impaired bitstreams, pre-parsed bitstream information into files in XML syntax, and well-known objective video quality measurement outputs. The database is continuously updated and enlarged using reproducible processing chains. Currently, more than 70,000 sequences are available for statistical analysis of video quality measurement algorithms. New research questions are posed as the database is designed to verify and validate models on a very large scale, testing and validating various scopes of applications, while subjective assessment has to be limited to a comparably small subset of the database. Special focus is given on the principles guiding the database development, and some results are given to illustrate the practical usefulness of such a database with respect to the detailed new research questions.

    Download full text (pdf)
    fulltext
  • 9.
    Barkowsky, Marcus
    et al.
    Université de Nantes, France.
    Sedano, Inigo
    TECHNALIA, Spain.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Leszczuk, Mikolaj Igor
    AGH University of Science and Technology, Poland.
    Staelens, Nicolas
    Ghent University, Belgium.
    Hybrid video quality prediction: Re-viewing video quality meausurement for widening application scope2014In: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721, Vol. 74, no 2, p. 323-Article in journal (Refereed)
    Abstract [en]

    A tremendous number of objective video quality measurement algorithms have been developed during the last two decades. Most of them either measure a very limited aspect of the perceived video quality or they measure broad ranges of quality with limited prediction accuracy. This paper lists several perceptual artifacts that may be computationally measured in an isolated algorithm and some of the modeling approaches that have been proposed to predict the resulting quality from those algorithms. These algorithms usually have a very limited application scope but have been verified carefully. The paper continues with a review of some standardized and well-known video quality measurement algorithms that are meant for a wide range of applications, thus have a larger scope. Their individual artifacts prediction accuracy is usually lower but some of them were validated to perform sufficiently well for standardization. Several difficulties and shortcomings in developing a general purpose model with high prediction performance are identified such as a common objective quality scale or the behavior of individual indicators when confronted with stimuli that are out of their prediction scope. The paper concludes with a systematic framework approach to tackle the development of a hybrid video quality measurement in a joint research collaboration.

    Download full text (pdf)
    fulltext
  • 10.
    Bosse, Sebastian
    et al.
    Fraunhofer, Germany.
    Brunnström, Kjell
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo. Mid Sweden University, Sweden.
    Arndt, Sebastian
    NTNU Norwegian University of Science and Technology, Norway.
    Martini, Maria G.
    Kingston University, UK.
    Ramzan, Naeem
    University of the West of Scotland, UK.
    Engelke, Ulrich
    CSIRO, Australia.
    A common framework for the evaluation of psychophysiological visualquality assessment2019In: Quality and User Experience, ISSN 2366-0139, E-ISSN 2366-0147, Vol. 4, no 3Article in journal (Refereed)
    Abstract [en]

    The assessment of perceived quality based on psychophysiological methods recently gained attraction as it potentiallyovercomes certain flaws of psychophysical approaches. Although studies report promising results, it is not possible toarrive at decisive and comparable conclusions that recommend the use of one or another method for a specific applicationor research question. The video quality expert group started a project on psychophysiological quality assessment to studythese novel approaches and to develop a test plan that enables more systematic research. This test plan comprises of a specificallydesigned set of quality annotated video sequences, suggestions for psychophysiological methods to be studied inquality assessment, and recommendations for the documentation and publications of test results. The test plan is presentedin this article.

    Download full text (pdf)
    fulltext
  • 11.
    Brotherton, Matthew
    et al.
    British Telecom, UK.
    Huynh-Thu, Quan
    Psytechnics Ltd, UK; University of Essex, UK.
    Hands, David
    British Telecom, UK.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Subjective Multimedia Quality Assessment2006In: IEICE Transactions on Fundamentals of Electronics Communications Computer Sciences, Vol. E89, no a11, p. 2920-2932Article in journal (Refereed)
    Abstract [en]

    The Video Quality Experts Group (VQEG) is preparing a programme of subjective multimedia quality tests. The results from these tests will be used to evaluate the performance of competing objective multimedia quality metrics. The reliability of the subjective test data is of great importance for VQEG's task. This paper provides an overview of VQEG's multimedia ad-hoc group. The work of this group will require subjective tests to be performed by laboratories located in Europe, Asia and North America. For VQEG's multimedia work to be successful, the subjective assessment methodology must be precisely defined and produce reliable and repeatable subjective quality data. Although international standards covering multimedia quality assessment methods are in force, there remains some uncertainty regarding the most effective approach to assessing the subjective quality of multimedia. A review of existing methods is provided. Two experiments are presented investigating the suitability of alternative subjective assessment methods (single-stimulus ACR and SAMVIQ). The results of these experiments are discussed within the context of the VQEG multimedia testing programme.

  • 12.
    Brunnström, Kjell
    et al.
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Allison, R. S.
    York University, Canada.
    Chandler, D. M.
    Shizuoka University, Japan.
    Colett, H.
    Intel Corp, USA.
    Corriveau, P.
    Intel Corp, USA.
    Daly, S.
    Dolby Laboratories, USA.
    Goel, J.
    Qualcomm, Canada.
    Knopf, J.
    Intel Corp, USA.
    Wilcox, L. M.
    York University, Canada.
    Yaacob, Y.
    Shizuoka University, Japan.
    Yang, S. -N
    Pacific University, USA.
    Zhang, Y.
    Shizuoka University, Japan.
    Industry and business perspectives on the distinctions between visually lossless and lossy video quality: Mobile and large format displays2017In: IS and T International Symposium on Electronic Imaging Science and Technology, 2017, p. 118-133Conference paper (Refereed)
    Abstract [en]

    This paper will explore the mobile and business perspectives of visually lossless image quality, as well as review recent scientific advances. It is the outcome from the Special Session on Visually Lossless Video Quality for Modern Devices: Research and Industry Perspectives organized at the Human Vision and Electronic Imaging 2017 by IS&T at San Francisco Airport, Burlingame, California, USA, Jan 29 - Feb 2, 2017. It summarizes four presentations and a panel discussion.

  • 13.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Ananth, Indirajith Vijai
    RISE, Swedish ICT, Acreo.
    Hedberg, Christer
    RISE, Swedish ICT, Acreo.
    Wang, Kun
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Andrén, Börje
    RISE, Swedish ICT, Acreo.
    Barkowsky, Marcus
    University of Nantes, France.
    Comparison between different rating scales for 3D TV2013In: Digest of Technical Papers - SID International Symposium, ISSN 0097-966X, Vol. 44, no 1, p. 509-512Article in journal (Refereed)
    Abstract [en]

    Stereoscopic 3D viewing experience has been studied quite intensively recently, but still the subjective test methods have not yet been settled. It has become clear that the 3D viewing experience cannot easily be described by just one scale. This paper describes a study where three different rating scales (Quality, Discomfort and Presence) are compared in a subjective test, combined with two viewing distances. The results shows that in a stereoscopic 3D video quality test targeting mainly coding distortions one scale such as video quality could be sufficient.

  • 14.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Schenkman, Bo
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Hamsis, Omar
    RISE Research Institutes of Sweden, Digital Systems, Prototyping Society.
    Recommended precautions because of Covid-19 for perceptual, behavioural,quality and user experience experimentswith test persons in indoor labs2020Report (Other academic)
    Abstract [en]

    Based on the recommendations from the Public Health Agency of Sweden(Folkhälsomyndigheten; FHM) and a set of internal rules from RISE, the followingrules are published for how to conduct experiments involving test persons in the timesof the pandemic Covid-19. The recommendations are for non-invasive and non-medicaltests, e.g. perceptual, consumer, ergonomic and human-computer interaction teststaking place in an indoor laboratory.

    Specifically, in this document we are specifying how experiments with test personstargeting audio and visual presentations should be done considering necessaryprecautions imposed by the Covid-19 pandemic. Laboratory experiments with testpersons, as it involves inviting people to the lab, require particular planning and carefulconsideration, if they are to be carried out safely because of the risks imposed by theCovid-19 pandemic. The safety aspects are valid for both the invited test persons andare equally important for the health of the test leaders.

    Download full text (pdf)
    fulltext
  • 15.
    Brunnström, Kjell
    et al.
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Barkowsky, Marcus
    Balancing type I errors and statistical power in video quality assessment2017In: IS and T International Symposium on Electronic Imaging Science and Technology, 2017, p. 91-96Conference paper (Refereed)
    Abstract [en]

    This paper analyzes how an experimenter can balance errors in subjective video quality tests between the statistical power of finding an effect if it is there and not claiming that an effect is there if the effect it is not there i.e. balancing Type I and Type II errors. The risk of committing Type I errors increases with the number of comparisons that are performed in statistical tests. We will show that when controlling for this and at the same time keeping the power of the experiment at a reasonably high level, it will require more test subjects than are normally used and recommended by international standardization bodies like the ITU. Examples will also be given for the influence of Type I error on the statistical significance of comparing objective metrics by correlation.

  • 16.
    Brunnström, Kjell
    et al.
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo. Mid Sweden University, Sweden.
    Barkowsky, Marcus
    University of Applied Sciences, Germany.
    Statistical quality of experience analysis: on planning the sample size and statistical significance testing2018In: Journal of Electronic Imaging (JEI), ISSN 1017-9909, E-ISSN 1560-229X, Vol. 27, no 5, article id 053013Article in journal (Refereed)
    Abstract [en]

    This paper analyzes how an experimenter can balance errors in subjective video quality tests betweenthe statistical power of finding an effect if it is there and not claiming that an effect is there if the effect is not there,i.e., balancing Type I and Type II errors. The risk of committing Type I errors increases with the number ofcomparisons that are performed in statistical tests. We will show that when controlling for this and at thesame time keeping the power of the experiment at a reasonably high level, it is unlikely that the number oftest subjects that are normally used and recommended by the International Telecommunication Union (ITU),i.e., 15 is sufficient but the number used by the Video Quality Experts Group (VQEG), i.e., 24 is more likelyto be sufficient. Examples will also be given for the influence of Type I error on the statistical significance ofcomparing objective metrics by correlation. We also present a comparison between parametric and nonparametricstatistics. The comparison targets the question whether we would reach different conclusions on the statisticaldifference between the video quality ratings of different video clips in a subjective test, based on thecomparison between the student T-test and the Mann–Whitney U-test. We found that there was hardly a differencewhen few comparisons are compensated for, i.e., then almost the same conclusions are reached. Whenthe number of comparisons is increased, then larger and larger differences between the two methods arerevealed. In these cases, the parametric T-test gives clearly more significant cases, than the nonparametrictest, which makes it more important to investigate whether the assumptions are met for performing a certaintest.

    Download full text (pdf)
    fulltext
  • 17.
    Brunnström, Kjell
    et al.
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo. Mid Sweden University, Sweden.
    Dima, Elijs
    Mid Sweden University, Sweden.
    Andersson, Mattias
    Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Qureshi, Tahir
    HIAB, Sweden.
    Johanson, Mathias
    Alkit Communications AB, Sweden.
    Quality of Experience of hand controller latency in a Virtual Reality simulator2019In: Human Vision and Electronic Imaging 2019 / [ed] Damon Chandler, Mark McCourt and Jeffrey Mulligan, 2019, article id 3068450Conference paper (Refereed)
    Abstract [en]

    In this study, we investigate a VR simulator of a forestry crane used for loading logs onto a truck, mainly looking at Quality of Experience (QoE) aspects that may be relevant for task completion, but also whether there are any discomfort related symptoms experienced during task execution. A QoE test has been designed to capture both the general subjective experience of using the simulator and to study task performance. Moreover, a specific focus has been to study the effects of latency on the subjective experience, with regards to delays in the crane control interface. A formal subjective study has been performed where we have added controlled delays to the hand controller (joystick) signals. The added delays ranged from 0 ms to 800 ms. We found no significant effects of delays on the task performance on any scales up to 200 ms. A significant negative effect was found for 800 ms added delay. The Symptoms reported in the Simulator Sickness Questionnaire (SSQ) was significantly higher for all the symptom groups, but a majority of the participants reported only slight symptoms. Two out of thirty test persons stopped the test before finishing due to their symptoms.

    Download full text (pdf)
    fulltext
  • 18.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Dima, Elijs
    Mid Sweden University, Sweden.
    Qureshi, Tahir
    HIAB AB, Sweden.
    Johanson, Mathias
    Alkit Communications AB, Sweden.
    Andersson, Mattias
    Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Latency impact on Quality of Experience in a virtual reality simulator for remote control of machines2020In: Signal processing. Image communication, ISSN 0923-5965, E-ISSN 1879-2677, Vol. 89, article id 116005Article in journal (Refereed)
    Abstract [en]

    In this article, we have investigated a VR simulator of a forestry crane used for loading logs onto a truck. We have mainly studied the Quality of Experience (QoE) aspects that may be relevant for task completion, and whether there are any discomfort related symptoms experienced during the task execution. QoE experiments were designed to capture the general subjective experience of using the simulator, and to study task performance. The focus was to study the effects of latency on the subjective experience, with regards to delays in the crane control interface. Subjective studies were performed with controlled delays added to the display update and hand controller (joystick) signals. The added delays ranged from 0 to 30 ms for the display update, and from 0 to 800 ms for the hand controller. We found a strong effect on latency in the display update and a significant negative effect for 800 ms added delay on latency in the hand controller (in total approx. 880 ms latency including the system delay). The Simulator Sickness Questionnaire (SSQ) gave significantly higher scores after the experiment compared to before the experiment, but a majority of the participants reported experiencing only minor symptoms. Some test subjects ceased the test before finishing due to their symptoms, particularly due to the added latency in the display update.

    Download full text (pdf)
    fulltext
  • 19.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Objective video quality assessment methods for Video assistant refereeing (VAR) Systems2021Report (Other academic)
    Abstract [en]

    This report describes the work and conclusions drawn after phase 4 in the project “Assessment methods for Video assistant refereeing (VAR) System”.

    The performance of six different video quality models have been evaluated, that were identified during phase 1, against the subjective video quality database that was created during phase 3. The results are slightly different for 1080p compared to 1080i. For 1080p the models VQM_VFD, SSIM and VMAF performs the best with Pearson Correlation Coefficients (PCC) above 0.9. For 1080i the PCC drops a bit overall and then VMAF and VQM_VFD are close in performance and performing the best. The overall performance for both formats VMAF an VQM_VFD stands out as the best models. In this comparison VQM_VFD has the added advantage to also be able to perform its own registration i.e. to fix any misalignment between the reference video and the distorted one.

    Download full text (pdf)
    fulltext
  • 20.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Objective video quality assessment methods for Video assistant refereeing (VAR) System: Phase 4 report on synchronizationand latency measurements2021Report (Other academic)
    Abstract [en]

    This report describes the work and conclusions drawn after phase 4 for the synchronization and latencymeasurements in the project “Assessment methods for Video Assistant Refereeing (VAR) System”.In phase 4, we have focused on up-dating the software so that it automatically detects: p (progressive),i (interlaced), and PsF (progressive segmented frame) formats and analyse the different video types. Wehave also the performed a field-test with an OB-van at an arena.

    Download full text (pdf)
    fulltext
  • 21.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Video quality based on a user study with video professionals for Video Assisted Refereeing (VAR) Systems2021Report (Other academic)
    Abstract [en]

    This document describes a user experiment with the purpose of finding a baseline quality that is suitable for system and a database for training and evaluating the objective quality measurement methods suitable for assessing the video quality of VAR systems. 

    A user experiment was performed involving 25 Swedish video experts. Three different video formats were incorporated 1080p, 1080i and 540i. The degradations were in most cased done using encoding with Motion JPEG (MJPEG) and H.264 in the bitrate range from 80 Mbit/s down to 10 Mbit/s. 

    MJPEG loses quality very fast and already at 80 Mbit/s it has significantly lower quality than the uncompressed reference and then for even lower bitrates the quality falls quickly to bad. On the other hand, H.264 was not found to be significant different from the uncompressed reference until the bitrate had dropped to 10 Mbit/s for 1080p. For 1080i 20 Mbit/s was also weakly and for 540i 20 Mbit/s was significantly lower for some of the scaling methods. For 1080i the deinterlacing requires careful consideration, since the deinterlacing scheme introduced received very low quality scores. For the scaling scheme lanczos was the best and bilinear the worst. 

    Requirement levels on bitrate and the encoders MJPEG and H.264 based on this experiment•    MJPEG require more than 120 Mbit/s•    H.264 require more than 50 Mbit/s

    Download full text (pdf)
    fulltext
  • 22.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Billingham, Johsan
    FIFA Fédération Internationale de Football Association, Switzerland.
    Wistel, Katharina
    FIFA Fédération Internationale de Football Association, Switzerland.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Ozolins, Oskars
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Evans, Nicolas
    FIFA Fédération Internationale de Football Association, Switzerland.
    Video expert assessment of high quality video for Video Assistant Referee (VAR): A comparative study2023In: Multimedia tools and applications, ISSN 1380-7501, E-ISSN 1573-7721Article in journal (Refereed)
    Abstract [en]

    The International Football Association Board decided to introduce Video Assistant Referee (VAR) in 2018. This led to the need to develop methods for quality control of the VAR-systems. This article focuses on the important aspect to evaluate the video quality. Video Quality assessment has matured in the sense that there are standardized, commercial products and established open-source solutions to measure it with objective methods. Previous research has primarily focused on the end-user quality assessment. How to assess the video in the contribution phase of the chain is less studied. The novelties of this study are two-fold: 1) The user study is specifically targeting video experts i.e., to assess the perceived quality of video professionals working with video production. 2) Six video quality models have been independently benchmarked against the user data and evaluated to show which of the models could provide the best predictions of perceived quality. The independent evaluation is important to get unbiased results as shown by the Video Quality Experts Group. An experiment was performed involving 25 video experts in which they rated the perceived quality. The video formats tested were High-Definition TV both progressive and interlaced as well as a quarters size format that was scaled down half the size in both width and height. The videos were encoded with both H.264 and Motion JPEG for the full size but only H.264 for the quarter size. Bitrates ranged from 80 Mbit/s down to 10 Mbit/s. We could see that for H.264 that the quality was overall very good but dropped somewhat for 10 Mbit/s. For Motion JPEG the quality dropped over the whole range. For the interlaced format the degradation that was based on a simple deinterlacing method did receive overall low ratings. For the quarter size three different scaling algorithms were evaluated. Lanczos performed the best and Bilinear the worst. The performance of six different video quality models were evaluated for 1080p and 1080i. The Video Quality Metric for Variable Frame Delay had the best performance for both formats, followed by Video Multimethod Assessment Fusion method and the Video Quality Metric General model. 

    Download full text (pdf)
    fulltext
  • 23.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Billingham, Johsan
    FIFA Fédération Internationale de Football Association, Switzerland.
    Wistel, Katharina
    FIFA Fédération Internationale de Football Association, Switzerland.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Ozolins, Oskars
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. KTH Royal Institute of Technology, Sweden.
    Evans, Nicolas
    FIFA Fédération Internationale de Football Association, Switzerland.
    Video quality of video professionals for Video Assisted Referee (VAR)2023In: IS and T International Symposium on Electronic Imaging Science and Technology, ISSN 2470-1173, Vol. 35, no 10, article id 259Article in journal (Refereed)
    Abstract [en]

    Changes in the footballing world’s approach to technology and innovation contributed to the decision by the International Football Association Board to introduce Video Assistant Referees (VAR). The change meant that under strict protocols referees could use video replays to review decisions in the event of a "clear and obvious error" or a "serious missed incident". This led to the need by Fédération Internationale de Football Association (FIFA) to develop methods for quality control of the VAR-systems, which was done in collaboration with RISE Research Institutes of Sweden AB. One of the important aspects is the video quality. The novelty of this study is that it has performed a user study specifically targeting video experts i.e., to measure the perceived quality of video professionals working with video production as their main occupation. An experiment was performed involving 25 video experts. In addition, six video quality models have been benchmarked against the user data and evaluated to show which of the models could provide the best predictions of perceived quality for this application. Video Quality Metric for variable frame delay (VQM_VFD) had the best performance for both formats, followed by Video Multimethod Assessment Fusion (VMAF) and VQM General model.

    Download full text (pdf)
    fulltext
  • 24.
    Brunnström, Kjell
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Ozolins, Oskars
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. KTH Royal Institute of Technology, Sweden.
    Billingham, Johsan
    FIFA Fédération Internationale de Football Association, Switzerland.
    Wistel, Katarina
    FIFA Fédération Internationale de Football Association, Switzerland.
    Evans, Nicolas
    FIFA Fédération Internationale de Football Association, Switzerland.
    Quality measurement methods for video assisting refereeing systems2023In: Sports Engineering, ISSN 1369-7072, E-ISSN 1460-2687, Vol. 26, no 1, article id 17Article in journal (Refereed)
    Abstract [en]

    Changes in the footballing world’s approach to technology and innovation, along with major advancements in broadcasting contributed to the decision by the International Football Association Board to introduce Video Assistant Referees in 2018. The change meant that under strict protocols referees could use video replays to review decisions in the event of a “clear and obvious error” or a “serious missed incident”. At the time of writing 48-Member Associations have introduced the Video Assistant Referees protocol in at least one of their tournaments and there are many technology providers who work with organisers to implement the Video Assistant Referees systems. To ensure that the use of Video Assistant Referees has a positive effect on the game, Fédération Internationale de Football Association collaborated with the RISE Research Institutes of Sweden to develop objective test methods that could be used to ensure that a system can provide an adequate solution. Each provider must be able to pass requirements that ensure that they can deal with the challenges of processing, coding, decoding, synchronising, and re-formatting of the broadcast feeds. This article will describe the development of the test methods and illustrate some initial results from a test event on Video Assistant Referees system candidates. The methods have shown to be robust and appropriate for their intended purpose and will be developed over the years to ensure the quality of Video Assistant Referees. The developed measurement methods are general and can be applied to other broadcast and video systems as well as to other sports. © 2023, The Author(s).

    Download full text (pdf)
    fulltext
  • 25.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo.
    Hands, D
    Speranza, F
    Webster, A
    VQEG Validation ITU Standardisation of Objective Perceptual Video Quality Metrics2009In: IEEE Signal Processing Magazine, Vol. 26, no 3, p. 96-Article in journal (Refereed)
    Download full text (pdf)
    fulltext
  • 26.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo.
    Schenkman, B
    Comparison of the predictions of a spatiotemporal model with the detection of distortion in small moving images2002In: Optical Engineering, Vol. 41, p. 711-22Article in journal (Refereed)
    Download full text (pdf)
    fulltext
  • 27.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo.
    Schenkman, B
    Jacobson, B
    Object Detection in Cluttered IR Images2003In: Optical Engineering, Vol. 42, p. 388-99Article in journal (Refereed)
    Download full text (pdf)
    fulltext
  • 28.
    Brunnström, Kjell
    et al.
    Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Muhammad, Imran
    HIAB AB, Sweden;Mid Sweden University, Sweden.
    Magnus, Pettersson
    HIAB AB, Sweden.
    Johanson, Mathias
    Alkit Communications AB, Sweden.
    Quality of Experience for a Virtual Reality simulator2018In: Human Vision and Electronic Imaging 2018 / [ed] Rogowitz, B.;Pappas, T.;De Ridder H., The Society for Imaging Science and Technology, 2018Conference paper (Refereed)
    Abstract [en]

    In this study, we investigate a VR simulator of a forestry crane used for loading logs onto a truck, mainly looking at Quality of Experience (QoE) aspects that may be relevant for task completion, but also whether there are any discomfort related symptoms experienced during task execution. The QoE test has been designed to capture both the general subjective experience of using the simulator and to study task completion rate. Moreover, a specific focus has been to study the effects of latency on the subjective experience, with regards both to delays in the crane control interface as well as lag in the visual scene rendering in the head mounted display (HMD). Two larger formal subjective studies have been performed: one with the VR-system as it is and one where we have added controlled delay to the display update and to the joystick signals. The baseline study shows that most people are more or less happy with the VR-system and that it does not have strong effects on any symptoms as listed in the Simulator Sickness Questionnaire (SSQ). In the delay study we found significant effects on Comfort Quality and Immersion Quality for higher Display delay (30 ms), but very small impact of joystick delay. Furthermore, the Display delay had strong influence on the symptoms in the SSQ, and causing test subjects to decide not to continue with the complete experiments. We found that this was especially connected to the longer added Display delays (≥ 20 ms).

    Download full text (pdf)
    fulltext
  • 29.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo.
    Stålenbring, D
    Pettersson, M
    Gustafsson, J
    The impact of transmission errors on progressive 720 lines HDTV coded with H.2642010In: Proceedings Volume 7527, Human Vision and Electronic Imaging XV; 75271J (2010), 2010, Vol. 7527Conference paper (Refereed)
  • 30.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo.
    Tavakoli, S
    Kulyk, V
    RISE, Swedish ICT, Acreo.
    Wang, Kun
    RISE, Swedish ICT, Acreo.
    Hedberg, Christer
    RISE, Swedish ICT, Acreo.
    Ananth, IV
    RISE, Swedish ICT, Acreo.
    Hussein, M
    RISE, Swedish ICT, Acreo.
    Andrén, Börje
    RISE, Swedish ICT, Acreo.
    Folkesson, M
    Barowski, M
    RISE, Swedish ICT, Acreo.
    3D Video Quality of Experience - Influence of Scale and Crosstalk Invited2014Conference paper (Refereed)
    Abstract [en]

    This paper gives an overview of three recent studies by the authors on the topic of 3D video Quality of Experience (QoE). Two of studies [1,2] investigated different psychological dimension that may be needed for describing 3D video QoE and the third the visibility and annoyance of crosstalk[3]. The results shows that the video quality scale could be sufficient for evaluating S3D video experience for coding and spatial resolution reduction distortions. It was also confirmed that with a more complex mixture of degradations more than one scale should be used to capture the QoE in these cases. The study found a linear relationship between the perceived crosstalk and the amount of crosstalk.

  • 31.
    Brunnström, Kjell
    et al.
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Tavakoli, Samira
    Technical University of Madrid, Spain.
    Søgaard, Jacob
    DTU Technical University of Denmark, Denmark.
    Compensating for Type-I errors in video quality assessment2015In: 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX), 2015, article id 7148080Conference paper (Refereed)
    Abstract [en]

    This paper analyzes the impact on compensating for Type-I errors in video quality assessment. A Type-I error is to incorrectly conclude that there is an effect. The risk increases with the number of comparisons that are performed in statistical tests. Type-I errors are an issue often neglected in Quality of Experience and video quality assessment analysis. Examples are given for the analysis of subjective experiments and the evaluation of objective metrics by correlation.

  • 32.
    Brunnström, Kjell
    et al.
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo. Mid Sweden University, Sweden.
    Wang, Kun
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Tavakoli, Samira
    Universidad Politecnica de Madrid, Spain.
    Andrén, Börje
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Symptoms analysis of 3D TV viewing based on Simulator SicknessQuestionnaires2017In: Quality and User Experience, ISSN 2366-0139, E-ISSN 2366-0147, Vol. 2Article in journal (Refereed)
    Abstract [en]

    Stereoscopic 3D TV viewing puts differentvisual demands on the viewer compared to 2D TV viewing.Previous research has reported on viewers’ fatigue anddiscomfort and other negative effects. This study is toinvestigate further how severe and what symptoms mayarise from somewhat longish 3D TV viewing. The MPEG3DV project is working on the next-generation videoencoding standard and in this process, MPEG issued a callfor proposal of encoding algorithms. To evaluate thesealgorithms a large scale subjective test was performedinvolving Laboratories all over the world [(MPEG 2011;Baroncini 2012)]. For the participating Labs, it wasoptional to administer a slightly modified Simulator SicknessQuestionnaire (SSQ) before and after the test. One ofthe SSQ data sets described in this article is coming fromthis study. The SSQ data from the MPEG test is the largestdata set in this study and also contains the longest viewingtimes. Along with the SSQ data from the MPEG test, wehave also collected questionnaire data in three other 3D TVstudies. We did two on the same 3D TV (passive filmpattern retarder) as in the MPEG test, and one was using aprojector system. As comparison SSQ data from a 2Dvideo quality experiment is also presented. This investigationshows a statistically significant increase in symptomsafter viewing 3D TV primarily related to the visual or Oculomotor system. Surprisingly, 3D video viewing usingprojectors did not show this effect.

    Download full text (pdf)
    fulltext
  • 33.
    Cuadra, Antonio
    et al.
    Indra, Spain.
    Cutanda, Mar
    Indra, Spain.
    Aurelius, Andreas
    RISE, Swedish ICT, Acreo.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Lopez De Vergara, Jorge E.
    Naudit, Spain.
    Varela, Martin
    VTT, Finland.
    Laulajainen, Jukka Pekka
    VTT, Finland.
    Morais, Anderson
    Telecom, France.
    Cavalli, Ana Rosa
    Telecom, France.
    Mellouk, Abdelhamid
    UPEC, France.
    Augustin, Brice
    UPEC, France.
    Perez-Mateos, Ismael
    DYCEC, Spain.
    Ecosystem for customer experience assurance2013In: 2013 International Conference on Smart Communications in Network Technologies, SaCoNeT 2013, 2013, article id 6654553Conference paper (Refereed)
    Abstract [en]

    Nowadays service providers focus all their effort on customers' satisfaction although determining the Quality of Experience (QoE) is not a trivial task. In addition, the evolution from traditional networks towards Next Generation Networks (NGN) is enabling service providers to deploy a wide range of multimedia services such as Internet Television (IPTV), Video on Demand (VoD), and multiplayer games services, all on the same underlying IP network. However, managing the satisfaction level of customers to provide a good user experience is not being an easy task due to the complexity of orchestrating network and customer data sources. This document proposes an ecosystem that allows managing customer experience in order to guarantee the quality levels delivered to end users, which is being implemented into the Celtic IPNQSIS project. The QoE ecosystem lies on a customer experience architecture formed by Data acquisition level, Monitoring level and Control Level. The work proposed in this paper will settle the basis of next generation Customer Experience Management systems.

  • 34.
    Cuadra-Sánchez, Antonio
    et al.
    Indra Sistemas S.A., Spain.
    Cutanda-Rodríguez, Mar
    Indra Sistemas S.A., Spain.
    Pérez-Mateos, Ismael
    DYCEC, Spain.
    Aurelius, Andreas
    RISE, Swedish ICT, Acreo.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Laulajainen, Jukka Pekka
    VTT Technical Research Centre of Finland, Finland.
    Varela, Martín
    VTT Technical Research Centre of Finland, Finland.
    López De Vergara, Jorge E.
    Naudit HPCN, Spain.
    A global customer experience management architecture2012In: 2012 Future Network and Mobile Summit, FutureNetw 2012, 2012, article id 6294185Conference paper (Refereed)
    Abstract [en]

    The quality of experience (QoE) is one of the main research lines in ITC industry, which seeks to manage quality as perceived by users. This document analyzes and describes requirements of a QoE driven management system architecture, which has been designed in the Celtic IPNQSIS project. The architecture is grouped into different levels: Data acquisition level, Monitoring level and Control Level. Each level comprises a specific set of capacities, such as Data collector, or Traffic Monitor amongst others. The architecture described in this paper constitutes the guidelines of the IPNQSIS project in terms of a QoE ecosystem that will settle the basis of global customer experience management architecture.

  • 35.
    Dima, Elijs
    et al.
    Mid Sweden University, Sweden.
    Brunnström, Kjell
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo. Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Andersson, Mattias
    Mid Sweden University, Sweden.
    Edlund, Joakim
    Mid Sweden University, Sweden.
    Johanson, Mathias
    Alkit Communications AB, Sweden.
    Qureshi, Tahir
    HIAB AB, Sweden.
    View Position Impact on QoE in an Immersive Telepresence System for Remote Operation2019In: 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), Berlin, Germany: IEEE , 2019Conference paper (Refereed)
    Abstract [en]

    In this paper, we investigate how different viewingpositions affect a user’s Quality of Experience (QoE) and performancein an immersive telepresence system. A QoE experimenthas been conducted with 27 participants to assess the generalsubjective experience and the performance of remotely operatinga toy excavator. Two view positions have been tested, an overheadand a ground-level view, respectively, which encourage relianceon stereoscopic depth cues to different extents for accurate operation.Results demonstrate a significant difference between groundand overhead views: the ground view increased the perceiveddifficulty of the task, whereas the overhead view increased theperceived accomplishment as well as the objective performanceof the task. The perceived helpfulness of the overhead view wasalso significant according to the participants.

    Download full text (pdf)
    fulltext
  • 36.
    Dima, Elijs
    et al.
    Mid Sweden University, Sweden.
    Brunnström, Kjell
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Andersson, Mattias
    Mid Sweden University, Sweden.
    Edlund, Joakim
    Mid Sweden University, Sweden.
    Johansson, Mathias
    Alkit Communications AB, Sweden.
    Qureshi, Tahir
    HIAB AB, Sweden.
    Joint effects of depth‑aiding augmentations and viewing positionson the quality of experience in augmented telepresence2020In: Quality and User Experience, ISSN 2366-0139, Vol. 5, article id 2Article in journal (Refereed)
    Abstract [en]

    Virtual and augmented reality is increasingly prevalent in industrial applications, such as remote control of industrial machinery,due to recent advances in head-mounted display technologies and low-latency communications via 5G. However, theinfluence of augmentations and camera placement-based viewing positions on operator performance in telepresence systemsremains unknown. In this paper, we investigate the joint effects of depth-aiding augmentations and viewing positionson the quality of experience for operators in augmented telepresence systems. A study was conducted with 27 non-expertparticipants using a real-time augmented telepresence system to perform a remote-controlled navigation and positioningtask, with varied depth-aiding augmentations and viewing positions. The resulting quality of experience was analyzed viaLikert opinion scales, task performance measurements, and simulator sickness evaluation. Results suggest that reducing thereliance on stereoscopic depth perception via camera placement has a significant benefit to operator performance and qualityof experience. Conversely, the depth-aiding augmentations can partly mitigate the negative effects of inferior viewingpositions. However the viewing-position based monoscopic and stereoscopic depth cues tend to dominate over cues basedon augmentations. There is also a discrepancy between the participants’ subjective opinions on augmentation helpfulness,and its observed effects on positioning task performance.

    Download full text (pdf)
    fulltext
  • 37.
    Eksvärd, Siri
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Falk, Julia
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Schenkman, Bo
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Evaluating Speech-to-Text Systems and AR-glasses: A study to develop a potential assistive device for people with hearing impairments2021Report (Other academic)
    Abstract [sv]

    Att ha en hörselskada eller att vara döv har flera konsekvenser på individens livskvalité. Det påverkar vardagen i stor utsträckning och gör det svårt att delta i konversationer. Det finns redan flera hjälpmedel för att underlätta vardagen för individer med hörselskador, exempelvis  hörapparater. Dock finns vissa utmaningar och problem med dessa. En möjlig lösning skulle kunna vara att använda Augmented Reality, eller förstärkt verklighet tillsammans med ett tal-till-text-system, där tal skulle kunna konverteras til text som exempelvis presenteras i AR-glasögon. Augmented Reality (AR) är en teknik som möjliggör att förstärka verkligheten genom att  datorgenerande information, till exempel visuella objekt, presenteras ovanpå verkligheten. En variant av en AR-teknologi är AR-glasögon, vilket innebär att datorgenerade objekt presenteras i glasögonen och sedermera förstärker användarens verklighet. Olika varianter av AR och AR-glasögon har studerats länge, men det inte förrän under den senaste tiden som kvalitén blivit tillräckligt bra för att kunna användas i vardagen. Idag finns flera olika AR-glasögon, med olika tekniska, ergonomiska, visuella och optiska egenskaper, där vissa glasögon kan vara mer eller mindre lämpade inom ett visst användningsområde. Glasögonens egenskaper påverkar även användarens upplevelse. Även om tekniken redan finns på marknaden kvarstår vissa problem, exempelvis belysning, bakgrund och att synligheten av de datorgenerade objekten påverkas av förhållanden i omgivningen. Då målet är att presentera text i AR-glasögonen är det viktiga att texten är synlig, läslig1 och läsbar2 under flera olika förhållanden, som varierande belysning och bakgrund. Dessutom bör hänsyn tas till hur texten presenteras, där faktorer som storlek, textfärg, bakgrundsfärg bakom text samt antal rader bör beaktas. Tal-till-text-system är också en teknik som fått genomslag under de senaste åren. Bland annat tack vare de stora framsteg som gjorts inom taligenkänning. På den engelskspråkiga marknaden är tekniken utbredd och det finns flera befintliga system, som Google Cloud Speech API, IBM Watson och Microsoft Azure. På den svenska marknaden finns dock få system och utvärderingar av dessa. Vid utvärderingar av tal-till-text-system bör man ta hänsyn till faktorer som korrekthet, fördröjning och robusthet under olika ljudnivåer, talhastigheter och dialekter. Således bör man utvärdera vilket svenskt tal-till-text-system som är lämpligast att använda för att översätta tal till text med hänsyn till ovannämnda aspekter. I detta examensarbete undersöks hur egenskaperna i två olika AR-glasögon påverkar användarupplevelsen med fokus på komfort, design, samt glasögonens optiska och visuella egenskaper. Vidare undersöks hur belysning och bakgrund påverkar läsligheten och läsbarheten av text, samt hur texten ska presenteras för att resultera i bäst läslighet och läsbarhet. Detta studeras genom användartester, där olika formateringar på texten presenteras under olika belysningar och med olika bakgrund. Detta följs av en kort enkät där deltagarna får besvara frågor gällande textformateringarna. Enkäten innehåller även frågor relaterade till glasögonens egenskaper och hur dessa påverkar användarupplevelsen för att kunna undersöka vilka glasögon som skulle vara mest lämpade för detföreslagna systemet. Avslutande genomförs en marknadsundersökning och litteraturundersökning över befintliga tal-till-text-system på den svenska marknaden. Resultatet från studien visar att belysning och bakgrund påverkar synligheten och sedermera läsligheten av texten som presenteras i glasögonen. Vidare påvisas det att textens formatering, med avseende på textfärg, textbakgrund, antal rader och storlek påverkar läsligheten. Resultatet visar även att AR-glasögonens egenskaper påverkar användarupplevelsen, läsligheten och läsbarheten. Vilka glasögon som är bäst lämpade tycks dock bero på individuella preferenser. Gällande tal-till-text-system, indentifieras fyra stycken som finns tillgängliga på den svenska marknaden. Utifrån utvärderingen av tillgängliga tal-till-text-system rekommenderas Google Cloud Speech API, baserat på tekniska egenskaper, robusthet och tillgänglighet.

    Download full text (pdf)
    fulltext
  • 38.
    Engelke, Ulrich
    et al.
    Commonwealth Scientific and Industrial Research Organisation, Australia.
    Darcy, Daniel P.
    Dolby Laboratories, USA.
    Mulliken, Grant H.
    Dolby Laboratories, USA.
    Bosse, Sebstian
    Fraunhofer Institute for Telecommunications, Germany.
    Martini, Maria G.
    Kingston University, UK.
    Arndt, Sebstian
    NTNU Norwegian University of Science and Technology, Norway.
    Antons, Jan Niklas
    Technische Universiät Berlin, Germany.
    Chan, Kit Y.
    Curtin University, Australia.
    Ramzan, Naeem
    University of the West of Scotland, UK.
    Brunnström, Kjell
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Psychophysiology-Based QoE Assessment: A Survey2017In: IEEE Journal on Selected Topics in Signal Processing, ISSN 1932-4553, E-ISSN 1941-0484, Vol. 11, no 1, p. 6-21, article id 7569001Article in journal (Refereed)
    Abstract [en]

    We present a survey of psychophysiology-based assessment for quality of experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarize multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardized experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research.

  • 39.
    Falk, Julia
    et al.
    RISE Research Institutes of Sweden.
    Eksvärd, Siri
    RISE Research Institutes of Sweden.
    Schenkman, Bo
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. KTH Royal Institute of Technology, Sweden.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Brunnström, Kjell
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Legibility and readability in Augmented Reality2021In: 2021 13th International Conference on Quality of Multimedia Experience (QoMEX), 2021, p. 231-236Conference paper (Refereed)
    Abstract [en]

    Digital technology offers multimodal presentation of information, that can be used for translating foreign languages or for alleviating hearing impairment or deafness communication problems. Today, there exists various aids that can be used for speech-to text translations, but there are some challenges with these. One potential solution for this is to make use of a combination of Augmented Reality (AR) and speech-to-text systems, where speech is converted into text that is then presented in AR-glasses. In AR, one crucial problem is the legibility and readability of text under different environmental conditions. Different types of AR glasses have different ergonomic characteristics, which implies that certain glasses might be more suitable for such a system than others. In this investigation, two different AR-glasses were evaluated based on among other things their optical, visual, and ergonomic characteristics. User tests were conducted to evaluate the legibility and readability of text under different environmental contexts. The results indicate that legibility and readability are affected by factors such as ambient illuminance, background properties and how the text is presented with respect to polarity, opacity, size, and number of lines. The characteristics of the glasses impacts the user experience, but which glasses that is preferred depends on the individual.

  • 40.
    Garcia, Marie Neige
    et al.
    Technische Universität Berlin, Germany.
    De Simone, Francesca
    Telecom ParisTech, France.
    Tavakoli, Samira
    Universidad Politecnica de Madrid, Spain.
    Staelens, Nicolas
    Ghent University, Belgium.
    Egger, Sebastian L.
    Austrian Institute of Technology, Austria.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Raake, Alexander
    Technische Universität Berlin, Germany.
    Quality of experience and HTTP adaptive streaming: A review of subjective studies2014In: 2014 6th International Workshop on Quality of Multimedia Experience, QoMEX 2014, 2014, p. 141-146, article id 6982310Conference paper (Refereed)
    Abstract [en]

    HTTP adaptive streaming technology has become widely spread in multimedia services because of its ability to provide adaptation to characteristics of various viewing devices and dynamic network conditions. There are various studies targeting the optimization of adaptation strategy. However, in order to provide an optimal viewing experience to the end-user, it is crucial to get knowledge about the Quality of Experience (QoE) of different adaptation schemes. This paper overviews the state of the art concerning subjective evaluation of adaptive streaming QoE and highlights the challenges and open research questions related to QoE assessment.

  • 41.
    Gutierrez, Jesus
    et al.
    University Politcnica de Madrid, Spain.
    Perez, Pablo
    Nokia Bell Labs, Spain.
    Orduna, Marta
    University Politcnica de Madrid, Spain.
    Singla, Ashutosh
    Technical University of Ilmenau, Germany.
    Cortes, Carlos
    University Politcnica de Madrid, Spain.
    Mazumdar, Pramit
    University degli Studi Roma Tre, Italy.
    Viola, Irene
    CWI, Netherlands.
    Brunnström, Kjell
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Battisti, Frederica
    Roma Tre University, Italy.
    Cieplinska, Natalia
    AGH University of Science and Technology, Poland.
    Juszka, Dawid
    AGH University of Science and Technology, Poland.
    Janowski, Lucjan
    AGH University of Science and Technology, Poland.
    Leszczuk, Mikoaj
    AGH University of Science and Technology, Poland.
    Adeyemi-ejeye, Femi
    University of Surrey, UK.
    Hu, Yaosi
    Wuhan University, China.
    Chen, Zhengzhong
    Wuhan University, China.
    Van Wallendael, Glenn
    Universiteit Gent, Belgium.
    Lambert, Peter
    Universiteit Gent, Belgium.
    Diaz, Cesar
    University Politecnica de Madrid, Spain.
    Hedlund, John
    RISE Research Institutes of Sweden.
    Hamsis, Omar
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Fremerey, Stephan
    Technical University of Ilmenau, Germany.
    Hofmeyer, Frank
    Technical University of Ilmenau, Germany.
    Raake, Alexander
    Technical University of Ilmenau, Germany.
    Cesar, Pablo
    CWI, Netherlands.
    Subjective evaluation of visual quality and simulator sickness of short 360 videos: ITU-T Rec. P.9192022In: IEEE transactions on multimedia, ISSN 1520-9210, E-ISSN 1941-0077, Vol. 24, p. 3087-3100Article in journal (Refereed)
    Abstract [en]

    Recently an impressive development in immersive technologies, such as Augmented Reality (AR), Virtual Reality (VR) and 360 video, has been witnessed. However, methods for quality assessment have not been keeping up. This paper studies quality assessment of 360 video from the cross-lab tests (involving ten laboratories and more than 300 participants) carried out by the Immersive Media Group (IMG) of the Video Quality Experts Group (VQEG). These tests were addressed to assess and validate subjective evaluation methodologies for 360 video. Audiovisual quality, simulator sickness symptoms, and exploration behavior were evaluated with short (from 10 seconds to 30 seconds) 360 sequences. The following factors' influences were also analyzed: assessment methodology, sequence duration, Head-Mounted Display (HMD) device, uniform and non-uniform coding degradations, and simulator sickness assessment methods. The obtained results have demonstrated the validity of Absolute Category Rating (ACR) and Degradation Category Rating (DCR) for subjective tests with 360 videos, the possibility of using 10-second videos (with or without audio) when addressing quality evaluation of coding artifacts, as well as any commercial HMD (satisfying minimum requirements). Also, more efficient methods than the long Simulator Sickness Questionnaire (SSQ) have been proposed to evaluate related symptoms with 360 videos. These results have been instrumental for the development of the ITU-T Recommendation P.919. Finally, the annotated dataset from the tests is made publicly available for the research community.

  • 42.
    Göreke, Hande
    et al.
    RISE Research Institutes of Sweden. Uppsala University, Sweden.
    Djupsjöbacka, Anders
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Schenkman, Bo
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. KTH Royal Institute of Technology, Sweden.
    Andrén, Börje
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems.
    Hermann, David S.
    Volvo Car Corporation, Sweden.
    Brunnström, Kjell
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Perceptual Judgments of Simulated Low Temperatures in LCD based Vehicle Displays2023In: Digest of Technical Papers - SID International Symposium, John Wiley and Sons Inc , 2023, Vol. 54, no 1, p. 595-598Conference paper (Refereed)
    Abstract [en]

    A well-known drawback with LCD-displays in cold is a slow pixel response leading to poor picture quality. Low temperatures can constitute a hazard in viewing important displays in cars. Perceptual experiments with 20 test-persons were conducted to find clear and acceptable ranges on screens simulating distortions in low temperatures. The results showed perception over clear and acceptable image quality was impaired beyond -20°C for the LCD-screen in the experiments. 

  • 43.
    Hermann, David
    et al.
    Volvo Car Corporation AB, Sweden.
    Djupsjöbacka, Anders
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Andrén, Börje
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo.
    Brunnström, Kjell
    RISE - Research Institutes of Sweden (2017-2019), ICT, Acreo. Mid Sweden University, Sweden.
    Rydell, Niclas
    TCO Development AB, Sweden.
    Display panel certification system for the vehicle industry2017In: Digest of Technical Papers - SID International Symposium, 2017, Vol. 48, no 1, p. 471-474Conference paper (Refereed)
    Abstract [en]

    The ever-increasing need for displaying in-vehicle visual information in a non-distracting way requires a high visual performance of automotive displays. For their procurement, deep technical and supply-chain knowledge is required. Therefore, based on our comparisons of in-vehicle and laboratory visual-performance measurements, we propose a certification system for automotive display panels.

  • 44.
    Kulyk, Valentin
    et al.
    Ericsson, Sweden.
    Tavakoli, Samira
    Universidad Politecnica de Madrid, Spain.
    Folkesson, Mats
    Ericsson, Sweden.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Wang, Kun
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Garcia, Narciso
    Universidad Politecnica de Madrid, Spain.
    3D video quality assessment with multi-scale subjective method2013In: 2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013 - Proceedings, 2013, p. 106-111, article id 6603219Conference paper (Refereed)
    Abstract [en]

    The stereoscopic 3D viewing experience has been studied quite intensively recently; however the subjective test methods to assess it have not been determined. It has become clear that the 3D viewing experience cannot easily be described by just one scale. This paper describes a cross-laboratory study employing the same test setup and materials where a multi-scale rating method was used. One of the laboratories used three scales: Depth Naturalness, Visual Comfort and Video Quality; while the other laboratory used these: 3D Realism, Depth Quantity and Video Quality. The results show that the Quality of Experience (QoE) of 3D video is a multidimensional concept and therefore more than one scale is in many cases needed for subjective assessment of the QoE of 3D video.

  • 45.
    Rafiei, Shirin
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Dima, Elijs
    Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Brunnström, Kjell
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Augmented Remote Operating System for Scaling in smart mining applications: Quality of Experience aspects2022In: Proceedings of Human Vision and Electronic Imaging 2022 / [ed] Damon Chandler, Mark McCourt, Jeffrey Mulligan, 2022, Vol. 34, article id HVEI-166Conference paper (Refereed)
    Abstract [en]

    Remote operation and Augmented Telepresence are fields of interest for novel industrial applications in e.g., construction and mining. In this study, we report on an ongoing investigation of the Quality of Experience aspects of an Augmented Telepresencesystem for remote operation. The system can achieve view augmentation with selective content removal and Novel Perspective view generation. Two formal subjective studies have been performed with test participants scoring their experience while using the system with different levels of view augmentation. The participants also gave free-form feedback on the system and their experiences. The first experiment focused on the effects of in-view augmentations and interface distributions on wall patterns perception. The second one focused on the effects of augmentations on depth and 3D environment understanding. The participants’ feedback from experiment 1 showed that the majority of participants preferred to use the original cam-era views and the Disocclusion Augmentation view instead of the novel Perspective views. Moreover, the Disocclusion Augmentation, which was shown in combination with other views seemed beneficial. When the views were isolated in experiment 2, the impact of the Disocclusion Augmentation view was found to be lower than the Novel Perspective views.

    Download full text (pdf)
    fulltext
  • 46.
    Rafiei, Shirin
    et al.
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Singhal, Chetna
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Indian Institute of Technology, India.
    Brunnström, Kjell
    RISE Research Institutes of Sweden, Digital Systems, Industrial Systems. Mid Sweden University, Sweden.
    Sjöström, Mårten
    Mid Sweden University, Sweden.
    Human Interaction in Industrial Tele-Operated Driving: Laboratory Investigation2023In: 2023 15th International Conference on Quality of Multimedia Experience (QoMEX), 2023, p. 91-94Conference paper (Refereed)
    Abstract [en]

    Tele-operated driving enables industrial operators to control heavy machinery remotely. By doing so, they could work in improved and safe workplaces. However, some challenges need to be investigated while presenting visual information from on-site scenes for operators sitting at a distance in a remote site. This paper discusses the impact of video quality (spatial resolution), field of view, and latency on users’ depth perception, experience, and performance in a lab-based tele-operated application. We performed user experience evaluation experiments to study these impacts. Overall, the user experience and comfort decrease while the users’ performance error increases with an increase in the glass-to-glass latency. The user comfort reduces, and the user performance error increases with reduced video quality (spatial resolution).

  • 47. Reiter, U
    et al.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    De Moor, K
    Mohamed-Chaker, L
    Pereira, M
    Pinheiro, A
    You, J
    Zgank, A
    Factors Influencing Quality of Experience2014In: Quality of Experience, 2014, p. 55-72Chapter in book (Other academic)
  • 48.
    Rizek, Hadi
    et al.
    RISE, Swedish ICT, Acreo.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Wang, Kun
    RISE, Swedish ICT, Acreo. Mid Sweden University, Sweden.
    Andrén, Börje
    RISE, Swedish ICT, Acreo.
    Johanson, Mathias
    Alkit Communications AB, Sweden.
    Subjective evaluation of a 3D videoconferencing system2014In: Proceedings of SPIE - The International Society for Optical Engineering, 2014Conference paper (Refereed)
    Abstract [en]

    A shortcoming of traditional videoconferencing systems is that they present the user with a flat, two-dimensional image of the remote participants. Recent advances in autostereoscopic display technology now make it possible to develop video conferencing systems supporting true binocular depth perception. In this paper, we present a subjective evaluation of a prototype multiview autostereoscopic video conferencing system and suggest a number of possible improvements based on the results. Whereas methods for subjective evaluation of traditional 2D videoconferencing systems are well established, the introduction of 3D requires an extension of the test procedures to assess the quality of depth perception. For this purpose, two depth-based test tasks have been designed and experiments have been conducted with test subjects comparing the 3D system to a conventional 2D video conferencing system. The outcome of the experiments show that the perception of depth is significantly improved in the 3D system, but the overall quality of experience is higher in the 2D system.

  • 49. Schmidmer, C
    et al.
    Takahashi, A
    Pinson, M
    Winkler, S
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Mu, M
    Mauthe, A
    An Interview with Video Quality Experts2009Other (Other academic)
    Abstract [sv]

    Artikel i nyhetsbrev

  • 50.
    Sedano, Inigo
    et al.
    TECHNALIA, Spain.
    Brunnström, Kjell
    RISE, Swedish ICT, Acreo.
    Kihl, Maria
    Lund University, Sweden.
    Aurelius, Andreas
    RISE, Swedish ICT, Acreo. Lund University, Sweden.
    Full-reference video quality metric assisted development of no-reference video quality metrics for real time network monitoring2014In: EURASIP Journal on Image and Video Processing, ISSN 1687-5176, E-ISSN 1687-5281, no 4Article in journal (Refereed)
    Abstract [en]

    High-quality video is being increasingly delivered over Internet Protocol networks, which means that network operators and service providers need methods to measure the quality of experience (QoE) of the video services. In this paper, we propose a method to speed up the development of no-reference bitstream objective metrics for estimating QoE. This method uses full-reference objective metrics, which makes the process significantly faster and more convenient than using subjective tests. In this process, we have evaluated six publicly available full-reference objective metrics in three different databases, the EPFL-PoliMI database, the HDTV database, and the Live Video Wireless database, all containing transmission distortions in H.264 coded video. The objective metrics could be used to speed up the development process of no-reference real-time video QoE monitoring methods that are receiving great interest from the research community. We show statistically that the full-reference metric Video Quality Metric (VQM) performs best considering all the databases. In the EPFL-PoliMI database, SPATIAL MOVIE performed best and TEMPORAL MOVIE performed worst. When transmission distortions are evaluated, using the compressed video as the reference provides greater accuracy than using the uncompressed original video as the reference, at least for the studied metrics. Further, we use VQM to train a lightweight no-reference bitstream model, which uses the packet loss rate and the interval between instantaneous decoder refresh frames, both easily accessible in a video quality monitoring system.

    Download full text (pdf)
    fulltext
12 1 - 50 of 70
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf