This document describes a user experiment with the purpose of finding a baseline quality that is suitable for system and a database for training and evaluating the objective quality measurement methods suitable for assessing the video quality of VAR systems. 

A user experiment was performed involving 25 Swedish video experts. Three different video formats were incorporated 1080p, 1080i and 540i. The degradations were in most cased done using encoding with Motion JPEG (MJPEG) and H.264 in the bitrate range from 80 Mbit/s down to 10 Mbit/s. 

MJPEG loses quality very fast and already at 80 Mbit/s it has significantly lower quality than the uncompressed reference and then for even lower bitrates the quality falls quickly to bad. On the other hand, H.264 was not found to be significant different from the uncompressed reference until the bitrate had dropped to 10 Mbit/s for 1080p. For 1080i 20 Mbit/s was also weakly and for 540i 20 Mbit/s was significantly lower for some of the scaling methods. For 1080i the deinterlacing requires careful consideration, since the deinterlacing scheme introduced received very low quality scores. For the scaling scheme lanczos was the best and bilinear the worst. 

Requirement levels on bitrate and the encoders MJPEG and H.264 based on this experiment•    MJPEG require more than 120 Mbit/s•    H.264 require more than 50 Mbit/s