无参考视频质量评价是指在不借助原始无损参考视频信息的条件下，对于给定的任意一段视频，直接评测出其质量程度。传统的无参考视频质量评价方法大都基于统计分析，绝大多数都针对特定的视频失真类型，对视频的时域信息考虑较少，导致现有的基于统计分析的方法应用范围局限，实时性较差。提出一种融合视频时空信息的基于卷积神经网络的无参考视频质量评价方法。该方法不针对特定失真类型。将方法分为空域和时域两部分进行处理，空域上提出一种基于卷积神经网络的方法学习空域失真特征，时域上设计一组基于邻帧块结构相似度的特征用以表征视频的时域失真信息。最后将视频的时空特征进行融合，送至线性回归模型进行视频质量的预测。实验表明，所提方法的多项指标均达到主流视频质量评价方法的性能，且方法运行速度大大提高，显示出较好的实时应用前景。
No-reference video quality assessment (NR-VQA) measures distorted videos quantitatively without the reference of original distorted-less videos. Most conventional NR-VQA methods are based on statistical analysis, and the majority of them are generally designed for specific types of distortions or consider less about the temporal information, which limits their application scenarios as well as their speeds. In this paper, we propose a spatio-temporal no-reference video quality assessment method based on convolutional neural network, which is not designed for specific types of distortions. We divide the method into spatial and temporal processes. We redesign a convolutional neural network in spatiality to learn the distortion features in frames. A group of SSIM-like features are exploited in temporality. Finally, we train a linear regression model using the spatio-temporal features to predict the video quality. Experiments demonstrate that the proposed method is similar to other state-of-the-art no-reference VQA methods in performance. Fourthermore, the proposed method runs much faster than other VQA methods, which makes the proposed method have better application prospects.
[1] Vu P V, Vu C T, Chandler M D. A spatiotemporal most-apparent-distortion model for video quality assessment[C]//201118th IEEE International Conference on Image Processing. Brussels:IEEE, 2011:2505-2508.
[2] Vu P V, Chandler D M. Vis3:an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices[J]. Journal of Electronic Imaging, 2014, 23(1):013016.
[3] Seshadrinathan K, Bovik A C. Motion tuned spatio-temporal quality assessment of natural videos[J]. IEEE Transactions on Image Processing, 2010, 19(2):335-350.
[4] Seshadrinathan R, Bovik A C. Video quality assessment by reduced reference spatio-temporal entropic differencing[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2013, 23(4):684-694.
[5] Zhu K F, Keisuke H, Asari V, et al. A no-reference video quality assessment based on laplacian pyramids[C]//201320th IEEE International Conference on Image Processing. Melbourne:IEEE, 2013:49-53.
[6] Lin X Y, Tian X, Chen Y W. No-reference video quality assessment based on region of interest[C]//20122nd International Conference on Consumer Electronics, Communications and Networks. Yichang:IEEE, 2012:1924-1927.
[7] Saad M A, Bovik A C, Charrier C. Blind prediction of natural video quality[J]. IEEE Transactions on Image Processing, 2014, 23(3):1352-1365.
[8] Mittal A, Soundararajan R, Bovik A C. Making a completely blind image quality analyzer[J]. IEEE Signal processing Letters, 2013, 22(3):209-212.
[9] Mittal A, Saad M, Bovik A C. Assessment of video naturalness using time-frequency statistics[C]//2014 IEEE International Conference on Image Processing. Paris:IEEE, 2014:571-574.
[10] Xu J T, Ye P, Liu Y, et al. No-reference video quality assessment via feature learning[C]//2014 IEEE International Conference on Image Processing. Paris:IEEE, 2014:491-495.
[11] Li Y M, Po L M, Cheung C H, et al. No-reference video quality assessment with 3d shearlet transform and convolutional neural networks[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(6):1044-1057.
[12] Kang L, Ye P, Li Y, et al. Convolutional neural network for no reference image quality assessment[C]//2014 IEEE International Conference on Computer Vision and Pattern Recognition. Columbus:IEEE, 2014:1733-1740.
[13] Seshadrinathan K, Bovik A C. Temporal hysteresis model of time varying subjective video quality[C]//2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Prague:IEEE, 2011:1153-1156.
[14] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transaction on Image Processing, 2004, 13(4):600-612.
[15] Seshadrinathan K, Soundararajan R, Bovik A C, et al. A subjective study to evaluate video quality assessment algorithms[C]//IS&T/SPIE Electronic Imaging. San Jose:IEEE, 2010:75270H.
[16] Sheikh H R, Bovik A C, Veciana G D. An information fidelity criterion for image quality assessment using natural scene statistics[J]. IEEE Transactions on Image Processing, 2005, 14(12):2117-2128.
