No-reference video quality assessment (NR-VQA) measures distorted videos quantitatively without the reference of original distorted-less videos. Most conventional NR-VQA methods are based on statistical analysis, and the majority of them are generally designed for specific types of distortions or consider less about the temporal information, which limits their application scenarios as well as their speeds. In this paper, we propose a spatio-temporal no-reference video quality assessment method based on convolutional neural network, which is not designed for specific types of distortions. We divide the method into spatial and temporal processes. We redesign a convolutional neural network in spatiality to learn the distortion features in frames. A group of SSIM-like features are exploited in temporality. Finally, we train a linear regression model using the spatio-temporal features to predict the video quality. Experiments demonstrate that the proposed method is similar to other state-of-the-art no-reference VQA methods in performance. Fourthermore, the proposed method runs much faster than other VQA methods, which makes the proposed method have better application prospects.
WANG Chunfeng
,
SU Li
,
HUANG Qingming
. Spatio-temporal-fused no-reference video quality assessment based on convolutional neural network[J]. Journal of University of Chinese Academy of Sciences, 2018
, 35(4)
: 544
-549
.
DOI: 10.7523/j.issn.2095-6134.2018.04.018
[1] Vu P V, Vu C T, Chandler M D. A spatiotemporal most-apparent-distortion model for video quality assessment[C]//201118th IEEE International Conference on Image Processing. Brussels:IEEE, 2011:2505-2508.
[2] Vu P V, Chandler D M. Vis3:an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices[J]. Journal of Electronic Imaging, 2014, 23(1):013016.
[3] Seshadrinathan K, Bovik A C. Motion tuned spatio-temporal quality assessment of natural videos[J]. IEEE Transactions on Image Processing, 2010, 19(2):335-350.
[4] Seshadrinathan R, Bovik A C. Video quality assessment by reduced reference spatio-temporal entropic differencing[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2013, 23(4):684-694.
[5] Zhu K F, Keisuke H, Asari V, et al. A no-reference video quality assessment based on laplacian pyramids[C]//201320th IEEE International Conference on Image Processing. Melbourne:IEEE, 2013:49-53.
[6] Lin X Y, Tian X, Chen Y W. No-reference video quality assessment based on region of interest[C]//20122nd International Conference on Consumer Electronics, Communications and Networks. Yichang:IEEE, 2012:1924-1927.
[7] Saad M A, Bovik A C, Charrier C. Blind prediction of natural video quality[J]. IEEE Transactions on Image Processing, 2014, 23(3):1352-1365.
[8] Mittal A, Soundararajan R, Bovik A C. Making a completely blind image quality analyzer[J]. IEEE Signal processing Letters, 2013, 22(3):209-212.
[9] Mittal A, Saad M, Bovik A C. Assessment of video naturalness using time-frequency statistics[C]//2014 IEEE International Conference on Image Processing. Paris:IEEE, 2014:571-574.
[10] Xu J T, Ye P, Liu Y, et al. No-reference video quality assessment via feature learning[C]//2014 IEEE International Conference on Image Processing. Paris:IEEE, 2014:491-495.
[11] Li Y M, Po L M, Cheung C H, et al. No-reference video quality assessment with 3d shearlet transform and convolutional neural networks[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(6):1044-1057.
[12] Kang L, Ye P, Li Y, et al. Convolutional neural network for no reference image quality assessment[C]//2014 IEEE International Conference on Computer Vision and Pattern Recognition. Columbus:IEEE, 2014:1733-1740.
[13] Seshadrinathan K, Bovik A C. Temporal hysteresis model of time varying subjective video quality[C]//2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Prague:IEEE, 2011:1153-1156.
[14] Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transaction on Image Processing, 2004, 13(4):600-612.
[15] Seshadrinathan K, Soundararajan R, Bovik A C, et al. A subjective study to evaluate video quality assessment algorithms[C]//IS&T/SPIE Electronic Imaging. San Jose:IEEE, 2010:75270H.
[16] Sheikh H R, Bovik A C, Veciana G D. An information fidelity criterion for image quality assessment using natural scene statistics[J]. IEEE Transactions on Image Processing, 2005, 14(12):2117-2128.
