Evaluation of VIF Lin Zhang, School of Software Engineering, Tongji University |
Introduction
VIF (Visual Information Fidelity) index is presented by Sheikh and Bovik on 2006 [1].
Source Code
The authors of VIF provide two different implementations of VIF, one in the wavelet domain and the other one in the pixel domain. The authors claim that pixel domain version performs slightly worse than its wavelet domain counterpart. We use the wavelet domain implementation here, which can be downloaded from here http://live.ece.utexas.edu/research/Quality/vifvec_release.zip. Please note that in our previous works, we used the VIF implementation provided in the MetriX MuX package, which will generate slightly different results from the ones reported here. This is because in MetriX MuX uses a different "color to grayscale" conversion strategy.
Usage Notes
1. Do not change the default parameter settings.
2. VIF can only deal with gray-scale images and the luminance range is [0, 255]. So, for color images, before calling VIF, you need to convert it to [0, 255] gray-scale version. Usually, this can be accomplished by the Matlab routine rgb2gray.
Evaluation Results
The results (in Matlab .mat format) are provided here. Each result file contains a n by 2 matrix, where n denotes the number of distorted images in the database. The first column is the VIF values, and the second column is the mos/dmos values provided by the database. For example, you can use the following matlab code to calculate the SROCC and KROCC values for VIF values obtained on the TID2008 database:
%%%%%%%%%%%%%%%
matData
= load('VIFOnTID.mat');
VIFOnTID = matData.VIFOnTID;
VIF_TID_SROCC = corr(VIFOnTID(:,1), VIFOnTID(:,2), 'type', 'spearman');
VIF_TID_KROCC = corr(VIFOnTID(:,1), VIFOnTID(:,2), 'type', 'kendall');
%%%%%%%%%%%%%%%
The source codes to calculate the PLCC and RMSE are also provided for each database. This needs a nonlinear regression procedure which is dependant on the initialization of the parameters. We try to adjust the parameters to get a high PLCC value. For different databases, the parameter initialization may be different. The nonlinear fitting function is of the form as described in [2].
Evaluation results of VIF on seven databases are given below. Besides, for each evaluation metric, we present its weighted-average value over all the testing datasets; and the weight for each database is set as the number of distorted images in that dataset.
Database |
Results |
Nonlinear fitting code | SROCC | KROCC | PLCC | RMSE |
TID2013 |
NonlinearFittingTID2013 | 0.6769 | 0.5147 | 0.7720 | 0.7880 | |
TID2008 |
NonlinearFittingTID | 0.7491 | 0.5860 | 0.8084 | 0.7899 | |
CSIQ |
NonlinearFittingCSIQ | 0.9195 | 0.7537 | 0.9277 | 0.0980 | |
LIVE |
NonlinearFittingLIVE | 0.9636 | 0.8282 | 0.9604 | 7.6137 | |
IVC |
NonlinearFittingIVC | 0.8964 | 0.7158 | 0.9028 | 0.5239 | |
Toyama-MICT |
NonlinearFittingMICT | 0.9077 | 0.7315 | 0.9138 | 0.5084 | |
A57 |
0.6223 |
0.4589 | 0.6915 | 0.1784 | ||
WIQ |
0.6918 |
0.5246 | 0.7605 | 14.8731 | ||
Weighted-Average |
|
Reference
[1] H.R. Sheikh and A.C. Bovik, "Image information and visual quality", IEEE Trans. on Image Processing, vol. 15, pp. 430-444, 2006.
[2] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006.
Created on: May. 08, 2011
Last update: Dec. 03, 2013