Evaluation of Information Content-Weighted SSIM (IW-SSIM) Lin Zhang, School of Software Engineering, Tongji University |
Introduction
Information Content Weighted SSIM (IW-SSIM) index is an extension of MS-SSIM by using spatially varied weights. It is presented by Dr. Zhou Wang et al. on 2011 [1].
Source Code
The authors provide their implementation, which can be downloaded here https://ece.uwaterloo.ca/~z70wang/research/iwssim/iwssim_iwpsnr.zip.
Usage Notes
1. Do not change the default parameter settings.
2. IW-SSIM can only deal with gray-scale images and the luminance range is [0, 255]. So, for color images, before calling IW-SSIM, you need to convert it to [0, 255] gray-scale version. Usually, this can be accomplished by the Matlab routine rgb2gray.
Evaluation Results
The results (in Matlab .mat format) are provided here. Each result file contains a n by 2 matrix, where n denotes the number of distorted images in the database. The first column is the IW-SSIM values, and the second column is the mos/dmos values provided by the database. For example, you can use the following matlab code to calculate the SROCC and KROCC values for IW-SSIM values obtained on the TID2008 database:
%%%%%%%%%%%%%%%
matData
= load('IWSSIMOnTID.mat');
IWSSIMOnTID= matData.IWSSIMOnTID;
IWSSIM_TID_SROCC = corr(IWSSIMOnTID(:,1), IWSSIMOnTID(:,2), 'type', 'spearman');
IWSSIM_TID_KROCC = corr(IWSSIMOnTID(:,1), IWSSIMOnTID(:,2), 'type', 'kendall');
%%%%%%%%%%%%%%%
The source codes to calculate the PLCC and RMSE are also provided for each database. This needs a nonlinear regression procedure which is dependant on the initialization of the parameters. We try to adjust the parameters to get a high PLCC value. For different databases, the parameter initialization may be different. The nonlinear fitting function is of the form as described in [2].
Evaluation results of IW-SSIM on seven databases are given below. Besides, for each evaluation metric, we present its weighted-average value over all the testing datasets; and the weight for each database is set as the number of distorted images in that dataset.
Database |
Results |
Nonlinear fitting code | SROCC | KROCC | PLCC | RMSE |
TID2013 |
NonlinearFittingTID2013 | 0.7779 | 0.5977 | 0.8319 | 0.6880 | |
TID2008 |
NonlinearFittingTID | 0.8559 | 0.6636 | 0.8579 | 0.6895 | |
CSIQ |
NonlinearFittingCSIQ | 0.9213 | 0.7529 | 0.9144 | 0.1063 | |
LIVE |
NonlinearFittingLIVE | 0.9567 | 0.8175 | 0.9522 | 8.3473 | |
IVC |
NonlinearFittingIVC | 0.9125 | 0.7339 | 0.9231 | 0.4686 | |
Toyama-MICT |
0.9202 | 0.7537 | 0.9248 | 0.4761 | ||
A57 |
0.8709 | 0.6842 | 0.9034 | 0.1054 | ||
WIQ |
0.7865 | 0.6038 | 0.8329 | 12.6765 | ||
Weighted-Average |
|
Reference
[1] Z. Wang and Q. Li, "Information content weighting for perceptual image quality assessment", IEEE Trans. on Image Processing, vol. 20, no. 5, pp. 1185-1198, 2011.
[2] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006.
Created on: May 09, 2011
Last update: Dec. 07, 2013