Evaluation of Information Content-Weighted SSIM (IW-SSIM)

Lin Zhang, School of Software Engineering, Tongji University


Introduction

Information Content Weighted SSIM (IW-SSIM) index is an extension of MS-SSIM by using spatially varied weights. It is presented by Dr. Zhou Wang et al. on 2011 [1].


Source Code

The authors provide their implementation, which can be downloaded here https://ece.uwaterloo.ca/~z70wang/research/iwssim/iwssim_iwpsnr.zip.


Usage Notes

1. Do not change the default parameter settings.

2. IW-SSIM can only deal with gray-scale images and the luminance range is [0, 255]. So, for color images, before calling IW-SSIM, you need to convert it to [0, 255] gray-scale version. Usually, this can be accomplished by the Matlab routine rgb2gray.


Evaluation Results

The results (in Matlab .mat format) are provided here. Each result file contains a n by 2 matrix, where n denotes the number of distorted images in the database. The first column is the IW-SSIM values, and the second column is the mos/dmos values provided by the database. For example, you can use the following matlab code to calculate the SROCC and KROCC values for IW-SSIM values obtained on the TID2008 database:

%%%%%%%%%%%%%%%

matData = load('IWSSIMOnTID.mat');
IWSSIMOnTID= matData.IWSSIMOnTID;
IWSSIM_TID_SROCC = corr(IWSSIMOnTID(:,1), IWSSIMOnTID(:,2), 'type', 'spearman');
IWSSIM_TID_KROCC = corr(IWSSIMOnTID(:,1), IWSSIMOnTID(:,2), 'type', 'kendall');

%%%%%%%%%%%%%%%

The source codes to calculate the PLCC and RMSE are also provided for each database. This needs a nonlinear regression procedure which is dependant on the initialization of the parameters. We try to adjust the parameters to get a high PLCC value. For different databases, the parameter initialization may be different. The nonlinear fitting function is of the form as described in [2].

Evaluation results of IW-SSIM on seven databases are given below. Besides, for each evaluation metric, we present its weighted-average value over all the testing datasets; and the weight for each database is set as the number of distorted images in that dataset.

Database

Results

Nonlinear fitting code SROCC KROCC PLCC RMSE

TID2013

IWSSIMOnTID2013

NonlinearFittingTID2013 0.7779 0.5977 0.8319 0.6880

TID2008

IWSSIMOnTID

NonlinearFittingTID 0.8559 0.6636 0.8579 0.6895

CSIQ

IWSSIMOnCSIQ

NonlinearFittingCSIQ 0.9213 0.7529 0.9144 0.1063

LIVE

IWSSIMOnLIVE

NonlinearFittingLIVE 0.9567 0.8175 0.9522 8.3473

IVC

IWSSIMOnIVC

NonlinearFittingIVC 0.9125 0.7339 0.9231 0.4686

Toyama-MICT

IWSSIMOnMICT

NonlinearFittingMICT

0.9202 0.7537 0.9248 0.4761

A57

IWSSIMOnA57

NonlinearFittingA57

0.8709 0.6842 0.9034 0.1054

WIQ

IWSSIMOnWIQ

NonlinearFittingWIQ

0.7865 0.6038 0.8329 12.6765

Weighted-Average

 

         

Reference                

[1] Z. Wang and Q. Li, "Information content weighting for perceptual image quality assessment", IEEE Trans. on Image Processing, vol. 20, no. 5, pp. 1185-1198, 2011.

[2] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006.


Created on: May 09, 2011

Last update: Dec. 07, 2013