Evaluation of RFSIM

Lin Zhang, School of Software Engineering, Tongji University

Introduction

RFSIM (Riesz Transforms based Feature Similarity) index is proposed by Zhang et al. in 2010 [1].

Source Code

The authors' implementation can be downloaded here http://sse.tongji.edu.cn/linzhang/IQA/RFSIM/RFSIM.htm.

Usage Notes

Just pass two image matrices to the RFSIM function. For color images, you do not need to convert it to grayscale since such a procedure is embedded in RFSIM.

Evaluation Results

The results (in Matlab .mat format) are provided here. Each result file contains a n by 2 matrix, where n denotes the number of distorted images in the database. The first column is the RFSIM values, and the second column is the mos/dmos values provided by the database. For example, you can use the following matlab code to calculate the SROCC and KROCC values for RFSIM values obtained on the TID2008 database:

%%%%%%%%%%%%%%%

matData = load('RFSIMOnTID.mat');
RFSIMOnTID= matData.RFSIMOnTID;
RFSIM_TID_SROCC = corr(RFSIMOnTID(:,1), RFSIMOnTID(:,2), 'type', 'spearman');
RFSIM_TID_KROCC = corr(RFSIMOnTID(:,1), RFSIMOnTID(:,2), 'type', 'kendall');

%%%%%%%%%%%%%%%

The source codes to calculate the PLCC and RMSE are also provided for each database. This needs a nonlinear regression procedure which is dependant on the initialization of the parameters. We try to adjust the parameters to get a high PLCC value. For different databases, the parameter initialization may be different. The nonlinear fitting function is of the form as described in [2].

Evaluation results of RFSIM on seven databases are given below. Besides, for each evaluation metric, we present its weighted-average value over all the testing datasets; and the weight for each database is set as the number of distorted images in that dataset.

Database

Results

 Nonlinear fitting code SROCC KROCC PLCC RMSE

TID2013

RFSIMOnTID2013

 NonlinearFittingTID2013 0.7744 0.5951 0.8333 0.6852

TID2008

RFSIMOnTID

 NonlinearFittingTID 0.8680 0.6780 0.8645 0.6746

CSIQ

RFSIMOnCSIQ

 NonlinearFittingCSIQ 0.9295 0.7645 0.9179 0.1042

LIVE

RFSIMOnLIVE

 NonlinearFittingLIVE 0.9401 0.7816 0.9354 9.6642

IVC

RFSIMOnIVC

 NonlinearFittingIVC 0.8192 0.6452 0.8361 0.6684

Toyama-MICT

RFSIMOnMICT

NonlinearFittingMICT

 0.7731 0.5752 0.7783 0.7857

A57

RFSIMOnA57

NonlinearFittingA57

 0.8215 0.6324 0.8475 0.1305

WIQ

RFSIMOnWIQ

NonlinearFittingWIQ

 0.7368 0.5493 0.8103 13.4241

Weighted-Average

　

 　 　 　 　 　

Reference                

[1] L. Zhang, L. Zhang, and X. Mou, "RFSIM: a feature based image quality assessment metric using Riesz transforms", in: Proc. ICIP, pp. 321-324, 2010.

[2] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006.

Created on: May. 08, 2011

Last update: Dec. 07, 2013