Evaluation of GSM

Lin Zhang, School of Software Engineering, Tongji University


Introduction

GSM (Gradient Similarity based Metric) index is presented by Liu et al. 2012 [1].


Source Code

Source code is provided by the creators at https://1drv.ms/u/s!Ag8dxliAh_DPgVvk_xjw63DnGVFZ?e=3tjM7c


Usage Notes

1. Do not change the default parameter settings.

2. imRef = rgb2gray(imRef);
    imDis = rgb2gray(imDis);
    metricValue = GSM(double(imRef),double(imDis));


Evaluation Results

The results (in Matlab .mat format) are provided here. Each result file contains a n by 2 matrix, where n denotes the number of distorted images in the database. The first column is the IFC values, and the second column is the mos/dmos values provided by the database. For example, you can use the following matlab code to calculate the SROCC and KROCC values for GSM values obtained on the TID2008 database:

%%%%%%%%%%%%%%%

matData = load('GSMOnTID.mat');
GSMOnTID= matData.GSMOnTID;
GSM_TID_SROCC = corr(GSMOnTID(:,1), GSMOnTID(:,2), 'type', 'spearman');
GSM_TID_KROCC = corr(GSMOnTID(:,1), GSMOnTID(:,2), 'type', 'kendall');

%%%%%%%%%%%%%%%

The source codes to calculate the PLCC and RMSE are also provided for each database. This needs a nonlinear regression procedure which is dependant on the initialization of the parameters. We try to adjust the parameters to get a high PLCC value. For different databases, the parameter initialization may be different. The nonlinear fitting function is of the form as described in [2].

Evaluation results of GSM on seven databases are given below. Besides, for each evaluation metric, we present its weighted-average value over all the testing datasets; and the weight for each database is set as the number of distorted images in that dataset.

Database

Results

Nonlinear fitting code SROCC KROCC PLCC RMSE

TID2013

GSMOnTID2013 NonlinearFittingTID2013 0.7946 0.6255 0.8464 0.6603

TID2008

GSMOnTID2008 NonlinearFittingTID2008 0.8504 0.6596 0.8422 0.7235

CSIQ

GSMOnCSIQ

NonlinearFittingCSIQ 0.9108 0.7374 0.8964 0.1164

LIVE

GSMOnLIVE NonlinearFittingLIVE 0.9561 0.8150 0.9512 8.4327

Weighted-Average

 

         

Reference                

[1] A. Liu, W. Lin, and M. Narwaria, "Image quality assessment based on gradient similarity," IEEE Trans. Image Process., vol. 21, no. 4, pp. 1500-1512, 2012.

[2] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms," IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006.


Created on: Dec. 04, 2013

Last update: Dec. 04, 2013