Evaluation of UQI

Lin Zhang, Dept. Computing, The Hong Kong Polytechnic University

Introduction

UQI (Universal Quality Index) is proposed by Wang and Bovik in 2002 [1].

Source Code

We use Dr. Zhou Wang's original implementation which can be downloaded here https://ece.uwaterloo.ca/~z70wang/research/quality_index/img_qi.m.

Usage Notes

UQI can only deal with gray-scale images and the luminance range is [0, 255]. So, for color images, before calling UQI, you need to convert it to [0, 255] gray-scale version. Usually, this can be accomplished by the Matlab routine rgb2gray.

Evaluation Results

The results (in Matlab .mat format) are provided here. Each result file contains a n by 2 matrix, where n denotes the number of distorted images in the database. The first column is the UQI values, and the second column is the mos/dmos values provided by the database. For example, you can use the following matlab code to calculate the SROCC and KROCC values for UQI values obtained on the TID2008 database:

%%%%%%%%%%%%%%%

matData = load('UQIOnTID.mat');
UQIOnTID= matData.UQIOnTID;
UQI_TID_SROCC = corr(UQIOnTID(:,1), UQIOnTID(:,2), 'type', 'spearman');
UQI_TID_KROCC = corr(UQIOnTID(:,1), UQIOnTID(:,2), 'type', 'kendall');

%%%%%%%%%%%%%%%

The source codes to calculate the PLCC and RMSE are also provided for each database. This needs a nonlinear regression procedure which is dependant on the initialization of the parameters. We try to adjust the parameters to get a high PLCC value. For different databases, the parameter initialization may be different. The nonlinear fitting function is of the form as described in [2].

Evaluation results of UQI on seven databases are given below. Besides, for each evaluation metric, we present its weighted-average value over all the testing datasets; and the weight for each database is set as the number of distorted images in that dataset.

Database

Results

 Nonlinear fitting code SROCC KROCC PLCC RMSE

TID2008

UQIOnTID

 NonlinearFittingTID 0.5851 0.4255 0.6643 1.0031

CSIQ

UQIOnCSIQ

 NonlinearFittingCSIQ 0.8098 0.6188 0.8312 0.1460

LIVE

UQIOnLIVE

 NonlinearFittingLIVE 0.8941 0.7100 0.8987 11.9823

IVC

UQIOnIVC

 NonlinearFittingIVC 0.8244 0.6252 0.8302 0.6792

Toyama-MICT

UQIOnMICT

NonlinearFittingMICT

 0.7028 0.5227 0.7164 0.8731

A57

UQIOnA57

NonlinearFittingA57

 0.4260 0.3330 0.6356 0.1897

WIQ

UQIOnWIQ

NonlinearFittingWIQ

 0.6084 0.4360 0.6974 16.4163

Weighted-Average

¡¡

 ¡¡ 0.7137 0.5398 0.7602 ¡¡

Reference                

[1] Z. Wang and A.C. Bovik, ¡°A universal image quality index,¡± IEEE Signal Process. Lett., vol. 9, no. 3, pp. 81-84, 2002.

[2] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 3440-3451, 2006.

Created on: May 08, 2011

Last update: Aug.03, 2011