Digital Video Quality E-Book

Stefan Winkler

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 250

Veröffentlichungsjahr: 2013

Beschreibung

Visual quality assessment is an interdisciplinary topic that links image/video processing, psychology and physiology. Many engineers are familiar with the image/video processing; transmission networks side of things but not with the perceptual aspects pertaining to quality.

Digital Video Quality first introduces the concepts of human vision and visual quality. Based on these, specific video quality metrics are developed and their design is presented. These metrics are then evaluated and used in a number of applications, including image/video compression, transmission and watermarking.

• Introduces the concepts of human vision and vision quality.
• Presents the design and development of specific video quality metrics.
• Evaluates video quality metrics in the context of image/video compression, transmission and watermarking.
• Presents tools developed for the analysis of video quality

Leseprobe

Contents

About the Author

Acknowledgements

Acronyms

1 Introduction

1.1.Motivation

1.2.Outline

2 Vision

2.1 Eye

2.2 Retina

2.3 Visual Pathways

2.4 Sensitivity to Light

2.5 Color Perception

2.6 Masking and Adaptation

2.7 Multi-channel Organization

2.8 Summary

3 Video Quality

3.1 Video Coding and Compression

3.2 Artifacts

3.3 Visual Quality

3.4 Quality Metrics

3.5 Metric Evaluation

3.6 Summary

4 Models and Metrics

4.1 Isotropic Contrast

4.2 Perceptual Distortion Metric

4.3 Summary

5 Metric Evaluation

5.1 Still Images

5.2 Video

5.3 Component Analysis

5.4 Summary

6 Metric Extensions

6.1 Blocking Artifacts

6.2 Object Segmentation

6.3 Image Appeal

6.4 Summary

7 Closing Remarks

7.1 Summary

7.2 Perspectives

Appendix: Color Space Conversions

References

Index

Copyright © 2005 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England

Telephone [phone]

Email (for orders and customer service enquiries): [email protected]

Visit our Home Page on www.wiley.com

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to ([phone].

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The Publisher is not associated with any product or vendor mentioned in this book.

This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Other Wiley Editorial Offices

John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA

Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA

Wiley–VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany

John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809

John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Library of Congress Cataloging-in-Publication Data

Winkler, Stefan.

Digital video quality: vision models and metrics / Stefan Winkler.

p. cm.

Includes bibliographical references and index.

ISBN 0-470-02404-6

1. Digital video. 2. Image processing—Digital techniques. 3. Imaging systems—Image quality. I. Title.

TK6680.5.W55 2005

006.6′96–dc22

2004061588

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN 0-470-02404-6

About the Author

O, what may man within him hide, Though angel on the outward side!

William Shakespeare

Stefan Winkler was born in Horn, Austria. He received the M.Sc. degree with highest honors in electrical engineering from the University of Technology in Vienna, Austria, in 1996, and the Ph.D. degree in electrical engineering from the École Polytechnique Fédérale de Lausanne (EPFL), Switzerland, in 2000 for work on vision modeling and video quality measurement. He also spent one year at the University of Illinois at Urbana-Champaign as a Fulbright student. He did internships at Siemens, ROLM, German Aerospace, Andersen Consulting, and Hewlett-Packard.

In January 2001 he co-founded Genimedia (now Genista), a company developing perceptual quality metrics for multimedia applications. In October 2002, he returned to EPFL as a post-doctoral fellow, and he also held an assistant professor position at the University of Lausanne for a semester. Currently he is Chief Scientist at Genista Corporation.

Dr Winkler has been an invited speaker at numerous technical conferences and seminars. He was organizer of a special session on video quality at VCIP 2003, technical program committee member for ICIP 2004 and WPMC 2004, and has been serving as a reviewer for several scientific journals. He is the author and co-author of over 30 publications on vision modeling and quality assessment.

Acknowledgements

I thank you most sincerely for your assistance; whether or no my book may be wretched, you have done your best to make it less wretched.

Charles Darwin

The basis for this book was my PhD dissertation, which I wrote at the Signal Processing Lab of the École Polytechnique Fédérale de Lausanne (EPFL) under the supervision of Professor Murat Kunt. I appreciated his guidance and the numerous discussions that we had. Christian van den Branden Lambrecht, whose work I built upon, was also very helpful in getting me started. I acknowledge the financial support of Hewlett-Packard for my PhD research.

I enjoyed working with my colleagues at the Signal Processing Lab. In particular, I would like to mention Martin Kutter, Marcus Nadenau and Pierre Vandergheynst, who helped me shape and realize many ideas. Yousri Abdeljaoued, David Alleysson, David McNally, Marcus Nadenau, Francesco Ziliani and my brother Martin read drafts of my dissertation chapters and provided many valuable comments and suggestions for improvement. Professor Jean-Bernard Martens from the Eindhoven University of Technology gave me a lot of feedback on my thesis. Furthermore, I thank all the people who participated in my subjective experiments for their time and patience.

Kambiz Homayounfar and Professor Touradj Ebrahimi created Genimedia and thus allowed me to carry on my research in this field and to put my ideas into products; they also encouraged me to work on this book. I am grateful to all my colleagues at Genimedia/Genista for the stimulating discussions we had and for creating such a pleasant working environment.

Thanks are due to the anonymous reviewers of the book for their helpful feedback. Simon Robins spent many hours with painstaking format conversions and more proofreading. I also thank my editor Simone Taylor for her assistance in publishing this book.

Last but not least, my sincere gratitude goes to my family for their continuous support and encouragement.

Acronyms

A word means just what I choose it to mean – neither more nor less.

Lewis Carroll

ACR

Absolute category rating

ANSI

American National Standards Institute

ATM

Asynchronous transfer mode

CIE

Commission Internationale de l’Eclairage

cpd

Cycles per degree

CRT

Cathode ray tube

CSF

Contrast sensitivity function

dB

Decibel

DCR

Degradation category rating

DCT

Discrete cosine transform

DMOS

Differential mean opinion score

DSCQS

Double stimulus continuous quality scale

DSIS

Double stimulus impairment scale

DVD

Digital versatile disk

DWT

Discrete wavelete transform

EBU

European Broadcasting Union

FIR

Finite impulse response

HDTV

High-definition television

HLS

Hue, lightness, saturation

HSV

Hue, saturation, value

HVS

Human visual system

IEC

International Electrotechnical Commission

IIR

Infinite impulse response

ISO

International Organization for Standardization

ITU

International Telecommunication Union

JND

Just noticeable difference

JPEG

Joint Picture Experts Group

kb/s

Kilobit per second

LGN

Lateral geniculate nucleus

Mb/s

Megabit per second

MC

Motion compensation

MOS

Mean opinion score

MPEG

Moving Picture Experts Group

MSE

Mean squared error

MSSG

MPEG Software Simulation Group

NTSC

National Television Systems Committee

NVFM

Normalization video fidelity metric

PAL

Phase Alternating Line

PDM

Perceptual distortion metric

PBDM

Perceptual blocking distortion metric

PSNR

Peak signal-to-noise ratio

RGB

Red, green, blue

RMSE

Root mean squared error

SID

Society for Information Display

SSCQE

Single stimulus continuous quality evaluation

SNR

Signal-to-noise ratio

TCP/IP

Transmission control protocol/internet protocol

VCD

Video compact disk

VHS

Video home system

VQEG

Video Quality Experts Group

1

Introduction

‘Where shall I begin, please your Majesty?’ he asked.

‘Begin at the beginning,’ the King said, gravely, ‘and go on till you come to the end: then stop.’

Lewis Carroll

1.1. MOTIVATION

Humans are highly visual creatures. Evolution has invested a large part of our neurological resources in visual perception. We are experts at grasping visual environments in a fraction of a second and rely on visual information for many of our day-to-day activities. It is not surprising that, as our world is becoming more digital every day, digital images and digital video are becoming ubiquitous.

In light of this development, optimizing the performance of digital imaging systems with respect to the capture, display, storage and transmission of visual information is one of the most important challenges in this domain. Video compression schemes should reduce the visibility of the introduced artifacts, watermarking schemes should hide information more effectively in images, printers should use the best half-toning patterns, and so on. In all these applications, the limitations of the human visual system (HVS) can be exploited to maximize the visual quality of the output. To do this, it is necessary to build computational models of the HVS and integrate them in tools for perceptual quality assessment.

The need for accurate vision models and quality metrics has been increasing as the borderline between analog and digital processing of visual information is moving closer to the consumer. This is particularly evident in the field of television. While traditional analog systems still represent the majority of television sets today, production studios, broadcasters and network providers have been installing digital video equipment at an ever-increasing rate. Digital satellite and cable services have been available for quite some time, and terrestrial digital TV broadcast has been introduced in a number of locations around the world. A similar development can be observed in photography, where digital cameras have become hugely popular.

The advent of digital imaging systems has exposed the limitations of the techniques traditionally used for quality assessment and control. For conventional analog systems there are well-established performance standards. They rely on special test signals and measurement procedures to determine signal parameters that can be related to perceived quality with relatively high accuracy. While these parameters are still useful today, their connection with perceived quality has become much more tenuous. Because of compression, digital imaging systems exhibit artifacts that are fundamentally different from analog systems. The amount and visibility of these distortions strongly depend on the actual image content. Therefore, traditional measurements are inadequate for the evaluation of these artifacts.

Given these limitations, researchers have had to resort to subjective viewing experiments in order to obtain reliable ratings for the quality of digital images or video. While these tests are the best way to measure ‘true’ perceived quality, they are complex, time-consuming and consequently expensive. Hence, they are often impractical or not feasible at all, for example when real-time online quality monitoring of several video channels is desired.

Looking for faster alternatives, the designers of digital imaging systems have turned to simple error measures such as mean squared error (MSE) or peak signal-to-noise ratio (PSNR), suggesting that they would be equally valid. However, these simple measures operate solely on a pixel-by-pixel basis and neglect the important influence of image content and viewing conditions on the actual visibility of artifacts. Therefore, their predictions often do not agree well with actual perceived quality.

These problems have prompted the intensified study of vision models and visual quality metrics in recent years. Approaches based on HVS-models are slowly replacing classical schemes, in which the quality metric consists of an MSE- or PSNR-measure. The quality improvement that can be achieved using an HVS-based approach instead is significant and applies to a large variety of image processing applications. However, the human visual system is extremely complex, and many of its properties are not well understood even today. Significant advancements of the current state of the art will require an in-depth understanding of human vision for the design of reliable models.

The purpose of this book is to provide an introduction to vision modeling in the framework of video quality assessment. We will discuss the design of models and metrics and show examples of their utilization. The models presented are quite general and may be useful in a variety of image and video processing applications.

1.2 OUTLINE

Chapter 2 gives an overview of the human visual system. It looks at the anatomy and physiology of its components, explaining the processing of visual information in the brain together with the resulting perceptual phenomena.

Chapter 3 outlines the main aspects of visual quality with a special focus on digital video. It briefly introduces video coding techniques and explores the effects that lossy compression or transmission errors have on quality. We take a closer look at factors that can influence subjective quality and describe procedures for its measurement. Then we review the history and state of the art of video quality metrics and discuss the evaluation of their prediction performance.

Chapter 4 presents tools for vision modeling and quality measurement. The first is a unique measure of isotropic local contrast based on analytic directional filters. It agrees well with perceived contrast and is used later in conjunction with quality assessment. The second tool is a perceptual distortion metric (PDM) for the evaluation of video quality. It is based on a model of the human visual system that takes into account color perception, the multi-channel architecture of temporal and spatial mechanisms, spatio-temporal contrast sensitivity, pattern masking and channel interactions.

Chapter 5 is devoted to the evaluation of the prediction performance of the PDM as well as a comparison with competing metrics. This is achieved with the help of extensive data from subjective experiments. Furthermore, the design choices for the different components of the PDM are analyzed with respect to their influence on prediction performance.

Chapter 6 investigates a number of extensions of the perceptual distortion metric. These include modifications of the PDM for the prediction of perceived blocking distortions and for the support of object segmentation. Furthermore, attributes of image appeal are integrated in the PDM in the form of sharpness and colorfulness ratings derived from the video. Additional data from subjective experiments are used in each case for the evaluation of prediction performance.

Finally, Chapter 7 concludes the book with an outlook on promising developments in the field of video quality assessment.

2

Vision

Seeing is believing.

English proverb

Vision is the most essential of our senses; 80–90% of all neurons in the human brain are estimated to be involved in visual perception (Young, 1991). This is already an indication of the enormous complexity of the human visual system. The discussions in this chapter are necessarily limited in scope and focus mostly on aspects relevant to image and video processing. For a more detailed overview of vision, the reader is referred to the abundant literature, e.g. the excellent book by Wandell (1995).