Digital Color E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 444

Veröffentlichungsjahr: 2013

Beschreibung

In this book the authors identify the basic concepts and recent advances in the acquisition, perception, coding and rendering of color. The fundamental aspects related to the science of colorimetry in relation to physiology (the human visual system) are addressed, as are constancy and color appearance. It also addresses the more technical aspects related to sensors and the color management screen. Particular attention is paid to the notion of color rendering in computer graphics. Beyond color, the authors also look at coding, compression, protection and quality of color images and videos. Individual chapters focus on the LMS specification, color constancy, color appearance models, rendering in synthetic image generation, image sensor technologies, image compression, and quality and secure color imaging. Ideal for researchers, engineers, Master's and PhD students, Digital Color: Acquisition, Perception, Encoding and Rendering offers a state of the art on all the scientific and technical issues raised by the different stages of the digital color process - acquisition, analysis and processing. Contents 1. Colorimetry and Physiology - The LMS Specification, Françoise Viénot and Jean Le Rohellec. 2. Color Constancy, Jean-Christophe Burie, Majed Chambah and Sylvie Treuillet. 3. Color Appearance Models, Christine Fernandez-Maloigne and Alain Trémeau. 4. Rendering and Computer Graphics, Bernard Péroche, Samuel Delepoulle and Christophe Renaud. 5. Image Sensor Technology, François Berry and Omar Ait Aider. 6. From the Sensor to Color Images, Olivier Losson and Eric Dinet. 7. Color and Image Compression, Abdelhakim Saadane, Mohamed-Chaker Larabi and Christophe Charrier. 8. Protection of Color Images, William Puech, Alain Trémeau and Philippe Carré. 9. Quality Assessment Approaches, Mohamed-Chaker Larabi, Abdelhakim Saadane and Christophe Charrier.

Leseprobe

Table of Contents

Foreword

Chapter 1. Colorimetry and Physiology - The LMS Specification

1.1. Physiological basis

1.2. The XYZ colorimetry: the benchmark model of CIE

1.3. LMS colorimetry

1.4. Colors in their context

1.5. Conclusion

1.6. Bibliography

Chapter 2. Color Constancy

2.1. Introduction

2.2. Theoretical preliminaries and problems

2.3. Color constancy models

2.4. Color correction algorithms

2.5. Comparison of color constancy algorithms

2.6. Conclusion

2.7. Bibliography

Chapter 3. Color Appearance Models

3.1. Introduction

3.2. The two perceptual phenomena of color appearance

3.3. The main components of a CAM

3.4. The CIECAM02

3.5. Conclusion

3.6. Bibliography

Chapter 4. Rendering and Computer Graphics

4.1. Introduction

4.2. Reflection and representation models of light sources

4.3. Simulation of light propagation

4.4. Display of results

4.5. Conclusion

4.6. Bibliography

Chapter 5. Image Sensor Technology

5.1. Photodetection principle

5.2. Imagers

5.3. Spectral sensitivity of imagers

5.4. Color acquisition systems

5.5. Through monochrome camera

5.6. Tri-sensor systems

5.7. Color camera based on color filter arrays

5.8. Variants of integrated sensors

5.9. Conclusion

5.10. Bibliography

Chapter 6. From the Sensor to Color Images

6.1. Introduction

6.2. Presentation and formalization of demosaicing

6.3. Demosaicing methods

6.4. Quality of the estimated image

6.5. Color camera calibration

6.6. Conclusion

6.7. Bibliography

Chapter 7. Color and Image Compression

7.1. Introduction

7.2. Fundamentals of image compression

7.3. Compression standards and color

7.4. Color Image Compression

7.5. General conclusion

7.6. Bibliography

Chapter 8. Protection of Color Images

8.1. Introduction

8.2. Protection and security of digital data

8.3. Color image watermarking

8.4. Protection of color images by selective encryption (SE)

8.5. Conclusion

8.6. Bibliography

Chapter 9. Quality Assessment Approaches

9.1. Introduction

9.2. Color fidelity metric

9.3. Subjective assessment of the quality

9.4. Objective evaluation of quality

First published 2012 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd 27–37 St George's Road London SW19 4EU UK

www.iste.co.uk

John Wiley & Sons, Inc. 111 River Street Hoboken, NJ 07030 USA

www.wiley.com

© ISTE Ltd 2012

The rights of Christine Fernandez-Maloigne, Frédérique Robert-Inacio and Ludovic Macaire to be identified as the author of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Cataloging-in-Publication Data

Digital color / edited by Christine Fernández-Maloine, Frédérique Robert-Inacio, Ludovic Macaire. p. cm. Includes bibliographical references and index.   ISBN 978-1-84821-346-3   1. Image converters. 2. Digital images. 3. Color vision. 4. Color photography--Digital techniques. I. Fernández-Maloine, Christine. II. Robert-Inacio, Frédérique. III. Macaire, Ludovic.   TK8316.D54 2012   006.6--dc23

2012005893

British Library Cataloguing-in-Publication Data

Foreword

The evolution of human vision occurred naturally with time. Some anthropologists suggest that the cells that are sensitive to color, cones, were developed when we were gatherers, in order to distinguish fruits from foliage. In fact, each of the three kinds of cones is sensitive to a wide spectrum of electromagnetic waves. The sum of these three spectra is what we call light. Color does not exist in nature but it is reconstituted, or learned, by our brain based on the quantities of light received on each of the three elements. Of course there are defects inherent to this chain, related to the imperfections of the eye (aging, color-blindness, etc.), or to the learning or the diversity of our cultures. In any case, naming a color is highly subjective. Moreover we denote by the same word a color that would be obtained either by adding the three cones excited by a continuous or discontinuous spectrum of a portion of light or by a pure ray. This is what specialists call metameric colors. They thus have different spectral compositions and we see them or denote them by the same color, the same word. Colors, in human vision, are also influenced by their environment which may cause optical illusions. But we are also able to name a color by the same word even if the lighting conditions change: this is color constancy. All this is already very difficult to understand.

It is along this path that the authors of this book on digital imaging color, like others before them, take us. It is a difficult path fraught with questions starting from image capture up to making a decision. It is a path where the authors have added the acquired knowledge and their own research, and where each element of the chain affects the other and vice versa. Of course, they rely on human vision that, as we have seen, has its defects, while scanning an image adds even more. The selection or the variability of the illuminant, the light/matter interaction from a macroscopic point of view (reflection, diffusion, diffraction, polarization, etc.), the particle size of the object, etc., make the spectrum received by the sensor already complex and influenced by all these physical phenomena. The different technologies of the digital sensor are also added to the complexity of obtaining an objective digital image.

Reader, all this is largely explained in this book which will guide you through this science of colorimetry associated with physiology - that is to say to our body functions, which is in this case our human visual system (HVS). Other explanations will guide you in choosing different models, often standardized by the CIE (International Commission on Illumination) through to the color appearance model used to approach the HVS. Every word of this model clearly demonstrates the difficulty of understanding the concept of colorimetry. It is a genuine aid that is proposed to base the choices of the sensor and the model on what we want, i.e. based on our application. Algorithms and hints are included to assist you and resolve issues, while the explanations at every turn of a page increase your understanding.

So if the image is digital why not create a resemblance to reality. A whole chapter is reserved for rendering and image synthesis. You will therefore find how to perform computationally some of the issues of the first section of this preface. Representation models of light sources up to spectrum rendering passing through the simulation of light propagation (ray tracing, etc.) make us travel in this world of virtual reality. Encryption, watermarking to protect authors will have almost no secret for you. If you want to know how to set the quality of an image and contrast it with the fidelity of an image, then approaches in this thriving field will bring you, if not solutions, at least the tools for you to build a good workshop. In contrast to the sensor, there is the visualization. So, how can we measure - for the sensor as well - how to calibrate it? Nothing is simple in this field since subjectivity is ultimately the teacher. At the end of the bench there is the user, the observer, and it is here that everything escapes us. We open the time to the reader and his memory. A second book1 by the same authors is also to be published that is complementary to this one, but with amore algorithmic and applicative vision.

I would like to extend thanks to Christine Fernandez-Maloigne (University of Poitiers), Frédérique Robert-Inacio (ISEN-Toulon) and Ludovic Macaire (University of Lille) for having successfully led this team of authors, each an expert in their field. In addition thank you to all of the authors for devoting your time and dispensing your knowledge to share with us this major work, which is obviously useful for our community and also beyond. This book is intended for all those who wish to develop a chain of acquisition, processing and analysis of color digital images. To you, reader, I wish you good reading and am sure you will find the answers to your problems in this field.

Pierre BONTON

University Blaise Pascal, Clermont-Ferrand II, France LASMEA Laboratory, CNRS JUR 6602 March 2012

1 C. FERNANDEZ-MALOIGNE, F. ROBERT-INACIO, L. MACAIRE, Digital Color Imaging, ISTE, London, John Wiley & sons, New York, 2012.

Chapter 1

Colorimetry and Physiology – The LMS Specification1

To test and improve the quality of color images, we need to know how the human visual system operates. Colorimetry is a method that quantitatively assesses the changes that the engineer makes to an image. Recent advances in this field are due to a better understanding of visual mechanisms.

This chapter first describes the physiological mechanisms that are transferred from the retina in the eye to the human brain, which produce the physiological perception of color. Then it presents two approaches to colorimetry: first, as recommended by the International Commission on Illumination (CIE), and second, deriving directly from the physiology of the visual system that results in the ability to specify stimuli and color differences. Finally, the chapter outlines the difficulties in defining the appearance of color and the advantages in modeling the human visual system.

1.1. Physiological basis

Light detected by the eye excites the photoreceptors that are photosensitive cells. It produces biochemical changes and yields a signal, which is relayed by different classes of post-synaptic retinal neurons. The post-synaptic retinal neurons are organized in a layer in radial and transversal directions. The information is conveyed along the radial direction of the receptors toward the bipolar cells and then toward the ganglion cells whose axons form the optic nerve.

Horizontal cells and amacrine cells form a transversal network, whose action modulates direct signals. Then, the signal travels in the form of trains of action potentials through the optic nerve toward the visual cortex, where a visual image is formed.

1.1.1. The photoreceptors

1.1.1.1. Spectral sensitivity of cones, the monovariant response of a photoreceptor

Each photon absorbed by a cone triggers a cascade of chemical reactions producing a signal at the output of the cone, regardless of the wavelength associated with that photon. As a result, the amplitude of the response of the cone to light depends only on the number of absorbed photons, and not on the wavelength associated with the photons. While light consist of wavelengths in the visible spectrum with wide energy distributions, the response of a single cone is monovariant, it varies only in amplitude. If the photon has energy close to that required for the isomerization of the photosensitive pigment included in the cone, this is absorbed by the cone. The probability of absorption is determined by the spectral sensitivity of the cone.

The in vitro measurements of the spectral sensitivity of the cones [DAR 83] showed the existence of three families of retinal cones with maxima at 419, 531 and 558 nm. This would correspond to in vivo measurements in a healthy eye, taking into account the filtering effect of the ocular media, which is about 440 nm for S cones sensitive to short wavelengths, 540 nm for M cones, sensitive to middle wavelengths,

Figure 1.1.Spectral sensitivity of the three families of cones

and 565 nm for L cones, sensitive to longer wavelengths. The spectral sensitivity curves are widely spread over the visible spectrum, with the M and L cones being close to each other. It should be noted that there is no retinal cone whose maximum sensitivity lies in the part of the spectrum that appears red to the eye (beyond about 620 nm), indicating that the retinal cones are not simple color receptors. Red, like other colors, is reconstructed by the visual system. The M and L cones give to the eye its maximum light sensitivity of around 555 nm.

1.1.1.2. The retinal mosaic

Through optical or electronic microscopy, it is observed that the retina consists of two types of morphologically distinct photoreceptors: rods and cones. Rods are responsible for the vision at low illumination, and conesathigher illumination. The numerical densityof conesismaximum in the fovea, that is to say in the central area of the retina, where images of the objects that we see are formed, and drops significantly toward the surrounding area. It is also possible to observe the cones in vivo, or at the back of the eye, using adaptive optics that neutralize the aberrations. We can also identify their corresponding families L, M or S [ROO 99, HOF 05]. Among the ten retinas that were examined, it was verified that the S cones were relatively few and that the numerical proportion of L and M cones was on average 2L for 1M, with surprising variations from person to person ranging from 1L for 2M to 16L for 1M, for normal color vision.

1.1.2. Retinal organization

The extraction of the color signal is achieved by comparing the signals from a family of cones with those from another family of cones. This comparison is carried out by post-synaptic retinal neurons.

1.1.2.1. Concept of receptive field of the neuron

Each neuron of the visual system, wherever it is located in the hierarchy of the processing, corresponds to a given area of space seen by the subject. It also corresponds to all the requested photoreceptors, within which the bio-electrical behavior of the neuron is changed. This area is called the receptive field of the neuron.

The receptive fields are small in the fovea, and larger as we move away from it. Each neuron is only a small, circular part of the visual field and the encoding of the signal responsible for each neuron depends on its immediate environment.

A retinal neuron does not perform an absolute coding of the light contained in its receptive field, but a coding related to the light status in the near vicinity. Only a differential signal (contrast) generates a signal in the neuron, which is transmitted to the next neurones in the hierarchy of visual information processing. The contrast may relate both to a difference in light or to a difference in spectral content of light.

1.1.2.2. Two parallel pathways from the retina to the cortex

The chemical contact between photoreceptors, bipolar cells, and horizontal cells is carried out at the terminal portion of the cone and is called “synapse”. The synapse type determines a fundamental functional dichotomy of the coding of the light signal. Some bipolar cells have synapses that maintain the polarity of the signal coming from the cone, others reverse it. The ON-bipolar cells indicate an increment of light at the center of the receptive field (relative to the surrounding fields). They initiate a neural pathway called the ON pathway. The OFF-bipolar cells indicate a decrease in light at the center of the receptive field (relative to the surrounding fields). They initiate a neural pathway called the OFF pathway. These ON and OFF pathways run in parallel across each unit area of the visual space by encoding all the variations of light and remain independent up to the cortex.

1.1.2.3. At the origin of konio, parvo and magnocellular pathways

Different types of bipolar cells are at the origin of three separate neurophysiological pathways from the retina to the cortex: the koniocellular pathway dedicated to spectral differentiation; the parvocellular pathway dedicated to spectral differentiation and light differentiation; and the magnocellular pathway dedicated to light differentiation.

Midget bipolar (MB) cells are the most numerous. They receive signals from cones L and M. They are distinguished by the type of synapse, one belonging to the ON pathway, others to the OFF pathway (see next section).

For ease of nomenclature, a neuron whose receptive field center is ON (responding to an increase in light), will be encoded by “+”, and a neuron whose receptive field center is OFF (responding to a decrease in light) will be encoded “−”. The letter following the sign denotes the majority cone type in the center, either L or M (S cones will be discussed in the next section). Implicitly, the area of the receptive field receives signals from the other family of cones, either M or L, on an antagonist mode. For example, a neuron denoted +L will transmit a signal if the center is brighter than the surrounding area and/or if the spectral composition in the center favors large wavelengths. Thus, the midget bipolar cells are of four main types +L, −L, +M and −M. These neurons are the essential elements of the parvocellular pathway and the main initiators of the spectrum of colors and the range of forms and details.

Specific bipolar cells of S cones are the essential elements of the koniocellular pathway. They perform an encoding of the spectral antagonism by contrasting the signals of short wavelengths (S) to those of larger wavelengths (L and M), but are not involved in the encoding of variations in brightness.

Diffused bipolar cells are not selective of the spectral origin. Some are ON, while others are OFF. These neurons are the essential elements of the magnocellular pathway, which initiates the perception of the motion, flicker and variations in brightness.

1.1.2.4. Functional characteristic of the parvocellular pathway (P)

Mostly, neurons in the parvocellular pathway are sensitive to both spectral variations and light variations, thus combining two perceptually distinct variables: variation in brightness (shape precursor) and variation in chromaticity (color precursor). On the same area of the image, these neurons have two functional organizations of receptive fields: a receptive field sensitive to light variations only and a receptive field sensitive to spectral variations. For example, a neuron +L, indicates an increment of light in the center of its receptive field AND/OR, for an equiluminance stimulation, the neuron indicates that the center is covered with a light of longer wavelength than the one illuminating the surrounding area.

1.1.2.5. Three neural pathways at the end of the retina

Given the broad band of spectral sensitivity of cones, cone antagonism allows us, in addition to a saving of messengers, to decorrelate the signals from the cones and to minimize redundancies within and among the three pathways: magno, parvo, and koniocellular [LEE 99, ZAI 97, SHE 08].

Each cone can simultaneously interact with many types of bipolar cells. This is as if the signal of a cone was specifically transcribed according to the specialization of the neuron that it contacts, thus creating multiple filters of a single piece of information generated by the interaction of the light with the photoreceptor.

In each channel, the ganglion cells collect the signals that come from the bipolar cells modulated by the amacrine cells. These are transmitted to the neurons in the lateral geniculate nucleus of the thalamus without significant functional change and with convergence rates that vary depending on the distance to the fovea. These signals are then projected on the primary visual cortex, where they are combined (summed and/or differentiated). As one goes up the hierarchy of processing, the size of receptive fields increases, taking into account larger areas of the visual field.

Figure 1.2.Schematic representation of three independent neural pathways from the retina to the cortex, from the cones. First level: synaptic level of 3 types of cones, L, M and S (here, minimal version of 3 cones). Second level: the receptive fields of the neurons in the parvocellular pathway combine the signals from L and M cones. Cone antagonism appears as a center-surround organization encoding both light contrast (+ and −) and spectral contrast (L and M) giving rise to four types of units (+L, −L, +M, −M). The receptive fields of neurons in the koniocellular pathway combine the signals of S cones with the signals of the cones that are sensitive to larger wavelengths (L and M) under the form of co-extensive receptive fields that encode the chromatic contrast (S vs LM). The receptive fields of neurons in the magnocellular pathway combine the signals from L and M cones, the center-surround organization encode the light contrast (+ and −). Third level: additive combination of receptive fields of units in the parvocellular pathway, resulting in selective units at the thin chromatic contrast and selective units at the thin light contrast. The encoding of the color contrast is enhanced and refined by the summation of parvo units with the koniocellular units. The encoding of the light contrast is enhanced and refined by the summation of parvo units with magnocellular units (for a color version of this figure, see www.iste.co.uk/fernandez/digicolor.zip)

1.1.3. Physiological modeling of visual attributes related to color

The De Valois and De Valois model [VAL 93] allows us to account for segregation of color and light information mixed in the early stages in the visual pathways. This model distinguishes several successive levels of development of the nerve signal leading to the perception of color.

1.1.3.1. Conesynoptic and antagonism level

This model is based on the existence of three types of cones: L, M and S. The S cones are arranged in a regular grid and are less numerous (for example, by proportion: 10 L for 5 M for 1 S).

Cone antagonism is differential encoding between the center and the surrounding area ofthe receptive field. Each cone features a large number of synaptic sites at its terminal portion. A minimal version of the model suggests that each cone establishes contact with at least eight bipolar cells, four with ON neurons (+L, +M, +S and +LMS) and four with OFF neurons (L, M, −S and −LMS).

1.1.3.2. Third level: perceptual antagonism

Cone antagonism of the second level provides relative, local information of a dichotomous nature (for example, areas of more light or; fewer large wavelengths). The third level reflects the separation of information regarding spectral content and that regarding light content. The signals reaching the cortex are summed taking into account the retinotopic organization (the initial vicinity on the retina is preserved). The summation of the signals of the second level sometimes enhances the contrast of light and sometimes the color contrast. When the units of different spectral origins have the same sign, for example: (+L) + (+M), the light contrast is enhanced. When they are of opposite sign, for example (+L) + (M), their joint action enhances the color contrast signal, forming a receptive field with two color antagonism between the center and the surrounding area, indicating thin variations of the color contrast.

Considering the retinotopic recovery in the cortex, the signals from the S cones are conveyed through the koniocellular pathway. The signals from the magnocellular pathway interact with the signals from parvocellular pathway, and then they refine the color signal and the light signal.

In the visual cortex, four types of information are available. Two relate to the spectral nature of the light and two to its relative intensity between the center and the surrounding area of the receptive field regardless of its spectral nature:

1) the spectral contrast, local, between the large and the shortest wavelengths (L vs M);

2) the spectral contrast, local, between the short wavelengths on the one hand and the largest wavelengths on the other hand (S vs L+M);

3) the local contrast of luminance between the center and the surrounding area of the receptive field of small size (parvocellular);

4) the local contrast of luminance between the center and the surrounding area of the receptive field of large size (magnocellular).

There is a specific associative cortical area for the processing of “color” information: the area V4. It processes signals from area V1. In this area V4, the receptive fields are large. Thus, the color signals can be interpreted based on the surrounding color context.

The modeling of color vision still needs to be developed to take into account the weight of the signals of the cones, the linear and/or nonlinear mode of summations [VAL 00, GEG 03], the role of the light and color adaptation, [VAL 08] and the nature of color constancy.

1.2. The XYZ colorimetry: the benchmark model of CIE

In colorimetry, light is a stimulus that elicits a perception of color. Colorimetry is a set of methods and data to universally specify colors.

There are two methods to reproduce colors. The additive mixture consists of adding lightbeams. Their images are projected on the retina and their effects are added. This is the method of displaying colors on screens. The subtractive mixture consists of superimposing absorbing dyes on a medium. A part of the radiation is subtracted from the light reflected by the media and, therefore, does not reach the retina. This is the process used in printers.

Colorimetry is based on visual experiments. Experiments show that the majority of colors may be reproduced by the addition of three additive primary colors, red, green and blue, in appropriate amounts. The experiment, called “color match”, also shows that colors have the properties of vectors, in particular the additive property. Therefore, we apply the laws of linear algebra to colors and additive mixtures of colors. The real additive primary colors consist of the base vectors [R], [G], and [B]. A color is defined by three scalar quantities R, G, and B, called tristimulus values (not to be confused with digital intensities encoding the color of a pixel).

Since the properties of additive mixtures of colors was replicated by different observers, it was decided to establish a reference color specification system. In 1931, the International Commission on Illumination (CIE) adopted the XYZ color system obtained through the linear transformation of the RGB color system. The primaries [X], [Y], and [Z] are virtual. They were chosen so that all colors, including monochromatic radiations, havepositive X, Y and Z tristimulus values, and so that the [Y] axis, alone, carries the luminance. Let and be the spectral tristimulus values or color matching functions. Given a radiation of spectral power distribution (or density) of energy Φ(λ), the tristimulus values X, Y and Z of this radiation are computed as follows:

[1.1]

[1.2]

[1.3]

In case of a display, the function Φ(λ) is given by the measurement of spectral radiance. With the adjustment factor 683, the tristimulus value Y corresponds to the visual luminance of the radiation.

For a reflective material of reflectance ρ(λ), which is illuminated by a source of spectral distribution of energy Φ(λ), the tristimulus values are compared with a white surface of unit reflectance. The tristimulus value Y is in this case equal to the reflectance:

[1.4]

[1.5]

[1.6]

1.3. LMS colorimetry

In colorimetry, light is a stimulus that elicits a perception of color. However, the color vision begins with the absorption of photons by the visual pigments contained in the cones of the retina. Thus, the light, as a stimulus, can be defined by the three signals L, M and S that are generated in the cones, which are the input signals in the visual system.

1.3.1. LMS fundamentals

1.3.1.1. Measurement of the spectral sensitivity of cones by psychophysical techniques

Since their spectral sensitivities overlap, it is impossible to obtain the response of a single family of cones to monochromatic radiation. Instead, a psychophysical method that takes advantage of the reduction to two families of cones among some color-blinds is used. These dichromats feature only S and L cones or S and M cones. The relative spectral sensitivity of L or M cones is measured by temporally alternating between two monochromatic radiations, one serving as a reference, and the other for testing. The pace is fast enough to exclude the S-cones’ response, but slow enough to leave a slight flicker. The flicker sensation is minimal when two radiations have the same visual efficiency. By doing this for all test wavelengths, we obtain the spectral sensitivity of the cone type (either L or M) that followed the flicker.

1.3.1.2. The photosensitive pigments and the cone fundamentals: definition and properties

The absorption phenomena of photons by the molecules of photosensitive pigments are quantum phenomena. It was found that the absorption spectra of all the photosensitive pigments of terrestrial vertebrates, measured in vitro, have almost the same shape on a graph whose horizontal axis is graduated in inverse of the wavelength.

The spectral sensitivity of the cones measured at the entrance of the cornea is called fundamental or cone fundamentals. It is the sensitivity of cones integrated in the eye, including all the filters such as the crystalline lens, the macular pigment and others, which absorb a fraction of the light entering the eye before it reaches the retina. Cone fundamentals should be linked to the color-matching functions through a linear relationship.

The objective measurement of the spectral sensitivity of the cones is recent. For many years, it has been accessible only by indirect experimental methods.

From the late 19th Century, by posing a few assumptions, it was possible to predict the fundamentals from color matches performed by normal and dichromat subjects. The assumptions were the absence of a family of cones among the dichromats, the need for obtaining positive cone spectral responses, and the likelihood of the shape of the absorption spectrum of photosensitive pigments.

The most accurate and complete approach is provided by colorimetry, where each monochromatic radiation is defined by its spectral tristimulus values . It remains to be seen whether the linear relationship maps these values to the spectral response of cone fundamentals.

1.3.1.3. The recommendations of the CIE

The International Commission on Illumination [COM 06] published tables giving the values of relative spectral sensitivity of the fundamentals L, M and S for a field of view of 10° of angular diameter and for a field of 2°, for an average young observer with normal color vision.

Figure 1.3.Experimental color-matching functions and fundamentals linked by a linear relationship

The recommendations incorporate the work of Stockman and Sharpe [STO 00] over several years.

For a field of view of 10° of visual angle, there is an area of diameter 10 cm seen , which is derived from color matches at 57 cm, the experiments performed on 49 observers by Stiles and Burch [STI 59]. These experimental color-matching functions obtained with real monochromatic primaries, red (645.2 nm), green (526.3 nm), and blue (444.4 nm), exhibit high quality. The fundamentals are obtained by linear transformation:

[1.7]

For a field of view of angular diameter 10°, the fundamentals exhibit a maximum of sensitivity at 568.6 nm, 541.3 nm and 447.9 nm.

The values of the color-matching function , tabulated by the CIE are slightly different from what would provide the linear transformation in equation [1.7], but the consequences for colorimetry are negligible. The tables are available on the Website http://www.cvrl.org/.

Unfortunately, for a field of view of angular diameter 2°, high-quality experimental data was not available. However, several physiological factors are altered with the eccentricity. It was decided to adopt a calculation scheme that enables us to predict the fundamentals consistent with known physiological and psychophysical data. The scheme consists of starting from fundamentals of a field of 10°, going up to the absorption of visual pigments included in the cones. This takes into account the absorption phenomena in several structures of the eye, the crystalline lens, the macular pigment and the photosensitive pigment density in the cones, then getting back through the opposite path in order to obtain the fundamentals for a field of view of 2° [COM 06, SCH 07].

Figure 1.4 shows the fundamental calculation scheme validated for a field of view of 2°, and this can be generalized to any field of view of angular diameter between 1 and 10°. The CIE gives all the numerical values and equations necessary for modeling the fundamentals.

Figure 1.4.Derivation procedure of cone fundamentals for a field of view of angular diameter 2°

1.3.1.4. Diversity of observers

About 8% of the male population manifests defects of color vision in various forms and degrees. Women are practically free from that. These subjects, called color-blinds confuse certain colors that most people see clearly. The defect is caused by the absence or modification of a family of photosensitive pigments.

Individual variations are twofold in normal color vision. The numerical variations of cone populations do not significantly affect color vision and color identification, but have an effect on the visibility and the relative brightness of colors. Spectral variations of fundamentals alter color vision. As part of the normal color vision underpinned by three families of cones, there are subfamilies of photosensitive pigments whose maximum spectral sensitivity are slightly offset in wavelength by a few nanometers. Transmission variations of ocular media can also alter the fundamentals.

Recently, according to the color matches from 47 subjects examined by Stiles and Burch, it was proposed to classify the actual observers in seven categories, each accepting color matches that other categories rejected [SAR 10]. These individual variations have an impact on the assessment of colors on novel LED type displays with narrow primaries and could explain discrepancies between individuals.

1.3.2. Application of LMS colorimetry

Many applications directly use the LMS specification of cone excitation. The LMS specification also opens interesting perspectives for analyzing signal processing in visual pathways: color discrimination and color appearance.

1.3.2.1. LMS specification

TheL, M, S specification of a stimulus φλ(λ), based on the amplitude of the excitation of three families of cones, is obtained by a calculation similar to that of tristimulus values X, Y, and Z:

[1.8]

1.3.2.2. The luminance in LMS colorimetry

1.3.2.3. Real color domain

All the colors ofthe real surfaces are included in aclosed volumeof the space, limited by the position of the optimal colors. In fact, it is physically impossible to achieve surfaces whose spectral reflection factor exceeds 100%. Once this limit is reached, the only way to increase the luminance factor Y of the surface is to broaden the reflection spectral band, which inevitably reduces the purity of color. In other words, a surface cannot be simultaneously very clear and highly saturated. The calculation shows that the optimal colors correspond to a set of reflectances with a square spectral profile and that they are represented by a convex surface in the color space.

1.3.2.4. The metamerism

As a physical quantity, light is defined by the distribution of monochromatic radiations within it. As a color stimulus, light is defined only by three numbers, related to the three signals generated in the cones. The visual system is unable to analyze the spectrum of light. This explains the metamerism or phenomenon of visual origin, through which two beams of light may appear while their spectra are different. The two lightbeams appear identical because the same amountofphotons is absorbed by the cones in both cases. This phenomenon is widely exploited by visualization technologies. For example, the apparent color

Figure 1.5.Optimal colors in isoluminant planes of the LMS color space. The tristimulus values L, M, S, were normalized as described in the discussion

of the clouds whose spectrum is naturally continuous is replicated by an additive mixture of primaries with discrete spectra.

With displays, yellow is obtained through an additive mixture of red and green. In the Rayleigh match [RAY 81], a metamer of the yellow radiation of 589 nm is obtained through an additive mixture of radiations of 670 nm and 546 nm. This is possible because the number of photons absorbed by the cones L, M or S is identical for the two metamers.

Let us pose the problem: What are the values of radiant flux 0(A) that provide a solution? We propose to solve the problem by using LMS colorimetry. The values will be rounded off to facilitate the demonstration.

Let us write the color equations:

For the Lcones:

[1.9]

For the M cones:

[1.10]

Figure 1.6.Rayleigh match

[1.11]

We have to solve the system of equations:

[1.12]

The solution gives the proportion of radiant flux at 670 nm and 546 nm that have to be added to get the yellow color of the radiation of589nm:

[1.13]

1.3.2.5. Simulation of the vision of colorblind people

Using LMS specification, it was possible to simulate what a dichromat who has no photosensitive pigment L (protanope case), M (deuteranope

Figure 1.7.Representation of colors confused by two types of dichromats in the LMS color space. The confusion lines are parallel to the axes L, M or S. The stimulus Qp is confused with Q by the protanope. The stimulus Qd is confused with Q by the deuteranope

case) or S (tritanope case) [BRE 97] sees. For a dichromat, all stimuli located in the LMS space along a line parallel to the axis representing the family of absent cones are confused. This is known as the confusion line (Figure 1.7). The color space for the dichromat is obtained by projecting parallel to that axis all the colors of the LMS space on the surface that carries apparently identical colors for him and for a normal subject.

1.3.2.6. The silent substitution

Due to their spectral overlap, it is impossible to stimulate a family of cones separately, even with a monochromatic radiation. One way to stimulate a single family of cones is to alternate in time two colors that are located on a parallel to the axis representing the excitation of this family of cones. Thus, the stimulation of the other two families of cones does not vary. They remain “silent”.

This technique can be extended to the stimulation of rods and melanopsin expressing ganglion cells.

1.3.2.7. Generalization of the LMS model

The LMS color model is based on a carefully established physiological basis. There is a linear relationship among the tristimulus values R, G, B, the tristimulus values X, Y, Z, and the signals L, M, S (Figure 1.8). LMS colorimetry, in future, could replace XYZ colorimetry each time the effect of a lightbeam on the visual system has to be monitored and analyzed. Pending recommendation of the CIE, we can adopt the transition matrix defined by Smith and Pokorny to transpose the tristimulus values of the XYZ space to the LMS space [SMI 75]. Factors kL and kM are included in the transformation such that the sum of the tristimulus values (L + M) gives the luminance Y directly. Rigorously, the transposition is applied to the tristimulus values corrected by Vos in 1978 [VOS 78]:

[1.14]

1.3.3. Color discrimination

1.3.3.1. The isoluminant chromaticity diagram from MacLeod-Boynton

Figure 1.8.Representation of the Q color and a monochromatic radiation E(λ) in the RGB, LMS, and XYZ color spaces. The figure is replicated in three copies to accentuate the three axes of each space, respectively. Within each space, the axes are independent and non-orthonormal. The representation in the plane of the figure makes it difficult to account for the 3D structure of each space

L cones and the vertical axis carries the variation of the excitation of the S cones alone, relative to luminance. So, by posing:

[1.15]

Any line of the L, M, or S space is projected along a line in the l, s chromaticity diagram. Specifically, the confusion lines of protanopes and deuteranopes are projected as straight beams that come from (1.0) and (0.0) respectively, and those of tritanopes as straight lines parallel to the s axis.

1.3.3.2. The orientation and size of the MacAdam ellipses

In the x, y chromaticity diagram, variations of size and orientation of MacAdam ellipses indicate the abilities of chromatic discrimination [MAC 42]. They reflect the activity of the two chromatic pathways [GRA 49]. The major ellipse axes that follow a beam of straight lines, which come from the lower left of the spectral locus, represent a variation of the excitation of S cones. The small axes are extended in directions

Figure 1.9.l,s isoluminant chromaticity diagram. The unity of the tristimulus value S is such that the s coordinate does not exceed the unity. White equi-energy, (black A). Protan and deutan confusion lines (—). MacAdam ellipses, arbitrary scale

representing the variations of excitation of L and M cones (Figure 1.9, inset). We can expect to obtain a uniform representation of small color differences by exploiting the retinal processes of post-synaptic encoding. Thus, in the chromaticity diagram from MacLeod and Boynton, the chromatic discrimination thresholds are organized in circles when they are obtained on a background of the same chromaticity and in regularly arranged ellipses when they are obtained on a gray background [KRA92] (Figure 1.9).

1.4. Colors in their context

Colorimetry specifies a lightbeam or a color stimulus presented in isolation. Once the context of the presentation (illumination or surround) changes or becomes complex as within an image, the appearance of the stimulus is changed. Therefore, the definition of the appearance of color is opposed to the specification of a single color.

1.4.1. CIECAM02

A color in isolation has several appearance attributes: hue, chroma or colorfulness, lightness or brightness. The hues form the color wheel within which four elementary hues have a special status and are opposed in pairs: red and green; blue and yellow. The CIECAM02 colorful appearance model described in Chapter 3 includes a color adaptation module, a light adaptation module, and a module for calculating the appearance attributes.

This is an empirical, operational model resulting from numerous visual observations. By introducing the nonlinearity of the signals that come from the cones, the CIECAM02 model applies to a range of light levels greater than those encountered in an office or on a screen. It predicts the appearance of an isolated color in various lighting conditions, weak or strong, artificial or natural. However, it does not reflect the complexity of the operation of the visual system. Thus, the specificity of S cone signals known in the earlier version of the Hunt model [HUN 95] is not taken into account. The phenomena of simultaneous contrast (described in Chapter 3) is not taken into account either.

1.4.2. Chromatic adaptation

When the lighting of a scene changes, either in luminance or in chromaticity, the appearance of color objects changes a little, as if the gain of the receptor mechanisms were adjusting to the ambient light. The chromatic adaptation mechanisms proposed in CIECAM02 have spectral sensitivity curves, which are fairly close to the fundamentals, and differ by a slightly sharpened bandwidth. Sensitivity peaks more distant from each other with slight negative sections. These signs indicate a slight antagonism activity in the chromatic adaptation, which occurs not only in cones, but also, to a lesser extent, at the level of retinal neurons in the parvo and koniocellular pathways.

1.4.3. Partitioning of the perceptual space by the elementary hues

The partitioning in the LMS space of the reddish colors and the greenish colors is neither flat nor continuous in the vicinity of the achromatic site. The same goes for the blue/yellow and white/black boundaries, which exclude a linear modeling of elementary hues from the cone signals. Furthermore, these boundaries are deformed when the test to be assessed appears on a colored background, which induces a chromatic adaptation. The model proposed by Chichilninsky and Wandell [CHI 99] requires a separation of the signals in increments and decrements. It includes a first step where the signals of the cones are distinguished according to their sign L+, L−, M+, M−, S+, S−, each undergoing a specific gain reflecting chromatic adaptation. In the second step where the signals of the cones are recombined linearly, the increments and decrements are again separated in pre-antagonist signals RG+, RG−, BY+, BY−, WB+, and WB−, before being recombined into antagonist signals.

1.4.3.1. The simultaneous contrast

The simultaneous contrast phenomenon (described in Chapter 3), according to which a surface changes its color depending on the background against which it is presented, has been widely studied. The law is stated as follows: Through their juxtaposition, two colors lose more or less of the color that is common to them ([CHE 39] Chapter IX). Therefore, a green-yellow turns green on a yellow background, and turns yellow on a green background. As opposed to the simplest assumptions, the colors induced by a background are not exactly complementary to the background, which points out the intervention of perceptual mechanisms beyond the receptors.

In the images, the contrast situations are very complex and the involved mechanisms are poorly known. Thus, in contrast to the perception of a natural scene in 3D [WEB 02], the color of a target is biased in the sense of simultaneous contrast in the digital image. The color contrasts could be treated independently, or even upstream of the color itself, which would explain our speed to decrypt the complexity of an image. To model the phenomena of color contrasts, it is necessary to introduce signal rectification processes after retinal processing (Shapiro, 2008) [SHA 08].

1.4.3.2. Association effect and assimilation

As opposed to the contrast, when the color of a surface tends to take the color of its surround, we talk of assimilation [KNO]. The phenomenon, easily observable with an environment of relatively high spatial frequency, results from association mechanisms whose explanation is still speculative. The white illusion in color, widely popularized, is dramatic [VIE 03].

1.5. Conclusion

This chapter gathers the physiological basis, colorimetric data and observations, which represent different approaches to color vision, and which reflect, each in their own way, the unique answer of the human visual system. Colorimetry provides a necessary yet insufficient answer on how to process an image. A color cannot be processed in isolation. The context, the surround, the conditions of adaptation and the expectation of the observer are the major parameters that significantly alter the appearance of the color. The quality of image processing comes from its compliance with the processes actually performed by the human visual system. Digital imaging can contribute to our understanding of the complexity of the HVS.

1.6. Bibliography

[BRE 97] BRETTEL H., VIÉNOT F., MOLLON J.D., “Computerized simulation of color appearance for dichromats”, Journal of the Optical Society of America A, vol. 14, no. 10, p. 2647–2655, OSA, 1997.

[CHE 39] CHEVREUL M., De la Loi du Contraste Simultané des Couleurs, Pitois-Levrault, Paris, 1839.

[CHI 99] CHICHILNISKY E.J., WANDELL B.A., “Trichromatic opponent color classification”, Vision Research, vol. 39, p. 3444–58, 1999.

[COM 06] COMMISSION INTERNATIONALE DE L'éCLAIRAGE, “Fundamental chromaticity diagram with physiological axes - part 1”, CIE, vol. 170-1-2006, 2006.

[DAR 83] DARTNALL H.J., BOWMAKER J.K., MOLLON J.D., “Human visual pigments: microspectrophotometric results from the eyes of seven persons”, Proc. R. Soc. Lond. B. Biol. Sci., vol. 220, no. 1218, p. 115–30, 1983.

[GEG 03] GEGENFURTNER K.R., KIPER D.C., “Color vision”, Annual Review of Neuroscience, vol. 26, no. 1, p. 181–206, 2003.

[GRA 49] GRAND Y.L., “Les seuils différentiels de couleurs dans la théorie de Young”, Revue d’optique, vol. 28, p. 261–278, 1949.

[HOF 05] HOFER H., SINGER B., WILLIAMS D., “Different sensations from cones with the same photopigment”, J. Vis., vol. 5, no. 5, p. 444–54, 2005.

[HUN 95] HUNT R., The Reproduction of Colour in Photography, Printing & Television: 5th ed., Fountain Press, Kingston-upon-Thames, UK, 1995.

[KNO] KNOBLAUCH K., SHEVELL S.K., CHALUPA L.M.,WERNER J.S. (eds.), The Visual Neurosciences, p. 892–907, MIT Press, Cambridge, USA.

[KRA 92] KRAUSKOPF J., GEGENFURTNER K.R., “Color discrimination and adaptation”, Vision Research, vol. 32, p. 2165–2175, 1992.

[LEE 99] LEE B., “Receptor inputs to primate ganglion cells”, GEGENFURTNER K., SHARPE L. (eds.), Color Vision: From Genes to Perception, p. 203–217, Cambridge University Press, NewYork, 1999.

[MAC 42] MACADAM D., “Visual sensitivities to color differences in daylight”, J. Opt. Soc. Am., vol. 32, no. 5, p. 247–273, OSA, 1942.

[MAC 79] MACLEOD D.I.A., BOYNTON R.M., “Chromaticity diagram showing cone excitation by stimuli of equal luminance”, J. Opt. Soc. Am., vol. 69, no. 8, p. 1183–1186, OSA, 1979.

[RAY 81] RAYLEIGH L., “Experiments on colour”, Nature, vol. 25, p. 64–66, 1881.

[ROO 99] ROORDA A., WILLIAMS D., “The arrangement of the three cone classes in the living human eye”, Nature, vol. 397(6719), p. 473–475, 1999.

[SAR 10] SARKAR A., BLONDé L., CALLET P.L., AUTRUSSEAU F., MORVAN P., STAUDER J., “Toward reducing observer metamerism issue in industrial applications: colorimetric observer categories and observer classification”, 18th Color Imaging Conference, Color Science and Engineering Systems, Technologies, and Applications, CIC18, San Antonio, Texas, USA, 2010.

[SCH 07] SCHANDA J., Colorimetry, Understanding the CIE system, Wiley and Sons, Hoboken, New Jersey, USA, 2007.

[SHA 08] SHAPIRO A.G., “Separating color from color contrast”, Journal of Vision, vol. 8, no. 1, p. 1–18, 2008.

[SHE 08] SHEVELL S., KINGDOM F., “Color in complex scenes”, Annual Review of Psychology, vol. 59, no. 1, p. 143–166, 2008.

[SMI 75] SMITH V., POKORNY J., “Spectral sensitivity of the foveal cone photopigments between 400 and 500 nm”, Vision Research, vol. 15, no. 2, p. 161–171, 1975.

[STI 59] STILES W., BURCH J., “NPL colour-matching investigation: final report (1958)”, Optica Acta, vol. 6, p. 1–26, 1959.

[STO 00] STOCKMAN A., SHARPE L.T., “The spectral sensitivities of the middle-and long-wavelength-sensitive cones derived from measurements in observers of known genotype”, Vision Research, vol. 40, no. 13, p. 1711–1737, 2000.

[VAL 08] VALBERG A., SEIM T., “Neural mechanisms of chromatic and achromatic vision”, Color Research Appl., vol. 33, p. 433–443, 2008.

[VAL 93] VALOIS R.D., VALOIS K.D., “A multi-stage color model”, Vision Research, vol. 33, no. 8, p. 1053–1065, 1993.

[VAL 00] VALOIS R.D., COTTARIS N., ELFAR S., MAHON L., WILSON J., “Some transformations of color information from lateral geniculate nucleus to striate cortex”, Proc. of the National Academy of Sciences of the USA, vol. 97, no. 9, p. 4997–5002, 2000.

[VIE 03] VIENOT F., ROHELLEC J.L., “Jeux de couleurs”, Pour la Science, Dossier hors-série, p. 20–27, 2003.

[VOS 78] VOS J ., “Colorimetric and photometric properties of a 2° fundamental observer”, Color Research and Application, vol. 3, p. 125–128, 1978.

[WEB 02] WEBSTER M.A., MALKOC G., BILSON A.C., WEBSTER S.M., “Color contrast and contextual influences on color appearance.”, Journal of Vision, vol. 2, no. 6, p. 505–519, 2002.

[ZAI 97] ZAIDI Q., “Decorrelation of L and M cone signals”, J. Opt. Soc. Am., vol. A14, p. 3430–3431, 1997.

1 Chapter written by Françoise VIÉNOT and Jean LE ROHELLEC.

Chapter 2

Color Constancy1

2.1. Introduction

In Chapter 1