Digital Image Interpolation in Matlab E-Book

Chi-Wah Kok

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley - IEEE
• Sprache: Englisch

Seitenzahl: 563

Veröffentlichungsjahr: 2018

Beschreibung

This book provides a comprehensive study in digital image interpolation with theoretical, analytical and Matlab® implementation. It includes all historically and practically important interpolation algorithms, accompanied with Matlab® source code on a website, which will assist readers to learn and understand the implementation details of each presented interpolation algorithm. Furthermore, sections in fundamental signal processing theories and image quality models are also included. The authors intend for the book to help readers develop a thorough consideration of the design of image interpolation algorithms and applications for their future research in the field of digital image processing.

• Introduces a wide range of traditional and advanced image interpolation methods concisely and provides thorough treatment of theoretical foundations
• Discusses in detail the assumptions and limitations of presented algorithms
• Investigates a variety of interpolation and implementation methods including transform domain, edge-directed, wavelet and scale-space, and fractal based methods
• Features simulation results for comparative analysis, summaries and computational and analytical exercises at the end of each chapter

Digital Image Interpolation in Matlab® is an excellent guide for researchers and engineers working in digital imaging and digital video technologies. Graduate students studying digital image processing will also benefit from this practical reference text. 

Leseprobe

Table of Contents

Cover

Dedication

About the Authors

Preface

Acknowledgments

Nomenclature

Abbreviations

About the Companion Website

1 Signal Sampling

1.1 Sampling and Bandlimited Signal

1.2 Unitary Transform

1.3 Quantization

1.4 Sampled Function Approximation: Fitting and Interpolation

1.5 Book Organization

1.6 Exercises

2 Digital Image

2.1 Digital Imaging in MATLAB

2.2 Current Pixel and Neighboring Pixels

2.3 Frequency Domain

2.4 2D Filtering

2.5 Edge Extraction

2.6 Geometric Transformation

2.7 Resize an Image

2.8 Color Image

2.9 Noise

2.10 Summary

2.11 Exercises

3 Image Quality

3.1 Image Features and Artifacts

3.2 Objective Quality Measure

3.3 Structural Similarity

3.4 Summary

3.5 Exercises

4 Nonadaptive Interpolation

4.1 Image Interpolation: Overture

4.2 Frequency Domain Analysis

4.3 Mystery of Order

4.4 Application: Affine Transformation

4.5 Summary

4.6 Exercises

5 Transform Domain

5.1 DFT Zero Padding Interpolation

5.2 Discrete Cosine Transform

5.3 DCT Zero Padding Image Interpolation

5.4 Overlapping

5.5 Multi‐Kernels

5.6 Iterative Error Correction

5.7 Summary

5.8 Exercises

6 Wavelet

6.1 Wavelet Analysis

6.2 Wavelet Image Interpolation

6.3 Cycle Spinning

6.4 Error Correction

6.5 Which Wavelets to Use

6.6 Summary

6.7 Exercises

7 Edge‐Directed Interpolation

7.1 Explicit Edge‐Directed Interpolation

7.2 Implicit Edge‐Directed Interpolation

7.3 Summary

7.4 Exercises

8 Covariance‐Based Interpolation

8.1 Modeling of Image Features

8.2 Interpolation by Autoregression

8.3 New Edge‐Directed Interpolation (NEDI)

8.4 Boundary Extension

8.5 Threshold Selection

8.6 Error Propagation Mitigation

8.7 Covariance Window Adaptation

8.8 Iterative Covariance Correction

8.9 Summary

8.10 Exercises

9 Partitioned Fractal Interpolation

9.1 Iterated Function System

9.2 Partitioned Iterative Function System

9.3 Encoding

9.4 Decoding

9.5 Decoding with Interpolation

9.6 Overlapping

9.7 Summary

9.8 Exercises

AppendixMATLAB Functions List

Bibliography

Index

End User License Agreement

List of Tables

Chapter 02

Table 2.1 Effect of the scaling parameters.

Chapter 03

Table 3.1 MSSIM value of Figure 3.5 a with different

and

values.

Chapter 04

Table 4.1 PSNR and SSIM of

Cat

image down‐sampled to

and rotated 9 times with angle

.

Chapter 08

Table 8.1 The PSNR and SSIM performance of a

Cat

image interpolation to size

using the NEDI and periodic extension with various variance threshold values.

Table 8.2 The PSNR and SSIM performance of a

Cat

image interpolation to size

using the MEDI and periodic extension with various variance threshold values.

Table 8.3 The PSNR and SSIM performance of a

Cat

image interpolation to size

using the EMEDI and periodic extension with various variance threshold values.

Table 8.4 The PSNR and SSIM performance of a

Cat

image interpolation to size

using the iMEDI and periodic extension with various variance threshold values

Chapter 09

Table 9.1 Approximation of

by Newton's method.

Table 9.2 Fractal image representation of the

Cat

image with range block block sizes of

and

, respectively, running

encodeFractal

with MATLAB R2016b on MS Windows 7 PC with Quad Core Xeon E5520 at 2.27 GHz and 8 GB RAM.

Table 9.3

Cat

image reconstructed from fractal image representation using range block block sizes of

and

, respectively, with blank initial reference image running

decodeFractal

with MATLAB R2016b on MS Windows 7 PC with Quad Core Xeon E5520 at 2.27 GHz and 8 GB RAM.

List of Illustrations

Chapter 01

Figure 1.1 (a) Spectrum of a bandlimited signal

with bandwidth

; (b) sampled with rate

with

can be recovered with a sinc filter with bandwidth

.

Figure 1.2 (a) Spectrum of a bandlimited signal

with bandwidth

; (b) sampled with rate

with

will suffer from spectrum overlap error, also known as aliasing noise, which makes it difficult to be recovered by a sinc filter with bandwidth

.

Figure 1.3 Sampling and quantization of a one‐dimensional continuous signal.

Figure 1.4 Signal sampling with different quantization bit depths.

Figure 1.5 Effects of sampling rate and number of quantization bit on the sampling quality of a one‐dimensional continuous signal: (a) a damped cosine function sampled with a particular sampling rate and number of quantization bit, (b) the same function sampled with doubled sampling rate but the same number of quantization bit length, and (c) the same function with the same sampling rate as that in (b) and increased number of quantization bit by one.

Figure 1.6 Reconstructing the analog signal from its samples through

interpolation

(solid line) and

fitting

by

least squares approximation

(dashed line).

Figure 1.7 Illustration of (a) impulse response of a zero‐order hold (ZOH) filter

; (b) the sampled signal

reconstructed using the ZOH filter

; (c) impulse response of a first‐order hold (FOH) filter

; (d) the reconstructed signal filtered by

.

Figure 1.8 Time and spectral responses for various reconstruction filters: (a) Sinc filter; (b) Zero‐order hold; and (c) First‐order hold.

Figure 1.9 Book organization.

Chapter 02

Figure 2.1 Illustration of capturing an image by digital camera. (

See insert for color representation of this figure

.)

Figure 2.2 Representation of the digital image

Cat

: (a) a grayscale printout of

Cat

, which is described by an

2D array within the computation system with each matrix element representing the intensity of a pixel taking a value in the quantizer (in this case, it is [0, 255] for

Cat

is a 8‐bit quantized image); (b) a pixel intensity map of the selected region in the image, where the pixel intensity at [101, 201] is 152; and the intensity variation across the complete image by viewing (c) the 2D vector mesh of the image on a plane with the height (

‐axis) being the pixel intensity (note the direction of the

‐axis is inverted to display the

Cat

more vividly) or through (d) the contour map, where the pixel with same intensities are located on the same contour lines.

Figure 2.3 Synthetic grayscale image: Letter

A

.

Figure 2.4 Current pixels and its neighborhood.

Figure 2.5 Calculating a 2D DFT by row–column technique; (a) Original image; (b) DFT of each row of (a); and (c) DFT of each column of (b).

Figure 2.6 (a) The four quadrants of the original spectrum of a digital image obtained by 2D DFT. (b) The centered spectrum obtained through swapping quadrants.

Figure 2.7 (a) A

portion of the letter

and (b) the associated 2D DFT spatial frequency magnitude.

Figure 2.8 Illustration of 2D image convolution.

Figure 2.9 Boundary extensions applied to finite length 2D image (a portion of the letter

A

in Figure 2.3 in (a)); (b) zero padding; (c) periodic extension; (d) symmetric extension; and (e) constant extension; with the sub‐image enclosed by the box being the original image shown in (a).

Figure 2.10 Cropping to avoid undefined boundary signal in 2D filtering.

Figure 2.11 The edge information contained in the block of the

Cat

image.

Figure 2.12 Two types of grayscale image brightness variations (a) and (c) and the corresponding 1D plots in (b) and (d), respectively.

Figure 2.13 The vertical derivative operator,

, and horizontal derivative operator,

, of (a) Robert operator, (b) Prewitt operator, and (c) Sobel operator.

Figure 2.14 The edge maps of

Cat

extracted by Sobel filtering at different thresholds: (a)

threshold

 = 80, (b)

threshold

 = 100, and (c)

threshold

 = 120.

Figure 2.15 Comparison of first‐order and second‐order derivative edge detection: (a) edge normal, (b) first‐order derivation, and (c) second‐order derivation.

Figure 2.16 Smoothing filters: (a) a

Laplacian of Gaussian (LoG) filter with

and (b) a

Gaussian filter with

.

Figure 2.17 The edge of

Cat

extracted by Laplacian of Gaussian (LoG) filter with different

threshold

and

: (a) at zero‐crossing points with

 = 0.8, (b)

threshold

 = 0.03 and

 = 0.8, (c)

threshold

 = 0.03 and

 = 1, and (d)

threshold

 = 0.03 and

 = 1.2.

Figure 2.18 Geometric transformation: (a) translation and (b) reflection.

Figure 2.19 Geometric transformation: scaling.

Figure 2.20 Geometric transformation: rotation.

Figure 2.21 (a) 1D interpolation estimates a point on a line. (b) 2D interpolation estimates a point on a surface.

Figure 2.22 Example illustrates image enlargement by interpolation: (a) the original

image, (b) the enlarged image with spatial resolution of

, (c) the zoom‐in of the enclosed region in (a), (d) unknown pixels are added before interpolation, and (e) the zoom‐in of the enclosed region in the enlarged image in (b).

Figure 2.23 Image decimation by filtering.

Figure 2.24 Sinc function filter.

Figure 2.25 Block averaging filter.

Figure 2.26 The effects of direct and block averaging down‐sampling on image reconstruction through interpolation. (a) original image block; (b) down‐sampled image blocks obtained from direct and averaging down‐samplings; (c) image blocks obtained by interpolation using nearest neighbor and linear interpolation methods.

Figure 2.27 Three separate RED, GREEN, and BLUE channels are combined to create a final, full‐color image. (

See insert for color representation of this figure

.)

Figure 2.28 Red, green, and blue samples obtained from

Bayer pattern

color filter. (

See insert for color representation of this figure

.)

Figure 2.29 Image noise and denoising: (a) impulse noise‐corrupted (also known as “Salt & Pepper Noise”)

Cat

image with SNR = 4.18 dB, (b) additive Gaussian noise‐corrupted

Cat

image with SNR = 4.18 dB, (c) denoised image (a) recovered by median filter of window size of

, (d) denoised image (a) recovered by Gaussian smoothing filter of size

and

(refer to Section 2.5.2.2 ), (e) denoised image (b) recovered by median filter of window size

, and (f) denoised image (b) recovered by Gaussian smoothing filter of size

and

, where the filters used for denoising can be referred to Section 2.5.2.2 .

Figure 2.30 The average spectral power at radian frequency

of the

Cat

image and additive Gaussian noise. At radian frequency

and larger, the noise power will dominate spectrum.

Chapter 03

Figure 3.1 A natural image

Cat

showing three basic image features: homogeneous area, texture area, and edges.

Figure 3.2 Image interpolation artifacts of the synthetic image letter

A

demonstrating (a) aliasing (jaggy), (b) blurring, and (c) edge halo and ringing.

Figure 3.3 Blurring effect of linear interpolation in one‐dimensional case: (a) original high‐resolution data points, (b) low‐resolution data obtained by subsampling, and (c) recovered data by linear interpolation.

Figure 3.4 Image interpolation quality computation.

Figure 3.5 The

Cat

image is down‐sampled by a scaling factor of 2 and then restored to its original size by the same scaling factor by different algorithms to produce (a) and (b) with the PSNR of both images close to 23.04 dB.

Figure 3.6 The sensitivity of SSIM toward

with varying

at different

(the solid lines) and the sensitivity of SSIM toward

with varying

at different

(the dashed lines). (

See insert for color representation of this figure

.)

Figure 3.7 The sensitivity of SSIM toward

with varying

(solid lines) and the sensitivity of SSIM toward

with varying

(dashed lines).

Chapter 04

Figure 4.1 General framework of image interpolation.

Figure 4.2 The nearest neighbor interpolation kernel in spatial domain.

Figure 4.3 A

image block interpolated by nearest neighbor method with interpolation ratio of 2 to achieve a

interpolated image.

Figure 4.4 Nearest neighbor interpolation: interpolated image of “letter

” and intensity maps of the enclosed diagonal and vertical edges.

Figure 4.5 Nearest neighbor interpolation of natural image

Cat

by a factor of 2 (PSNR = 25.98 dB): (a) the full interpolated image, (b) zoom‐in portion of cat's whiskers in original image, and (c) zoom‐in portion of cat's whiskers in interpolated image.

Figure 4.6 The plot of (a) linear interpolation kernel and (b) bilinear interpolation kernel in spatial domain.

Figure 4.7 Spatial weighting map of the bilinear interpolation for the pixel

located at

.

Figure 4.8 Bilinear interpolation: interpolated image of “letter

” and intensity map of the enclosed diagonal and vertical edges.

Figure 4.9 Bilinear interpolation of natural image

Cat

by a factor of 2 (PSNR = 28.39 dB): (a) the full interpolated image, (b) zoom‐in portion of cat's whiskers in original image, (c) zoom‐in portion of cat's whiskers in interpolated image, (d) zoom‐in portion of cat's ear in original image, and (e) zoom‐in portion of cat's ear in interpolated image.

Figure 4.10 Bicubic interpolation: estimation of unknown pixels lying on the exact surface defined by the neighboring 16 pixels. (

For color interpretation please refer the digital version of this figure

.)

Figure 4.11 Basic function of cubic convolution.

Figure 4.12 The 2D bicubic interpolation kernel in spatial domain.

Figure 4.13 Neighboring pixels of bilinear and bicubic interpolation.

Figure 4.14 Bicubic interpolation: interpolated image of “letter

” and intensity maps of the enclosed diagonal and vertical edges.

Figure 4.15 Bicubic interpolation of natural image

Cat

by a factor of 2 (PSNR = 24.9 dB): (a) the full interpolated image, (b) zoom‐in portion of

Cat

's whiskers in original image, (c) zoom‐in portion of

Cat

's whiskers in interpolated image, (d) zoom‐in portion of

Cat

's ear in original image, and (e) zoom‐in portion of

Cat

's ear in interpolated image.

Figure 4.16 The spatial (a) and frequency response (b) of the nearest neighbor, bilinear, and bicubic interpolation kernels. (

See insert for color representation of this figure

.)

Figure 4.17 A visual representation of a situation where linear interpolation is superior to higher‐order interpolating polynomials. The function to be fit (an edge) undergoes an abrupt change at

. Parts (a)–(c) indicate that the abrupt change induces oscillations in interpolating polynomials. In contrast, because it is limited to third‐order curves with smooth transitions, a linear interpolation (d) provides a much more acceptable approximation.

Figure 4.18 Illustration of different spline functions: (a) first‐order spline, (b) second‐order spline, and (c) cubic spline.

Figure 4.19 Example of affine transformation: (a) input image and (b) transformed image.

Figure 4.20 Image rotation through pixel mapping from original coordinate grid to a rotated grid: (a) rotated

Cat

image, (b) missing pixel after rotation, and (c) illustration of pixel mapping in image rotation.

Figure 4.21

Cat

image rotated with

obtained from (a) nearest neighbor interpolation by

rotatenn(f,a)

with

f

being the

Cat

image matrix and

and (b) bilinear interpolation by

rotatebi(f,a)

.

Figure 4.22 Image interpolation algorithm structural integrity evaluation by considering the image rotated 9 times with angle

for (a) nearest neighbor, (b) bilinear, and (c) bicubic.

Chapter 05

Figure 5.1 Spectra of a bandlimited signal

with bandwidth

sampled at sampling rate of (a)

, (b)

, and (c)

.

Figure 5.2 The interpolation kernel

in Eq. ( 5.14 ) with

is plotted using solid line, while the sinc function sinc

is plotted with dashed line.

Figure 5.3 The spectral representation of the natural image

Cat

: (a) the original image and (b) the zero‐padded spectral representation of a

image interpolation. (

Note

:

fftshift

is applied to obtain this spectral plot. Readers may want to review Example 2.1.)

Figure 5.4 A 2

interpolated

Cat

image obtained by zero‐padded DFT (PSNR = 26.9 dB, SSIM = 0.8053 dB): (a) the full interpolated image, (b) the zoom‐in portion of cat's whiskers in original image, and (c) zoom‐in portion of cat's whiskers in interpolated image. Note that the heavy oscillations (ringing) around the cat's whiskers shown in the zoom‐in image (c) are the result of

Gibbs phenomenon

.

Figure 5.5 Illustration of heavy oscillations (ringing) at the borders of the DFT zero‐padded interpolated signal: (a) the original ramp signal with 32 uniformly sampled samples and (b) the 8

interpolated ramp signal.

Figure 5.6 The 8

interpolated signal of the ramp signal shown in Figure 5.5 a by zero‐padded DCT. Note the missing high frequency oscillations at the borders in the interpolated ramp signal compared with the one shown in Figure 5.5 b.

Figure 5.7 (a) Subdividing the cat image into non‐overlapping

blocks. (b) Plots of the DCT coefficients of each

transformation.

Figure 5.8 Framework of the zero‐padded DCT interpolation of an image: (a) subdividing the

Cat

image into

blocks, (b) applying 2D DCT to each of the

block, (c) zero‐padded 2D DCT spectral image each block from size of

to form the interpolated image in spectral domain, and (d) converting the spectral image in (c) back into a 2

interpolated image in spatial domain by inverse DCT with block size

.

Figure 5.9 (a) Image doubling by zero padding

and (b) with

DCT block

Cat

image; the zoom‐in sub‐images are displayed in the middle together with that from the original high‐resolution image.

Figure 5.10 Illustration of different block partition methods: (a) the original “non‐shifted” image

, (b) the “horizontal shifted” image

, (c) the “vertical shifted” image

, and (d) the “diagonal shifted” image

.

Figure 5.11 The zoom‐in portion of the eye of the

Cat

image extracted from the zero‐padded DCT interpolated images

gF0p

,

gF1p

,

gF2p

, and

gF3p

, where the same region is studied in Figure 5.9 : (a) the non‐shifted interpolated image

gF0p

, (b) the horizontally shifted interpolated image

gF1p

, (c) the vertically shifted interpolated image

gF2p

, and (d) the diagonally shifted interpolated image

gF3p

.

Figure 5.12 Interpolated image obtained from averaging the zero‐padded

block DCT interpolation with non‐shift and vertically shifted (by four pixels) images: (a) the interpolated image with averaging, (b) the zoom‐in image of the interpolated image obtained from the interpolated image of “non‐shifted” image, and (c) the zoom‐in image of the interpolated image with averaging.

Figure 5.13 Framework of DCT zero padding image interpolation with mean filter applied to multiple overlap images.

Figure 5.14 (a) Interpolated image obtained from averaging the DCT zero‐padded

block DCT interpolation between non‐shifted, vertically shifted, horizontally shifted, and diagonally shifted (by four pixels) images: (a) the interpolated image, (b) the zoom‐in image of the selected region taken from the interpolated image of “non‐shifted” image, and (c) the same zoom‐in image of the selected region taken from the interpolated image shown in (a).

Figure 5.15 (a) A

obtained by averaging the zero‐padded DCT interpolated image and EIDCT interpolated image with block size

in both interpolation methods. (b) The zoom‐in image of the portion enclosed in (a). (c) The zoom‐in image obtained from the portion enclosed in (a) with interpolated image obtained from zero‐padded DCT and EIDCT using

block size.

Figure 5.16 Gradually fitting a metal sheet to a given surface.

Figure 5.17 Error compensation signal flow block diagram.

Figure 5.18 Zero‐padded DCT interpolation with block size

and error compensation at the (a) 0th iteration (PSNR = 26.0916 dB and mean absolute difference with low‐resolution image 6.4421). (b) 1st iteration (PSNR = 25.4664 dB and mean absolute difference with low‐resolution image 6.5636). (c) 46th iteration (PSNR = 25.9732 dB and mean absolute difference with low‐resolution image 5.6423).

Chapter 06

Figure 6.1 Illustration of 1D DWT decomposition. The bandwidth of the resulting signal is marked with “BW.”

Figure 6.2 Spectral response of a typical perfect reconstruction two‐channel subband filter bank.

Figure 6.3 Multi‐resolution space representation.

Figure 6.4 Illustration of 2D wavelet decomposition of the

Cat

image. (a) Filter bank structure for 2D wavelet image transformation, (b) the corresponding second‐level wavelet decomposition, and (c) the subband stitched together to show the invariance of the overall spatial size, (d) second‐level dyadic decomposition, with (e) the subband images from first‐ and second‐level decomposition stitched together (the subband images look darker because brightness scaling has been applied to squeeze the dynamic range of all subband images to be displayed in the same figure).

Figure 6.5 Image interpolation by wavelet coefficient estimation.

Figure 6.6

Cat

image interpolated with Haar wavelet zero padding and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the original high‐resolution

Cat

image.

Figure 6.7 (a)

Cat

image interpolated with Haar wavelet with high frequency subband images obtained from bilinear interpolation of lower‐level high frequency subband images and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the interpolated image obtained by Haar wavelet zero padding.

Figure 6.8 Wavelet zero padding image interpolation with alternate zero wavelet coefficient insertion.

Figure 6.9 (a)

Cat

image interpolated with alternate zero padding wavelet interpolation and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the original high‐resolution

Cat

image.

Figure 6.10 (a)

Cat

image interpolated with alternate zero padding wavelet interpolation and subband images divided by 1.5 and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the interpolated image obtained by alternate zero padding wavelet without scaling the subband images.

Figure 6.11 (a)

Cat

image interpolated with alternate zero padding with regularity‐preserving subband scaling wavelet image interpolation and subband images scaled to satisfy across scale subband image energy ratio and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the original high‐resolution

Cat

image.

Figure 6.12 WZP‐CS image interpolation signal flow diagram.

Figure 6.13 (a)

Cat

image interpolated with wavelet zero padding cyclic spinning image interpolation with intermediate image generated by WRAZP image interpolation through MATLAB function

wzpcs

and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the original high‐resolution

Cat

image.

Figure 6.14 Wavelet low‐resolution image cyclic spinning image interpolation signal flow diagram.

Figure 6.15 (a)

Cat

image interpolated with wavelet low‐resolution subband cyclic spinning with intermediate image generated by

wrazp

image interpolation through MATLAB function

wlrcs

and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the high‐resolution

Cat

image obtained from

wzpcs

.

Figure 6.16 (a) Interpolated

Cat

image obtained by averaging

wzpcs

and

wlrcs

and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the original high‐resolution

Cat

image.

Figure 6.17 Image interpolation by wavelet multi‐resolution synthesis with error correction. (

Note

: WI can be any wavelet interpolation methods except

wzp

.)

Figure 6.18 (a) Interpolated

Cat

image obtained by error correcting (

wecor

) the (

wlrcs

) wavelet low‐resolution image cyclic spinning high‐resolution image using intermediate image generated by WRAZP image interpolation (

wlrcs

) and the zoom‐in sub‐images of the

Cat

whiskers in (c) and ear in (e), while shown in (b) and (d) are the sub‐images of the same area extracted from the original high‐resolution

Cat

image.

Figure 6.19 Laplacian pyramid transform of the

Cat

.

Figure 6.20 Example of reordering of DCT coefficient to form multi‐resolution image representation. (

See insert for color representation of this figure

.)

Chapter 07

Figure 7.1 Framework of general explicit edge‐directed interpolation [3 ].

Figure 7.2 The center‐on‐surround‐off filter and Laplacian‐of‐Gaussian filter.

Figure 7.3 Lattices for edge‐directed interpolation.

Figure 7.4 Estimation errors aroused by edge map.

Figure 7.5 Illustration of Canny edge interpolation (CEI): (a) high‐resolution image block obtained from up‐sampling a low‐resolution image block (the dark pixels are the original low‐resolution pixels, and the gray pixels are newly added pixels with intensity estimated by bilinear interpolation) and (b) post‐processing on edge pixels (white pixels): replacement of pixel intensity according to the edge orientation.

Figure 7.6 The

training window of ELA method.

Figure 7.7 The sliding training window in directional‐orientation interpolation (DOI), where the training window size is

and the search range is

.

Figure 7.8 1D error‐amended sharp edge (EASE).

Figure 7.9 2D error‐amended sharp edge (EASE).

Figure 7.10

interpolated image of “letter A” using EASE and the associated intensity maps of the enclosed diagonal and vertical edges.

Figure 7.11

interpolated

Cat

image by EASE with PSNR = 28.16 dB and SSIM = 0.8429 and the associated zoom‐in images of the whiskers of the

Cat

image taken from (b) the original image, (c) the bicubic interpolated image, and (d) the EASE interpolated image.

Chapter 08

Figure 8.1 3D visualization of image containing edge: (a) original image with a selected portion under illustration, (b) the zoom‐in 3D map, and (c) the intensity plot of the selected portion.

Figure 8.2 Coefficients of linear prediction.

Figure 8.3 Distribution of pixels in up‐sampled natural image

Cat

obtained by covariance based interpolation: (a) by considering full‐ranked

, (b) by considering both full‐ranked

and

with

and (c) by considering both full‐ranked

and

with

.

Figure 8.4 Three types of to‐be‐interpolated pixels in NEDI after resampling.

Figure 8.5 Illustration of pixels considered in MATLAB program of NEDI.

Figure 8.6 The rotated prediction windows for the estimation of unknown pixels at

in NEDI.

Figure 8.7 The rotated prediction windows for the estimation of unknown pixels at

in NEDI.

Figure 8.8 Interpolation of synthetic image letter

A

by different interpolation methods: (a) NEDI interpolated image without boundary extension, where the intensity map of the region enclosed by the box will be investigated; the intensity map of the selected region in (b) the original image; (c) the NEDI interpolated image; (d) the bilinear interpolated image; and (e) the zero‐padded wavelet interpolated image.

Figure 8.9 Boundary extension in NEDI: (a) original

Cat

image with extended boundary (i.e. in MATLAB

fext

), (b) the NEDI interpolated image showing the extended boundary, and (c) the NEDI interpolated image with the boundary pixels being removed.

Figure 8.10 NEDI of natural image

Cat

by a factor of 2 and with threshold

(PSNR = 27.6027 dB, SSIM = 0.8248): (a) the full interpolated image, (b) zoom‐in portion of the ear of the

Cat

image in original image, (c) zoom‐in portion of the ear of the

Cat

image in the bilinear interpolated image, and (d) zoom‐in portion of the ear of the

Cat

image in the NEDI interpolated image.

Figure 8.11 The zoom‐in portion of the up‐sampled

Cat

image taken from the forehead of the

Cat

image, in which the image is interpolated by NEDI using different thresholds

and boundary width of 8 pixels: (a)

; (b)

; and (c)

. The corresponding distribution showing pixels interpolated by covariance‐based interpolation is shown in the same column as that of the zoom‐in image for comparison, where the white pixels in the distribution maps representing the pixels are interpolated by covariance‐based interpolation.

Figure 8.12 Illustration of the covariance window and local block of the second step of the MEDI method.

Figure 8.13 A

MEDI interpolated

Cat

image with periodic boundary extension at

and

(PSNR = 27.8014 dB, SSIM = 0.8148). (a) The interpolated image, (b) zoom‐in portion of the ear of the original high‐resolution

Cat

image, (c) zoom‐in portion of the ear of the NEDI interpolated

Cat

image, and (d) zoom‐in portion of the ear of the MEDI interpolated

Cat

image.

Figure 8.14 The zoom‐in portion of the ear of the MEDI interpolated

Cat

image subject to different thresholds, namely, (a)

, (b)

, and (c)

, where the boundary extension in all images is

.

Figure 8.15 Illustration of covariance mismatch between prediction window and local covariance window corresponding to the gray pixel in (a); (b) prediction window contains Edge 1 only; (c) mean covariance window contains both Edge 1 and Edge 2.

Figure 8.16 Directional adaptive prediction windows with elongation along the edge direction but reducing the width in perpendicular to the edge, where the edge are oriented in

,

,

, and

[68 ].

Figure 8.17 Directional adaptive prediction windows that are spatially “wide” in shape: (a) original prediction window and (b)–(e) the modified arrow‐shaped prediction windows in different orientations but in the same size [68 ].

Figure 8.18 Windows of iNEDI [5 ], where the solid line windows are the low‐resolution window and the dashed line windows is the mean covariance window.

Figure 8.19 Illustration of different placements of covariance estimation window with respect to a fixed prediction window: (a) the placement adopted by the NEDI and (b)–(e) the other four placement variations.

Figure 8.20 Illustration of the spatial adaptation of the five mean covariance windows adopted in the EMEDI for type 0 pixel. Noted the window

winnum=0

is same as that in the original NEDI. (

See insert for color representation of this figure

.)

Figure 8.21 A

EMEDI interpolated

Cat

image with variance threshold set at

and with the application of periodic boundary extension (PSNR = 27.8396 dB, SSIM = 0.8147). (a) The interpolated image, (b) zoom‐in portion of the ear of the NEDI interpolated

Cat

image, (c) zoom‐in portion of the ear of the MEDI interpolated

Cat

image, and (d) zoom‐in portion of the ear of the EMEDI interpolated

Cat

image.

Figure 8.22 Illustration of the examples of the biased covariance windows for high‐ and low‐resolution image pixels.

Figure 8.23 iMEDI interpolation of the

Cat

image by a factor of 2 and with threshold

(PSNR = 27.9552 dB, SSIM = 0.8201) with five iterations: (a) the interpolated image, (b) zoom‐in portion of the ear of the NEDI interpolated

Cat

image, (c) the EMEDI interpolated

Cat

image, and (d) the sixth iteration iMEDI interpolated

Cat

image.

Figure 8.24 Covariance energy difference maps of the iMEDI interpolated

Cat

image at (a) the third iteration, (b) the fifth iteration, and (c) the sixth iteration.

Figure 8.25 The iMEDI interpolation suppresses the ringing artifact when compared with that of EMEDI: (a) the EMEDI interpolated

Cat

image and (d) the iMEDI interpolated

Cat

image, where the same scaling factor of 2 and threshold

are applied. (b) and (c) are the corresponding zoom‐in portion of the hairs of both interpolated images.

Chapter 09

Figure 9.1 Three different range block partition methods for the

Cat

image with the edges of the

Cat

overlay the partitions ‐ (a) uniform, (b) quadtree, and (c) HV partition – to show their relationships.

Figure 9.2 The seven different isometries – affine transformations.

Figure 9.3 Generation of

codebook

that contains the four rotation affine transform down‐sampled domain blocks extracted from the original image. (

See insert for color representation of this figure

.)

Figure 9.4 Fractal encoding by matching range blocks with grayscale transformed domain blocks from the codebook.

Figure 9.5

Cat

image of size

encoded with

range block fractal and decoded under different iterations with constant initial image being a 0 matrix: (a) the third iteration, (b) the fifth iteration, (c) the seventh iteration, and (d) the twentieth iteration.

Figure 9.6 Comparison of

Cat

image of size

generated by (a) fractal decoding with 20 iterations and

blocksize=8

; (b) 8 × 8 DCT that retain the low frequency 4 × 4 DCT coefficients and zero padded all other high frequency coefficients.

Figure 9.7 Comparison of fractal decoded

Cat

image of size

obtained from the same fractal codebook but with different initial images at the twentieth iteration, where the same block size of

is applied in the decoding: (a) initial image with all pixel intensity at 0 and (b) the original

Cat

image as initial image, where both the decoded images have the same PSNR and SSIM at 26.54 dB and 0.726, respectively.

Figure 9.8

Cat

image of size

with (a)

range block fractal encoding and decoded with 30 iterations and a 0 matrix initial image, and the associated zoom‐in image block of the whiskers of the

Cat

, and (b) the same block from

iterations fractal decoded image with

range block.

Figure 9.9 Fractal zoom‐in

Cat

image in (a) same size as that of original image (i.e.

) and (b) an enlargement to the size of

, with both images decoded with identical range block block size of

, dark image as initial image, and the number of iteration is 20 (PSNR = 25.2715 dB, SSIM = 0.6290).

Figure 9.10 Fractal interpolated

Cat

image by a factor of 2 to

at the fifteenth iteration with range block block size of

interleaved with low‐resolution

Cat

image pixel layer (PSNR = 26.41 dB, SSIM = 0.7231): (a) the full interpolated image; zoom‐in portion of the whiskers of (b) the original

Cat

image, (d) the interpolated image by

interpFractal

, and (f) the interpolated image by

exactinterpFractal

; zoom‐in portion of the eye of (c) the original

Cat

image, (e) the interpolated image by

interpFractal

, and (g) the interpolated image by

exactinterpFractal

.

Figure 9.11

interpolated

Cat

image by mean filtering a set of four cyclic spin fractal decoding images (original plus three other shifted images) with doubled range block block size (i.e.

) and interleaved low‐resolution image pixels using

exactinterpFractal

function (PSNR = 27.39 dB, SSIM = 0.7662): (a) the full interpolated image; zoom‐in portion of the whiskers of the

Cat

image obtained from (b)

exactinterpFractal

without mean filtering (same as Figure 9.10 c) and (c) mean filtering shifted fractal decoded images; zoom‐in portion of the eye of the

Cat

image obtained from (d)

exactinterpFractal

without mean filtering (same as Figure 9.10 g) and (e) mean filtering shifted fractal decoded images.

Guide

Cover

Table of Contents

Begin Reading

Pages

C1

iv

v

xiii

xv

xvi

xvii

xviii

xix

xx

xxi

xxii

xxiii

xxiv

xxv

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

291

292

293

294

295

295

296

297

298

299

299

300

301

302

303

304

305

E1

Digital Image Interpolation in MATLAB®

Chi-Wah Kok andWing-Shan Tam

Canaan Semiconductor LimitedHong KongChina

Copyright

This edition first published 2019

© 2019 John Wiley & Sons Singapore Pte. Ltd

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Chi‐Wah Kok and Wing‐Shan Tam to be identified as the authors of this work has been asserted in accordance with law.

Registered Offices

John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

John Wiley & Sons Singapore Pte. Ltd., 1 FusionopolisWalk, #07‐01 Solaris South Tower, Singapore 138628

Editorial Office

The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print‐on‐demand. Some content that appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This work's use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software.

While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging‐in‐Publication Data

Names: Kok, Chi‐Wah, author. | Tam, Wing‐Shan, author.

Title: Digital image interpolation in MATLAB® / Dr. Chi‐Wah Kok, Canaan Semiconductor Limited,

Hong Kong, China, Dr. Wing‐Shan Tam, Canaan Semiconductor Limited,

Hong Kong, China.

Description: First edition. | Hoboken, NJ : John Wiley & Sons, Inc., 2019. |

Includes bibliographical references and index. |

Identifiers: LCCN 2018043062 (print) | LCCN 2018045690 (ebook) | ISBN

9781119119630 (Adobe PDF) | ISBN 9781119119647 (ePub) | ISBN 9781119119616

(hardcover)

Subjects: LCSH: Image processing–Digital techniques–Data processing. |

Interpolation. | MATLAB.

Classification: LCC TA1632 (ebook) | LCC TA1632 .K63 2019 (print) | DDC

006.6/86–dc23

LC record available at https://lccn.loc.gov/2018043062

Cover Design: Wiley

Cover Image: Courtesy of Chi‐Wah Kok and Wing‐Shan Tam

To my love Annie from Ted for putting up with it all once again

To mom Gloria Lee and the memory of dad, Simon Tam, dedicated by Wing‐Shan

About the Authors

Chi‐Wah Kok was born in Hong Kong. He was granted with a PhD degree from the University of Wisconsin–Madison. Since 1992, he has been working with various semiconductor companies, research institutions, and universities, which include AT&T Labs Research, Holmdel, SONY U.S. Research Labs, Stanford University, Hong Kong University of Science and Technology, Hong Kong Polytechnic University, City University of Hong Kong, Lattice Semiconductor, etc. In 2006, he founded Canaan Semiconductor Ltd., a fabless IC company with products in mixed‐signal IC, high performance audio amplifier, and high‐power MOSFETs and IGBTs. Dr. Kok embraces new technologies to meet the fast‐changing market requirements. He has extensively applied signal processing techniques to improve the circuit topologies, designs, and fabrication technologies within Canaan. This includes the application of semidefinite programming to circuit design optimization, abstract algebra in switched capacitor circuit topologies, and nonlinear optimization method to optimize high voltage MOSFET layout and fabrication. He was MPEG (MPEG 4) and JPEG (JPEG 2000) standards committee member. He is the founding editor in chief of the journal Solid State Electronics Letters since 2017. He is also the author of three books by Prentice Hall and Wiley‐IEEE and has written numerous papers on digital signal processing, multimedia signal processing, and CMOS circuits, devices, fabrication process, and reliability.

Wing‐Shan Tam was born in Hong Kong. She received her PhD degree in electronic engineering from the City University of Hong Kong. She has been working in different telecommunication and semiconductor companies since 2004 and is currently the engineering manager of Canaan Semiconductor Ltd., where she works on both advanced CMOS sensor design and high‐power device structure and process development. Dr. Tam has participated in professional services actively, in which she has been the researcher in different universities since 2007. She has been the invited speaker for different talks and seminars in numerous international conferences and renowned universities. She has served as guest editor in several journals published by IEEE and Elsevier. She is the founding editor of the journal Solid State Electronics Letters since 2017. She is the co‐author of another Wiley‐IEEE technology textbook and research papers with award quality. Her research interests include image interpolation algorithm, color enhancement algorithm and mixed‐signal integrated circuit design for data conversion and power management, and device fabrication process and new device structure development.

Preface

The process of deriving real‐world application from scientific knowledge is usually a very, very long process. However, with the advancement in complementary metal oxide semiconductor (CMOS) image sensor, and its application in handheld device, image interpolation has rapidly migrated from complex mathematics and academic publications to everyday applications in smartphones, laptops and tablets. Image interpolation has become a red‐hot research topic in both academia and industry. One of the highly cited academic works in image interpolation is authored by Dr. Tam, which is an excerpt from her master thesis. Her work is also the origin of this book. However, this book is not intended to be a memoir of the work done by Dr. Tam and her research group; it is intended to be the course materials for senior‐ and graduate‐level courses, training materials for engineers, and also a reference text for readers who are working in the field of digital imaging.

All the image interpolation algorithms discussed in this book will include both theories, where detailed analytic analysis are derived, and implementations through MATLAB into useful tools. Numerous algorithms are reviewed in this book together with detailed discussions on their origins, performances, and limitations. We are particularly happy with the numerical simulations presented for all the algorithms described in this book to clarify the observable but difficult to explain image interpolation artifacts, as the author shares the well‐known Chinese saying that a picture is worth a thousand words. Furthermore, many of our unpublished works are included in this book, where new algorithms are developed to overcome various limitations.

This book is authored as much as it is collected. We have tried our best to cite references whenever we are aware of related works on the topics. However, we suspect that some topics may have been independently studied by many individuals, and thus we might have missed their citation. Over 30 years of research works are collected in one place, and we presented each selected topics in a self‐contained format. If you are interested in further reading on any of these topics, you should look into the cited references and the Summary sections at the end of each chapter in this book. On a subject such as this one, which has been continuously investigated for over half a century, inevitably a number of valuable research results are not included in this book. It is nonetheless expected that the contents of this book will enable the careful readers to independently explore the more advanced image interpolation/processing technique.

Although much of the materials covered by this book are new to most students, our goal is to provide a working knowledge of various image interpolation algorithms without the need for additional course work besides freshman‐level engineering mathematics and a junior‐level matrix lab programming. To perform numerical simulation using computer, we must use a language that a computer can understand. This is why we choose to use MATLAB in this book, because MATLAB is not only a computer language. MATLAB, which is built with matrix data structure, is also a language of arithmetic. Once the MATLAB implementation of the algorithms have been learned, it will be fairly straightforward to implement them in other computer languages and VHDL for hardware synthesis. While almost all the MATLAB example codes presented in this book are co‐developed from the basic and do not require any toolbox to run with, in Chapter 6, the author just cannot resist to make use of the wavelet toolbox developed by Prof. T.Q. Nguyen of UCSD who is also the PhD adviser of Dr. Kok back in the University of Wisconsin–Madison. The toolbox has made everything easy, which definitely helped the readers to understand the topics and ease their practical implementation tremendously.

The book is divided into nine chapters. Chapter 1 provides an account of basic signal processing and mathematical tools used in subsequent chapters. It also serves the purpose of getting the readers to be familiar with the mathematical notations adopted in the book. Chapter 2 introduces the important concepts of digital imaging and the operations that are useful to image interpolation algorithms. The quality and performance measures between the processed image and the original image are presented in Chapter 3. The human visual system that is first discussed in Chapter 2 will be extended here for the discussion of the structural similarity quality index. The nonparametric image interpolation algorithm developed around algebraic functions are presented in Chapter 4. This chapter ends with a discussion on the deficiency of nonadaptive interpolation methods. Chapter 5 discusses the interpolation by Fourier and other orthogonal series. We are particularly interested in interpolating image in the discrete cosine transform domain, which is motivated by current trends in international image compression and storage standards. The blocking noise resulted from transform domain zero padding interpolation with small block size is alleviated by variations of overlap and add interpolation techniques. An iterative algorithm is presented to improve the least squares solution of the conventional transform coefficients zero padding image interpolation algorithm. Note that iterative image interpolation algorithms are considered to be offline image interpolation algorithms. More about iterative interpolation algorithm that helps to maintain the original pixel values while improving the performance of the non‐iterative image interpolation algorithms will be presented in subsequent chapters. Chapter 6 extends the block‐based transform domain image interpolation to the wavelet domain. A number of the techniques presented in previous chapters are applicable to the wavelet domain image interpolation too, and various researchers have been given them different names in the literature. The performance of wavelet image interpolation can be improved by exploiting the scale‐space relationships obtained by multi‐resolution analysis through wavelet transform (a version of the human visual system). The explicit edge detection‐based image interpolation methods discussed in Chapter 7 interpolate the image according to the edge‐directed image perception property of human visual system. Various edge‐directed interpolation methods will be discussed where edges are explicitly obtained by various edge detection methods discussed in Chapter 2, and implicit edge detection methods that the nature of the pixels to be interpolated is determined in the course of the estimation. The chapter concludes with discussions on the pros and cons of edge‐directed image interpolation algorithm using explicit edge detection. Another type of edge‐detected image interpolation method will be presented in Chapter 8, which is based on the edge geometric duality where a covariance‐based implicit edge location and estimation method will interpolate the image along the edge to achieve good visual quality. Digital signal processing theory tells us that there is always room to improve the solutions of any estimation problem. Various improvements to the edge‐directed interpolation problem will be discussed in this chapter to improve the preservation of edge geometric duality between the original image and the interpolated image, to reduce the interpolation error propagation by removing inter‐processing dependence, and finally to improve the estimation solution through an iterative re‐estimation algorithm. The book changes its course from linear statistical‐based interpolation technique to fractal interpolation in Chapter 9.

It should be noticed that fractal is usually not considered to be a statistical‐based interpolation algorithm. On the other hand, the generation of fractal map is based on similarity between image features, where the similarity is computed or classified via the statistics of the image or image blocks. Finally, an iterative algorithm is presented to improve the fractal image interpolation algorithm with the constraint that the original low‐resolution image is the pivot of the interpolated image, i.e. the location and intensity invariance of the low‐resolution image in the interpolation image is guaranteed. The advantage of such algorithmic constraint not only allows the preservation of the original low‐resolution image pixel values in the interpolated image but also ensures the highest preservation of the structure property of the interpolated image. As a result, fractal image interpolation has been embedded in a number of successful image processing softwares. The book concludes with an appendix that lists all the MATLAB source codes discussed in the book.

Many people have contributed, directly or indirectly, over a long period of time, to the subjects presented in this book. Their contributions are cited appropriately in this book, and also in the Summary section at the end of each chapter. The Summary sections also aimed to detail the state‐of‐the‐art development with respect to the topics discussed in each chapter. The exercises presented in the Exercise sections are essential parts of this text and often provide a discovery‐like experience regarding the associated topics. It is our hope that the exercises will provide general guidelines to assist the readers to design new image interpolation algorithms for their own applications. The readers' effort spent on tackling the exercises will help them to develop a thorough consideration on the design of image processing algorithms for their future career in research and development in the field.

The book is definitely not meant to represent a comprehensive history about the development of image interpolation algorithms. On the other hand, it does provide a not so short review, which chronologically follows the evolution of some of the image interpolation algorithms that have direct implications on commercially available image processing softwares. In particular, we avoided with our best effort to provide a comprehensive survey of every image interpolation algorithms in literature and market. Instead, our selection of topics is on the importance of the algorithms with respect to their applications in image processing softwares in today's or near‐future market. Our hope is that the book offers the readers a range of interesting topics and the current state‐of‐the‐art image interpolation methods. In simple terms, image interpolation is an open problem that has no definite winner. Analyzing the design and performance trade‐offs and proposing a range of attractive solutions to various image interpolation problems are the basic aims of this book. The book will underline the range of design considerations in an unbiased fashion, and the readers will be able to glean information from it in order to solve their own particular image interpolation problems. Most of all, we hope that the readers will find it an enjoyable and relatively effortless reading, providing them with intellectual stimulation.

Hong Kong, August 2018

Chi‐Wah Kok

Wing‐Shan Tam

Acknowledgments

Dr. Kok would like to thank his wife Dr. Annie Ko, an extraordinary woman with abiding faith in Christianity. He has acknowledged her in his previous book for her enormous contributions to his life – and still do. He thanks her for her encouragement, and she created enough time for him to write the book while being granted with tenure and awarded the best teaching award in her university. She has been his inspiration and motivation for continuing to improve his knowledge and move his career forward.

Dr. Kok would also like to thank Dr. Cindy Tam for allowing him to put up with far too many side projects while writing this book. He appreciates her belief in him to provide the leadership and knowledge to make this book a reality. She has provided research insights along the way, working with him to complete each chapter with the appropriate MATLAB sources and analytic details through revision and re‐revision, pouncing on obscurities, decrying omissions, correcting misspelling, redrawing figures, and often making her life very much more difficult by his unrelenting insistence that the text and figures could be more literate, accurate, and intelligible. He is very pleased to see his illegible red marginalia have found their way into the text of this book. The last but not the least, he would like to thank her for contributing the beautiful photo of “BeBe” both as the designated simulation image source for all the examples and also the cover image of the book. This lovely cat is Dr. Tam's domestic cat, and the best model for image interpolations, because it contains all the necessary image features that can demonstrate the visual artifacts and performance of various image interpolation algorithms.

Dr. Tam is glad to write her second book with the topic on image interpolation, the same topic as her master thesis. This book project gives her precious opportunities to review the work done in her early years of research and a chance to refresh her knowledge with the ongoing technology development and to explore new research breakthroughs in the field. An interesting research topic always begins with some extraordinary idea. Dr. Tam would like to thank her best mentor and collaborator, Dr. Kok, who introduced and inspired her in this interesting topic.

Dr. Tam would not be able to finish her master thesis, and all other industrial and research projects, without the patience and guidance of Dr. Kok. Though sometimes the collaboration is challenging and bumpy, Dr. Tam believes all the experience and knowledge gained from their collaboration have laid the cornerstone for her future, both personally and professionally.

Dr. Tam would not be able to continue her research career without the love and support from her family. She would like to thank her mother, Gloria, for her love and support, offering her a warm shelter to rest her tired and frustrated body and mind for all these years, and her father, Simon, now in heaven watching and praying for her. Dr. Tam has inherited her father's spirit in striving for perfection, which keeps her moving and be a better researcher.

Her father would be happy to see the publication of her second book and all her research papers. Thanks also go to her sister Candy, brother‐in‐law Kelvin, niece Clarice, and nephew Kayven who have brought much happiness and laughter to her, the natural booster to keep her energetic year round.