Image Segmentation E-Book

Tao Lei

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 617

Veröffentlichungsjahr: 2022

Beschreibung

Image Segmentation Summarizes and improves new theory, methods, and applications of current image segmentation approaches, written by leaders in the field The process of image segmentation divides an image into different regions based on the characteristics of pixels, resulting in a simplified image that can be more efficiently analyzed. Image segmentation has wide applications in numerous fields ranging from industry detection and bio-medicine to intelligent transportation and architecture. Image Segmentation: Principles, Techniques, and Applications is an up-to-date collection of recent techniques and methods devoted to the field of computer vision. Covering fundamental concepts, new theories and approaches, and a variety of practical applications including medical imaging, remote sensing, fuzzy clustering, and watershed transform. In-depth chapters present innovative methods developed by the authors--such as convolutional neural networks, graph convolutional networks, deformable convolution, and model compression--to assist graduate students and researchers apply and improve image segmentation in their work. * Describes basic principles of image segmentation and related mathematical methods such as clustering, neural networks, and mathematical morphology. * Introduces new methods for achieving rapid and accurate image segmentation based on classic image processing and machine learning theory. * Presents techniques for improved convolutional neural networks for scene segmentation, object recognition, and change detection, etc. * Highlights the effect of image segmentation in various application scenarios such as traffic image analysis, medical image analysis, remote sensing applications, and material analysis, etc. Image Segmentation: Principles, Techniques, and Applications is an essential resource for undergraduate and graduate courses such as image and video processing, computer vision, and digital signal processing, as well as researchers working in computer vision and image analysis looking to improve their techniques and methods.

Leseprobe

Image Segmentation

Principles, Techniques, and Applications

Tao Lei

Shaanxi University of Science and Technology

Xi’an, China

Asoke K. Nandi

Brunel University London

Uxbridge, UK

This edition first published 2023© 2023 John Wiley & Sons Ltd

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/ permissions.

The right of Tao Lei and Asoke K. Nandi to be identified as the authors of this work has been asserted in accordance with law.

Registered OfficesJohn Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USAJohn Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

Editorial OfficeThe Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print‐on‐demand. Some content that appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of WarrantyIn view of ongoing research, equipment modifications, changes in governmental regulations, and the constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader is urged to review and evaluate the information provided in the package insert or instructions for each chemical, piece of equipment, reagent, or device for, among other things, any changes in the instructions or indication of usage and for added warnings and precautions. While the publisher and author have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness fora particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and author endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging‐in‐Publication data applied forISBN: 9781119859000

Cover Design: WileyCover Image: © d3sign/Getty Images

ToMy parents, my wife Yan Lu, and our daughter—Lu Lei.Tao Lei

My wife, Marion, and our children—Robin, David, and Anita Nandi.Asoke K. Nandi

About the Authors

Tao Lei received a PhD in information and communication engineering from Northwestern Polytechnical University, Xi'an, China, in 2011. From 2012 to 2014, he was a postdoctoral research fellow with the School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China. From 2015 to 2016, he was a visiting scholar with the Quantum Computation and Intelligent Systems group at University of Technology Sydney, Sydney, Australia. From 2016 to 2019, he was a postdoctoral research fellow with the School of Computer Science, Northwestern Polytechnical University, Xi'an, China. He has authored and coauthored 80+ research papers published in IEEE TIP, TFS, TGRS, TGRSL, ICASSP, ICIP, and FG.

He is currently a professor with the School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology. His current research interests include image processing, pattern recognition, and machine learning. Professor Lei is an associate editor of Frontiers in Signal Processing; he is also a guest editor of Remote Sensing and IEEE JESTAR. He is a senior member of IEEE and CCF.

Asoke K. Nandi received the degree of PhD in physics from the University of Cambridge (Trinity College), Cambridge. He held academic positions in several universities, including Oxford, Imperial College London, Strathclyde, and Liverpool, as well as Finland Distinguished Professorship in Jyvaskyla (Finland). In 2013 he moved to Brunel University London, to become the Chair and Head of Electronic and Computer Engineering.

In 1983 Professor Nandi jointly discovered the three fundamental particles known as W+, W−, and Z0, providing the evidence for the unification of the electromagnetic and weak forces, for which the Nobel Committee for Physics in 1984 awarded the prize to his two team leaders for their decisive contributions. His current research interests lie in the areas of signal processing and machine learning, with applications to communications, gene expression data, functional magnetic resonance data, machine condition monitoring, and biomedical data. He has made many fundamental theoretical and algorithmic contributions to many aspects of signal processing and machine learning. He has much expertise in big data, dealing with heterogeneous data and extracting information from multiple data sets obtained in different laboratories and different times. Professor Nandi has authored over 600 technical publications, including 260 journal papers as well as five books, entitled Condition Monitoring with Vibration Signals (Wiley, 2020), Automatic Modulation Classification: Principles, Algorithms and Applications (Wiley, 2015), Integrative Cluster Analysis in Bioinformatics (Wiley, 2015), Blind Estimation Using Higher‐Order Statistics (Springer, 1999), and Automatic Modulation Recognition of Communications Signals (Springer, 1996). The h‐index of his publications is 80 (Google Scholar) and his Erdös number is 2.

Professor Nandi is a Fellow of the Royal Academy of Engineering (UK) and of seven other institutions. Among the many awards he received are the Institute of Electrical and Electronics Engineers (USA) Heinrich Hertz Award in 2012, the Glory of Bengal Award for his outstanding achievements in scientific research in 2010, from the Society for Machinery Failure Prevention Technology, a division of the Vibration Institute (USA) in 2000, the Water Arbitration Prize of the Institution of Mechanical Engineers (UK) in 1999, and the Mountbatten Premium of the Institution of Electrical Engineers (UK) in 1998. Professor Nandi is an IEEE Distinguished Lecturer (2018–2019). Professor Nandi is the Field Chief Editor of Frontiers in Signal Processing journal.

Preface

Image segmentation is one of the most challenging frontier topics in computer vision. It provides an important foundation for image analysis and image description, as well as image understanding. The basic task of image segmentation is to segment an image into several regions that are nonoverlapping, with these regions having accurate boundaries. The current task of image segmentation not only requires accurate region division but also requires a label output on different regions, that is, semantic segmentation. With the development of computer vision and artificial intelligence techniques, the roles and importance of image segmentation have grown significantly. Image segmentation has been applied widely in various fields, such as industry of detection, intelligent transportation, biological medicine, agriculture, defense, and remote sensing. At present, a large number of image segmentation techniques have been reported, and many of them have been successfully applied to actual product development. However, the fast development of imaging and artificial intelligence techniques requires image segmentation to deal with increasingly more complex tasks. These complex tasks require more effective and efficient image segmentation techniques. For that reason, there is a growing body of literature resulting from efforts in research and development by many research groups around the world. Although there are many publications on image segmentation, there is only a few collections of recent techniques and methods devoted to the field of computer vision.

This book attempts to summarize and improve principles, techniques, and applications of current image segmentation. It can help researchers, postgraduate students, and practicing engineers from colleges, research institutes, and enterprises to understand the field quickly. Firstly, based on this book, researchers can quickly understand the basic principles of image segmentation and related mathematical methods such as clustering, mathematical morphology, and convolutional neural networks. Secondly, based on classic image processing and machine learning theory, the book introduces a serious of recent methods to achieve fast and accurate image segmentation. Finally, the book introduces the effect of image segmentation in various application scenarios such as traffic, medicine, remote sensing, and materials. In brief, the book aims to inform, enthuse, and attract more researchers to enter the field and thus develop further image segmentation theory and applications.

Chapter1 is a brief introduction to image segmentation and its applications in various fields including industry, medicine, defense, and environment. Besides, an example is presented to help readers understand image segmentation quickly.

Chapter2 is concerned with principles of clustering. Three clustering approaches that are concerned closely with image segmentation, are presented, i.e. k‐means clustering, fuzzy c‐means clustering, spectral clustering, and gaussian mixed model.

Chapter3 is concerned with principles of mathematical morphology since it is important in image processing, especially watershed transform, which is popular for image segmentation. In this chapter, morphological filtering, morphological reconstruction, and the watershed transform are presented. Besides, multivariate mathematical morphology is presented since it is important for multichannel image processing, which can help image segmentation for multichannel images.

Chapter4 is concerned with principles of neural networks since they are important in image processing, especially convolutional neural networks, which are popular for image segmentation. In this chapter, artificial neural networks, convolutional neural networks, and graph convolutional networks are presented.

Chapter5 introduces a fast image segmentation approach based on fuzzy clustering. This chapter illustrates related works with improved FCM and presents two strategies: local spatial information integration and membership filtering, which achieves better segmentation results.

Chapter6 introduces a fast and robust image segmentation approach based on the watershed transform. This chapter illustrates related works with seeded image segmentation and presents an adaptive morphological reconstruction method that can help the watershed transform to achieve better segmentation results.

Chapter7 introduces a fast image segmentation approach based on superpixel and the Gaussian mixed model (GMM). This chapter illustrates related works with superpixel algorithms and presents the idea of combing superpixel and GMM for image segmentation.

Chapter8 introduces the application of image segmentation for traffic scene segmentation. This chapter illustrates related works with traffic scene semantic segmentation and presents the idea of multi‐scale information fusion combing nonlocal network for traffic scene semantic segmentation.

Chapter9 introduces the application of image segmentation for medical images. This chapter illustrates related works with liver and liver‐tumor segmentation and presents two approaches for liver and liver‐tumor segmentation including lightweight V‐net and deformable context encoding network.

Chapter10 introduces the application of image segmentation for remote sensing. This chapter illustrates related works with change detection and presents two approaches for change detection including unsupervised change detection and end‐to‐end change detection for very high resolution (VHR) remote sensing images.

Chapter11 introduces the application of image segmentation for material analysis. This chapter presents three applications for different material analysis including metallic materials, foam materials, and ceramics materials.

This book is up‐to‐date and covers a lot of the advanced techniques used for image segmentation, including recently developed methods. In addition, this book provides new methods, including unsupervised clustering, watershed transform, and deep learning for image segmentation, which covers various topics of current research interest. Additionally, the book will provide several popular image segmentation applications including traffic scene, medical images, remote sensing images, and scanning electron microscope images. A work of this magnitude will, unfortunately, contain errors and omissions. We would like to take this opportunity to apologize unreservedly for all such indiscretions in advance. We welcome comments and corrections; please send them by email to [email protected] or by any other means.

Acknowledgment

It should be remarked that some of the research results reported in this book have been sourced from refereed publications, arising from projects originally funded by the Royal Society (UK) grant (IEC\NSFC\170 396) and NSFC (China) grant (61811530325).

SEPTEMBER 2022

TAO LEI AND ASOKE K. NANDIXI'AN, CHINA, AND LONDON, UK

List of Acronyms

ADAM

Adaptive moment estimation

AFCF

Automatic fuzzy clustering framework for image segmentation

AlexNet

ImageNet classification with deep convolutional neural networks

AMR

Adaptive morphological reconstruction

AMR‐RMR‐WT

Fast and Automatic Image Segmentation Using Superpixel‐Based Graph Clustering

AMR‐SC

Spectral clustering based on pre‐segmentation of AMR‐WT

ASD

Average symmetric surface distance

ASPP

Atrous spatial pyramid pooling

ASSD

Average symmetric surface distance

AutoML

Automatic machine learning

BDE

Boundary displacement error

BP

Back-Propagation

CNN

Convolutional neural network

CPU

Central processing unit

CS

Comparison scores

CT

Computed Tomography

CV

Segmentation covering

DC

Deformable convolution

DSC

Depthwise separable convolution

DWT

Discrete wavelet transform

EGC

Eigen‐value gradient clustering

ELU

Exponential Linear Unit

ELSE

Edge‐based level‐set

EM

Expectation–maximization

EnFCM

Enhanced Fuzzy c‐means clustering

FAS‐SGC

Fast and automatic image segmentation algorithm employing superpixel‐based graph clustering

FCM

Fuzzy c‐means clustering

FCN

Full convolution network

FCN‐PP

Fully convolutional network within pyramid pooling

FN

False negative fraction

FP

False positive fraction

GB

Gigabyte

GCE

Global consistency error

GELU

Gauss error linear element

GFLOPs

Giga Floating‐point Operations Per Second

GL‐graph

Global/regional affinity graph

GMM

Gaussian mixture model

GPU

Graphics Processing Unit

GT

Ground truths

HC

Hierarchical clustering

HD

Hausdorff distance

H‐DenseUNet

Hybrid densely connected UNet

HMRF

Hidden Markov random field

HSR

High spatial resolution

HSV

Hue‐Saturation‐Value

HU

Hounsfield

IDWT

Inverse discrete wavelet transform

IOU

Intersection over Union

IRB

Inverted residual bottleneck

ISBI

International symposium on biomedical imaging

Ladder‐ASPP

Ladder‐atrous‐spatial‐pyramid‐pooling

LIM

Landslide inventory mapping

LiTS

Liver Tumor Segmentation Challenge

LM

Landslide mapping

LMSE

Linear Mean Squared Error

LSC

Linear Spectral Clustering

LV‐Net

Lightweight V‐Net

MCG

Multiscale combinatorial grouping

MGR

Morphological gradient reconstruction

MGR‐WT

Morphological Gradient Reconstruction based Watershed Transform

MIoU

Mean intersection over union

MMF

A new approach to morphological color image processing

MMG

Multiscale morphological gradient algorithm

MMG‐WT

Multiscale morphological gradient for watersheds

MMGR

Multiscale morphological gradient reconstruction

MMGR‐WT

Novel WT based on MMGR

MMR

Morphological reconstruction

MP

McCulloch Pitts

MPA

Mean Pixel Accuracy

MR

Morphological reconstruction

MRI

Magnetic Resonance Imaging

MRF

Markov random fields

MRI

Magnetic Resonance Imaging

MSD

Maximum symmetric surface distance

MSE

Mean square error

MSR

Multi Scale Retinex

MSSD

Maximum symmetric surface distance

NAS

Neural Architecture Search

NLP

Natural language processing

NP

Non‐deterministic Polynomial

OBEM

Object‐based expectation maximization

OE

Overall error

OEF

Oriented edge forests

OG

Original gradient

PA

Pixel Accuracy

PC

Personal Computer

PCA

Principal components analysis

PDE

Partial differential equation

PP

Pyramid pooling

PReLU

Parametric ReLU

PRI

Probabilistic rand index

PW

Power watershed

ReLU

Rectified linear unit

RGB

Red‐Green‐Blue

RGD

Random gradient descent

RLSE

Region‐based level‐set

RMSE

Root mean square error

RMSD

Root mean square deviation

RReLU

Randomized ReLU

RVD

Relative volume difference

RW

Random Walker

RWT

Robust watershed transform

SC

Spectral clustering

SCG

Single‐scale combinatorial grouping

SE

Structured edge

SEM

Scanning electron microscope

SEs

Structuring elements

SGD

Stochastic Gradient Descent

SH

Superpixel Hierarchy

SLIC

Simple linear iterative clustering

SOTA

State‐of‐the‐art

SPP

Spatial Pyramid Pooling

SVMs

Support vector machines

TM

Trademark

TP

True positive fraction

UAV

Unmanned Aerial Vehicle

VHR

Very high‐resolution

VI

variation of information

ViT

Vision Transformer

VO

Vector ordering

VOC

Visual Object Classes

VOE

Volume Overlap Error

WT

Watershed transformation

YIQ

National Television Standards Committee

3‐D

Three‐dimensional

Part 1Principles

1Introduction

Since the birth of computers in 1950s, people have been using computers to process multimedia information such as text, graphics, and images. With the rapid development of computer technology, signal processing, and imaging technology, image processing technology has been boosted in recent years. Humans use their senses such as vision, hearing, and touch to obtain external information. Among all these senses, vision is the most momentous way for obtaining information since it can often capture more information than hearing and touch. Therefore, images play a vital role in people's perception of the world.

Image processing technology mainly includes image transformation [1], image restoration [2], image compression [3], image segmentation [4], target detection and recognition [5], and image classification [6]. In image processing, image segmentation plays a crucial role. The main reason is that image segmentation is the foundation of high‐level tasks in computer vision. With the continuous improvement of image processing technology, image segmentation is developed from the early edge detection, region merging and division, and contour extraction to the current semantic segmentation. Currently, image segmentation technology has been widely used in industrial detection [7], medical image analysis [8], remote sensing earth observation [9], intelligent driving [10], and other fields. In view of the expeditious development and comprehensive application of image segmentation technology, this book will focus on the basic theoretical knowledge related to image segmentation, the introduction of mainstream algorithms of image segmentation, and specific applications of image segmentation, as shown in Figure 1.1.

1.1 Preliminary Concepts

Image segmentation is dividing an image into nonoverlapping regions. Early image segmentation is relatively simple, which mainly tackles some problems coming from industrial defect detection, target detection, and image enhancement tasks. Due to the limitation of early imaging technology, early digital images have some characteristics of low resolution, blur, noise corruption, and so on. However, since image segmentation can effectively extract the contour and other details of an image, it can be used to enhance visual effects of the image. For example, in industrial detection, the industrial camera is deployed on the pipeline, so the camera imaging results have a fixed background. Under this circumstance, it is relatively easy to use an image segmentation technique to obtain the target and defect area.

For the above example, although this binary image segmentation technology is easy to implement, it is not universal, which means it is unsuitable for images with complex background. The main reason is that this kind of image segmentation method only depends on low‐level features of images to make decisions. As low‐level features of images have a weak ability of representation for real semantic information, these binary image segmentation methods usually provide poor segmentation results in practical applications. With the rapid development of imaging technology, the image quality has improved, such as higher image resolution, more bands, and richer colors. However, with the improvement of artificial intelligence technology, the task of image analysis has become more complex requiring higher segmentation accuracy, faster inference speed, and less resource consumption. For example, in automatic driving, the semantic understanding of the scene is the basic requirement, along with the speed and video information, which requires that the segmentation time of an image should be less than 1/24 second. Therefore, the current task of image segmentation is more sophisticated and thus face many challenges.

Figure 1.1 The application of image segmentation in different scenes. (a) Natural image.(b) Remote sensing image. (c) Scanning electron microscope image. (d) Medical image.