Architecture-Aware Optimization Strategies in Real-time Image Processing E-Book

Chao Li

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch
• Veröffentlichungsjahr: 2017

Seitenzahl: 187

Beschreibung

In the field of image processing, many applications require real-time execution, particularly those in the domains of medicine, robotics and transmission, to name but a few. Recent technological developments have allowed for the integration of more complex algorithms with large data volume into embedded systems, in turn producing a series of new sophisticated electronic architectures at affordable prices. This book performs an in-depth survey on this topic. It is primarily written for those who are familiar with the basics of image processing and want to implement the target processing design using different electronic platforms for computing acceleration. The authors present techniques and approaches, step by step, through illustrative examples. This book is also suitable for electronics/embedded systems engineers who want to consider image processing applications as sufficient imaging algorithm details are given to facilitate their understanding.

Leseprobe

Table of Contents

Cover

Title

Copyright

Preface

1 Introduction of Real-time Image Processing

1.1. General image processing presentation

1.2. Real-time image processing

2 Hardware Architectures for Real-time Processing

2.1. History of image processing hardware platforms

2.2. General-purpose processors

2.3. Digital signal processors

2.4. Graphics processing units

2.5. Field programmable gate arrays

2.6. SW/HW codesign of real-time image processing

2.7. Image processing development environment description

2.8. Comparison and discussion

3 Rapid Prototyping of Parallel Reconfigurable Instruction Set Processor for Efficient Real-Time Image Processing

3.1. Context and problematic

3.2. Related works

3.3. Design exploration framework

3.4. Case study: RISP conception and synthesis for spatial transforms

3.5. Hardware implementation of spatial transforms on an FPGA-based platform

3.6. Discussion and conclusion

4 Exploration of High-Level Synthesis Technique

4.1. Introduction of HLS technique

4.2. Vivado_HLS process presentation

4.3. Case of HLS application: FPGA implementation of an improved skin lesion assessment method

4.4. Discussion

5 CDMS4HLS: A Novel Source-To-Source Compilation Strategy for HLS-Based FPGA Design

5.1. S2S compiler-based HLS design framework

5.2. CDMS4HLS compilation process description

5.3. CDMS4HLS compilation process evaluation

5.4. Discussion

6 Embedded Implementation of VHR Satellite Image Segmentation

6.1. LSM description

6.2. Implementation and optimization presentation

6.3. Experiment evaluation

6.4. Discussion and conclusion

7 Real-time Image Processing with Very High-level Synthesis

7.1. VHLS motivation

7.2. Image processing from Matlab to FPGA-RTL

7.3. VHLS process presentation

7.4. VHLS implementation issues

7.5. Future work for real-time image processing with VHLS

Bibliography

Index

End User License Agreement

List of Tables

2 Hardware Architectures for Real-time Processing

Table 2.1. Compatibility evaluation of fundamental development environments for different hardware devices

3 Rapid Prototyping of Parallel Reconfigurable Instruction Set Processor for Efficient Real-Time Image Processing

Table 3.1. IR assembly and parallelism extraction results: acceleration corresponds to number of sequential cycle per number of parallel cycle for one block of 2D-DCT processing

Table 3.2. Used hardware resources and operating frequency of each RISP processor

Table 3.3. Macroblock distribution of four video sequences using the HEVC decoding process

Table 3.4. IDCT processing time in seconds for one frame of different video formats: these results were obtained using five soft-core processors (four RISPs and one MicroBlaze). Bold values indicate that the processing speed is equal or superior to 25 frames/s

Table 3.5. IDCT processing used hardware resources and operating frequency: these results were obtained by using five soft-core processors (four RISPs and one MicroBlaze)

4 Exploration of High-Level Synthesis Technique

Table 4.1. Characteristic evaluation of five HLS tools

Table 4.2. Operations-cores mapping of the scheduling schematic in Figure 4.5

Table 4.3. Size of search spaces for skin parameters (see [JOL 13])

Table 4.4. Necessary instruction number comparison between original and optimized KM functions

Table 4.5. Population parameter configuration for KMGA and HCR-KMGA

Table 4.6. Implementation acceleration ratio comparison: the clock of CPU and FPGA are, respectively, 2.4 GHz and 50 MHz

Table 4.7. Used hardware resources estimation of FPGA-KMGA and FPGA-HCR-KMGA implementations on Virtex7-XC7VX1140T of Xilinx

5 CDMS4HLS: A Novel Source-To-Source Compilation Strategy for HLS-Based FPGA Design

Table 5.1. Symbolic expression manipulation strategies

Table 5.2. Performances and resources consumption evaluation (FI, function inline; LM, loop manipulation; SEM, symbolic expression manipulation; LU, loop unwinding; MM, memory manipulation)

6 Embedded Implementation of VHR Satellite Image Segmentation

Table 6.1. Optimization evaluation: original corresponds to fpga_hls_ori implementation while LU corresponds to fpga_hls_opt implementation

Table 6.2. Running time comparison (s/pixel)

Guide

Cover

Table of Contents

Begin Reading

Pages

C1

iii

iv

v

ix

x

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

141

142

143

144

145

146

147

148

149

150

151

153

154

155

156

157

158

159

160

161

163

164

G1

G2

G3

G4

G5

e1

Series EditorHenri Maître

Architecture-Aware Optimization Strategies in Real-time Image Processing

Chao Li

Souleymane Balla-Arabe

Fan Yang-Song

First published 2017 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd 27-37 St George’s Road London SW19 4EU UKwww.iste.co.uk

John Wiley & Sons, Inc. 111 River Street Hoboken, NJ 07030 USAwww.wiley.com

© ISTE Ltd 2017

The rights of Chao Li, Souleymane Balla-Arabe and Fan Yang-Song to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Control Number: 2017948985

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library

ISBN 978-1-78630-094-2

Preface

In the image processing field, a lot of applications require real-time execution in the domains of medical technology, robotics and transmission, etc. Recently, real-time image processing fields have made a lot of progress. Technological developments allow engineers to integrate more complex algorithms with large data volumes onto embedded systems, and produce series of new sophisticated electronic architectures at an affordable price. At the same time, industrial and academic researchers have proposed new methods and provided new tools in order to facilitate real-time image processing realization at different levels. It is necessary to perform a deepened educational and synthetic survey on this topic. We will present electronic platforms, methods, and strategies to reach this objective.

This book consists of seven chapters ranging from the fundamental conceptual introduction of real-time image processing to future perspectives in this area. We describe hardware architectures and different optimization strategies for real-time purposes. The latter consists of a survey of software and hardware co-design tools at different levels. Two real-time applications will be presented in detail in order to illustrate the proposed approaches.

The major originalities of this book include (1) algorithm architecture mapping: we select methods and tools that treat simultaneously the application and its electronic platform in order to perform fast and optimal design space exploration (DSE), (2) each approach will be illustrated by concrete examples and (3) two of the chosen algorithms have been only recently advanced in their domain.

This book is written primarily for those who are familiar with the basics of image processing and want to implement the target image processing design using different electronic platforms for computing acceleration. It accomplishes this by presenting the techniques and approaches step by step, the algorithm and architecture conjointly, and by notions of description and example illustration. This concerns both the software engineer and the hardware engineer.

This book will also be adequate for those who are familiar with programming and applying embedded systems to other problems and are considering image processing applications. Much of the focus and many of the examples are taken from image processing applications. Sufficient detail is given to make algorithms and their implementation clear.

Chao LISouleymane BALLA-ARABEFan YANGAugust 2017

1Introduction of Real-time Image Processing

1.1. General image processing presentation

The traditional view of an image derives heavily from experience in photography, television and the like. This means that an image is a two-dimensional (2D) structure, a representation and also a structure with meaning to a visual response system. This view of an image only accepts spatial variation (a static image). In parallel, a dynamic image has spatial and temporal variation. In most contexts, this is usually referred to as video. This more complex structure needs to be viewed as a sequence of images each representing a particular instance in time. On the other hand, an image can be formed by taking a sampling plane through that volume and so the variation in three dimensions is observed. This may be referred to as a volume image. An image linked to a volume that changes with time is a further possibility. This has particular significance in medical imaging applications [MYE 09].

Image processing is a method to convert an image into digital form and perform some operations on it in order to get an improved image or to extract some useful information from it. A digital image described in a 2D space is usually considered as 2D signals while applying already set signal methods to it. Image processing forms a core research area within engineering and computer science too.

Image processing is an enabling technology for a wide range of applications including remote sensing, security, image data sets, digital television, robotics and medical imaging, etc. The goal of this technology can be divided into three levels: low, medium and high. Low-level image processing techniques are mathematical or logical operators that perform simple processing tasks. This “processing” level possesses both image_in and image_out. Medium-level image processing combines the simple low-level operators to perform feature extraction and pattern recognition functions. Using an image_in, this “analysis” level produces measurements_out (parameters). High-level image processing uses combinations of medium-level functions to perform interpretation. This “understanding” level treats measurements_in for high-level description_out. Usually, apart from these three levels, it is also necessary to apply preprocessing techniques that are designed to remove distortions introduced by sensors.

Figure 1.1.Overview of the typical image acquisition process (see [MOE 12])

Before any image processing can commence, an image must be captured by a camera and converted into a manageable entity. This is the image acquisition process (see Figure 1.1), which consists of three steps; energy reflected from the object of interest, an optical system that focuses on the energy and finally a sensor that measures the amount of energy. In order to capture an image, a camera requires some sort of measurable energy. The energy of interest in this context is light or electromagnetic waves. Each wave can have different wavelengths (or different energy levels or different frequencies). The human visual spectrum is in the range of approximately 400–700 nm.

After having illuminated the object of interest, the light reflected from the object now has to be captured by the camera composed of an optical system and an image sensor. The role of the first is to focus the light reflected from the object onto the second (a material sensitive to the reflected light). An image sensor consists of a 2D array of cells. Each of these cells is denoted a pixel and is capable of measuring the amount of incident light and convert that into a voltage, which in turn is converted into a digital number. The more incident light, the higher the voltage and the higher the digital number.

In order to transform the information from the sensor into an image, each cell content is now converted into a pixel value in the range: (0, 255). Such a value is interpreted as the amount of light hitting a cell. This is denoted the intensity of a pixel. It is visualized as a shade of gray denoted gray-level value ranging from black (0) to white (255). Standardly, a monochromatic, static image corresponds to a matrix of m rows and n columns. Therefore, the camera records m × n pixels of 8 bit values.

In order to capture a color image, the color camera must record three matrices of three primary colors red, green and blue. Recently, a lot of applications are realized using multispectral image processing, since the multispectral cameras are more available with reasonable prices. According to application needs, multispectral images are captured using different wavelengths (bands). They can be considered as a cube formed by 2D gray-level images. Figure 1.2 displays two typical test images in the area of image processing research. Figure 1.3 gives two multispectral cube examples; the right image is captured for a skin lesion assessment application.

Figure 1.2.Lena (gray-level image) and landscape (color image). For a color version of the figure, seewww.iste.co.uk/li/image.zip

Figure 1.3.Two multispectral image cubes. For a color version of the figure, seewww.iste.co.uk/li/image.zip

Certain tools are central to the processing of digital images. These include mathematical tools, statistical descriptions and manipulative tools. We can cite some more used such as convolution, filtering in spatial and frequency domains, morphological operations and image transforms, etc. The types of basic operations that can be applied to digital images to transform an input image A(m, n) into an output image B(m, n) (or another representation) can be classified into three categories:

– Point: the output value at a specific matrix coordinate is dependent only on the input value at that same coordinate. In this case, generic operation complexity per pixel is constant.

– Local: the output value at a specific coordinate is dependent on the input values in the

neighborhood