Neuro-inspired Information Processing E-Book

Alain Cappy

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 324

Veröffentlichungsjahr: 2020

Beschreibung

With the end of Moore's law and the emergence of new application needs such as those of the Internet of Things (IoT) or artificial intelligence (AI), neuro-inspired, or neuromorphic, information processing is attracting more and more attention from the scientific community. Its principle is to emulate in a simplified way the formidable machine to process information which is the brain, with neurons and artificial synapses organized in network. These networks can be software – and therefore implemented in the form of a computer program – but also hardware and produced by nanoelectronic circuits. The �material� path allows very low energy consumption, and the possibility of faithfully reproducing the shape and dynamics of the action potentials of living neurons (biomimetic approach) or even being up to a thousand times faster (high frequency approach). This path is promising and welcomed by the major manufacturers of nanoelectronics, as circuits can now today integrate several million neurons and artificial synapses.

Leseprobe

Table of Contents

Cover

Acknowledgments

Introduction

1 Information Processing

1.1. Background

1.2. Information processing machines

1.3. Information and energy

1.4. Technologies of the future

1.5. Microprocessors and the brain

1.6. Conclusion

2 Information Processing in the Living

2.1. The brain at a glance

2.2. Cortex

2.3. An emblematic example: the visual cortex

2.4. Conclusion

3 Neurons and Synapses

3.1. Background

3.2. Cell membrane

3.3. Membrane at equilibrium

3.4. The membrane in dynamic state

3.5. Synapses

3.6. Conclusion

4 Artificial Neural Networks

4.1. Software neural networks

4.2. Hardware neural networks

4.3. Conclusion

References

Index

End User License Agreement

List of Tables

Chapter 1

Table 1.1. Scaling rules for a MOSFET transistor, and consequence of different c...

Table 1.2. The different approaches for improving the energy efficiency of infor...

Table 1.3. Comparison of the physical characteristics of the human brain with th...

Table 1.4. Comparison of human brain/microprocessor in information processing

Chapter 3

Table 3.1. Extra- and intracellular ionic concentrations

Table 3.2. Numerical parameters of the Hodgkin–Huxley model

Table 3.3. Parameters of the Morris–Lecar model

Chapter 4

Table 4.1. Comparison of several properties of biological and artificial systems

Table 4.2. Width of transistors and value of capacitances of the circuit in Figu...

List of Illustrations

Chapter 1

Figure 1.1. Block diagram of information processing systems. For a color version...

Figure 1.2. Different information encoding types

Figure 1.3. The Turing machine and its practical realization (http://aturingmach...

Figure 1.4. von Neumann architecture

Figure 1.5. Physical and electrical diagrams of NMOS and PMOS transistors. Curre...

Figure 1.6. Inverter composed of NMOS and PMOS transistors and forming a CMOS pa...

Figure 1.7. Principle of downscaling. All dimensions and voltages are divided by...

Figure 1.8. Evolution in the number of transistors, clock frequency and total po...

Figure 1.9. Switching of a CMOS inverter. Charge and discharge currents of the c...

Figure 1.10. Drain–source current of an NMOS as a function of the control voltag...

Figure 1.11. Inverter loaded by two logic gates. C

g

is the input capacitance of ...

Figure 1.12. Charge of an RC circuit by a voltage variable over time

Figure 1.13. Drain–source current as a function of control voltage in logarithmi...

Figure 1.14. Bloch sphere

Figure 1.15. (a) Schematic diagram of the CNOT gate; (b) CNOT gate conversion ta...

Figure 1.16. (a) Diagram of interconnections within an integrated circuit; (b) a...

Chapter 2

Figure 2.1. The lobes of the left hemisphere and cross-sectional view of the bra...

Figure 2.2. Organization of the cortex in six layers and cortical columns

5

Figure 2.3. Detail of cortical layers, L1–L6. Columns and mini-columns. For a co...

Figure 2.4. Hierarchical organization of the cortex. Each box in the diagram rep...

Figure 2.5. Internal and external connections of a cortical column. The black ar...

Figure 2.6. Organization of the human visual system. The eye diameter has been a...

Figure 2.7. Organization of visual area V1. The color-sensitive cells are locate...

Chapter 3

Figure 3.1. Structure of a neuron. The diameter of the soma is of the order of a...

Figure 3.2. Network of synapses between pre-neurons (dotted lines) and a post-ne...

Figure 3.3. Details of a synapse. The intracellular media of the two neurons are...

Figure 3.4. Mechanism of ion channel opening by neurotransmitters

Figure 3.5. Structure of the plasma membrane surrounding the neuron. For a color...

Figure 3.6. The ATP sodium–potassium pump, or ‘NaK’ pump. For a color version of...

Figure 3.7. Modeling of ionic currents: simplifying hypotheses and boundary cond...

Figure 3.8. Simplified equivalent circuit of the membrane in equilibrium. For a ...

Figure 3.9. Representation of resting potential V

r

as a function of G

K

/G

Na

. The ...

Figure 3.10. Equivalent circuit of the membrane in dynamic state. For a color ve...

Figure 3.11. Simplified equivalent circuit of the membrane in dynamic state

Figure 3.12. Other equivalent circuit of the membrane in dynamic state. The arro...

Figure 3.13. Rule for the variation of the activation coefficient, n, leading to...

Figure 3.14. Coefficients n

ss

, m

ss

and h

ss

, extracted from expressions [3.24]. T...

Figure 3.15. Time constants τ

n

τ

m

and τ

h

, extracted from expressions [3.24]. The...

Figure 3.16. Membrane potential, V

m

(t), in response to a current impulse of 3.5 ...

Figure 3.17. Activation coefficients, n(t), m(t) and h(t), in response to a curr...

Figure 3.18. Sodium, G

Na

(t), and potassium, G

K

(t), conductances in response to a...

Figure 3.19. Sodium, potassium and leakage currents in response to a current imp...

Figure 3.20. Membrane potential in response to a current pulse of 6 μA/cm

2

appli...

Figure 3.21. Coefficient h as a function of n in the event of a spike. The red d...

Figure 3.22. Nullclines of n (blue line) and of V

m

Iex=0 (red dotted line) and I...

Figure 3.23. Nullclines of n and V

m

(Iex=6μA/cm

2

) and Vm(t), n(t) cycles resulti...

Figure 3.24. Isoclines n and V

m

for different values of excitation current, Iex....

Figure 3.25. Action potentials of the ML model for two excitation current values...

Figure 3.26. Equivalent circuit of a non-myelinated axon

Figure 3.27. Axon represented by an RC line. For a color version of the figure, ...

Figure 3.28. Axon represented by an active line. Conductances G

Na

and G

K

are gov...

Figure 3.29. Generation of a spike at the soma-axon junction (red disc). Propaga...

Figure 3.30. Equivalent circuit of a myelinated axon. For a color version of the...

Figure 3.31. Shape of Excitation current, I

ex

(t), in red, and associated respons...

3.32. Excitatory postsynaptic potentials in red, inhibitory in green. For a colo...

Figure 3.33. Mathematical models of EPSPs given by expressions [3.50a, b, c]. Fo...

Figure 3.34. EPSP of an excitatory synapse for different synaptic weight values....

Figure 3.35. Principle of STDP, according to Bi and Poo. w is the synaptic weigh...

Chapter 4

Figure 4.1. Signal-flow graph of the perceptron. Coefficients w

i

are the synapti...

Figure 4.2. Signal-flow graph of the multilayer perceptron or feedforward networ...

Figure 4.3. Principle of a convolutional network

Figure 4.4. Signal-flow graph of a hidden-layer recurrent neural network. The re...

Figure 4.5. Graph of an “Elman”-type recurrent network. For a color version of t...

Figure 4.6. Graph of a “Hopfield”-type recurrent network, for which the intercon...

Figure 4.7. Principle of reservoir computing. The black arrows represent fixed-w...

Figure 4.8. Modeling of the Pavlov experiment with three neurons (N

1

“food”, N

2

...

Figure 4.9. Current control by ion channel in a biological membrane and by a tra...

Figure 4.10. Two “current-mirror” circuits enabling the relationship between I

1

...

Figure 4.11. Diagram of a DPI composed of a differential pair and two current mi...

Figure 4.12. Diagram of a DPI neuron. The yellow block models the leakage conduc...

Figure 4.13. Circuit diagram, transfer function V

out

(V

in

) and voltage gain Gv of...

Figure 4.14. Transfer function, V

out

(V

in

), of a subthreshold inverter (V

dd

=200 m...

Figure 4.15. Reduction of the effective subthreshold slope of a transistor throu...

Figure 4.16. Electrical circuit of an axon-hillock artificial neuron

Figure 4.17. Signal-flow graph of the amplifier (a), ideal transfer function (b)...

Figure 4.18. Time variation of potentials V

in

and V

out

in the case of the simpli...

Figure 4.19. Time variation of potentials V

in

and V

out

, obtained by SPICE simula...

Figure 4.20. Block diagram of the electronic circuit simulating the ML model

Figure 4.21. Complete schematic diagram of the eight-transistor electronic circu...

Figure 4.22. Simplified diagram of the ML neuron comprising only six transistors...

Figure 4.23. Temporal variation in membrane potential and sodium and potassium c...

Figure 4.24. Comparison between a very complete biomimetic neuron model (top) an...

Figure 4.25. Dendritic tree of post-neuron, j. V

mi

(or V

mk

, respectively) repres...

Figure 4.26. Definition of the memristance or fourth element based on state vari...

Figure 4.27. V(t)–I(t) curves when I = I

0

sin(ωt), for a resistance, R, an induct...

Figure 4.28. Architecture for the electronic implementation of the STDP. The two...

Figure 4.29. Principle of implementation of the STDP. (a) Scenario where t

pre

< ...

Figure 4.30. Circuit enabling highlighting of the stochastic resonance. Transcon...

Figure 4.31. Response of the neuron as a function of noise power. The membrane p...

Figure 4.32. Simulation of a cortical column by a reservoir-computing architectu...

Figure 4.33. General architecture of the TrueNorth chip (a) and detailed archite...

Guide

Cover

Table of Contents

Begin Reading

Pages

v

ii

iii

iv

ix

xi

xii

xiii

xiv

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

219

220

221

223

224

225

226

227

To my mentors, Professors Georges Salmer and Eugène Constant, who passed on to me their passion for research into semiconductor device physics

To Nadine, Hélène and Pierre

Series Editor

Robert Baptist

Neuro-inspired Information Processing

Alain Cappy

First published 2020 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd27-37 St George’s RoadLondon SW19 4EUUK

www.iste.co.uk

John Wiley & Sons, Inc.111 River StreetHoboken, NJ 07030USA

www.wiley.com

© ISTE Ltd 2020

The rights of Alain Cappy to be identified as the author of this work have been asserted by him in accordance with the Copyright, Designs and Patents Act 1988.

Library of Congress Control Number: 2019957598

British Library Cataloguing-in-Publication Data

A CIP record for this book is available from the British Library

ISBN 978-1-78630-472-8

Acknowledgments

I wish to thank my colleagues, Virginie Hoel, Christophe Loyez, François Danneville, Kevin Carpentier and Ilias Sourikopoulos, who have accompanied my work on neuro-inspired information processing. This book would not have been possible without our numerous discussions on this new research theme.

I would also like to thank Marie-Renée Friscourt for her diligent and efficient proofreading of the manuscript, and for the many insightful remarks made for the benefit of its improvement.

Introduction

The invention of the junction transistor in 1947 was undoubtedly the most significant innovation of the 20th Century, with our day-to-day lives coming to entirely depend on it. Since this date, which we will come back to later, the world has “gone digital”, with virtually all information processed in binary form by microprocessors.

In order to attain the digital world we know today, several steps were essential, such as the manufacture of the first integrated circuit in 1958. It soon became apparent that integrated circuits not only enabled the processing of analog signals, such as those used in radio, but also digital signals. Such digital circuits were used in the Apollo XI mission that led humankind onto the moon, on July 21, 1969. Astronauts only had very limited computing means at their disposal to achieve this spectacular feat. The flight controller was a machine that we might consider very basic by today’s standards. Composed of 2,800 integrated circuits, each comprising two three-input “NOR” gates, 2,048 words RAM1 and 38,000 words ROM2 for programs, it worked at a clock frequency of 80 kHz and weighed no more than 32 kg for 55 W power consumption.

The exploit was thus essentially based on “human” or “cortical” processing of information: processing power, too often advanced today, is not always the sine qua non condition for success!

In order to reduce the weight of processing systems, while improving their performance, it is necessary to incorporate a large number of logic gates into the same circuit. In 1971, this integration pathway led to a veritable revolution: the development of the first microprocessor. Since then, digital information processing technologies have witnessed tremendous progress, in terms of both their technical performance and their impact on society.

The world in which we live has become one of a “data deluge”, a term coined to describe the massive growth in the volume of data generated, processed and stored by digital media (audio and video), business transactions, social networks, digital libraries, etc. Every minute, for example, the Internet handles almost 200 million e-mails, 40 million voice messages, 20 million text messages, and 500,000 tweets. In 2016, the size of the digital universe, defined as the amount of data created, digitized and stored by human beings, was estimated at 16 ZB3 (zettabytes) and this figure is predicted to double every two years, i.e. 44 ZB in 2020 and 160 ZB in 2025. What a leap in just half a century!

This progression, symbolized by the famous “Moore’s law”4 , which predicted the doubling of microprocessor power5 every 18 months, occurred at constant price, i.e. the price of a modern microprocessor is much the same as that of the 1971 microprocessor, even though performance has been improved by more than five orders of magnitude.

This remarkable evolution was only made possible by the existence of a universal model of information processing machines, the Turing machine, and a technology capable of physically implementing these machines, that of semiconductor devices. More specifically, the “binary coding/Von Neumann architecture/CMOS technology” triplet has been the dominant model of information processing systems since the early 1970s.

Yet two limits have been reached at present: that of miniaturization, with devices not exceeding several nanometers in size, and that of power dissipated, with a barrier of the order of 100 Watts when the processor is working intensely.

As long as performance improved steadily, the search for new information processing paradigms was not ever a priority. With the foreseeable saturation of processor performance in the medium term, and also with the emergence of new application domains such as connected objects and artificial intelligence, the question of an information processing paradigm possessing both (i) high energy efficiency and (ii) superior performance in relation to current systems, in order to resolve certain types of problems, is resurfacing as a matter of some urgency.

This book, dedicated to neuro-inspired6 information processing, reflects these considerations. Its purpose is to offer students and researchers interested in this fascinating topic, a general overview of the current knowledge and state of the art, while heightening awareness of the innumerable questions posed and problems that remain unresolved.

Associating neuroscience, information technology, semiconductor physics and circuit design as well as mathematics and information theory, the subject matter addressed covers a wide variety of fields.

To enable the reader to progress uninterrupted through this book, they are regularly reminded of the basic concepts, or referred to the list of reference documents provided. Wherever possible, mathematical models of the phenomena studied are proposed, in order to enable an analysis that while simplified, offers a quantitative picture of the influence of the various parameters. This thinking aid using analytical formulations is, we believe, the condition for sound understanding of the physics of the phenomena involved.

This book is organized into four essentially independent chapters:

–

Chapter 1

introduces the basic concepts of electronic information processing, in particular coding, memorization, machine architecture and CMOS technology, which constitutes the hardware support for such processing. As one of the objectives of this book is to expand on the link between information processing and energy consumption, various ways of improving the performance of current systems are presented – particularly neuro-inspired processing, the central topic of this book. A fairly general comparison of the operating principles and performance of a modern microprocessor and of the brain is also presented in this chapter.

–

Chapter 2

is dedicated to the known principles of the functioning of the brain, and in particular those of the cerebral cortex, also known as “gray matter”. In this part, the approach is top-down, i.e. the cortex is first looked at from a global, functional perspective before we then study its organization as a basic processing unit, the cortical columns. An emblematic example, vision and the visual cortex, is also described to illustrate these different functional aspects.

–

Chapter 3

offers a detailed exploration of neurons and synapses, which are the building blocks of information processing in the cortex. Based on an in-depth analysis of the physical principles governing the properties of biological membranes, different mathematical models of neurons are described, ranging from the most complex to the simplest phenomenological models. Based on these models, the response of neurons and synapses to various stimuli is also described. This chapter also explores the principles of propagation of action potentials, or spike, along the axon, and examines how certain learning rules can be introduced into synapse models.

– Finally,

Chapter 4

covers artificial neural and synaptic networks. The two major approaches to creating these networks, using software or hardware, are presented, together with their respective performance. A state of the art is also given for each approach. In this chapter, we show the benefits of hardware in the design and creation of networks of artificial neural and synaptic networks with ultra-low power and energy consumption, and examples of artificial neural networks ranging from the very simple to the highly complex are described.

1

Memory that can be both written to and read from.

2

Read-only memory.

3

Zetta = 10

21

and one byte is made up of 8 bits.

4

Gordon Moore co-founded Intel in 1968.

5

Represented by the number of logic gates per circuit.

6

Also referred to as “bio-inspired”.

1Information Processing

1.1. Background

Since the beginning, humankind has forever created and used various techniques to process, transmit and memorize information, a vast domain summed up by the term “information processing”. This concept is very different from that of intelligence, and the reader should take care not to confuse these two ideas. This is a conflation commonly witnessed today owing to the buzz around the idea of artificial intelligence, which often only covers Big Data processing. Misuse of this term has seen the concepts of “intelligence” and “processing power” come to be erroneously employed interchangeably.

Intelligence is a much more complex matter that is beyond the scope of this book, referring instead to adaptation and imagination capacity. Moreover, Einstein is credited with the maxim: “The true sign of intelligence is not knowledge, but imagination.”

Generally speaking, information processing can be broken down into several phases, set out in Figure 1.1.

Let us elaborate on this figure a little. One might say that for a piece of information (a sign, sound, color, etc.) existing in the real world, the first operation necessary before any processing is its acquisition through encoding.

Figure 1.1.Block diagram of information processing systems. For a color version of this figure, see www.iste.co.uk/cappy/neuro.zip

Our senses, such as our vision and hearing, use sensors, in this case the retina and the cochlea, whose role it is to encode visual and auditory information so that they can be processed by the brain. If this same visual and auditory information from the real world is captured by a camera and a microphone, the encoding will be different and may be analog or binary. Once encoded, the information can be processed. Processing machines and their internal organization can be very diverse, ranging from the brain to computers, but all possess a device enabling information memorization and communication, whether this be within the machine itself, or with a similar machine. After processing, the information, which exists in the machine’s encoding system, will need to be decoded in order for it to be interpretable in the real world. Decoding is, for example, the role of the computer screen displaying a computation result.

Let us briefly develop on these basic concepts.

1.1.1. Encoding

All processing requires a prior information encoding operation. To illustrate this concept, let us imagine that the information in question is the temperature, T, of a physical environment.

Figure 1.2.Different information encoding types

Encoding can be analog (Figure 1.2a), in which case, the “temperature” magnitude will be represented after encoding by a continuous value1x (e.g. voltage or current), such that:

[1.1]

where is the encoding transfer function. If, for example, is a linear function, we will have x = α.T + β, where α and β are real constants.

When encoding is analog, the information processing must also be analog, in which case we speak of analog computers.

Another way to process this information is to code it in binary. In this case, we choose a pitch ΔT, and T will be represented by a number, M, of N bits such that:

[1.2]

One of the N bits (Figure 1.2b) will only be able to take on two values, arbitrarily noted 0 and 1. Binary coding offers numerous advantages: it enables the use of Boolean algebra and the set of associated algorithms for processing encoded information, and can easily be implemented with physical systems in two entirely distinct states. The switch, in particular, is either open (state 0) or closed (state 1).

Other encoding methods also exist. In the brain, for example, the organ focused on in this book, information is both time and frequency encoded by electrical impulses known as “action potential”, also referred to below as “spikes” (Figure 1.2c). In the case of frequency encoding, for example, the average frequency, < F >, of the impulses will be linked to the magnitude to be encoded by an expression of the type:

[1.3]

Where is the encoding transfer function. In the case of a linear transfer function, the frequency of the impulses will be given, for example, by < F > = γ.T + δ, where γ and δ are real constants.

In summary, encoding can be analog, i.e. represented by a continuous value, and can also consist of an alphabet, i.e. limited to a set of signs such as binary code {0,1} of Boolean algebra or the four bases of DNA {A, T, G, C}, which encode the genome. Lastly, it can be event-based and linked to the appearance of a temporally-determined phenomenon.

There is a close link between the choice of encoding and that of the technology of the processing machine that is to process the encoded information: analog or binary encoding for electronic processing, and pulse encoding for the brain.

1.1.2. Memorization

The artist who, over 30,000 years ago, painted the “Horses panel” in France’s Chauvet-Pont-d’Arc Cave (Chauvet 1994) was not only processing the information of the world around him, but also enabling it to be analogously memorized. The memorization technique used, rock painting, is simple but of the highest quality, because the message has been conserved right up to the present day!

Throughout history, information memorization has experienced two major revolutions: printing in the 15th Century and digital technology in the mid-20th Century. We can thus observe three characteristic periods. Before Gutenberg and the development of printing, a great number of techniques were used to store information, among them paint, clay, wax and paper. The printed book would come to dominate information memorization for almost five centuries, with this domination today replaced by digital technology. Nevertheless, in the digital world of today, there is a tendency to equate “memory” with “digital memory”, yet there are still many analog memories in existence, including not only painting and photography, but also sound on a vinyl record.

Regardless of the technology used, the memorization process consists of the same three phases: writing information on the memorization medium, storing it, and retrieving or reading it. The performance of a particular information memorization technology is characterized by quality criteria such as read and write speeds, storage durability, etc. Each of these performance elements is closely linked to both the quality of the encoding process and the machine processing the memorized information.

1.2. Information processing machines

In this section, we will turn our attention to artifacts, i.e. human-made machines that are used for information processing. This section focuses on dominant technology composed of binary coding, John von Neumann’s processing architecture and semiconductor devices for their material implementation.

1.2.1. The Turing machine

Alan Turing, the brilliant 20th-Century mathematician, used abstract and theory to demonstrate that any computing machine could be reduced to three elements (Figure 1.3):

– a tape of infinite length divided into squares, each containing a symbol (e.g. 0 or 1 in binary language) or a “blank” where the square has not yet been written. The tape represents the input/output peripheral device;

– a read/write head, which reads or writes the symbols and can move one square to the right or left;

– a state register, which memorizes the current state of the machine. There is a finite number of possible states. A “start state” is the initial state of the machine before executing a program;

– at each stage a program, or table of actions, defines whether the machine is to read or write and whether it is to move to the right or to the left, and specifies the new state.

The machine operates as follows: if the machine is in a state En, and the symbol read is s, i.e. the square beneath the read/write head contains the symbol s, then it writes the symbol e and moves in direction r (to the right) or l (to the left), and its new state is En+1.

Turing demonstrated that in binary logic, two functions simply needed to be created in order to perform any computational algorithm: one operating on one bit (the inverter) and the other on two bits (OR, AND, EXCLUSIVE OR, etc.).

Figure 1.3.The Turing machine and its practical realization (http://aturingmachine.com/)

The Turing machine was defined at a time when computers as we know them today did not yet exist. Thus, it is first and foremost an abstract tool, although there have been practical implementations of the machine, such as that shown in Figure 1.3. It is a universal machine model, which can compute anything a physical computer can. Conversely, any problem that cannot be solved by a Turing machine cannot be solved by a physical computer either, whatever its technology, and however powerful it is. The Turing machine concept has been extended to quantum computers, as we will see in section 1.4.2.1.

1.2.2. von Neumann architecture

Drawing on the machine concept proposed by Turing, John von Neumann proposed a specific architecture well suited to binary computing that can be emulated by an electronic circuit. It is composed of the following ingredients (Figure 1.4):

– a control unit, which manages data exchanges between blocks;

– an arithmetic logic unit, which performs operations;

– a memory, containing data as well as programs. Part of this memory, called the mass memory, can be external to the processing circuitry or the processor;

– input/output devices, such as a keyboard and screen, which enable the processor to exchange information and data with the outside world.

This remarkable architecture is generic and is used for all information processing machines and computers marketed to date.

To ensure the ease of exchange between blocks, the information exchanged (addresses, data or instructions) is in the form of words, which in principle are of fixed length. The first ever microprocessor, introduced by Intel in 1971, worked with 4-bit words. This means that in all exchanges, the 4 bits were transmitted in parallel. In order to increase their processing power, modern processors work with words 64 bits in length. The bits are transmitted in parallel by lines known as “buses”: the control bus, address bus and data bus.

Figure 1.4.von Neumann architecture

The machine’s operation is quite straightforward. Let us imagine that it is used to produce the sum, C, of two numbers, A and B: C = A + B.

First of all, numbers A and B need to be contained in the memory, either because they have been computed beforehand in one of the program’s instructions, or as they have been typed out on the keyboard, which is an input device. Before the “sum” operation, they are memorized at addresses MA and MB of the memory and the length of this data is known, for example N bits. Establishing the sum comes down to the following elementary operations:

1) retrieve

A

from the memory at address

M

A

and temporarily store in the arithmetic logic unit (ALU). This operation therefore involves memory reading and writing in the ALU;

2) similarly, retrieve

B

and store in the ALU;

3) once numbers

A

and

B

are available in the ALU, perform the “sum” instruction, which is in fact a micro-program performing the sum of

A

and

B

bit-to-bit, starting from the least significant bit and ending with the most significant. It ought to be recalled here that data

A

and

B

are composed of the same number of bits. When the bit-to-bit sum is performed, the sum number,

C

, is available in the ALU;

4) write the number

C

at the free address,

M

C

, in the memory;

5) where appropriate, display

C

on the screen (output device).

The example we have described here demonstrates that the elementary operations are accompanied by incessant memory readings and writings. We will return to this characteristic of von Neumann architecture later.

1.2.3. CMOS technology

To produce a machine using binary coding and reproducing von Neumann’s architecture, devices presenting two very distinct physical states are required. The most straightforward idea within this field is the switch, which can only be in open state (I = 0, whatever the value of V) or closed state (V = 0, whatever the value of I). Before the introduction of semiconductor devices, electromechanical relays or vacuum tubes were used as switches. The ENIAC (Electronic Numerical Integrator and Computer), the ancestor of modern computers, consisted, for example, of 17,468 vacuum tubes, 7,200 diodes, 1,500 relays, 70,000 resistors, 10,000 capacitors and approximately 5 million welds performed by hand (ENIAC 1945)! Fortunately, a revolutionary technology would come to replace the relays and vacuum tubes: semiconductor materials and devices.

As far back as the late 1940s, researchers at Bell Laboratories demonstrated that the conductivity of crystals of certain materials, such as germanium or silicon, which they named “semiconductors”, could be controlled through the addition of carefully-chosen impurities. This discovery rapidly came to be used to design and manufacture active semiconductor devices, i.e. capable of generating voltage, current, or power gains. With the first manufactured devices, bipolar junction transistors, mainly being used as analog amplifiers for radio applications, the first binary information processing applications came about with TTL (transistor–transistor logic) circuits. Given how complex and energy intensive this logic approach is, it will never be used to perform complex functions.

It was not until the early 1960s that another type of transistor, the MOSFET (metal–oxide–semiconductor field-effect transistor), saw the light of day. It was the use of the MOSFET that enabled the creation of the first microprocessors in the early 1970s, and these same devices are still used in today’s most modern processors.

NOTE.– A detailed description of the physics of this device would be beyond the scope of this book, but readers wishing to know more can refer to the numerous written books and online courses to further their knowledge.

Let us examine the transistor from a functional perspective. The MOS transistor has three electrodes: the gate, the source and the drain. The input value or control is the voltage applied between the gate and source (Vgs) and the controlled value, or output value, is current Ids flowing between the drain and source (Figure 1.5). Therefore, the transistor is basically a voltage-current transducer.

For any MOS transistor, there are two distinct modes of operation:

– below a control voltage, the “threshold”,

V

th

, the transistor is turned off and only a very low parasitic current, or leakage current, passes through it. This mode is essentially that of an open switch;

– beyond the threshold voltage,

V

th

, the transistor is turned on. The greater the control voltage, the higher the current passing through it (

Figure 1.5

). In this mode, the transistor can be assimilated to a (fairly good) closed switch.

Figure 1.5.Physical and electrical diagrams of NMOS and PMOS transistors. Current–voltage characteristic of a MOS transistor. Vth is the threshold voltage. For a color version of this figure, see www.iste.co.uk/cappy/neuro.zip

As such, MOSFETs are fairly good switches and can be associated as complementary pairs (an N-channel MOSFET and a P-channel MOSFET; Figure 1.5) to form a CMOS (complementary MOS) pair, the favored solution for creating logic gates in binary information processing circuits, particularly microprocessors.

In a CMOS inverter (Figure 1.6), NMOS and PMOS transistors have their gates connected to the input voltage, Vin, and their drains connected to the output voltage, Vout. It operates as follows:

– if

V

in

= 0

, the NMOS transistor is turned off (open switch) because

V

gs

= 0

and therefore

V

gs

< V

th

, and the PMOS is turned on (closed switch) because

V

sg

= V

dd

; therefore,

V

sg

> V

th

. Consequently,

V

out

= V

dd

. It is also to be noted that no current (other than an inevitable leakage current) circulates within the circuit between the power supply,

V

dd

, and the mass (potential 0) because the NMOS transistor is turned off;

– if

V

in

= V

dd

, the NMOS transistor is turned on (closed switch) because

V

gs

= V

dd

and therefore

V

gs

> V

th

, and the PMOS is turned off (open switch) because

V

sg

= 0; therefore,

V

sg

< V

th

. We will therefore have

V

out

= 0

and the current between

V

dd

and the mass is null, because the PMOS transistor is turned off;

– the transition between these two extreme states occurs at approximately

V

in

=V

dd

/2

, because in this case, both the transistors are turned on.

Figure 1.6.Inverter composed of NMOS and PMOS transistors and forming a CMOS pair. Form of input and output voltages. CL is the load capacitance of the gate

The significant feature of the CMOS approach is that in stable states not dependent on time, i.e. for constant values of Vin (Vin = 0 or Vin = Vdd), and therefore of Vout (Vout = Vdd or Vout = 0), the circuit consumes no electrical power2, because the current provided by the power supply is null. This property is valid not only for an inverter, but also for any multi-input CMOS logic gate. Thus, leakages aside, a logic gate using CMOS technology only consumes electrical energy during transitions between two stable states. We will expand at length on the properties of CMOS inverters, which are key devices in information processing, later in this chapter and again in Chapter 4.

Moreover, NMOS and PMOS transistors have a remarkable property (Dennard et al. 1974): downscaling (Figure 1.7), which can be expressed as follows:

If we divide all of the dimensions of a MOSFET transistor and the applied voltages by the same factor, k, the transistor speed is multiplied by k, the integration density, i.e. the number of transistors that can be integrated per unit surface area is multiplied by k2, the power dissipated per transistor is divided by k2 and the transit time3 is divided by k.

Figure 1.7.Principle of downscaling. All dimensions and voltages are divided by the same factor, k, while the semiconductor doping is multiplied by k. For a color version of this figure, see www.iste.co.uk/cappy/neuro.zip

This property, resulting from the physical equations describing MOS devices, is summarized in Table 1.1, which presents the effects of downscaling on a circuit’s main parameters.

Table 1.1.Scaling rules for a MOSFET transistor, and consequence of different circuit parameters

Parameter

Abbreviation

Scale factor

Gate length

L

g

1/k

Oxide thickness

t

ox

1/k

Gate width

W

1/k

Supply voltage

V

dd

1/k

Semiconductor doping

N

d

or

N

a

k

Maximum drain current

Id

max

1/k

Transistor surface area

S

1/k

2

Number of transistors per unit surface area

N

k

2

Gate capacitance

C

g

1/k

Transit time

τ

1/k

Maximum frequency

F

max

k