122,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
A bridge between the application of subspace-based methods for parameter estimation in signal processing and subspace-based system identification in control systems Model-Based Processing: An Applied Subspace Identification Approach provides expert insight on developing models for designing model-based signal processors (MBSP) employing subspace identification techniques to achieve model-based identification (MBID) and enables readers to evaluate overall performance using validation and statistical analysis methods. Focusing on subspace approaches to system identification problems, this book teaches readers to identify models quickly and incorporate them into various processing problems including state estimation, tracking, detection, classification, controls, communications, and other applications that require reliable models that can be adapted to dynamic environments. The extraction of a model from data is vital to numerous applications, from the detection of submarines to determining the epicenter of an earthquake to controlling an autonomous vehicles--all requiring a fundamental understanding of their underlying processes and measurement instrumentation. Emphasizing real-world solutions to a variety of model development problems, this text demonstrates how model-based subspace identification system identification enables the extraction of a model from measured data sequences from simple time series polynomials to complex constructs of parametrically adaptive, nonlinear distributed systems. In addition, this resource features: * Kalman filtering for linear, linearized, and nonlinear systems; modern unscented Kalman filters; as well as Bayesian particle filters * Practical processor designs including comprehensive methods of performance analysis * Provides a link between model development and practical applications in model-based signal processing * Offers in-depth examination of the subspace approach that applies subspace algorithms to synthesized examples and actual applications * Enables readers to bridge the gap from statistical signal processing to subspace identification * Includes appendices, problem sets, case studies, examples, and notes for MATLAB Model-Based Processing: An Applied Subspace Identification Approach is essential reading for advanced undergraduate and graduate students of engineering and science as well as engineers working in industry and academia.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 639
Veröffentlichungsjahr: 2019
Ähnliche
Table of Contents
Cover
Preface
References
Acknowledgements
Glossary
1 Introduction
1.1 Background
1.2 Signal Estimation
1.3 Model‐Based Processing
1.4 Model‐Based Identification
1.5 Subspace Identification
1.6 Notation and Terminology
1.7 Summary
MATLAB Notes
References
Problems
2 Random Signals and Systems
2.1 Introduction
2.2 Discrete Random Signals
2.3 Spectral Representation of Random Signals
2.4 Discrete Systems with Random Inputs
2.5 Spectral Estimation
2.6 Case Study: Spectral Estimation of Bandpass Sinusoids
2.7 Summary
Matlab Notes
References
Problems
3 State‐Space Models for Identification
3.1 Introduction
3.2 Continuous‐Time State‐Space Models
3.3 Sampled‐Data State‐Space Models
3.4 Discrete‐Time State‐Space Models
3.5 Gauss–Markov State‐Space Models
3.6 Innovations Model
3.7 State‐Space Model Structures
3.8 Nonlinear (Approximate) Gauss–Markov State‐Space Models
3.9 Summary
MATLAB Notes
References
Problems
4 Model‐Based Processors
4.1 Introduction
4.2 Linear Model‐Based Processor: Kalman Filter
4.3 Nonlinear State‐Space Model‐Based Processors
4.4 Case Study: 2D‐Tracking Problem
4.5 Summary
MATLAB Notes
References
Problems
5 Parametrically Adaptive Processors
5.1 Introduction
5.2 Parametrically Adaptive Processors: Bayesian Approach
5.3 Parametrically Adaptive Processors: Nonlinear Kalman Filters
5.4 Parametrically Adaptive Processors: Particle Filter
5.5 Parametrically Adaptive Processors: Linear Kalman Filter
5.6 Case Study: Random Target Tracking
5.7 Summary
MATLAB Notes
References
Problems
6 Deterministic Subspace Identification
6.1 Introduction
6.2 Deterministic Realization Problem
6.3 Classical Realization
6.4 Deterministic Subspace Realization: Orthogonal Projections
6.5 Deterministic Subspace Realization: Oblique Projections
6.6 Model Order Estimation and Validation
6.7 Case Study: Structural Vibration Response
6.8 Summary
MATLAB Notes
References
Problems
7 Stochastic Subspace Identification
7.1 Introduction
7.2 Stochastic Realization Problem
7.3 Classical Stochastic Realization via the Riccati Equation
7.4 Classical Stochastic Realization via Kalman Filter
7.5 Stochastic Subspace Realization: Orthogonal Projections
7.6 Stochastic Subspace Realization: Oblique Projections
7.7 Model Order Estimation and Validation
7.8 Case Study: Vibration Response of a Cylinder: Identification and Tracking
7.9 Summary
MATLAB NOTES
References
Problems
8 Subspace Processors for Physics‐Based Application
8.1 Subspace Identification of a Structural Device
8.2 MBID for Scintillator System Characterization
8.3 Parametrically Adaptive Detection of Fission Processes
8.4 Parametrically Adaptive Processing for Shallow Ocean Application
8.5 MBID for Chirp Signal Extraction
References
Appendix A: Probability and Statistics Overview
A.1 Probability Theory
A.2 Gaussian Random Vectors
A.3 Uncorrelated Transformation: Gaussian Random Vectors
A.4 Toeplitz Correlation Matrices
A.5 Important Processes
References
Appendix B: Projection Theory
B.1 Projections: Deterministic Spaces
B.2 Projections: Random Spaces
B.3 Projection: Operators
References
Appendix C: Matrix Decompositions
C.1 Singular‐Value Decomposition
C.2 QR‐Decomposition
C.3 LQ‐Decomposition
References
Appendix D: Output‐Only Subspace Identification
References
Index
End User License Agreement
List of Tables
Chapter 2
Table 2.1 Properties of covariance and spectral functions.
Table 2.2 Linear system with random inputs: covariance/spectrum relationships.
Chapter 3
Table 3.1 Gauss–Markov representation.
Table 3.2 ARMAX representation.
Table 3.3 Approximate nonlinear Gauss–Markov model.
Chapter 4
Table 4.1 Linear Kalman filter algorithm (predictor–corrector form).
Table 4.2 Heuristic notions for Kalman filter tuning.
Table 4.3 Statistical tuning tests for Kalman filter.
Table 4.4 Steady‐state Kalman filter algorithm.
Table 4.5 Linearized Kalman filter (LZKF) algorithm.
Table 4.6 Extended Kalman filter (EKF) algorithm.
Table 4.7 Iterated–extended Kalman filter (IEKF) algorithm.
Table 4.8 Discrete unscented Kalman filter (UKF) algorithm.
Table 4.9 Bootstrap particle filter (PF) SIR algorithm.
Chapter 5
Table 5.1 Augmented extended Kalman filter algorithm.
Table 5.2 Parametrically adaptive innovations model (RPE) algorithm.
Table 5.3 PF Performance towed array problem.
Chapter 6
Table 6.1 Ho–Kalman (SVD) realization algorithm
Table 6.2 Canonical realization algorithm.
Table 6.3 Multivariable output error state‐space algorithm (MOESP).
Table 6.4 Numerical algorithm for subspace state‐space system identification (N4...
Table 6.5 Deterministic subspace identification: three‐mass mechanical system.
Table 6.6 Deterministic subspace identification: eight‐mass mechanical system.
Chapter 7
Table 7.1 Iterative KSP algorithm (Riccati equation).
Table 7.2 Stochastic realization algorithm: Riccati equation approach.
Table 7.3 Stochastic realization algorithm: Kalman filter approach.
Table 7.4 Iterative KSP algorithm (Kalman filter).
Table 7.5 Stochastic realization algorithm: PAST OUTPUT‐MOESP (PO‐MOESP) approac...
Table 7.6 Stochastic subspace identification: six (6) sinusoidal system.
Table 7.7 Numerical algorithm for (stochastic) subspace state‐space system ident...
Table 7.8 Stochastic realization: statistical tests.
Table 7.9 Stochastic subspace identification: 3‐mass mechanical system.
Table 7.10 Stochastic subspace identification: six (6) modal frequencies.
Chapter 8
Table 8.1 Modal identification ensemble statistics.
2
Table B.1 Matrix Projections.
List of Illustrations
Chapter 1
Figure 1.1 Processing of a
deterministic
signal. (a) Raw data and spectrum with...
Figure 1.2 Processing of a
random
signal and noise. (a) Raw data and spectrum w...
Figure 1.3 Model‐based signal processing “model staircase”: Step
– simple mod...
Figure 1.4 Model‐based signal processing of a noisy measurement conceptually il...
Figure 1.5 Model‐based signal processing “model” for an RC‐circuit.
Figure 1.6 Model‐based processor for RC‐circuit: raw data (solid), true measure...
Figure 1.7 System identification problem: input (deterministic/random), disturb...
Figure 1.8 Model‐based identification of RC‐circuit parameters:
,
,
and
.
Figure 1.9 Identification of a sinusoidal model from noise‐free data: spectral ...
Chapter 2
Figure 2.1 Random signal comparison: random signal and spectrum and the determi...
Figure 2.2 Random signal processing: (a) Data and spectrum. (b) Covariance and ...
Figure 2.3 Ensemble of random walk realizations.
Figure 2.4 Signal and noise subspaces with two harmonic signal vectors and one ...
Figure 2.5 Bandpass sinusoidal problem (
,
,
,
,
Hz) spectral estimation: (a) ...
Chapter 3
Figure 3.1 Gauss–Markov model of a discrete process.
Figure 3.2 Gauss–Markov simulation of first‐order process: (a) state/measuremen...
Figure 3.3 Linearization of a deterministic system using the reference trajecto...
Chapter 4
Figure 4.1 Predictor–corrector form of the Kalman filter: Timing diagram.
Figure 4.2 LKF design for
‐circuit problem. (a) Estimated state (voltage) and ...
Figure 4.3 Nonlinear trajectory estimation: linearized Kalman filter (LZKF). (a...
Figure 4.4 Nonlinear trajectory estimation: extended Kalman filter (EKF). (a) E...
Figure 4.5 Nonlinear trajectory estimation: Iterated–extended Kalman filter (IE...
Figure 4.6 Nonlinear trajectory estimation: unscented Kalman filter (UKF). (a) ...
Figure 4.7 Particle filter representation of “empirical” posterior probability ...
Figure 4.8 PF surface (
vs
vs
) representation of posterior probability dis...
Figure 4.9 Nonlinear trajectory Bayesian estimation: Particle filter (PF) desig...
Figure 4.10 PMF data: Kullback–Leibler divergence and Hellinger distance calcul...
Figure 4.11 Tanker ground track geometry for the harbor docking application: (a...
Figure 4.12 EKF design for harbor docking problem (input known). (a)
‐position...
Figure 4.13 LZKF design for harbor docking problem (input known). (a)
‐positio...
Figure 4.14 UKF design for harbor docking problem (input unknown). (a)
‐positi...
Chapter 5
Figure 5.1 Parametrically adaptive processor structure illustrating the couplin...
Figure 5.2 Nonlinear (uncertain) trajectory estimation: parametrically adaptive...
Figure 5.3 Nonlinear (uncertain) trajectory estimation: Parametrically adaptive...
Figure 5.4 Nonlinear (uncertain) trajectory estimation: PF algorithm. (a) Estim...
Figure 5.5 PF posterior distribution estimation. (a) Estimated state posterior....
Figure 5.6 PF ensemble estimation for a
‐member realizations. (a) Ensemble sta...
Figure 5.7 Parametrically adaptive innovations state‐space processor: RC‐circui...
Figure 5.8 Synthetic aperture sonar tracking problem: simulated target motion f...
Figure 5.9 Particle filter bearing estimates for four targets in random motion:...
Figure 5.10 Particle filter predicted measurement estimates for four channel hy...
Figure 5.11 Particle filter classical “sanity” performance metrics: zero‐mean/w...
Figure 5.12 Particle filter instantaneous posterior bearing estimates: (a)
ta...
Chapter 6
Figure 6.1 Deterministic realization problem: classical approach and subspace a...
Figure 6.2 SVD realization. (a) Estimated singular values of the Hankel matrix....
Figure 6.3 The three‐output mechanical system under investigation.
Figure 6.4 Impulse response and spectrum of three‐output mechanical system. (a)...
Figure 6.5 SVD realization of three‐output mechanical system. (a) Singular valu...
Figure 6.6 Orthogonal projection operator: orthogonal complement operation.
Figure 6.7 Oblique projections: Projection
of future outputs (
) onto past da...
Figure 6.8 Extraction of future state and shifted state sequences: (a) Oblique ...
Figure 6.9 Pseudo‐random response of three‐output mechanical system. (a) Synthe...
Figure 6.10 SVD realization of three‐output mechanical system. (a) Power spectr...
Figure 6.11 SVD realization of three‐output mechanical system. (a) Raw and esti...
Figure 6.12 Structural vibrations of eight‐mode mechanical system: (a) accelero...
Figure 6.13 Structural vibration subspace identification eight‐mode mechanical ...
Figure 6.14 Subspace identification of eight‐mode mechanical system. (a) Raw da...
Chapter 7
Figure 7.1 Stochastic realization problem: (a) classical and (b) Subspace appro...
Figure 7.2 SVD order estimation of an unknown system from the Hankel covariance...
Figure 7.3 Simulated sinusoids in noise (
). (a) Noisy measurement. (b) Spectra...
Figure 7.4 Subspace identification of sinusoids in noise (
). (a) Spectral esti...
Figure 7.5 Comparison of identified sinusoids in noise (
). (a) Impulse respons...
Figure 7.6 Projections: orthogonal projection of
onto the joint subspaces
f...
Figure 7.7 Response of
‐output mechanical system to white noise excitation (
)...
Figure 7.8 N4SIDsubspace identification of
‐output mechanical system excited b...
Figure 7.9 N4SID‐subspace validation of
‐output mechanical system: signal esti...
Figure 7.10 N4SID‐subspace validation of
‐output mechanical system. (a) Channe...
Figure 7.11 Cylindrical object (copper pipe) vibrational response experiment: p...
Figure 7.12 Analysis of cylindrical object (pipe) single‐channel data: Fourier ...
Figure 7.13 Multichannel cylindrical object data: (a) Preprocessed triaxial cha...
Figure 7.14 Cylindrical object multichannel identification results: (a) Identif...
Figure 7.15 Stochastic realization of cylindrical object (pipe) power spectral:...
Figure 7.16 Subspace identification of cylindrical object,
‐mode model: (a) Es...
Figure 7.17 Postprocessing of modal frequency tracking (ensemble) statistics: (...
Figure 7.18 Postprocessed modal frequency tracking of cylindrical object data f...
Chapter 8
Figure 8.1 Structural device experimental setup: Motor‐driven stinger random vi...
Figure 8.2 Subspace identification of
‐mode model: (a) Estimated transfer func...
Figure 8.3 Postprocessing modal frequency tracking ensemble statistics: (a) Mod...
Figure 8.4 Postprocessed modal frequency tracking of (a)
‐mode, (b)
‐mode, (c...
Figure 8.5 Scintillator/photomultiplier system with equivalent circuit model.
Figure 8.6 Typical scintillator/photomultiplier response: (a) Pulse. (b) Bode p...
Figure 8.7 Cascaded scintillator/photomultiplier system model: (a) Diagram. (b)...
Figure 8.8 Deterministic state‐space model output for cascaded scintillator/pho...
Figure 8.9 Stochastic state‐space model output for cascaded scintillator/photom...
Figure 8.10 Subspace‐identified scintillator model: (a) Estimated and raw respo...
Figure 8.11 Parameter estimation for scintillator voltage response.
Figure 8.12 Estimated outputs for cascaded scintillator/photomultiplier system ...
Figure 8.13 Performance metrics for Kalman filter: (a) Innovation (
out). (b) ...
Figure 8.14 Kalman filter estimation for scintillator voltage response: (a) Pho...
Figure 8.15 Kalman filter performance for scintillator voltage response: (a) In...
Figure 8.16 As each individual interarrival is extracted, it is discriminated, ...
Figure 8.17 Fission detection/estimation processing: Count data, fission parame...
Figure 8.18 Sequential Bayesian (log‐likelihood) threat detection for
)=(
,
)...
Figure 8.19 Family of sequential Bayesian detection ROC curves for various SNRs...
Figure 8.20 Sequential Bayesian parameter estimates (arrows) and absolute RMS e...
Figure 8.21 Posterior PMF of various slices (
interarrivals) illustrating the ...
Figure 8.23 Modal function tracking with adaptive modal coefficient estimation:...
Figure 8.24 Adaptive modal coefficient parameter estimation data (MODEL) from t...
Figure 8.22 Raw/enhanced pressure‐field (DATA) data from the Hudson Canyon expe...
Figure 8.25 Raw pressure‐field data/enhanced data (DATA) from the Hudson Canyon...
Figure 8.26 Modal function tracking for adaptive wavenumber estimation: Hudson ...
Figure 8.27 Probability mass function (PMF) posterior estimation (mode
) surfa...
Figure 8.28 Probability mass function (PMF) posterior estimation (wavenumber
)...
Figure 8.29 True chirp and FSK spectra (20–30 kHz sweep): (a) Chirp spectrum. (...
Figure 8.30 Chirp data: (a) True chirp signal and disturbance with spectrum. (b...
Figure 8.31 Model‐based CHIRP identification: (a) Estimated chirp. (b) Linear c...
Figure 8.32 FSK data: (a) True FSK signal and disturbance with spectrum. (b) No...
Figure 8.33 Model‐based identification of FSK signal with disturbances: (a) Tru...
Guide
Cover
Table of Contents
Begin Reading
Pages
xiii
xiv
xv
xvi
xvii
xviii
20
21
xxi
xxiii
xxiv
xxv
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
185
186
187
188
189
190
191
192
193
194
195
197
196
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
286
285
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
362
361
363
364
365
366
367
368
369
370
371
372
373
374
375
376
378
377
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
477
478
479
480
481
482
483
484
485
486
487
488
489
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
Model-Based Processing
An Applied Subspace Identification Approach
James V. Candy
Lawrence Livermore National Laboratory University of California, Santa Barbara
Copyright
This edition first published 2019
© 2019 John Wiley & Sons, Inc.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.
The right of James V. Candy are to be identified as the author of this work has been asserted in accordance with law.
Registered Office
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
Editorial Office
111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print‐on‐demand. Some content that appears in standard print versions of this book may not be available in other formats.
Limit of Liability/Disclaimer of Warranty
MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This work's use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
Library of Congress Cataloging‐in‐Publication Data applied for
ISBN: 9781119457763
Cover design by Wiley
Cover image: Courtesy of Patricia Candy
Dedication
Jesus replied, “I am the way, the truth and the life. No one comes to the Father except through me.” (John: 14:6)
Preface
This text encompasses the basic idea of the model‐based approach to signal processing by incorporating the often overlooked, but necessary, requirement of obtaining a model initially in order to perform the processing in the first place. Here we are focused on presenting the development of models for the design of model‐based signal processors (MBSP) using subspace identification techniques to achieve a model‐based identification (MBID) as well as incorporating validation and statistical analysis methods to evaluate their overall performance 1. It presents a different approach that incorporates the solution to the system identification problem as the integral part of the model‐based signal processor (Kalman filter) that can be applied to a large number of applications, but with little success unless a reliable model is available or can be adapted to a changing environment 2. Here, using subspace approaches, it is possible to identify the model very rapidly and incorporate it into a variety of processing problems such as state estimation, tracking, detection, classification, controls and communications to mention a few 3,4. Models for the processor evolve in a variety of ways, either from first principles accompanied by estimating its inherent uncertain parameters as in parametrically adaptive schemes 5 or by extracting constrained model sets employing direct optimization methodologies 6, or by simply fitting a black‐box structure to noisy data 7,8. Once the model is extracted from controlled experimental data, or a vast amount of measured data, or even synthesized from a highly complex truth model, the long‐term processor can be developed for direct application 1 . Since many real‐world applications seek a real‐time solution, we concentrate primarily on the development of fast, reliable identification methods that enable such an implementation 9–11. Model extraction/development must be followed by validation and testing to ensure that the model reliably represents the underlying phenomenology – a bad model can only lead to failure!
System identification 6 provides solutions to the problem of extracting a model from measured data sequences either time series, frequency data or simply an ordered set of indexed values. Models can be of many varieties ranging from simple polynomials to highly complex constructs evolving from nonlinear distributed systems. The extraction of a model from data is critical for a large number of applications evolving from the detection of submarines in a varying ocean, to tumor localization in breast tissue, to pinpointing the epicenter of a highly destructive earthquake, or to simply monitoring the condition of a motor as it drives a critical system component 1 . Each of these applications require an aspect of modeling and fundamental understanding (when possible) of the underlying phenomenology governing the process as well as the measurement instrumentation extracting the data along with the accompanying uncertainties. Some of these problems can be solved simply with a “black‐box” representation that faithfully reproduces the data in some manner without the need to capture the underlying dynamics (e.g. common check book entries) or a “gray‐box” model that has been extracted, but has parameters of great interest (e.g. unknown mass of a toxic material). However, when the true need exists to obtain an accurate representation of the underlying phenomenology like the structural dynamics of an aircraft wing or the untimely vibrations of a turbine in a nuclear power plant, then more sophisticated representations of the system and uncertainties are clearly required. In cases such as these, models that capture the dynamics must be developed and “fit” to the data in order to perform applications such as condition monitoring of the structure or failure detection/prediction of a rotating machine. Here models can evolve from lumped characterizations governed by sets of ordinary differential equations, linear or nonlinear, or distributed representations evolved from sets of partial differential equations. All of these representations have one thing in common, when the need to perform a critical task is at hand – they are represented by a mathematical model that captures their underlying phenomenology that must somehow be extracted from noisy measurements. This is the fundamental problem that we address in this text, but we must restrict our attention to a more manageable set of representations, since many monographs have addressed problem sets targeting specific applications 12,13.
In fact, this concept of specialty solutions leads us to the generic state‐space model of systems theory and controls. Here the basic idea is that all of the theoretical properties of a system are characterized by this fundamental set of models that enables the theory to be developed and then applied to any system that can be represented in the state‐space. Many models naturally evolve in the state‐space, since it is essentially the representation of a set of th‐order differential equations (ordinary or partial, linear or nonlinear, time (space) invariant or time (space) varying, scalar or multivariable) that are converted into a set of first‐order equations, each of which is a state. For example, a simple mechanical system consisting of a single mass, spring, damper construct is characterized by a set of second‐order, linear, time‐invariant, differential equations that can simply be represented in state‐space form by a set of two first‐order equations, each one representing a state: one for displacement and one for velocity 12 . We employ the state‐space representation throughout this text and provide sufficient background in Chapters 2 and 3.
System identification is broad in the sense that it does not limit the problem to various classes of models directly. For instance, for an unknown system, a model set is selected with some perception that it is capable of representing the underlying phenomenology adequately, then this set is identified directly from the data and validated for its accuracy. There is clearly a well‐defined procedure that captures this approach to solve the identification problem 6 –15. In some cases, the class structure of the model may be known a priori, but the order or equivalently the number of independent equations to capture its evolution is not (e.g. number of oceanic modes). Here, techniques to perform order estimation precede the fitting of model parameters first, then are followed by the parameter estimation to extract the desired model 14. In other cases, the order is known from prior information and parameter estimation follows directly (e.g. a designed mechanical structure). In any case, these constraints govern the approach to solving the identification problem and extracting the model for application. Many applications exist, where it is desired to monitor a process and track a variety of parameters as they evolve in time, (e.g. radiation detection), but in order to accomplish this on‐line, the model‐based processor must update the model parameters sequentially in order to accomplish its designated task. We develop these processors for both linear and nonlinear models in Chapters 4 and 5.
Although this proposed text is designed primarily as a graduate text, it will prove useful to practicing signal processing professionals and scientists, since a wide variety of case studies are included to demonstrate the applicability of the model‐based subspace identification approach to real‐world problems. The prerequisite for such a text is a melding of undergraduate work in linear algebra (especially matrix decomposition methods), random processes, linear systems, and some basic digital signal processing. It is somewhat unique in the sense that many texts cover some of its topics in piecemeal fashion. The underlying model‐based approach of this text is the thread that is embedded throughout in the algorithms, examples, applications, and case studies. It is the model‐based theme, together with the developed hierarchy of physics‐based models, that contributes to its uniqueness coupled with the new robust, subspace model identification methods that even enable potential real‐time methods to become a reality. This text has evolved from four previous texts, 1 , 5 and has been broadened by a wealth of practical applications to real‐world, model‐based problems. The introduction of robust subspace methods for model‐building that have been available in the literature for quite a while, but require more of a systems theoretical background to comprehend. We introduce this approach to identification by first developing model‐based processors that are the prime users of models evolving to the parametrically adaptive processors that jointly estimate the signals along with the embedded model parameters 1 , 5 . Next, we introduce the underlying theory evolving from systems theoretic realizations of state‐space models along with unique representations (canonical forms) for multivariable structures 16,17. Subspace identification is introduced for these deterministic systems. With the theory and algorithms for these systems in hand, the algorithms are extended to the stochastic case, culminating with a combined solution for both model sets, that is, deterministic and stochastic.
In terms of the system identification area, this text provides the link between model development and practical applications in model‐based signal processing filling this critical gap, since many identification texts dive into the details of the algorithms without completing the final signal processing application. Many use the model results to construct model‐based control systems, but do not focus on the processing aspects. Again the gap is filled in the signal processing community, by essentially introducing the notions and practicalities of subspace identification techniques applied to a variety of basic signal processing applications. For example, spectral estimation, communications, and primarily physics‐based problems, which this text will demonstrate in the final chapters. It is especially applicable for signal processors because they are currently faced with multichannel applications, which the state‐space formulations in this text handle quite easily, thereby opening the door for novel processing approaches. The current texts are excellent, but highly theoretical, attempting to provide signal processors with the underlying theory for the subspace approach 9 – 12 . Unfortunately, the authors are not able to achieve this, in my opinion, because the learning curve is too steep and more suitable for control system specialists with a strong systems theoretical background. It is difficult for signal processors to easily comprehend, but by incorporating the model‐based signal processing approach, which is becoming more and more known and utilized by the signal processing community as the connection will enable the readers to gently “bridge the gap” from statistical signal processing to subspace identification for subsequent processing especially in multichannel applications. This is especially true with readers familiar with our previous texts in model‐based processing 1 , 6 . It will also have an impact in the structural dynamics area due to our case studies and applications introducing structural/test engineers to the model‐based identifiers/processors 16 . They already apply many of these identification techniques to their problem sets.
The approach we take is to introduce the concept of subspace identification by first discussing the ideas of signal estimation, identification to model‐based signal processing (MBSP) leading to the concept of model‐based identification (MBID) 1 , 5 . Here the model set is defined, and a variety of techniques ranging from the black‐box approach to well‐defined structural models employing parameter estimation techniques are developed. After introducing these concepts in the first chapter, random signals and systems are briefly discussed leading to the concept of spectral estimation, which provides an underlying cornerstone of the original identification problem.
Next, state‐space models are introduced in detail evolving from continuous‐time, sampled‐data to discrete‐time systems leading to the stochastic innovations model linking the classical Wiener filter to the well‐known Kalman filter 2 . With this in hand, multivariable (multiple‐input/multiple‐output) systems are developed simply as time series, to sophisticated canonical forms, leading to the matrix fraction transfer function descriptions. Chapter 3 is concluded with approximate nonlinear Gauss–Markov representations in state‐space form.
Model‐based processors are highlighted in the next two chapters 4 and 5 ranging from developments of the linear representations leading to the optimal Kalman filter 2 . Next the suite of nonlinear processors is developed initiated by the linearized processor leading to the special cases of the extended and unscented Kalman filters and culminating with the novel particle filter evolving from the Bayesian approach 5 . These techniques are extended to the joint signal/parameter estimation problem to create the parametric adaptive processors. Throughout these chapters, examples and case studies are introduced to solidify these fundamental ideas.
Next, we introduce the foundations for the heart of the text – subspace identification first constrained to deterministic systems. Here we develop the fundamental realization problem that provides the basis of subspace identification using Hankel matrices. Many of the underlying systems theoretical results introduced by Kalman 18 in the 1960s are captured by properties of the Hankel matrix. The problem is extended to the deterministic identification problem by incorporating input/output sequences 15 . Perhaps one of the most important contributions to realization theory is the concept of a balanced realization enabling the evolution of robust algorithms. All of these concepts are carefully developed in this chapter. Canonical realizations, that is, the identification of models in unique canonical forms is an important concept in identification 16 – 18 . Here, much of this effort has been ignored over the years primarily because the concept of a unique representation can lead to large errors when identifying the model. However, it is possible to show that they can also be considered a viable approach, since they transform the Hankel array to the so‐called structural matrix enabling both the order and parameters to be identified simultaneously, leading to an invariant system description 17 ,19. Finally, we introduce the ideas of projection theory showing how orthogonal/oblique projections lead to popular deterministic identification techniques 9 – 11 . Chapter 6 is concluded with a detailed case study on the identification application for a mechanical/structural system.
Chapter 7 is the extension of the deterministic identification problem to the stochastic case. Here, in the realization context, covariance matrices replace impulse response matrices, while deterministic input/output sequences are replaced with noisy multichannel sequences – the real‐world problem. As in Chapter 6, we develop stochastic realization theory starting with the “indirect” realization approach 4 based on covariance matrices for infinite and finite sequences to develop the basis of stochastic realization theory. The main ideas evolve from the work of Akaike 20,21 and the development of predictor spaces leading to the fundamental results from the systems theoretic viewpoint. The optimal solution to this problem proceeds from classical spectral factorization techniques leading to the steady‐state Kalman filter and the fundamental innovations model that is an integral part of subspace realizations 1 20 –29. Next, subspace methods are reintroduced for random vector spaces and provided as solutions to the stochastic realization problem followed by the so‐called combined subspace technique extracting both deterministic and stochastic models simultaneously 9 – 13 . This chapter concludes with a case study discussing the design of a processor to detect modal anomalies in an unknown cylindrical structure.
The text concludes with a chapter describing sets of real‐world applications of these techniques. The applications range from failure detection, to the threat detection of fission sources, to the identification of chirp signals for radar/sonar application, to the parametrically adaptive processor design for localization and tracking in the ocean environment, and to the design of an MBP chirp‐based signals as well as a critical radiation system – the scintillator.
Appendices are included for critical review as well as problem sets and notes for the MATLAB software used in the signal processing/controls/identification areas at the end of each chapter.
References
1 Candy, J. (2006).
Model‐Based Signal Processing
. Hoboken, NJ: Wiley/IEEE Press.
2 Kalman, R. (1960). A new approach to linear filtering and prediction problems.
Trans. ASME J. Basic Eng.
82: 34–45.
3 Van Der Veen, A., Deprettere, E., and Swindlehurst, A. (1993). Subspace‐based methods for the identification of linear time‐invariant systems.
Proc. IEEE
81 (9): 1277–1308.
4 Viberg, M. (1995). Subspace based signal analysis using singular value decomposition.
Automatica
31 (12): 1835–1851.
5 Candy, J. (2016).
Bayesian Signal Processing: Classical, Modern and Particle Filtering Methods
, 2e. Hoboken, NJ: Wiley/IEEE Press.
6 Ljung, L. (1999).
System Identification: Theory for the User
, 2e. Englewood Cliffs, NJ: Prentice‐Hall.
7 Ljung, L. and Soderstrom, T. (1983).
Theory and Practice of Recursive Identification
. Cambridge: MIT Press.
8 Soderstrom, T. and Stoica, P. (1989).
System Identification
. New York: Academic Press.
9 van Overschee, P. and De Moor, B. (1996).
Subspace Identification for Linear Systems: Theory, Implementation, Applications
. Boston, MA: Kluwer Academic Publishers.
10 Katayama, T. (2005).
Subspace Methods for System Identification
. London: Springer.
11 Verhaegen, M. and Verdult, V. (2007).
Filtering and System Identification: A Least‐Squares Approach
. Cambridge: Cambridge University Press.
12 Juang, J. (1994).
Applied System Identification
. Upper Saddle River, NJ: Prentice‐Hall PTR.
13 Aoki, M. (1990).
State Space Modeling of Time Series
, 2e. London: Springer.
14 Norton, J. (1986).
An Introduction to Identification
. New York: Academic Press.
15 Ho, B. and Kalman, R. (1966). Effective reconstruction of linear state variable models from input/output data.
Regelungstechnik
14: 545–548.
16 Luenberger, D. (1967). Canonical forms for linear multivariable systems.
IEEE Trans. Autom. Control
AC‐12: 290–293.
17 Candy, J., Warren, M., and Bullock, T. (1977). Realization of an invariant system description from Markov sequences.
IEEE Trans. Autom. Control
AC‐23 (12): 93–96.
18 Chen, C. (1984).
Linear System Theory and Design
. New York: Holt, Rinehart & Winston.
19 Guidorzi, R. (1975). Canonical structures in the identification of multivariable systems.
Automatica
11: 361–374.
20 Akaike, H. (1974). Stochastic theory of minimal realization.
IEEE Trans. Autom. Control
19: 667–674.
21 Akaike, H. (1975). Markovian representation of stochastic processes by canonical variables.
SIAM J. Control
13 (1): 162–173.
22 Faurre, P. (1976). Stochastic realization algorithms. In:
System Identification: Advances and Case Studies
(ed. R. Mehra and D. Lainiotis), 1–23. New York: Academic Press.
23 Larimore, W. (1990). Canonical variate analysis in identification, filtering and adaptive control. In:
Proceedings of the 29th Conference on Decision and Control
, Hawaii, USA, 596–604.
24 Tse, E. and Wiennert, H. (1975). Structure determination and parameter identification for Multivariable stochastic linear systems.
IEEE Trans. Autom. Control
20: 603–613.
25 Glover, K. and Willems, J. (1974). Parameterizations of linear dynamical systems: canonical forms and identifiability.
IEEE Trans. Autom. Control
19: 640–646.
26 Denham, M. (1974). Canonical forms for identification of multivariable linear systems.
IEEE Trans. Autom. Control
19: 646–656.
27 Candy, J., Bullock, T., and Warren, M. (1979). Invariant system description of the stochastic realization.
Automatica
15: 493–495.
28 Candy, J., Warren, M., and Bullock, T. (1978). Partial realization of invariant system descriptions.
Int. J. Control
28 (1): 113–127.
29 Sullivan, E. (2015).
Model‐Based Processing for Underwater Acoustic Arrays
. New York: Springer.
James V. Candy
Danville, CA
Acknowledgements
The support and encouragement of my wife, Patricia, is the major motivational element needed to undertake this endeavor. My family, extended family, and friends having endured the many regrets, but still offer encouragement in spite of all of my excuses. Of course, the constant support of my great colleagues and friends, especially Drs. S. Lehman, I. Lopez, E. Sullivan, and Mr. B. Beauchamp, who carefully reviewed the manuscript and suggested many improvements, cannot go without a hearty acknowledgment.
Glossary
ADC
analog‐to‐digital conversion
AIC
Akaike information criterion
AR
autoregressive (model)
ARMA
autoregressive moving average (model)
ARMAX
autoregressive moving average exogenous input (model)
ARX
autoregressive exogenous input (model)
AUC
area‐under‐curve (ROC curve)
BSP
Bayesian signal processing
BW
bandwidth
CD
central difference
CDF
cumulative distribution
CM
conditional mean
CRLB
Cramer–Rao lower bound
C‐Sq
Chi‐squared (distribution or test)
CT
continuous‐time
CTD
concentration–temperature–density (measurement)
CVA
canonical variate analysis
EKF
extended Kalman filter
EM
expectation–maximization
FPE
final prediction error
GLRT
generalized likelihood ratio test
G‐M
Gaussian mixture
GM
Gauss–Markov
G‐S
Gaussian sum
HD
Hellinger distance
HPR
high probability region
IEKF
iterated–extended Kalman filter
i.i.d.
independent‐identically distributed (samples)
KD
Kullback divergence
KL
Kullback–Leibler
KLD
Kullback–Leibler divergence
KSP
Kalman–Szego–Popov (equations)
LD
lower diagonal (matrix) decomposition
LE
Lyapunov equation
LKF
linear Kalman filter
LMS
least mean square
LS
least‐squares
LTI
linear, time‐invariant (system)
LZKF
linearized Kalman filter
MA
moving average (model)
MAICE
minimum Akaike information criterion
MAP
maximum a posteriori
MATLAB
mathematical software package
MBID
model‐based identification
MBP
model‐based processor
MBSP
model‐based signal processing
MC
Monte Carlo
MDL
minimum description length
MIMO
multiple‐input/multiple‐output (system)
MinE
minimum probability of error
ML
maximum likelihood
MOESP
multivariable output error state‐space algorithm
MMSE
minimum mean‐squared error
MSE
mean‐squared error
MV
minimum variance
N4SID
numerical algorithm for subspace state‐space system identification
NMSE
normalized mean‐squared error
N‐P
Neyman–Pearson (detector)
ODP
optimal decision (threshold) point
probability density function (continuous)
P‐E
probability‐of‐error (detector)
PEM
prediction error method
PF
particle filter
PI‐MOESP
past‐input multivariable output error state‐space algorithm
PMF
probability mass function (discrete)
PO‐MOESP
past‐output multivariable output error state‐space algorithm
PSD
power spectral density
RC
resistor capacitor (circuit)
REBEL
recursive Bayesian estimation library
RLC
resistor–inductor–capacitor (circuit)
RLS
recursive least‐squares
RMS
root mean‐squared
RMSE
root minimum mean‐squared error
ROC
receiver operating characteristic (curve)
RPE
recursive prediction error
RPEM
recursive prediction error method
SID
subspace identification
SIR
sequential importance sampling‐resampling
SIS
sequential importance sampling
SMC
sequential Markov chain
SNR
signal‐to‐noise ratio
SPRT
sequential probability ratio test
SPT
sigma‐point transformation
SSIS
sequential sampling importance sampling
SSP
state‐space processor
SSQE
sum‐squared error
SVD
singular‐value (matrix) decomposition
UD
upper diagonal matrix decomposition
UKF
unscented Kalman filter
UT
unscented transform
WSSR
weighted sum‐squared residual statistical test
W‐test
whiteness test
Z
Z‐transform
Z‐M
1Introduction
In this chapter, we introduce the idea of model‐based identification, starting with the basic notions of signal processing and estimation. Once defined, we introduce the concepts of model‐based signal processing, that lead to the development and application of subspace identification. Next, we show that the essential ingredient of the model‐based processor is the “model” that must be available either through the underlying science (first principles) or through the core of this text – model‐based identification.
1.1 Background
The development of processors capable of extracting information from noisy sensor measurement data is essential in a wide variety of applications, whether it be locating a hostile target using radar or sonar systems or locating a tumor in breast tissue or even locating a seismic source in the case of an earthquake. The nondestructive evaluation (NDE) of a wing or hull of a ship provides a challenging medium even in the simplest of arrangements requiring sophisticated processing especially if the medium is heterogeneous. Designing a controller for a smart car or a drone or for that matter a delicate robotic surgical instrument also depends on providing enhanced signals for feedback and error corrections. Robots replacing humans in assembly lines or providing assistance in mundane tasks must sense their surroundings to function in a such a noisy environment. Most “hi‐tech” applications require the incorporation of “smart” processors capable of sensing their operational environment, enhancing noisy measurements and extracting critical information in order to perform a preassigned task such as detecting a hostile target and launching a weapon or detecting a tumor and extracting it. In order to design a processor with the required capability, it is necessary to utilize as much available a priori information as possible. The design may incorporate a variety of disciplines to achieve the desired results. For instance, the processor must be able to sense the operational environment, whether it be highly cluttered electromagnetic propagation at an airport or a noisy ocean acoustic environment in a busy harbor. Array radiation measurements in the case of an active radar system targeting signals of great interest can detect incoming threats, while passive listening provided by an acoustic array aids in the detection of submarines or similarly tumors in the human body for ultrasonics as well. The ability of the processor to operate effectively in such harsh environments requires more and more sophistication, rather than just simple filtering techniques. It is here that we address not only the need, but also the a priori requirements for a design. For instance, the detection and localization of a quiet diesel submarine cannot be achieved without some representation of the noisy, varying ocean incorporated in the processing scheme. How does such information get embedded? This is the question for not only the signal processor, but also the ocean acoustician and sensor designer to ponder. The solution boils down to the melding of this information enabling the development of a processor capable of performing well. So we see that except in an exceptional case, the knowledge of the underlying phenomenology that governs just how a signal propagates in an uncertain medium or environment coupled with that of how a sensor can make a reasonable measurement to provide the desired information and a processor capable of extracting that information defines a “team” consisting of a phenomenologist, sensor designer and signal processor that can enable a solution to the problem at hand. In this text, we discuss such an approach that incorporates all of these capabilities. We start with the basic processor and then progress to a scheme capable of incorporating the underlying phenomenology, measurement systems, and uncertainties into the processor. In order to do so, we start with defining signal processing and signal estimation, followed by the fundamental model‐based signal processor and then approaches to obtain the required model from experimental as well as application data sets.
1.2 Signal Estimation
Signal processing is based on one fundamental concept – extracting critical information from uncertain measurement data [1,2]. Processing problems can lead to some complex and intricate paradigms to perform this extraction especially from noisy, sometimes inadequate measurements. Whether the data are created using a seismic geophone sensor from a monitoring network or an array of hydrophone transducers located on the hull of an ocean‐going vessel, the basic processing problem remains the same – extract the useful information. Techniques in signal processing (e.g. filtering, Fourier transforms, time–frequency and wavelet transforms) are effective; however, as the underlying process generating the measurements becomes more complex, the resulting processor may require more and more information about the process phenomenology to extract the desired information. The challenge is to formulate a meaningful strategy that is aimed at performing the processing required, even in the face of these high uncertainties. This strategy can be as simple as a transformation of the measured data to another domain for analysis or as complex as embedding a full‐scale propagation model into the processor [3]. For example, think of trying to extract a set of resonances (damped sinusoids) from accelerometer time series. It is nearly impossible to calculate zero‐crossings from the time series, but it is a simple matter to transform the data to the spectral domain using a Fourier transform and then applying the property that sinusoids are impulse‐like in Fourier space facilitating their extraction through peak detection. Finding a sinusoidal source propagating in the ocean is another matter that is quite complex due to the attenuation and dispersion characteristics of this harsh, variable environment. Here, a complex propagation model must be developed and applied to “unravel” the highly distorted data to reveal the source – a simple Fourier transform will no longer work. The aims of both approaches are the same – to extract the desired information and reject the extraneous and, therefore, develop a processing scheme to achieve this goal. The underlying signal processing philosophy is a “bottoms‐up” perspective enabling the problem to dictate the solution, rather than vice versa.
More specifically, signal processing forms the basic nucleus of many applications. It is a specialty area that many researchers/practitioners apply in their daily technical regimen with great success such as the simplicity in Fourier analysis of resonance data or in the complexity of analyzing the time–frequency response of dolphin sounds. Applications abound with unique signal processing approaches offering solutions to the underlying problem. For instance, the localization of a target in the hostile underwater ocean acoustic environment not only challenges the phenomenologist but also taxes the core of signal processing basics, thereby requiring that more sophistication and a priori knowledge be incorporated into the processor. This particular application has led to many advances both in underwater signal processing and in the development of a wide variety of so‐called model‐based or physics‐based processors. A prime example of this technology is the advent of the model‐based, matched‐field processor [ 3 –5] that has led not only to a solution of the target localization problem, but also to many applications in other areas such as nondestructive evaluation and biomedical imaging. Therefore, the conclusion remains the same, signal processing is a necessary ingredient as a working tool that must be mastered by phenomenologists to extract the useful information from uncertain measurements. In fact, we define signal processing as a set of techniques “to extract the desired information and reject the extraneous from uncertain measurement data.”
Signal processing relies on any prior knowledge of the phenomenology generating the underlying measurements. Characterizing this phenomenology and propagation physics along with the accompanying measurement instrumentation and noise are the preliminaries that all phenomenologists must tackle to solve such a processing problem. In many cases, this is much easier said than done. The first step is to determine what the desired information is and typically this is not the task of the signal processor, but that of the phenomenologist performing the study. In our case, we assume that the investigation is to extract information stemming from signals emanating from a source, whether it be an autonomous unmanned vehicle (AUV) on the highway or passively operating in the deep ocean, or a vibrating structure responding to ground motion. Applications can be very complex especially in the case of ultrasound propagating through complex media such as tissue in biomedical applications or through heterogeneous materials of critical parts in nondestructive evaluation (NDE) investigations or photons emanating from a radiation source [6]. In any case, the processing usually involves manipulating the measured data to extract the desired information, such as location and tracking of the AUV, failure detection for the structure, or tumor/flaw detection, and localization in both biomedical and NDE [3] .
If a measured signal is free from extraneous variations and is repeatable from measurement to measurement, then it is defined as a deterministic signal (Chapter 2). However, if it varies extraneously and is no longer repeatable, then it is defined as random signal. This text is concerned with the development of processing techniques to extract pertinent information from random signals utilizing any a priori information available. We call these techniques signal estimation or signal enhancement, and we call a particular algorithm a signal estimator or just estimator. Symbolically, we use the “caret” (^) notation to annotate an estimate (e.g. s → ŝ). Sometimes, estimators are called filters (e.g. Wiener filter) because they perform the same function as a deterministic (signal) filter except for the fact that the signals are random; that is, they remove unwanted disturbances. Noisy measurements are processed by the estimator to produce “filtered” data. To solidify these concepts, consider the following examples.
Example 1.1
A deterministic signal composed of two sinusoids: the information at 10 Hz and the disturbance at 20 Hz, with its corresponding Fourier spectrum shown in Figure 1.1a. From a priori information, it is known that the desired signal has no frequencies above 15 Hz; however, the raw spectrum reveals the disturbance at 20 Hz. Since the data are deterministic, a low‐pass filter with a cutoff frequency of 12.5 Hz is designed to extract the desired information (10 Hz signal) and reject the extraneous (20 Hz disturbance). The filtered data are shown in Figure 1.1 b, where we can see the filtered signal and the resulting spectrum.
Figure 1.1 Processing of a deterministic signal. (a) Raw data and spectrum with signal at Hz and disturbance at Hz. (b) Processed data extracting the Hz signal (desired) and rejecting the extraneous ( Hz disturbance).
Consider the output of the estimation filter designed to eliminate random noise from a transient signal; that is, the estimator is a function of the signal and noise
Consider the following example to illustrate this processor.
Example 1.2
A random pulse‐like signal contaminated by noise is shown in Figure 1.2a. Here, we see the measured data along with its Fourier spectrum. We design a signal estimator to extract the desired signal and remove the noise. The processed data are shown in Figure 1.2 b, where we observe the results of the estimation filter and the corresponding enhanced spectrum. Here, we see how the filtered response has eliminated the random noise. We discuss the concepts of signal estimation using the modern parametric design methods in Chapter 4.
If the estimation filter or more commonly the estimator employs a model of phenomenology or process under investigation, then it is considered a model‐based processor. For example, suppose we use a so‐called Gauss–Markov model (see Chapter 3) in our estimator design, that is, if we use
then the resulting signal estimator,
is called a model‐based signal processor and in this case a Kalman filter. Model‐based signal processing is discussed in Chapter 4. We shall see that random signals can be characterized by stochastic processes (Chapter 2), transformed to equivalent deterministic representations (covariance and power spectrum) and processed (model‐based processors, Chapters 4 and 5) much the same as a deterministic signal.
Figure 1.2 Processing of a random signal and noise. (a) Raw data and spectrum with noise. (b) Processed data extracting the signal (estimate) and rejecting the extraneous (noise).
Estimation can be thought of as a procedure made up of three primary parts:
Criterion function
Models
Algorithm.
The criterion function can take many forms and can also be classified as deterministic or stochastic. Models represent a broad class of information, formalizing the a priori knowledge about the process generating the signal, measurement instrumentation, noise characterization, and underlying probabilistic structure. Finally, the algorithm or technique chosen to minimize (or maximize) the criterion can take many different forms depending on (i) the models, (ii) the criterion, and (iii) the choice of solution. For example, one may choose to solve the well‐known least‐squares problem recursively or with a numerical‐optimization algorithm. Another important aspect of most estimation algorithms is that they provide a “measure of quality” of the estimator. Usually, what this means is that the estimator also predicts vital statistical information about how well it is performing.
Intuitively, we can think of the estimation procedure as follows:
The specification of a criterion
The selection of models from a priori knowledge
The development and implementation of an algorithm.
Criterion functions are usually selected on the basis of information that is meaningful about the process or the ease with which an estimator can be developed. Criterion functions that are useful in estimation can be classified as deterministic and probabilistic. Some typical functions are as follows:
Deterministic:
Squared error
Absolute error
Integral absolute error
Integral squared error
Probabilistic:
Maximum likelihood
Maximum a posteriori (Bayesian)
Maximum entropy
Minimum (error) variance.
Models can also be deterministic as well as probabilistic; however, here we prefer to limit their basis to knowledge of the process phenomenology (physics) and the underlying probability density functions as well as the necessary statistics to describe the functions. Phenomenological models fall into the usual classes defined by the type of underlying mathematical equations and their structure, namely, linear or nonlinear, differential or difference, ordinary or partial, time invariant or varying. Usually, these models evolve to a stochastic model by the inclusion of uncertainty or noise processes.
Finally, the estimation algorithm can evolve from various influences. A preconceived notion of the structure of the estimator heavily influences the resulting algorithm. We may choose, based on computational considerations, to calculate an estimate recursively, rather than as a result of a batch process because we require an online, pseudo‐real‐time estimate. Also each algorithm must provide a measure of estimation quality, usually in terms of the expected estimation error. This measure provides a means for comparing estimators. Thus, the estimation procedure is a combination of these three major ingredients: criterion, models, and algorithm.
1.3 Model‐Based Processing
Another view of the underlying processing problem is to decompose it into a set of steps that capture the strategic essence of the processing scheme. Inherently, we believe that the more “a priori” knowledge about the measurement and its underlying phenomenology we can incorporate into the processor, the better we can expect the processor to perform – as long as the information that is included is correct! One strategy called the model‐based approach provides the essence of model‐based signal processing (MBSP) [3] . Some believe that all signal processing schemes can be cast into this generic framework. Simply, the model‐based approach is “incorporating mathematical models of both physical phenomenology and the measurement process (including noise) into the processor to extract the desired information.” This approach provides a mechanism to incorporate knowledge of the underlying physics or dynamics in the form of mathematical process models along with measurement system models and accompanying noise as well as model uncertainties directly into the resulting processor. In this way, the model‐based processor (MBP) enables the interpretation of results directly in terms of the problem physics. It is actually a modeler's tool enabling the incorporation of any a priori information about the problem to extract the desired information. This approach of selecting the appropriate model is depicted in the signal processing staircase of Figure 1.3. We note that as we progress up the “modeling” steps to increase the (SNR), the complexity of the model increases to achieve the desired results. This is our roadmap [3]
