Autonomous Learning Systems E-Book

Plamen Angelov

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 389

Veröffentlichungsjahr: 2012

Beschreibung

Autonomous Learning Systems is the result of over a decade of focused research and studies in this emerging area which spans a number of well-known and well-established disciplines that include machine learning, system identification, data mining, fuzzy logic, neural networks, neuro-fuzzy systems, control theory and pattern recognition. The evolution of these systems has been both industry-driven with an increasing demand from sectors such as defence and security, aerospace and advanced process industries, bio-medicine and intelligent transportation, as well as research-driven – there is a strong trend of innovation of all of the above well-established research disciplines that is linked to their on-line and real-time application; their adaptability and flexibility.

Providing an introduction to the key technologies, detailed technical explanations of the methodology, and an illustration of the practical relevance of the approach with a wide range of applications, this book addresses the challenges of autonomous learning systems with a systematic approach that lays the foundations for a fast growing area of research that will underpin a range of technological applications vital to both industry and society. 

Key features: 

• Presents the subject systematically from explaining the fundamentals to illustrating the proposed approach with numerous applications.
• Covers a wide range of applications in fields including unmanned vehicles/robotics, oil refineries, chemical industry, evolving user behaviour and activity recognition.
• Reviews traditional fields including clustering, classification, control, fault detection and anomaly detection, filtering and estimation through the prism of evolving and autonomously learning mechanisms.
• Accompanied by a website hosting additional material, including the software toolbox and lecture notes.

Autonomous Learning Systems provides a ‘one-stop shop’ on the subject for academics, students, researchers and practicing engineers. It is also a valuable reference for Government agencies and software developers.

Leseprobe

Contents

Cover

Title Page

Copyright

Forewords

Adrian Stoica

Vladik Kreinovich

Arthur Kordon

Lawrence O. Hall

Preface

About the Author

Chapter 1: Introduction

1.1 Autonomous Systems

1.2 The Role of Machine Learning in Autonomous Systems

1.3 System Identification – an Abstract Model of the Real World

1.4 Online versus Offline Identification

1.5 Adaptive and Evolving Systems

1.6 Evolving or Evolutionary Systems

1.7 Supervised versus Unsupervised Learning

1.8 Structure of the Book

Part I: Fundamentals

Chapter 2: Fundamentals of Probability Theory

2.1 Randomness and Determinism

2.2 Frequentistic versus Belief-Based Approach

2.3 Probability Densities and Moments

2.4 Density Estimation – Kernel-Based Approach

2.5 Recursive Density Estimation (RDE)

2.6 Detecting Novelties/Anomalies/Outliers using RDE

2.7 Conclusions

Chapter 3: Fundamentals of Machine Learning and Pattern Recognition

3.1 Preprocessing

3.2 Clustering

3.3 Classification

3.4 Conclusions

Chapter 4: Fundamentals of Fuzzy Systems Theory

4.1 Fuzzy Sets

4.2 Fuzzy Systems, Fuzzy Rules

4.3 Fuzzy Systems with Nonparametric Antecedents (AnYa)

4.4 FRB (Offline) Classifiers

4.5 Neurofuzzy Systems

4.6 State Space Perspective

4.7 Conclusions

Part II: Methodology of Autonomous Learning Systems

Chapter 5: Evolving System Structure from Streaming Data

5.1 Defining System Structure Based on Prior Knowledge

5.2 Data Space Partitioning

5.3 Normalisation and Standardisation of Streaming Data in an Evolving Environments

5.4 Autonomous Monitoring of the Structure Quality

5.5 Short- and Long-Term Focal Points and Submodels

5.6 Simplification and Interpretability Issues

5.7 Conclusions

Chapter 6: Autonomous Learning Parameters of the Local Submodels

6.1 Learning Parameters of Local Submodels

6.2 Global versus Local Learning

6.3 Evolving Systems Structure Recursively

6.4 Learning Modes

6.5 Robustness to Outliers in Autonomous Learning

6.6 Conclusions

Chapter 7: Autonomous Predictors, Estimators, Filters, Inferential Sensors

7.1 Predictors, Estimators, Filters – Problem Formulation

7.2 Nonlinear Regression

7.3 Time Series

7.4 Autonomous Learning Sensors

7.5 Conclusions

Chapter 8: Autonomous Learning Classifiers

8.1 Classifying Data Streams

8.2 Why Adapt the Classifier Structure?

8.3 Architecture of Autonomous Classifiers of the Family AutoClassify

8.4 Learning AutoClassify from Streaming Data

8.5 Analysis of AutoClassify

8.6 Conclusions

Chapter 9: Autonomous Learning Controllers

9.1 Indirect Adaptive Control Scheme

9.2 Evolving Inverse Plant Model from Online Streaming Data

9.3 Evolving Fuzzy Controller Structure from Online Streaming Data

9.4 Examples of Using AutoControl

9.5 Conclusions

Chapter 10: Collaborative Autonomous Learning Systems

10.1 Distributed Intelligence Scenarios

10.2 Autonomous Collaborative Learning

10.3 Collaborative Autonomous Clustering, AutoCluster by a Team of ALSs

10.4 Collaborative Autonomous Predictors, Estimators, Filters and AutoSense by a Team of ALSs

10.5 Collaborative Autonomous Classifiers AutoClassify by a Team of ALSs

10.6 Superposition of Local Submodels

10.7 Conclusions

Part III: Applications of ALS

Chapter 11: Autonomous Learning Sensors for Chemical and Petrochemical Industries

11.1 Case Study 1: Quality of the Products in an Oil Refinery

11.2 Case Study 2: Polypropylene Manufacturing

11.3 Conclusions

Chapter 12: Autonomous Learning Systems in Mobile Robotics

12.1 The Mobile Robot Pioneer 3DX

12.2 Autonomous Classifier for Landmark Recognition

12.3 Autonomous Leader Follower

12.4 Results Analysis

Chapter 13: Autonomous Novelty Detection and Object Tracking in Video Streams

13.1 Problem Definition

13.2 Background Subtraction and KDE for Detecting Visual Novelties

13.3 Detecting Visual Novelties with the RDE Method

13.4 Object Identification in Image Frames Using RDE

13.5 Real-Time Tracking in Video Streams Using ALS

13.6 Conclusions

Chapter 14: Modelling Evolving User Behaviour with ALS

14.1 User Behaviour as an Evolving Phenomenon

14.2 Designing the User Behaviour Profile

14.3 Applying AutoClassify0 for Modelling Evolving User Behaviour

14.4 Case Studies

14.5 Conclusions

Chapter 15: Epilogue

15.1 Conclusions

15.2 Open Problems

15.3 Future Directions

Appendices

Appendix A: Mathematical Foundations

A.1 Probability Distributions

A.2 Basic Matrix Properties

Appendix B: Pseudocode of the Basic Algorithms

B.1 Mean Shift with Epanechnikov Kernel

B.2 AutoCluster

B.3 ELM

B.4 AutoCluster

B.5 AutoSense

B.6 AutoClassify0

B.7 AutoClassify1

B.8 AutoControl

References

Glossary

Index

This edition first published 2013 © 2013 John Wiley & Sons, Ltd

Registered office

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data:

Angelov, Plamen P. Autonomous learning systems : from data streams to knowledge in real-time / Plamen P. Angelov. pages cm Includes bibliographical references and index. ISBN 978-1-119-95152-0 (cloth)

1. Self-organizing systems. 2. Machine learning. I. Title. Q325.A54 2013 006.3′1–dc23 2012025907

A catalogue record for this book is available from the British Library.

ISBN: 978-1-119-95152-0

Preface

This book comes as a result of focused research and studies for over a decade in the emerging area that is on the crossroads of a number of well-known and well-established disciplines, such as (Figure 1):

Figure 1 Autonomous learning systems theory is build upon other well-established areas of research (the list is, of course, not exhaustive)

machine learning (ML);system engineering (specifically, system identification), SI;data mining, DM;statistical analysis, SA;pattern recognition including clustering, classification, PR;fuzzy logic and fuzzy systems, including neurofuzzy systems, FL;and so on.

On the one hand, there is a very strong trend of innovation of all of the above well-established branches of research that is linked to their online and real-time application; their adaptability, flexibility and so on (Liu and Meng, 2004; Pang, Ozawa and Kasabov, 2005; Leng, McGuinty and Prasad, 2005). On the other hand, a very strong driver for the emergence of autonomous learning systems (ALS) is industry, especially defence and security, but also aerospace and advanced process industries, the Internet, eHealth (assisted living), intelligent transport, and so on. The demand in defence was underpinned recently by a range of multimillion research and development projects funded by DARPA, USA (notably, two Grand Challenge competitions (Buehler, Iagnemma and Singh, 2010)); by MoD and BIS, UK (Defence Technology Centre on Systems Engineering and Autonomous Systems; ASTRAEA and GAMMA multimillion programmes, in which the author has played the research provider role, being the technical lead for several projects) and similar programmes in other European countries (France, Sweden, Spain, Czech Republic, Russia), and Israel. Major global companies have established their own programmes, such as IBM's autonomous computing initiative (IBM, 2009) and BT's intelligent network of autonomous elements (Detyenecki and Tateson, 2005). The International Neural Network Society (INNS) has established in 2011 a section on Autonomous Machine Learning of which the author is a founding member, together with scientists such as Bernard Widrow – the father of the famous least mean squares (LMS) algorithm.

This book attempts to address these challenges with a systematic approach that can be seen as laying the foundations of what can become a fast-growing area of research that can underpin a range of technological applications so needed by industry and society. The author does not claim that this represents a finished and monolithic theory; this is rather a catalyser for future developments, an inoculum, a vector pointing the direction rather than a full solution of the problems.

An important aim of preparing this book was also to make it a one-stop shop for students, researchers, practicing engineers, computer specialists, defence and industry experts and so on that starts with the motivation, presents the concept of the approach, describes details of the theoretical methodology based on a rigorous mathematical foundation, presents a wide range of applications, and more importantly, provides illustrations and algorithms that can be used for further research. The software (subject to a license) can be downloaded from the author's web site: http://www.lancs.ac.uk/staff/angelov/Downloads.htm. It is covered by USA patent # 2010-0036780, granted 21 August 2012 (priority date 1 Nov. 2006) and two pending patent applications and distributed by the spin-out company of Lancaster University called EntelSenSys Ltd. (www.entelsensys.com). From the same web site there will also be available for the readers for this book a set of lecture notes that will be a useful tool for delivering specialised short courses or an advanced Master level module as a part of various related programmes that cover the topics of machine learning, pattern recognition, control systems, computational intelligence, data mining, systems engineering, and so on.

The book was initially planned at the end of 2006 during the very successful IEEE Symposium on Evolving Fuzzy Systems held in Ambleside in the Lake District, UK but the turn of events (as usually happens) postponed its appearance by more than five years, which gave an opportunity for the concepts to mature and evolve further and new results and applications to be added.

It would not have become a reality without the support of the colleagues and collaborators, students, associates and visitors of the author. In the hope not to miss someone this includes Prof. Ronald Yager (Iona College, NY, USA), Dr. Dimitar Filev (Ford, MI, USA), Prof. Nikola Kasabov (Auckland University, New Zealand), Prof. Fernando Gomide (University of Campinas, Brazil), Dr. Xiaowei Zhou (HW Communications, UK), Dr. Jose Antonio Iglesias (University Carlos III, Madrid, Spain), Dr. Jose Macias Hernandez (CEPSA, Tenerife, Spain), Dr. Arthur Kordon (The Dow Chemical, TX, USA), Dr. Edwin Lughofer (Johan Kepler University, Linz, Austria), Prof. Igor Skrjanc (University of Ljubljana, Slovenia), Prof. Frank Klawonn (Ostfalia University of Applied Sciences, Germany), Mr. Jose Victor Ramos (University of Coimbra, Portugal), Dr. Ana Cara Belen (University of Granada), Mr. Javier Andreu (Lancaster University), Mr. Pouria Sadeghi-Tehran (Lancaster University), Mr. Denis Kolev (Rinicom Ltd.), Mrs. Rashmi Dutta Baruah (Lancaster University), Mr. Ramin Ramezani (Imperial College, London), Mr. Julio Trevisan (Lancaster University), and many more.

The feedback on the manuscript by Professor Vladik Kreinovich (University of Texas, USA) who is also President of the North American Fuzzy Information Processing Society (NAFIPS); Dr. Adrian Stoica, Senior Research Scientist at the Autonomous Systems Division, NASA Jet Propulsion Laboratory, Pasadena, CA, USA, Dr. Arthur Kordon, Team Leader at Dow Chemical, TX, USA; as well as from Dr. Larry Hall, Distinguished Professor and Chair at the Department of Computer Science and Engineering, University of South Florida, USA was also instrumental to improve and smooth out the presentation and remove some omissions.

Plamen Angelov November 2009–May 2012 Lancaster, UK

About the Author

The author, Dr Plamen Angelov, is a Reader in Computational Intelligence and coordinator of the Intelligent Systems Research Area at Infolab21, Lancaster University, UK. He is a Senior Member of the IEEE and of INNS (International Neural Networks Society) and Chair of the Technical Committee on Evolving Intelligent Systems, Systems, Man and Cybernetics Society, IEEE. He is also a member of the UK Autonomous Systems National Technical Committee and a founding member of the Centre of Excellence in CyberSecurity officially recognised by UK GCHQ for the period 2012–2017.

He has authored or coauthored over 160 peer-reviewed publications in leading journals (50+) peer-reviewed conference proceedings, three patents, two research monographs, half a dozen edited books, and has an active research portfolio in the area of computational intelligence and autonomous system modelling, identification, and machine learning. He has internationally recognised pioneering results into online and evolving methodologies and algorithms for knowledge extraction in the form of human-intelligible fuzzy rule-based systems and autonomous machine learning.

Dr. Angelov is a very active researcher leading numerous projects (over fifteen in the last five to six years) funded by UK and EU research councils, industry, HM Government, including UK Ministry of Defence (total funding of the order of tens of millions pounds with well over £1M for his group alone). His research contributes to the competitiveness of the industry, defence and quality of life and was recognised by ‘The Engineer Innovation and Technology 2008 Award in two categories: i) Aerospace and Defence and ii) The Special Award.

Dr. Angelov is also the founding Editor-in-Chief of Springer's journal on Evolving Systems and Associate Editor of the leading international scientific journals in this area, including IEEE Transactions on Systems, Man and Cybernetics, IEEE Transactions on Fuzzy Systems, Elsevier's Fuzzy Sets and Systems, Soft Computing, Journal on Automation, Mobile Robotics and Intelligent Systems Journal on Advances in Aircraft and Spacecraft Science and several others. He also chairs annual conferences organised by IEEE on Evolving and Adaptive Intelligent Systems, will be General co-Chair of the prime conferences on neural networks (IJCNN-2013, Dallas, Texas, August, Texas and fuzzy systems, FUZZ-IEEE-2014, June 2014, Beijing, China and on Cybernetics, CYBCO-2013, Lausanne, Switzerland), acted as Visiting Professor (in Brazil, Germany, Spain) regularly gives invited and plenary talks at leading conferences, universities and companies More information can be found at his web site www.lancs.ac.uk/staff/angelov.

The evolving face of the author (who is, of course an autonomous learning and evolving system himself) can be seen below: