115,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Learn to bridge the gap between machine learning and metaheuristic methods to solve problems in optimization approaches
Few areas of technology have greater potential to revolutionize the globe than artificial intelligence. Two key areas of artificial intelligence, machine learning and metaheuristic computation, have an enormous range of individual and combined applications in computer science and technology. To date, these two complementary paradigms have not always been treated together, despite the potential of a combined approach which maximizes the utility and minimizes the drawbacks of both.
Machine Learning and Metaheuristic Computation offers an introduction to both of these approaches and their joint applications. Both a reference text and a course, it is built around the popular Python programming language to maximize utility. It guides the reader gradually from an initial understanding of these crucial methods to an advanced understanding of cutting-edge artificial intelligence tools.
The text also provides:
- Treatment suitable for readers with only basic mathematical training
- Detailed discussion of topics including dimensionality reduction, clustering methods, differential evolution, and more
- A rigorous but accessible vision of machine learning algorithms and the most popular approaches of metaheuristic optimization
Machine Learning and Metaheuristic Computation is ideal for students, researchers, and professionals looking to combine these vital methods to solve problems in optimization approaches.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 732
Veröffentlichungsjahr: 2024
Ähnliche
Table of Contents
Cover
Table of Contents
Title Page
Copyright
About the Authors
Preface
Acknowledgments
Introduction
1 Fundamentals of Machine Learning
1.1 Introduction
1.2 Different Types of Machine Learning Approaches
1.3 Supervised Learning
1.4 Unsupervised Learning
1.5 Reinforcement Learning
1.6 Which Algorithm to Apply?
1.7 Recommendation to Build a Machine Learning Model
References
2 Introduction to Metaheuristics Methods
2.1 Introduction
2.2 Classic Optimization Methods
2.3 Descending Gradient Method
2.4 Metaheuristic Methods
2.5 Exploitation and Exploration
2.6 Acceptance and Probabilistic Selection
2.7 Random Search
2.8 Simulated Annealing
References
3 Fundamental Machine Learning Methods
3.1 Introduction
3.2 Regression
3.3 Classification
3.4 Decision Trees
3.5 Bayesian Classification
3.6 k-Nearest Neighbors (k-NN)
3.7 Clustering
3.8 Hierarchical Clustering
3.9 K-Means Algorithm
3.10 Expectation-Maximization Method
References
4 Main Metaheuristic Techniques
4.1 Introduction
4.2 Genetic Algorithms
4.3 Particle Swarm Optimization (PSO)
4.4 Differential Evolution (DE) Algorithm
References
5 Metaheuristic Techniques for Fine-Tuning Parameter of Complex Systems
5.1 Introduction
5.2 Differential Evolution (DE)
5.3 Adaptive Network-Based Fuzzy Inference System (ANFIS)
5.4 Differential Evolution for Fine-Tuning ANFIS Parameters Setting
References
6 Techniques of Machine Learning for Producing Metaheuristic Operators
6.1 Introduction
6.2 Hierarchical Clustering
6.3 Chaotic Sequences
6.4 Cluster-Chaotic-Optimization (CCO)
6.5 Computational Procedure
6.6 Implementation of the CCO Algorithm in MATLAB
6.7 Spring Design Optimization Problem Using the CCO Algorithm in MATLAB
References
7 Techniques of Machine Learning for Modifying the Search Strategy
7.1 Introduction
7.2 Self-Organization Map (SOM)
7.3 Evolutionary-SOM (EA-SOM)
7.4 Computational Procedure
7.5 Implementation of the EA-SOM Algorithm in MATLAB
7.6 Gear Design Optimization Problem Using the EA-SOM Algorithm in MATLAB
References
8 Techniques of Machine Learning Mixed with Metaheuristic Methods
8.1 Introduction
8.2 Flower Pollination Algorithm (FPA)
8.3 Feedforward Neural Networks (FNNs)
8.4 Training an FNN Using FPA
References
9 Metaheuristic Methods for Classification
9.1 Introduction
9.2 Crow Search Algorithm (CSA)
9.3 CSA for Nearest-Neighbor Method (k-NN)
9.4 CSA for Logistic Regression
9.5 CSA for Fisher Linear Discriminant
9.6 CSA for Naïve Bayes Classification
9.7 CSA for Support Vector Machine
References
10 Metaheuristic Methods for Clustering
10.1 Introduction
10.2 Cuckoo Search Method (CSM)
10.3 Search Strategy for CSM
10.4 Computational Procedure
10.5 Implementation of the CSM in MATLAB
10.6 Cuckoo Search Method for K-Means
References
11 Metaheuristic Methods for Dimensional Reduction
11.1 Introduction
11.2 Ant Colony Optimization (ACO)
11.3 Dimensionality Reduction
11.4 ACO for Feature Selection
References
12 Metaheuristic Methods for Regression
12.1 Introduction
12.2 Genetic Algorithm (GA)
12.3 Neural Network Regression with Artificial Genetic
12.4 Linear Regression Employing an Artificial Genetic
References
Index
End User License Agreement
List of Tables
Chapter 2
Table 2.1 Characteristics of the solutions used in the numerical example for...
Table 2.2 Parameters of the simulated annealing method implemented in code 2...
Chapter 3
Table 3.1 The frequency distribution of counts within the training set of fr...
Table 3.2 Training dataset for a numerical example of the Naïve Bayes method...
Table 3.3 Confusion matrix for a two-class classifier.
Table 3.4 Data used in the numerical example for classification.
Table 3.5 Data used in the numerical example for prediction.
Table 3.6 Data used in the numerical example for Ward’s method.
Table 3.7 Data used in the numerical example for the use of the MATLAB funct...
Chapter 8
Table 8.1 Types of pollination.
Chapter 9
Table 9.1 Test scores and results of the applicants.
Chapter 10
Table 10.1 Typical parameter configuration for the CSM.
List of Illustrations
Chapter 1
Figure 1.1 Peirce’s triangle symbolizes a framework outlining the connection...
Figure 1.2 Representation of deductive and inductive learning in the context...
Figure 1.3 Hierarchical classification of machine learning approaches.
Figure 1.4 Structure for supervised learning.
Figure 1.5 Effects of the clustering process under unsupervised learning.
Figure 1.6 Generalized representation of a system utilizing reinforcement le...
Figure 1.7 Visual guide to the comprehensive process of machine learning. Th...
Chapter 2
Figure 2.1 Graphical representation of the optimization problem formulated i...
Figure 2.2 Graphical representation of the numerical calculation of the grad...
Figure 2.3 Graphical representation of the function
Figure 2.4 Path of the solutions obtained by the gradient descent algorithm ...
Figure 2.5 Examples of objective functions that (a) are multimodal or (b) ar...
Figure 2.6 Optimization process from the point of view of metaheuristic algo...
Figure 2.7 Representación gráfica de la función
f
(
x
1
,
x
2
)
.
Figure 2.8 Solution map on (a) contour with level values and (b) with graysc...
Figure 2.9 Solution map on (a) the contour with the values and (b) the cooli...
Chapter 3
Figure 3.1 Ideal line and the observed data generated by adding noise to the...
Figure 3.2 Decision tree that separates the data into the specified classes,...
Figure 3.3 Distribution of an artificial dataset of 24 data points. The blac...
Figure 3.4 Relationship among the variables in the Iris flower dataset.
Figure 3.5 The scatter plot of the dataset features of the three iris specie...
Figure 3.6 Feature distribution of the dataset according to petal length vs ...
Figure 3.7 Structure of the classification process as executed by the result...
Figure 3.8 Classification regions produced by the Naïve Bayes method conside...
Figure 3.9 Visual representation of the k-Nearest Neighbors (k-NN) classific...
Figure 3.10 Hypothetical dendrogram considered as an example.
Figure 3.11 Partitioning clustering process considering three groups.
Figure 3.12 Produced cluster from de data distribution. (a) Initial data dis...
Figure 3.13 Produced clusters from the numerical example.
Figure 3.14 Illustrative dendrogram that emerges as the outcome of hierarchi...
Figure 3.15 Data distribution (a) and region of influence under the first it...
Figure 3.16 Clustering of data after one iteration of the K-means algorithm....
Figure 3.17 Result of running Code 3.6 showing the example data of the K-mea...
Figure 3.18 Clustering process using the expectation-maximization (EM) metho...
Figure 3.19 Generation of the data set, (a) mixture probability distribution...
Figure 3.20 Results of clustering with the EM method, (a) mixture probabilit...
Chapter 4
Figure 4.1 Operational process diagram of genetic algorithms.
Figure 4.2 Representation of a Potential Solution for the can design optimiz...
Figure 4.3 A randomly generated group of six cans and the calculation of the...
Figure 4.4 A roulette wheel representation for five solutions, segmented bas...
Figure 4.5 Execution of the Roulette-Wheel Selection mechanism.
Figure 4.6 Variety of crossover mechanisms.
Figure 4.7 Process of mutation.
Figure 4.8 Cost function example for the implementation of the binary GA.
Figure 4.9 The outcome produced upon executing Program 4.1.
Figure 4.10 Impact of the Simulated Binary Crossover (SBX) Operation with (a...
Figure 4.11 Impact of the Simulated Binary Crossover (SBX) Operation with (a...
Figure 4.12 The Ellipsoidal Probability Distribution in the UNDX Method.
Figure 4.13 Different Sets of solutions generated across two generations usi...
Figure 4.14 Creation of Solutions Using the PCX Crossover Method.
Figure 4.15 Distinct Sets of Solutions Generated by the PCX Crossover: (a) u...
Figure 4.16 Impact of mutation operations in the scenarios of (a) uniform an...
Figure 4.17 Impact of Simulated Binary Mutation operations in scenarios of (...
Figure 4.18 Example of a cost function for implementing the real-parameter g...
Figure 4.19 Progression of the optimization process conducted by the GA, dis...
Figure 4.20 Visual representation of the disparity between two randomly sele...
Figure 4.21 Visual depiction of combining the scaled difference of two vecto...
Figure 4.22 Visual depiction of the replacement of the original vector by th...
Figure 4.23 Visual depiction of the mutation operator employed in the differ...
Figure 4.24 Visual depiction of the crossover mechanism employed in the sear...
Chapter 5
Figure 5.1 Graphical description of two vectors randomly selected by the DE ...
Figure 5.2 Graphical representation of the substitution of element 1 by the ...
Figure 5.3 Differential Evolution (DE) main functioning flowchart.
Figure 5.4 Differential Evolution (DE) optimal value to sphere function.
Figure 5.5 Adaptive Network-based Fuzzy Inference System (ANFIS) structure....
Figure 5.6 Flowchart of the ANFIS hybridization structure.
Figure 5.7 ANFIS_DE response after training method regarding original data....
Chapter 6
Figure 6.1 Dendrogram structure.
Figure 6.2 Output from Algorithm 6.1. (a) Shows the produced dendrogram from...
Figure 6.3 Chaotic map of the ICMIC.
Figure 6.4 Local attraction movement. (a) Cluster with a high density and (b...
Figure 6.5 Process of local perturbation for a generic individual configurat...
Figure 6.6 Extra-cluster procedure. (a) Global attraction movement and (b) g...
Figure 6.7 The graph of the Ackley function produced by CCO.
Figure 6.8 Design parameters of the simple spring design problem.
Chapter 7
Figure 7.1 Representation of a Self-Organization Map (SOM).
Figure 7.2 Representation of a SOM considering initial weights for each inpu...
Figure 7.3 Topological structure of a Self-Organization Map (SOM). (a) Recta...
Figure 7.4 Output from Algorithm 7.1. (a) Shows the datapoints and the netwo...
Figure 7.5 Effect of the data-reduction and clustering in EA-SOM. (a) Benchm...
Figure 7.6 Visual representation of
δ
considering a benchmark function,...
Figure 7.7 The graph of the Bird function produced by EA-SOM.
Figure 7.8 Design parameters of the gear train design problem.
Chapter 8
Figure 8.1 Flower pollination algorithm (FPA) flowchart.
Figure 8.2 The Rosenbrock fitness function employed for the optimization pro...
Figure 8.3 Schematic of the perceptron.
Figure 8.4 Feedforward neural network.
Figure 8.5 Flowchart of the training process using the flower pollination al...
Chapter 9
Figure 9.1 Crow search algorithm (CSA) flowchart.
Figure 9.2 Sphere test function graphical representation.
Figure 9.3 Graphical result of classification of CSA-k-NN.
Figure 9.4 Logistic function or sigmoidal.
Figure 9.5 Logistic regression of accepted, rejected positions, and the deci...
Figure 9.6 Two-class data projected in
w
.
Figure 9.7 (a) Original data distribution, (b) data representation after the...
Figure 9.8 Training data distribution used in Algorithm 9.5.
Figure 9.9 Hyperplane adjustment considering the support vectors represented...
Figure 9.10 Result of classification using the CSA_NN and the percentage of ...
Chapter 10
Figure 10.1 Flowchart of the computational process for the CSM.
Figure 10.2 The graph of the Schwefel function produced by CSM.
Figure 10.3 Data separated by two clusters by the K-Means algorithm.
Figure 10.4 Data separated by two clusters by the CSM-KM algorithm. In Figur...
Chapter 11
Figure 11.1 The ACO pheromone representation consisting of
k
n
-dimensional s...
Figure 11.2 Ant colony optimization (ACO) flowchart.
Figure 11.3 The Rastrigin fitness function employed for the optimization pro...
Figure 11.4 Graph-like approach of ACO to the feature selection problem.
Figure 11.5 Flowchart for the ACO-based feature selection method.
Chapter 12
Figure 12.1 Roulette wheel selection based on fitness value information.
Figure 12.2 Genetic algorithm (GA) flowchart.
Figure 12.3 Fitness function employed for the optimization process, and best...
Figure 12.4 Graphical representation of the regression using a NN optimized ...
Figure 12.5 Graphical representation of the linear regression using the GA....
Guide
Cover
Table of Contents
Series Page
Title Page
Copyright
About the Authors
Preface
Acknowledgments
Introduction
Begin Reading
Index
End User License Agreement
Pages
ii
iii
iv
xi
xii
xiii
xiv
xv
xvi
xvii
xix
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
297
298
299
300
301
302
303
304
305
306
307
308
309
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
IEEE Press445 Hoes LanePiscataway, NJ 08854
IEEE Press Editorial BoardSarah Spurgeon, Editor-in-Chief
Moeness Amin
Jón Atli Benediktsson
Adam Drobot
James Duncan
Ekram Hossain
Brian Johnson
Hai Li
James Lyke
Joydeep Mitra
Desineni Subbaram Naidu
Tony Q. S. Quek
Behzad Razavi
Thomas Robertazzi
Diomidis Spinellis
Machine Learning and Metaheuristic Computation
Erik Cuevas, Jorge Galvez, Omar Avalos, and Fernando Wario
University of Guadalajara, Mexico
Copyright © 2025 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.
Library of Congress Cataloging-in-Publication Data Applied for:
Hardback ISBN: 9781394229642
Cover Design: WileyCover Image: © Doris – Magic Artist/Shutterstock
About the Authors
Erik Cuevas is currently working as a professor in the Department of Electronics at the University of Guadalajara, Mexico. He completed his BE in electronics and communication from the University of Guadalajara in 1996, and his postgraduate degree ME in applied electronics from ITESO, Guadalajara, Mexico, in 1998. He received his PhD in artificial intelligence from Freie Universität Berlin, Germany, in 2007. Dr. Cuevas currently serves as an associate editor in Applied Soft Computing, Applied Mathematical Modelling, Artificial Intelligence Review, International Journal of Machine Learning and Cybernetics, ISA Transactions, Neural Processing Letters, and Mathematics and Computers in Simulation. His research interests include metaheuristics and evolutionary computation in a wide range of applications such as image processing and machine learning.
Jorge Galvez serves as a full-time professor within the Department of Innovation Based on Information and Knowledge at the University Center for Exact Sciences and Engineering (CUCEI) of the University of Guadalajara, Mexico. In 2011, he earned a Bachelor of Science degree in informatics. In 2015, he obtained a Master of Science degree in electronics and computing. In 2019, he completed his Doctor of Philosophy degree in electronics and computer science from Universidad de Guadalajara. Since 2019, he has been a member of the Mexican National Research System (SNII I). His research agenda includes machine learning, deep learning, evolutionary computation, computer vision, complex systems, and robotics. Additionally, he serves as an editor for the Computation and Informatics section of the journal ReCIBE, which is published by the University of Guadalajara.
Omar Avalos is a professor in the Department of Electronics at the University of Guadalajara, Mexico. In 2013, he completed the BS degree in Communications and Electronics, and the MSc and PhD degrees in electronics and computer science from the Universidad de Guadalajara in 2015 and 2019, respectively. During his formation, he participated in different projects such as the Autonomus-Mini in collaboration with the Freie Universität Berlin and the development of a humanoid robot, just to mention a few. He is a member of the Mexican National Research System (SNI I). His research interests include different applications of Artificial Intelligence in different engineering fields, such as biomedical applications, improvement of energy consumption, system identification, and image processing.
Fernando Wario is an associate professor at the University of Guadalajara, Mexico, and an associate researcher at the Institute for Cognitive Sciences and Technologies (ISTC) of the Italian National Research Council (CNR). Fernando earned his PhD in computer science from the Freie Universtität Berlin in 2018, funded by a German Academic Exchange Service (DAAD) scholarship. Previous to his PhD studies, Fernando earned an MSc in electronic and computer science engineering delivered by the Universidad de Guadalajara in 2010 and a Bachelor in electronic engineering from the same institution in 2005. The research interest of Fernando spans over computational ethology, computer vision, heuristic optimization, complex systems, and swarm intelligence.
Preface
Machine Learning and Metaheuristic Computation represent two rapidly expanding domains within the realm of artificial intelligence, each boasting a vast array of applications in computer science. Machine learning, on one hand, encompasses a variety of deterministic and statistical methodologies designed to extract and interpret information from data. This field primarily focuses on identifying and analyzing patterns, a process crucial for understanding complex datasets. Metaheuristic computation, conversely, deals with optimization techniques that leverage collective insights from multiple search agents. This approach is instrumental in enhancing the precision of locating optimal solutions for diverse problems. While machine learning and metaheuristic computation are traditionally associated with different scientific communities, they share a fundamental reliance on advanced computational paradigms. When integrated, these fields can effectively complement each other, mitigating their individual limitations and amplifying their strengths. This synergy not only bridges the gap between two distinct areas of study but also unlocks new potentials in artificial intelligence research and applications.
Metaheuristic schemes have increasingly been utilized to augment machine learning methods, capitalizing on the fact that many processes of machine learning can be considered as optimization problems. This synergy has catalyzed the rapid development of a variety of classifiers and data analysis techniques. The fusion of machine learning and metaheuristic approaches has led to significant advancements in the field, enabling more sophisticated and effective methods for data identification and analysis. Conversely, machine learning techniques have been instrumental in the design and enhancement of metaheuristic algorithms. During the execution of metaheuristic methods, a substantial amount of data, including agent positions and corresponding objective values, is generated. This data trove offers a wealth of information, often underutilized. However, the application of machine learning techniques to this data has opened new avenues. By analyzing and extracting latent insights from the operational data of metaheuristic methods, Machine learning can significantly enhance the efficiency and effectiveness of these algorithms. This interplay between machine learning and metaheuristic schemes exemplifies a virtuous cycle, where each field informs and improves the other, leading to more refined and powerful computational tools.
This book aims to serve as a vital bridge connecting the communities of machine learning and metaheuristic methods, addressing a significant knowledge gap between these two fields. For enthusiasts and practitioners in machine learning who may not be well-versed in metaheuristic computation, this book is an essential resource. It elucidates that metaheuristic techniques are not merely theoretical constructs but practical tools capable of addressing and solving substantial problems frequently encountered in the realm of machine learning. These insights are particularly beneficial for those looking to expand their toolkit beyond conventional machine learning methodologies. Conversely, for those immersed in the world of metaheuristic computation, the book offers a valuable perspective on how machine learning problems can be effectively reformulated as optimization tasks. This recontextualization is key for metaheuristic practitioners aiming to apply their expertise to the nuanced challenges of machine learning. Overall, the book strives to foster a mutual understanding and collaboration between these two communities, highlighting how their combined strengths can lead to groundbreaking advancements in the field of artificial intelligence.
This book stands out not only for its comprehensive coverage of the most pivotal methods and techniques in machine learning and metaheuristic approaches but also for its extensive inclusion of MATLAB code and practical implementations. This aspect is considered the cornerstone of the book’s value. It acknowledges a crucial truth: even readers with a robust mathematical background may struggle to fully grasp a specific method or algorithm until they see it actualized in code. Implementations in the code serve to dissolve any ambiguities or uncertainties that might arise from theoretical descriptions alone, making the material significantly more accessible and clearer. This approach ensures that the learning process is not just theoretical, but also practical and intuitive. As readers progress through the book, moving from fundamental to more advanced methodologies, the computational focus – exemplified through the implemented code – aids in crystallizing their understanding of the different models. This not only reinforces their mathematical concepts but also provides a hands-on experience that is critical in comprehending and applying machine learning and metaheuristic strategies effectively.
This book uniquely positions itself not just as a reference text, but also as a comprehensive course, offering flexibility in how readers engage with its content. It is structured to allow random access, meaning readers can dive into specific chapters or themes as needed, without necessarily following a linear progression. Unlike other books that focus solely on theory and suggest that readers implement algorithms from scratch in exercises, this book takes a different approach. Drawing from our teaching experience, we’ve found that students learn more rapidly and effectively when they have access to existing code that they can modify and experiment with. This practical, hands-on methodology is central to our book’s design.
Many books in the field of machine learning and metaheuristic optimization are crafted with an audience that has advanced training in mathematical sciences in mind. However, after a thorough review of several related texts, it became evident that there is a demand for a more inclusive approach that caters to a wider array of readers and students. Recognizing this gap, the proposed book aims to bridge it by offering a broader and less technically dense exploration of these subjects. While it covers many of the same topics found in other works, this book shifts the focus towards a more accessible presentation. Emphasis is placed on the description, implementation, and practical application of methods rather than delving deeply into the mathematical intricacies. This approach makes the book more approachable for those who may not have a strong background in mathematics, broadening its appeal and utility. It aims to demystify complex concepts and make the fascinating world of machine learning and metaheuristic optimization accessible to a more diverse readership, encouraging engagement and learning across a broader spectrum of individuals.
To start learning machine learning and metaheuristic methods can be overwhelming considering the great number of mathematical concepts and computational techniques. Some very practical books try to satisfy this question by providing chapters where they indicate how to use off-the-shelf recipes for toy problems. However, what happens if some assumptions of the black-box model are not fulfilled. Is it possible to trust its results? Under such conditions, it is necessary to modify or adapt the model. To do this, it is important that a book provides the conceptual tools to appreciate and identify the underlying mathematics. The objective of this book is to be in the middle point, which means to give a rigorous but accessible vision of most of the machine learning algorithms and the most popular approaches of metaheuristic optimization.
Despite the advanced mathematical concepts underpinning machine learning and metaheuristic methods, it is entirely feasible to optimize systems or train models without an in-depth understanding of matrix algebra and calculus. For numerous readers, engaging with machine learning and metaheuristic techniques through programming rather than through complex mathematical formulations presents a more achievable goal. This approach aligns precisely with one of the key objectives of this book: to utilize MATLAB as a means to inspire readers to seamlessly integrate machine learning and optimization systems into their work. The strategy is to introduce and incorporate mathematical concepts only as necessary, easing the reader into the theoretical aspects without overwhelming them from the outset. This methodology not only makes the subject matter more accessible but also demonstrates the practical application of these concepts, thereby demystifying the process and encouraging a broader audience to delve into the fascinating world of machine learning and metaheuristics with confidence and curiosity.
Written from a teaching perspective, this book is primarily designed to function as a comprehensive textbook for undergraduate and postgraduate students specializing in science, electrical engineering, or computational mathematics. Its content and structure make it highly suitable for a range of courses, including artificial intelligence, machine learning, electrical engineering, evolutionary computation, and metaheuristics. The book’s utility extends beyond academic settings, as it also serves as a valuable reference for researchers in the machine learning and metaheuristic communities. The material, with its detailed yet accessible approach, is poised to enhance their knowledge and support their ongoing projects. Furthermore, the book is a significant resource for engineers and application developers. Recognizing the diverse needs and time constraints of professionals in these fields, the book is carefully structured to allow each chapter to be read independently. This feature enables readers to quickly access specific, relevant information, making it immensely practical for solving real-world industrial problems. Whether it is for academic purposes, research, or practical applications in various industries, this book offers a wealth of information that is easily navigable and highly applicable to a wide range of challenges and projects in the realms of machine learning and metaheuristics.
This book is organized into three distinct parts, each focusing on a crucial aspect of the interplay between machine learning and metaheuristic algorithms. The first part lays the foundation by discussing the main methods and algorithms that underpin both machine learning and metaheuristic approaches. This section serves as both an introduction and a deep dive into the fundamental concepts, providing readers with a solid understanding of the principles that guide these two areas of study. Moving forward, the second part shifts focus on machine learning techniques, specifically how they are applied within algorithms to enhance their search strategies. This part of the book is particularly insightful for those interested in how machine learning can be leveraged to optimize and refine the problem-solving processes inherent in algorithmic searches. Finally, the third section completes the circle by exploring the main algorithms and metaheuristic mechanisms that are employed to improve machine learning techniques. This includes a detailed examination of how metaheuristics can be tailored to improve the efficiency, accuracy, and overall performance of machine learning models. Together, these sections provide a comprehensive overview that not only demystifies complex concepts but also illustrates the synergistic potential of combining machine learning with metaheuristic algorithms to tackle a wide array of challenges in computational science and engineering.
The journey of writing this book was enriched and made possible by the invaluable contributions of numerous colleagues and students, each bringing forth a wealth of ideas and critical insights. In these lines, we extend our deepest gratitude to our esteemed colleagues and the dedicated students at the University of Guadalajara. Their thorough evaluation and constructive criticism have been instrumental in shaping the content and direction of this work.
23 June 2024
Erik Cuevas, Jorge Galvez, Omar Avalos,
and Fernando Wario
University of Guadalajara, Mexico
Acknowledgments
The journey of writing this book was enriched and made possible by the invaluable contributions of numerous colleagues and students, each bringing forth a wealth of ideas and critical insights. Their dedication and intellectual generosity have significantly enhanced the depth and quality of this work. In these lines, we extend our deepest gratitude to our esteemed colleagues and the dedicated students at the University of Guadalajara. Their rigorous evaluation and constructive criticism have been instrumental in shaping both the content and direction of this book. We are particularly grateful for their willingness to engage in meaningful dialogue, challenge assumptions, and provide diverse perspectives that have broadened our understanding and approach. We would also like to express our sincere appreciation to Nandhini Karuppiah, Victoria Bradshaw, and Aileen Storry for their exceptional efforts in the planning, development, and production of this book. Their hard work and commitment to excellence have been crucial in bringing this project to fruition.
Introduction
This book aims to serve as a significant link between the machine learning and metaheuristic methods communities, addressing the considerable knowledge gap between these two disciplines. Many books about machine learning and metaheuristic optimization have been written for readers with advanced mathematical backgrounds. However, the proposed book takes a different approach, providing a broader and less mathematically dense exploration of these topics. While this book covers many of the same subjects as other works, it focuses on a more accessible presentation. The emphasis is on describing, implementing, and applying methods in practical situations, rather than delving deeply into intricate mathematical concepts. This approach makes the book more suitable for individuals who may not have a strong background in mathematics, thus increasing its appeal and usefulness. The book aims to simplify complex ideas and make the fascinating world of machine learning and metaheuristic optimization accessible to a wide range of readers, encouraging engagement and learning across a diverse spectrum of individuals.
This book is distinguished not only by its comprehensive treatment of the most essential methods and techniques in machine learning and metaheuristic approaches but also by its extensive inclusion of MATLAB code and practical applications. This aspect is regarded as the cornerstone of the book’s value proposition, acknowledging an important truth: even readers with a strong mathematical foundation may struggle to fully comprehend a specific method or algorithm until they witness its implementation in code. The inclusion of code serves to dispel any ambiguities or uncertainties that may arise from theoretical descriptions alone, making the material considerably more accessible and clearer.
This book, written from a teaching perspective, serves as an in-depth textbook for undergraduate and graduate students specializing in science, electrical engineering, or computational mathematics. Its content and structure make it suitable for a broad range of applications, such as artificial intelligence, machine learning, electrical engineering, evolutionary computation, and metaheuristics. The book’s utility extends beyond academic settings, as it also serves as a valuable reference for researchers in machine learning and metaheuristic communities. The material, with its detailed yet accessible approach, is poised to enhance their knowledge and support ongoing projects. The book also serves as a significant resource for engineers and application developers, recognizing the diverse needs and time constraints of professionals in these fields. The book’s structure allows each chapter to be read independently, enabling readers to quickly access specific relevant information, making it practical for solving real-world industrial problems. This book offers a wealth of information that is easily navigable and highly applicable to a wide range of challenges and projects in the realms of machine learning and metaheuristics, whether for academic purposes, research, or practical applications in various industries.
1Fundamentals of Machine Learning
1.1 Introduction
In this chapter, we delve into the essential concepts of machine learning, aiming to offer a clear and detailed understanding of its key principles and approaches. We begin by exploring the basics, including data collection and preprocessing, and then advance to more complex topics such as choosing the right algorithms, training processes, and evaluating model performance. The chapter’s structure methodically unpacks machine learning’s complexities, ensuring that readers not only comprehend its theoretical foundations but also understand how to apply these concepts practically in diverse situations. This journey through machine learning will provide readers with a solid base of knowledge, encompassing critical areas like supervised and unsupervised learning, the importance of model optimization, and the pivotal role of data in creating precise and dependable models. This foundational understanding is crucial for anyone aspiring to explore machine learning more deeply or to effectively implement its techniques across various fields.
Machine learning (Alpaydin 2021) represents a fusion of disciplines, embodying the intersection and synergy of computer science, statistics, neurobiology, and control theory. This multidisciplinary field has emerged as a cornerstone in numerous domains, fundamentally altering the landscape of software programming. In the past, the pivotal question in computing was, “How to program a computer?” However, with the advent of machine learning, this query has evolved dramatically. Now, the question at the forefront is, “How will computers program themselves?” This shift signifies a profound change in our approach to computing, where the focus is on designing algorithms that enable machines to learn from and adapt to data, rather than just executing predefined instructions. This evolution not only expands the capabilities of computers but also reshapes our understanding of programming and problem-solving in the digital age.
Machine learning stands as a fundamental method that imparts computers with their own form of intelligence, mirroring the learning processes observed in humans. This field inherently interconnects with and draws parallels to the study and research of human learning. Just as the human brain and its complex network of neurons lay the groundwork for human insight and understanding, Artificial Neural Networks (ANNs) serve a similar role in the realm of computers. These ANNs form the crux of decision-making activities within machines, enabling them to process, analyze, and learn from data in a manner akin to human cognitive processes. This similarity underscores the deep, intrinsic relationship between machine learning and human learning, highlighting how machines are designed to emulate and learn from the intricate patterns of human intelligence and decision-making.
Machine learning enables us to discover models (Mahesh 2020) that describe a given set of data, essentially establishing a link between input variables and output variables within a system. This process often involves hypothesizing the presence of a mechanism responsible for the parametric generation of data. However, the precise values of these parameters are typically unknown and need to be inferred. To achieve this, machine learning utilizes a variety of statistical techniques, including Induction, Deduction, and Abduction. These methods collectively contribute to the process of understanding and interpreting the data. Induction allows for generalizing from specific instances to broader rules, Deduction involves applying general rules to specific instances, and Abduction aids in forming plausible hypotheses that explain the observed data. This interplay of techniques is crucial for developing accurate and reliable models that can effectively represent the underlying patterns and relationships within the data, as illustrated in Figure 1.1.
Induction, a key process in the realm of scientific inquiry, involves extracting general laws or principles from a specific set of observed data. This method stands in contrast to deduction, where the goal is to predict the value of specific variables based on pre-established general laws. Induction is foundational to the scientific method, serving as the primary mechanism through which general laws are derived. These laws, often articulated in mathematical language, are not assumed a priori but are instead developed through careful observation and analysis of phenomena. By observing patterns, trends, and relationships within the data, scientists use induction to formulate broad, overarching theories that can explain and predict a wide range of phenomena. This approach is central to the advancement of scientific knowledge, allowing for the conversion of discrete, individual observations into universally applicable laws and principles.
The process of observation in scientific and analytical contexts involves the measurement of various variables, leading to the acquisition of data that characterizes the observed phenomena. This data, rich in details and insights, forms the basis for developing models that can interpret and make sense of these observations. Once established, these models possess the capability to make predictions about new, unseen data. This entire procedure, where one starts with a collection of observations and progresses to making predictions for new scenarios, is known as inference. Inference is a fundamental component in many fields, particularly in statistics and machine learning, where it plays a crucial role in extrapolating from known data to predict future events, trends, or behaviors. It embodies the essence of learning from experience and applying that knowledge to new, often uncharted, situations.
Figure 1.1 Peirce’s triangle symbolizes a framework outlining the connections among various patterns of reasoning.
Inductive learning, a fundamental approach in the realm of knowledge acquisition and machine learning, begins with observations derived from the surrounding environment. This method involves analyzing and understanding these observations to generalize and extract broader knowledge or patterns. The goal of inductive learning is to develop insights or models that are not only applicable to the initially observed cases but also hold validity for scenarios that have not yet been encountered. Essentially, it’s a process of learning from specific instances and then applying that learning to make predictions or inferences about future, unseen situations. While there’s an inherent uncertainty in assuming that these generalizations will always apply to new cases, the strength of inductive learning lies in its ability to adapt and learn from ongoing experiences, constantly refining its understanding and predictions. Thus, inductive learning represents a hopeful endeavor to extrapolate from known data to the unknown, continuously expanding the frontiers of knowledge and prediction.
Inductive learning, a key methodology in understanding and interpreting data, can be broadly categorized into two distinct types:
Learning by Example
(Menon et al.
2013
): This approach is centered around gaining knowledge from a specific set of examples. It involves analyzing “positive examples,” which are instances embodying the concept that needs to be learned, and “negative examples,” which are instances that do not represent the concept. By examining these examples, the learning process discerns the defining characteristics of the concept, distinguishing what qualifies as an instance and what does not. This method is particularly prevalent in supervised learning scenarios in machine learning, where the model learns to classify or predict outcomes based on labeled training data.
Learning Regularity
(Gauci and Stanley
2008
): Unlike learning by example, learning regularity does not focus on a specific concept. Instead, the objective here is to identify patterns, trends, or common characteristics within the provided instances. This type of learning looks for underlying regularities or consistencies in the data that might not be immediately apparent. It’s more about uncovering hidden structures or relationships within the data rather than classifying or categorizing it. This form of inductive learning is often used in unsupervised learning scenarios, where the data does not come with predefined labels or categories.
Both types of inductive learning are crucial for understanding complex datasets and are widely applied in various fields, from artificial intelligence to statistical analysis, each serving different purposes but under the same structure of learning from and making inferences based on observed data.
Machine learning systems often outperform traditional algorithms in various complex scenarios, raising a natural question: Why is this the case? The reasons behind the limitations of traditional algorithms are multifaceted:
Difficulty in Problem Formalization
: Traditional algorithms require explicit programming based on a well-defined set of rules. However, many real-world problems are not easily formalized into computational steps. For instance, most people can recognize a friend’s voice but would struggle to describe the exact computational process to identify the speaker from a sound recording. This gap between human cognitive abilities and algorithmic description is where machine learning excels, as it learns to perform tasks without explicit programming.
High Number of Variables at Play
: In problems like character recognition from documents, the sheer number of variables involved makes the task daunting for traditional algorithms. Specifying all potentially relevant parameters is not only complex but may also vary significantly across different contexts or languages. Machine learning, on the other hand, can handle high-dimensional data and learn from it, making it more adaptable to such complex scenarios.
Figure 1.2 Representation of deductive and inductive learning in the context of machine learning.
Lack of Theory
: Some domains, such as predicting financial market performance, lack precise mathematical laws or theories. Traditional algorithms often depend on established theories or models, which, when absent, can lead to their failure. Machine learning can work around this by identifying patterns and making predictions based on data, without relying on pre-existing theoretical frameworks.
Need for Customization
: Traditional algorithms typically don’t account for individual user preferences, which can vary widely. What is considered an important feature by one user might be irrelevant to another. Machine learning systems, particularly those involving personalization, can learn and adapt to individual user preferences, distinguishing between what is interesting and uninteresting for each user.
In summary, the flexibility, adaptability, and ability to learn from data make machine learning systems more effective in scenarios where traditional algorithms struggle due to complexity, lack of theory, and the need for customization.
In the context of machine learning, deductive and inductive learning represent two fundamental approaches to how algorithms process and learn from data. Deductive learning is a top-down approach, where learning begins with a general theory or set of rules and then applies these to specific instances. In this method, the algorithm uses pre-existing knowledge or hypotheses to make predictions or decisions, following a logical progression from the general to the specific. It’s akin to applying a known formula to solve a new problem. Inductive learning, on the other hand, is a bottom-up approach. It starts with specific examples or data points and works to identify general patterns or rules. This approach does not begin with a preconceived theory but rather builds one based on the observation of individual instances. Machine learning models using inductive learning, like neural networks, are particularly adept at recognizing patterns in complex data sets and making predictions for new, unseen data. Both approaches are integral to the field of machine learning, each offering distinct ways of interpreting data and making decisions. Figure 1.2 shows a representation of deductive and inductive learning in the context of machine learning.
1.2 Different Types of Machine Learning Approaches
The effectiveness of machine learning largely hinges on the sophistication and continual evolution of its algorithms. These algorithms can be broadly categorized into different types based on the nature of the learning signal or the type of feedback the system uses. Understanding these categories is key to appreciating the versatility and power of machine learning:
Supervised Learning
: In supervised learning (Muhammad and Yan
2015
), the algorithm is trained on a labeled dataset, where each input data point is paired with a corresponding output (or label). The primary objective is to generate a function that maps inputs to desired outputs. This approach is akin to learning with a teacher who provides examples of correct input-output pairs. The algorithm uses these examples to learn the underlying structure of the data and to make predictions or decisions for new, unseen data. Supervised learning is extensively used for tasks like classification and regression, where the goal is to construct predictive models.
Unsupervised Learning
: Unlike supervised learning, unsupervised learning (Celebi and Aydin
2016
) algorithms deal with input data that is not labeled. The goal here is to explore the structure of the data and identify patterns or features that can describe the data’s underlying characteristics. Since there are no specific output variables to predict or classify, unsupervised learning is more about discovering hidden structures in the data. This approach is fundamental for tasks like clustering, dimensionality reduction, and associative rule learning. A typical application of unsupervised learning can be seen in search engines, where algorithms work to organize and categorize vast amounts of data without explicit human intervention.
Reinforcement Learning
: Reinforcement learning (Sugiyama
2015
) is a distinct paradigm where the learning process is driven by interactions with an environment. In this approach, an algorithm, typically referred to as an agent, learns to make decisions by performing actions and observing the results of these actions. The algorithm receives feedback in the form of rewards or penalties, which guide it towards the most effective strategies over time. The unique aspect of reinforcement learning is its focus on sequential decision-making, where the outcomes of previous actions influence future decisions. This method is widely used in dynamic environments, like game playing, robotics, and navigation, where the algorithm must continually adapt to changing conditions.
Each of these machine learning approaches offers a unique way of understanding and interacting with data, and they are chosen based on the specific requirements and constraints of the problem at hand. Their diversity and adaptability are what make machine learning a powerful and versatile tool across various domains.
Figure 1.3 illustrates a hierarchical classification of machine learning approaches, categorizing them into three primary types: supervised learning, unsupervised learning, and reinforcement learning. Each of these categories is further distinguished by their key methodologies or applications. Supervised learning, which involves training algorithms on labeled datasets to predict outcomes or classify data, is characterized mainly by techniques like regression and classification. These methods enable the algorithm to establish a relationship between input and output variables for predictive modeling or to categorize data into different classes based on learned patterns. Unsupervised learning, on the other hand, focuses on identifying patterns or structures in unlabeled data, primarily through clustering. Clustering algorithms group data based on similarities or patterns without prior knowledge of what these groups might represent. Finally, reinforcement learning, a distinct approach that emphasizes learning through interaction with an environment, is widely recognized for its application in deep learning. In this context, deep learning models, particularly those involving neural networks, are trained to make a series of decisions, improving their performance based on the feedback received in the form of rewards or penalties. Each of these approaches represents a unique facet of machine learning, demonstrating the field’s versatility and breadth in solving various types of problems.
Figure 1.3 Hierarchical classification of machine learning approaches.
1.3 Supervised Learning
Supervised learning (Muhammad and Yan 2015) is a pivotal technique in machine learning, designed to enable computer systems to automatically solve relevant tasks. The process begins by defining a set of input data, denoted as set I, which usually consists of vectors. This input data represents the information the system will learn from. Next, the set of desired output data is established, referred to as set O. These outputs are the correct answers or results that the system aims to predict or classify based on the inputs. The core of supervised learning involves defining a function f, which effectively maps each element of the input set I to its corresponding element in the output set O. This mapping is crucial as it forms the basis of the learning process. The combination of input data with their corresponding outputs constitutes what is known as a training set. This training set is used to “teach” the machine learning model, allowing it to learn the relationship between inputs and outputs. The model undergoes a training phase where it adjusts its parameters to minimize errors in its predictions or classifications. This entire workflow of supervised learning, from input data to the definition of the function and the creation of a training set, is succinctly illustrated in the subsequent Figure 1.4, providing a clear visual representation of how this process unfolds.
Supervised learning algorithms operate on a fundamental premise given a sufficient number of examples; they can develop a derived function fB that closely approximates a target function fA. This approximation is key to the success of these algorithms. If the derived function fB accurately approximates the desired function fA, it should be able to produce output responses that are similar to those generated by fA when presented with new input data. The underlying concept here is that similar inputs will lead to similar outputs. This assumption, although not universally valid in the real world, holds true in most scenarios.
Figure 1.4 Structure for supervised learning.
However, the efficacy of supervised learning algorithms is heavily contingent on the quality and quantity of the input data they are trained on. If the training set is too small, the algorithm may not “experience” enough variety in the data to make accurate predictions or classifications for new, unseen inputs. This is akin to having an inadequate learning experience. On the other hand, an excessively large training set can lead to its own set of challenges. Processing a vast amount of inputs can slow down the algorithm, and the derived function B might become overly complex, potentially leading to issues like overfitting, where the model performs well on the training data but poorly on new data.
Therefore, the balance and quality of the input data are crucial. They determine not just the performance of the algorithm in terms of speed and efficiency but also its ability to generalize well from the training data to real-world situations. This delicate balance between the quantity of data and the complexity of the derived function is a critical aspect of designing and implementing effective supervised learning algorithms.
Experience with supervised learning algorithms reveals a significant sensitivity to noise in the data. Even a small amount of incorrect or misleading data can severely compromise the reliability of the entire system. This susceptibility means that the presence of anomalies or errors in the training set can lead the algorithm to make incorrect predictions or decisions, impacting its overall effectiveness. Supervised learning also allows for the categorization of problems based on the nature of the output data. When the output is categorical, such as determining whether a data point belongs or does not belong to a certain class, the task is known as a classification problem. This involves categorizing data into discrete groups or classes. On the other hand, if the output is a continuous real value within a certain range, the task is termed a regression problem. In regression, the algorithm predicts a continuous quantity, such as a price or a temperature. This distinction between classification and regression is fundamental in supervised learning, guiding the choice of algorithms and methods used to tackle different types of predictive modeling tasks. Understanding whether a problem is a classification or regression problem is crucial for applying the appropriate techniques and achieving accurate results.
Supervised learning, a branch of machine learning, can be innovatively applied to enhance metaheuristic methods. Metaheuristics are high-level problem-solving frameworks designed to guide underlying heuristics in exploring and exploiting the search space of complex optimization problems. Integrating supervised learning with metaheuristics creates a synergy that can significantly improve the performance and efficiency of these algorithms. Here’s how supervised learning can be applied in this context:
Parameter Tuning
: Metaheuristic algorithms often come with several parameters that need fine-tuning for optimal performance. Supervised learning can be used to learn the best parameter settings for a given problem. By training a model on different instances of the problem with various parameter configurations and their respective performances, the algorithm can predict the most effective parameters for new instances.
Selection of Heuristics
: In many cases, a metaheuristic framework can use different heuristics or strategies at different stages of the search process. Supervised learning can help in dynamically selecting the most appropriate heuristic based on the current state of the search. This selection is done by training a model on historical data that maps states of the problem to the most effective heuristics.
Solution Improvement
: Supervised learning can be employed to refine the solutions generated by metaheuristics. For instance, a predictive model can be trained on high-quality solutions to learn their characteristics. This model can then guide the search process toward regions of the search space that are more likely to contain high-quality solutions.
Predictive Termination
: Often, it’s challenging to determine when a metaheuristic algorithm should stop searching for a better solution. Supervised learning models can be trained to predict the likelihood of finding a significantly better solution based on the current state of the search, thereby optimizing the computational effort.
Learning Problem-Specific Characteristics
: For problems that have various instances with different characteristics, supervised learning can help in understanding these variations. This knowledge can then be used to adapt the metaheuristic strategy to be more effective for specific types of instances.
By combining the adaptive and predictive capabilities of supervised learning with the exploratory strength of metaheuristics, these methods can become more robust, efficient, and effective in solving a wide range of optimization problems. This integration marks a significant step towards creating intelligent systems that can learn from past experiences and adapt their strategies accordingly.
1.4 Unsupervised Learning
Unsupervised learning (Celebi and Aydin 2016), a distinct branch of machine learning, focuses on extracting information automatically from databases without prior knowledge or labels about the data. This process stands in contrast to supervised learning, where models are trained with data that is clearly labeled or categorized. In unsupervised learning, there’s no predefined information about the classes to which the data points belong or any specific output associated with a given input. The primary objective is to explore the data and identify inherent structures or patterns. One common goal is to discover groups or clusters of data points that share similar characteristics, a process known as clustering. These algorithms work by identifying commonalities and differences within the data, grouping similar items together while separating dissimilar ones.
Unsupervised learning algorithms have a wide array of applications, with search engines being a notable example. In the context of search engines, these algorithms use one or more keywords to generate a relevant list of links or results. They analyze vast amounts of web data, understanding and categorizing content, to provide users with links that are most pertinent to their search queries. This capability to sift through and make sense of large, unstructured datasets without explicit guidance makes unsupervised learning an invaluable tool in areas where understanding complex patterns and relationships within data is crucial.
Figure 1.5 Effects of the clustering process under unsupervised learning.
