46,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Serie: Tech Today
- Sprache: Englisch
The definitive guide to leveraging AWS for generative AI
GenAI on AWS: A Practical Approach to Building Generative AI Applications on AWS is an essential guide for anyone looking to dive into the world of generative AI with the power of Amazon Web Services (AWS). Crafted by a team of experienced cloud and software engineers, this book offers a direct path to developing innovative AI applications. It lays down a hands-on roadmap filled with actionable strategies, enabling you to write secure, efficient, and reliable generative AI applications utilizing the latest AI capabilities on AWS.
This comprehensive guide starts with the basics, making it accessible to both novices and seasoned professionals. You'll explore the history of artificial intelligence, understand the fundamentals of machine learning, and get acquainted with deep learning concepts. It also demonstrates how to harness AWS's extensive suite of generative AI tools effectively. Through practical examples and detailed explanations, the book empowers you to bring your generative AI projects to life on the AWS platform.
In the book, you'll:
- Gain invaluable insights from practicing cloud and software engineers on developing cutting-edge generative AI applications using AWS
- Discover beginner-friendly introductions to AI and machine learning, coupled with advanced techniques for leveraging AWS's AI tools
- Learn from a resource that's ideal for a broad audience, from technical professionals like cloud engineers and software developers to non-technical business leaders looking to innovate with AI
Whether you're a cloud engineer, software developer, business leader, or simply an AI enthusiast, Gen AI on AWS is your gateway to mastering generative AI development on AWS. Seize this opportunity for an enduring competitive advantage in the rapidly evolving field of AI. Embark on your journey to building practical, impactful AI applications by grabbing a copy today.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 538
Veröffentlichungsjahr: 2025
Ähnliche
Table of Contents
Cover
Table of Contents
Title Page
Copyright
Dedication
Acknowledgments
About the Authors
Foreword
Introduction
Chapter 1: A Brief History of AI
The Precursors of the Mechanical or “Formal” Reasoning
The Digital Computer Era
Cybernetics and the Beginning of the Robotic Era
Birth of AI and Symbolic AI (1955–1985)
Subsymbolic AI Era (1985–2010)
Deep Learning and LLM (2010–Present)
Key Takeaways
Chapter 2: Machine Learning
What Is Machine Learning?
Implementation of Machine Learning
Key Takeaways
References
Chapter 3: Deep Learning
Deep Learning vs. Machine Learning
The History of Deep Learning
Understanding Deep Learning
Neural Network Architectures
Putting It All Together
Deep Learning on AWS
Key Takeaways
References
Chapter 4: Introduction to Generative AI
Generative AI Core Technologies
Transformer Architecture: Deep Dive
Terminology in Generative AI
Prompt Engineering
Generative AI Business Value
Building Value Within Your Enterprises
Summary
References
Chapter 5: Introduction to Foundation Models
Definition and Overview of Foundation Models
Types of Foundation Models
The Large Language Model (LLM)
Applications of Foundation Models
Challenges and Considerations
Key Takeaways
References
Chapter 6: Introduction to Amazon SageMaker
Data Preparation and Processing
Model Development
MLOps with Amazon SageMaker
Boost Your Generative AI Development with SageMaker JumpStart
No-Code ML with Amazon SageMaker Canvas
Amazon Bedrock
Choosing the Right Strategy for the Development of Your Generative AI Application with Amazon SageMaker
Conclusion
References
Chapter 7: Generative AI on AWS
AWS Services for Generative AI
Generative AI on AWS: The Fundamentals
Amazon Bedrock
Model Evaluation in Amazon Bedrock
Common Approaches to Customizing Your FMs
Amazon Bedrock Prompt Management
Amazon Bedrock Flows
Data Automation in Amazon Bedrock
GraphRAG in Amazon Bedrock
Knowledge Bases in Amazon Bedrock
Agents for Amazon Bedrock
Guardrails for Amazon Bedrock
Security in Amazon Bedrock
Amazon Q
Summary
Chapter 8: Customization of Your Foundation Model
Introduction to LLM Customization
Customizing Foundation Models with AWS
Conclusion
Chapter 9: Retrieval-Augmented Generation
What Is RAG?
Building a RAG Solution
RAG on AWS
Custom Data Pipeline to Build RAG
Case Studies and Applications
Challenges and Future Directions
Example Notebooks
References
Chapter 10: Generative AI on AWS Labs
Lab 1: Introduction to Generative AI with Bedrock
Lab 2: Dive Deep into Gen AI with Amazon Bedrock
Lab 3: Building an Agentic LLM Assistant on AWS
Lab 4: Retrieval-Augmented Generation Workshop
Lab 5: Amazon Q for Business
Lab 6: Building a Natural Language Query Engine for Data Lakes
Summary
Reference
Chapter 11: Next Steps
The Future of Generative AI: Key Dimensions and Staying Informed
Technical Evolution and Capabilities
Application Domains
Ethical and Societal Implications
Staying Current in the Rapidly Evolving AI Landscape
Glossary
Index
End User License Agreement
List of Illustrations
Chapter 1
Figure 1-1: Drawing of the top view of the Pascaline and overview of its mechani...
Figure 1-2: Drawing of the Stepped reckoner.
Figure 1-3: Alan Turing.
Figure 1-4: ENIAC in building 328 at the Ballistic Research Laboratory (BRL).
Figure 1-5: Unimate pouring coffee for a human, 1967.
Figure 1-6: Shakey the Robot at the Computer History Museum.
Figure 1-7: Dartmouth workshop commemorative plaque.
Figure 1-8: A conversation with ELIZA.
Chapter 2
Figure 2-1: Supervised learning, which implies to train a model with a labeled d...
Figure 2-2: Unsupervised learning, which implies a model that uses unlabeled dat...
Chapter 3
Figure 3-1: Deep learning: A subset of machine learning.
Figure 3-2: CV vs. NLP.
Figure 3-3: From academic activity period to GPU era.
Figure 3-4: Detailed history of deep learning.
Figure 3-5: The rise of the GPUs.
Figure 3-6: Illustration of a neuron.
Figure 3-7: Summation function.
Figure 3-8: Weights and bias.
Figure 3-9: Deep learning vs. machine learning.
Figure 3-10: An overview of the perceptron.
Figure 3-11: FeedForward networks and backpropagation.
Figure 3-12: Comparing loss functions.
Figure 3-13: Main types of neural networks.
Chapter 4
Figure 4-1: Architecture—Building a GAN model with PyTorch and Amazon SageMaker.
Figure 4-2: VAE architecture: Encoding input data into a latent distribution and...
Figure 4-3: RNN overview.
Figure 4-4: Transformer Architecture: Simplified diagram.
Figure 4-5: Transformer Architecture: Original model architecture.
Figure 4-6: Transformer Architecture: Tokenization process.
Figure 4-7: Transformer Architecture: Embedding layer.
Figure 4-8: Transformer Architecture: Embedding layer.
Figure 4-9: AWS Console – Machine Learning Services – Amazon Bedrock.
Figure 4-10: AWS Console – Amazon Bedrock home page.
Figure 4-11: AWS Console – Amazon Bedrock – Chat playground with Claude 3 Haiku.
Figure 4-12: AWS Console – Amazon Bedrock – Example of sentiment analysis.
Figure 4-13: Prompt in an LLM.
Figure 4-14: AWS Console – Amazon Bedrock – Example of Prompt Engineering.
Figure 4-15: AWS Console – Amazon Bedrock – Example of Prompt Engineering – In-Co...
Figure 4-16: AWS Console – Amazon Bedrock – Inference Configuration.
Figure 4-17: Output from model’s softmax layer.
Figure 4-18: Output from model’s softmax layer – Top P.
Figure 4-19: Output from Model’s softmax layer – Top K.
Figure 4-20: Comparison of different prompting techniques – tree of thoughts.
Figure 4-21: Advanced prompting techniques – RAG – overview.
Figure 4-22: Advanced prompting techniques – RAG – conceptual flow.
Figure 4-23: Automatic Reasoning and Tool-use – ART.
Figure 4-24: Reasons for biases in Prompt Engineering.
Figure 4-25: Getting business value from Generative AI.
Chapter 5
Figure 5-1: Traditional ML models vs. foundation models.
Figure 5-2: Approach for foundation models.
Figure 5-3: The role of foundation models in advancing NLP, LLMs, and computer v...
Figure 5-4: Overview of foundation models.
Figure 5-5: Criteria for choosing a foundation model.
Figure 5-6: Evolution of the number of parameters used for the training of found...
Figure 5-7: Approaches for the foundation model customization.
Figure 5-8: Decision tree of prompting methods.
Figure 5-9: Different modalities of the foundation models.
Figure 5-10: Generate a text with a foundation model.
Figure 5-11: Generate an image with a foundation model.
Figure 5-12: The Intersection of natural language processing and other data scien...
Figure 5-13: Evolution of natural language processing techniques.
Figure 5-14: Implementing a Sequence-to-Sequence model using LSTM-based Recurrent...
Figure 5-15: Use cases and examples of LLMs categorized to their architecture (en...
Figure 5-16: Transformer architecture.
Figure 5-17: Large language models: a subset of foundation models.
Figure 5-18: Tailoring foundation models: strategies for customization and fine-t...
Chapter 6
Figure 6-1: Amazon SageMaker service at every stage of the model creation.
Figure 6-2: Amazon SageMaker Unified Studio.
Figure 6-3: SageMaker Unified Studio build menu.
Figure 6-4: Amazon SageMaker AI studio home page.
Figure 6-5: Architecture diagram for SageMaker model training.
Figure 6-6: SageMaker JumpStart home page.
Figure 6-7: SageMaker Canvas home page.
Figure 6-8: Choosing the right strategy for the development of a GenAI applicati...
Chapter 7
Figure 7-1: Generative AI stack of tools and services on AWS.
Figure 7-2: Using foundation models with Bedrock.
Figure 7-3: Foundation model providers with Amazon Bedrock.
Figure 7-4: Model evaluations with Amazon Bedrock.
Figure 7-5: How prompt management works.
Figure 7-6: How prompt management works.
Figure 7-7: Visualizing Gen AI development workflows with Bedrock flows.
Figure 7-8: How knowledge bases work.
Figure 7-9: Pre-Processing.
Figure 7-10: Runtime execution – augmentation of user prompt.
Figure 7-11: Knowledge Base – Create a Knowledge Base via AWS Console.
Figure 7-12: Construction of agent via build-time API operations.
Figure 7-13: Runtime processing of agents with Amazon Bedrock.
Figure 7-14: AWS’s stack of generative AI services.
Figure 7-15: Amazon Q – built-in connectors.
Figure 7-16: Amazon Q – safety and security.
Figure 7-17: Amazon Q for Business.
Figure 7-18: Amazon Q in QuickSight.
Figure 7-19: Storytelling with Amazon Q in QuickSight.
Figure 7-20: Improving the story with Amazon Q in QuickSight.
Figure 7-21: AI answering questions of data on demand.
Figure 7-22: Amazon Q in Connect.
Chapter 8
Figure 8-1: Bedrock interface for a continued pre-training job.
Figure 8-2: Bedrock interface for deploying a customized model.
Chapter 9
Figure 9-1: RAG compared to other approaches for customizing foundation models.
Figure 9-2: Overview of the interactions between RAG modules.
Figure 9-3: Retrieval, augmentation, and generation modules.
Figure 9-4: Data processing for RAG.
Figure 9-5: Key architecture components.
Chapter 10
Figure 10-1: Image generation exercise.
Figure 10-2: Multimodal chatbot.
Figure 10-3: Document summarization.
Figure 10-4: Amazon Bedrock workshop.
Figure 10-5: Architecture of building an agentic LLM assistant on AWS.
Figure 10-6: Comparison of various RAG approaches.
Figure 10-7: Comparison of various RAG approaches.
Figure 10-8: Example prompt.
Figure 10-9: Amazon Q business – adding a data source.
Figure 10-10: Amazon Q business – adding PDF file.
Figure 10-11: Amazon Q business – updated response.
Figure 10-12: Data Genie – sample web interface.
Figure 10-13: Answer to the question.
List of Tables
Chapter 3
Table 3-1: The Basic Concepts of Neural Networks
Table 3-2: Some Common Activation Functions
Table 3-3: Parameters vs. Hyperparameters
Table 3-4: Types of Hyperparameters
Table 3-5: Comparison of Different Optimization Algorithms
Chapter 5
Table 5-1: Example of zero-shot learning
Table 5-2: Example of few-shot learning
Table 5-3: Example of chain-of-thought prompting
Chapter 7
Table 7-1: List of Amazon Titan LLMs available on Amazon Bedrock
Table 7-2: List of Anthropic’s LLMs Available on Amazon Bedrock
Table 7-3: Comparison of Cohere’s Models on Amazon Bedrock
Table 7-4: Comparison of LLAMA models on Amazon Bedrock
Table 7-5: Comparison of Mistral Models
Chapter 8
Table 8-1: Choosing a Customization Technique
Table 8-2: Customization Costs
Table 8-3: Hyperparameters for Model Customization
Chapter 9
Table 9-1: Knowledge Bases for Amazon Bedrock/Amazon Q vs. Custom RAG Pipeline
Guide
Cover
Table of Contents
Title Page
Copyright
Dedication
Acknowledgments
About the Authors
Foreword
Introduction
Begin Reading
Glossary
Index
End User License Agreement
Pages
i
ii
iii
iv
xiii
xiv
xv
xvi
xvii
xviii
xix
xx
xxi
xxii
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
GenAI on AWS®
A Practical Approach to Building Generative AI Applications on AWS
Olivier Bergeret, Asif Abbasi, Joel Farvault
Copyright © 2025 by John Wiley & Sons Inc. All rights reserved, including rights for text and data mining and training of artificial intelligence technologies or similar technologies
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
Published simultaneously in Canada.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our website at www.wiley.com.
Library of Congress Cataloging-in-Publication: 2025902947
Print ISBN: 9781394281282
ePDF ISBN: 9781394281305
epub ISBN: 9781394281299
OBook ISBN: 9781394344499
Cover Design: Wiley
Cover Image: © IMOGI/Getty Images
To my wife, Hifza Abbasi,
For twenty years of unwavering patience, steadfast perseverance, and boundless support.
This journey – and this book – would not have been possible without you.
– Asif Abbasi
To my wife Céline and my beautiful daughters Léna, Manon, Nolwenn, who fill my heart with joy each and every day.
– Olivier Bergeret
To my wife Emilie and my kids Janna, Mahe, Kadeen, Isia, who patiently supported me during the evenings and moments I have devoted to the writing of this book. I would never have succeeded without their support.
– Joel Farvault
Acknowledgments
Collaborating on this study guide has been an incredible journey, and I am immensely grateful to have worked alongside the exceptional professionals from Wiley and Amazon who made this possible.
First and foremost, my heartfelt thanks go to my co-authors, Olivier Bergeret and Joel Farvault. Olivier, you’ve been more than a mentor – you’ve been the calm in the chaos of our intense schedules. Your knack for distilling complex decisions into clear solutions is nothing short of wizardry, and your advice has been invaluable.
A special thanks to my colleague Joel, whose ability to juggle multiple books and countless activities with such poise continues to amaze me.
To Adam Driver, my mentor and an inspirational leader at Amazon – thank you for your unwavering guidance and for setting a leadership example that motivates us all. Your support throughout this process has been immeasurable.
To Julien Lepine, I deeply appreciate the opportunity you gave me to join this incredible team eight years ago. Your exemplary leadership, mentorship, and unwavering support have been invaluable throughout all the remarkable moments we’ve shared.
A huge shoutout to Kenyon Brown for trusting us with the opportunity to write about this critical topic and to David Clark for your remarkable patience and dedication. Your commitment to raising the bar has been instrumental. I’d also like to thank Moses Ashirvad for keeping us aligned and on track when the going got tough.
Finally, to my incredible UAE team – Asif Mujawar, Nickson Dicson, and Yasser Hassan – thank you for your patience, understanding, and continuous encouragement as I balanced this project with everything else.
This book is a product of collaboration, inspiration, and a whole lot of caffeine. Thank you to everyone who made it happen.
– Asif Abbasi
I would like to express my deepest gratitude to Asif Abbasi who proposed this book and led this incredible journey from A to Z with an immense energy and invaluable expertise. I would like to thank Joel Farvault for joining this adventure and bringing your irreplaceable expertise.
A special thanks to Wiley and Kenyon Brown for believing in this project, and David Clark for your insightful feedback and dedication to shaping this manuscript.
This book would not be the same without these nine exceptional years at Amazon Web Services, serving incredible leaders; many thanks to Johannes Borch, Stephan Hadinger, Philippe Desmaison, Julien Lepine, Adam Driver, and Siva Raghupathy for your mentorship and unconditional support throughout this journey. A big thank you to my colleagues from all horizons, service teams, specialist teams, and solution architect teams; without your great work and the incredible success of our services this book would not exist.
Finally, I would like to express my sincere gratitude to the cybernetic scientist, teacher, and journalist Albert Ducrocq, who inspired me to pursue science, robotics, cybernetics, and artificial intelligence. Although he is no longer with us, our passionate discussions from 1997 to 1999 on cybernetics, space, autonomous aircraft, and spacecraft still resonate with me and inspire me every day.
– Olivier Bergeret
First and foremost, I want to express my deepest gratitude to Olivier and Asif, whose unwavering support and guidance were instrumental in bringing this book to life. Throughout the writing process, their insights, encouragement, and patience helped me navigate through moments of creative blockage and self-doubt.
I extend my sincere thanks to the Wiley team – David Clark, Kenyon Brown, and Moses Ashirvad – for their continuous support and patient feedback throughout this project.
I am profoundly grateful to my colleagues at Amazon Web Services. Special thanks to Annabelle Pinel, my awesome sales partner who encouraged me in this endeavor. I also want to thank Lior Perez, Ioan Catana, Shikha Verma, Leonardo Gomez, Fabrizio Napolitano, Praveen Kumar, and Tej Luthra for their invaluable support and engaging discussions on Generative AI. My appreciation also goes to the managers who supported my learning journey: Simon Treacy, Asif Mujawar, Aurelia Pinhel, and Jeetesh Srivastva.
Finally, I warmly thank the IA School, École Supérieure de Génie Informatique, and Neoma Business School, particularly Othman Boujena and Charles-David Wajnberg, for their continuous support and encouragement.
– Joel Farvault
About the Authors
Olivier Bergeret is a technical leader at Amazon Web Services (AWS) working on database and analytics services. He has more than 25 years of experience in data engineering and analytics. Since joining AWS in 2015, he’s supported the launch of most of AWS AI services including Amazon SageMaker and AWS DeepRacer. He is a regular speaker and presenter at various data, AI, and cloud events such as AWS re:Invent, AWS Summits, and third-party conferences.
Joel Farvault is a Principal Solutions Architect in Analytics at Amazon Web Services (AWS), bringing more than 25 years of expertise in enterprise architecture, data strategy, and analytics. Throughout his career, he has successfully driven data transformation initiatives, with a particular focus on fraud analytics, business intelligence, and data governance. In addition to his professional role at AWS, Joel is a lecturer in Data Analytics, teaching at the IA School, Neoma Business School, and the École Supérieure de Génie Informatique (ESGI). Joel has earned multiple certifications from AWS.
Asif Abbasi is a Principal Solutions Architect specializing in Analytics at Amazon Web Services (AWS), with a career spanning more than 24 years in enterprise architecture, data strategy, and advanced analytics. Asif has been working with AWS since 2018 and is known for his ability to lead complex data transformation initiatives. Asif has deep expertise in data analytics, business intelligence, and enterprise architecture with special focus on large scale data processing and Generative AI. Asif has previously written acclaimed books on Apache Spark and the AWS Certified Data Analytics Specialty Exam. He holds master’s degrees in Computer Science and Business Administration.
Foreword
As the world continues to witness the rapid advancements in artificial intelligence, the emergence of Generative AI has undoubtedly captivated the imagination of technologists, entrepreneurs, and innovators alike. This powerful and transformative technology holds the key to unlocking unprecedented levels of creativity, efficiency, and problem-solving capabilities – and the time has never been better to harness its potential.
In this book, GenAI on AWS, the authors have meticulously crafted a comprehensive guide that not only introduces the fundamental concepts of Generative AI but also delves deep into the practical applications and implementation strategies on the Amazon Web Services (AWS) platform. This is a timely and invaluable resource for anyone seeking to leverage the power of Generative AI to drive innovation and transformation within their organizations.
The book’s structure takes readers on a compelling journey, beginning with a concise yet insightful exploration of the history of artificial intelligence, followed by a deep dive into the core principles of machine learning and deep learning. This foundational knowledge lays the groundwork for the authors’ in-depth examination of Generative AI, where they expertly unpack the various models, use cases, and interaction patterns that define this transformative technology.
The true value of this book, however, lies in its seamless integration of Generative AI concepts with the robust and comprehensive suite of services offered by AWS. The authors have meticulously mapped out the AWS ecosystem, highlighting the specific tools and capabilities that empower developers, data scientists, and business leaders to harness the full potential of Generative AI. From Amazon SageMaker’s advanced model training and deployment capabilities to the groundbreaking offerings of Amazon Bedrock and Amazon CodeWhisperer, this book equips readers with the knowledge and hands-on guidance needed to build practical, scalable, and secure Generative AI applications.
Equally important, the authors address the crucial considerations of customizing and fine-tuning foundation models, as well as the ethical implications and governance requirements associated with the responsible deployment of Generative AI. This holistic approach ensures that readers not only acquire the technical skills but also develop a well-rounded understanding of the broader landscape, empowering them to make informed decisions and navigate the complexities of this rapidly evolving field.
Whether you are an aspiring data scientist, a seasoned software engineer, or a forward-thinking business leader, GenAI on AWS is an indispensable resource that will equip you with the knowledge and tools to harness the transformative power of Generative AI on the AWS platform. Dive in, and prepare to embark on a journey of innovation, creativity, and unprecedented possibilities.
– Adam Driver
Introduction
Artificial intelligence (AI) is revolutionizing the way we live and work, unlocking possibilities that were once considered the realm of science fiction. From automating mundane tasks to generating highly creative outputs, AI is shaping industries and empowering businesses to achieve unprecedented efficiency and innovation.
This book, GenAI on AWS, is designed for readers who may have little to no prior knowledge of AI but are eager to learn. It is a step-by-step guide that takes you from the very basics of AI to the cutting-edge world of Generative AI, enabling you to build, customize, and implement models on AWS for your business workloads.
One of the biggest challenges in learning AI today is the sheer complexity of most resources. Many texts start with advanced topics – like diving straight into the intricacies of attention mechanisms – leaving readers overwhelmed and disconnected. We aim to bridge this gap by using simple, natural language and accessible examples to help you grasp the foundational concepts before tackling advanced topics.
Understanding Generative AI is no longer optional; it is a necessity for professionals across all industries. Whether you work in marketing, finance, healthcare, or technology, Generative AI can help you unlock insights, automate creativity, and solve complex problems. By the end of this book, you will have the knowledge and tools to not only understand the hype around Generative AI but also leverage it effectively using Amazon Web Services like SageMaker, foundation models, and more.
Here’s what you can expect as we embark on this journey:
A brief history of AI
An introduction to machine learning (ML) and deep learning
Core concepts of Generative AI and foundation models
How AWS empowers businesses to adopt Generative AI
Hands-on labs and customizations for real-world scenarios
What Does This Book Cover?
This book covers the essential topics required to understand and implement Generative AI on AWS, guiding readers from foundational knowledge to hands-on application. Below is a breakdown of the chapters:
Chapter 1
: A Brief History of AI
This chapter provides an overview of the evolution of artificial intelligence, from its early conceptual beginnings to its current state. It explores the major breakthroughs and trends that have shaped AI, setting the stage for understanding its capabilities and applications in modern technology.
Chapter 2
: Machine Learning
Here, readers will learn the basics of machine learning, including key concepts, types of ML models, and the training process. The chapter also introduces the role of data in building ML models and explains how ML serves as the foundation for more advanced AI techniques.
Chapter 3
: Deep Learning
This chapter delves into the field of deep learning, covering neural networks, how they function, and why they are crucial to advancements in AI. It explains concepts such as layers, activation functions, and backpropagation while simplifying the technical jargon for beginners.
Chapter 4
: Introduction to Generative AI
This chapter focuses on what makes Generative AI unique. It introduces generative models, including Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and transformers, and highlights their ability to create new data based on learned patterns.
Chapter 5
: Introduction to Foundation Models
Readers will explore foundation models – large pre-trained models that can perform a wide range of tasks. This chapter explains how these models are trained, fine-tuned, and used as building blocks for solving complex business problems.
Chapter 6
: Introduction to Amazon SageMaker
This chapter introduces Amazon SageMaker, AWS’s fully managed service for building, training, and deploying machine learning models. Readers will learn about SageMaker’s key features and how it simplifies the AI/ML workflow.
Chapter 7
: Generative AI on AWS
This chapter explains how AWS enables businesses to leverage Generative AI. It covers the tools, services, and infrastructure provided by AWS to implement Generative AI solutions, highlighting best practices for scalability and cost-effectiveness.
Chapter 8
: Customization of Your Foundation Model
Readers will learn how to adapt pre-trained foundation models to meet specific needs. This chapter provides a step-by-step guide to fine-tuning and customizing models for various workloads, enabling businesses to optimize Generative AI for their unique requirements.
Chapter 9
: Retrieval-Augmented Generation
This chapter takes an in-depth look at RAG, an advanced technique that combines retrieval mechanisms with generative models to create highly accurate and context-aware outputs.
Chapter 10
: Generative AI on AWS Labs
Practical experience is key to mastering any concept. This chapter provides hands-on exercises and real-world use cases to help readers apply their knowledge and develop their own Generative AI solutions using AWS tools.
Chapter 11
: Next Steps
The final chapter summarizes key learnings from the book and offers guidance on where to go next in your AI journey. Whether it’s pursuing advanced projects, certifications, or exploring other Amazon Web Services, readers will be equipped to continue their growth in the field of Generative AI.
With this structured approach, the book ensures that readers build a strong foundation while progressively advancing to complex topics, empowering them to implement Generative AI solutions effectively and confidently.
Who Should Read This Book
As the title implies, this book is intended for people who want to understand and implement Generative AI using AWS. Such individuals probably fall into three main categories:
AI enthusiasts and professionals who have experience with machine learning or traditional AI models but have not yet explored Generative AI in detail. Many professionals fall into this category as the rapid evolution of Generative AI has made it a specialized area of study.
AWS users who are familiar with Amazon Web Services and cloud computing but wish to expand their expertise into the field of Generative AI. These readers are likely comfortable with AWS infrastructure but need guidance on how to apply it to advanced AI workloads and utilize services like Amazon SageMaker effectively.
Highly motivated beginners who may have limited experience with AI or AWS but are eager to learn. These are the readers who will dive deep into the foundational chapters to build their understanding from the ground up and leverage AWS tools to create their own Generative AI models.
For all these groups, the learning curve can be steep. As of now, resources that provide a cohesive introduction to Generative AI while focusing on AWS tools are scarce. Most materials either assume prior expertise or delve directly into technical jargon, leaving many learners struggling to grasp fundamental concepts. This book aims to fill that gap by offering a structured, beginner-friendly guide to Generative AI on AWS.
While there are excellent tutorials and documentation available on AWS and AI, they are often fragmented and assume varying levels of expertise. This book brings together all the key elements – from understanding the basics of AI, machine learning, and deep learning to mastering advanced techniques in Generative AI using AWS. We will not only explore how to use Amazon Web Services but also ensure you understand the principles behind Generative AI, so you can confidently build and customize your models for real-world applications.
The resources and guidance in this book will empower you to go beyond just following tutorials. By the end of this journey, you will have the tools to:
Build foundational knowledge of AI and its evolution into Generative AI.
Understand and work with cutting-edge foundation models.
Harness Amazon Web Services like Amazon SageMaker to implement Generative AI for business use cases.
Develop and fine-tune your own models tailored to specific workloads.
Whether you want to follow this book sequentially, tackling each chapter step by step, or focus on specific areas of interest, the content is designed to accommodate different learning paths. Each chapter builds on the last while also providing standalone value, allowing you to progress at your own pace.
Generative AI is shaping the future of industries worldwide, and understanding its potential is essential for every professional. This book will help you gain the skills to navigate this exciting field and turn its potential into actionable solutions for your career or business. Let’s embark on this journey together!
Reader Support for This Book
Companion Hands-on Workshops
The book mentions some additional workshops and technical artifacts. All these workshops are available at: https://workshops.aws/
Chapter 1A Brief History of AI
Artificial Intelligence: “The conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”
– First definition coming from the Dartmouth Summer Research Project proposal in 1956
Defining Artificial Intelligence (AI) is not easy as AI is a young discipline. AI is undoubtedly a bold, exciting new world, where the lines between humans and machines blur, leading us to question the very nature of intelligence itself. In addition, AI is recognized as a collection of scientific disciplines including mathematical logic, statistics, probabilities, computational neurobiology, and computer science that aims to perform tasks commonly associated with the human cognitive abilities such as the ability to reason, discover meaning, generalize, or learn from past experiences.
Interestingly, AI founders weren’t just computer scientists. They were philosophers, mathematicians, neuroscientists, logicians, and economists. Shaping the course of AI required them to integrate a wide range of problem-solving techniques. These tools spanned from formal logic and statistical models to artificial neural networks and even operations research. This multidisciplinary approach became the key to solving intricate problems that AI posed.
The Precursors of the Mechanical or “Formal” Reasoning
One of these precursors was the French philosopher and theologist René Descartes (1596–1650), who wrote in 1637 his Discourse on the Method,1 one of the most influential works in the history of modern philosophy, and important to the development of natural science. He discussed in Part V the conditions required for an animal or a machine to demonstrate an intelligent being. This was one of the earliest examples of philosophical discussion about artificial intelligence. He envisioned later in his Meditations on First Philosophy2 (1639) the possibility of having machines being composed like humans but with no mind.
In 1666, the German polymath Gottfried Wilhelm Leibniz (1646–1716) published a work entitled On the Combinatorial Art3 in which he expressed his strong belief that all human reasoning can be represented mathematically and reduced to a calculation. To support this vision, he conceptualized and described in his writings a Calculus ratiocinator: a theoretical universal logical calculation method to make these calculations feasible and a Characteristica Universalis: a universal and formal language to express mathematical, scientific, and metaphysical concepts.
At the same time, Blaise Pascal (1623–1662) built in 1641 one of the first working calculating machines called the “Pascaline,” shown in Figure 1.1, which could perform additions and subtractions. Inspired by this work, Leibniz built his “Stepped reckoner” (1694), as shown in Figure 1.2, a more sophisticated mechanical calculator that could not only add and subtract but also multiply and take the square root of a number.
Figure 1-1: Drawing of the top view of the Pascaline and overview of its mechanism.
1779, Oeuvres de Blaise Pascal, Chez Detune, La Haye, Public Domain.
Figure 1-2: Drawing of the Stepped reckoner.
Hermann Julius Meyer /Wikimedia Commons/Public domain.
After the initial developments in mechanical calculation, further advancements were made in the early nineteenth century. In 1822, English mathematician Charles Babbage (1791–1871) designed the Difference Engine, an automatic mechanical calculator intended to tabulate polynomial functions. The Difference Engine was conceived as a room-sized machine, but it was never constructed in Babbage’s lifetime.
Building on the foundations laid by Leibniz, George Boole published in 1854 The Laws of Thought4 presenting the concept that logical reasoning could be expressed mathematically through a system of equations. Now known as Boolean algebra, Boole’s breakthrough established the basis for computer programming languages. Additionally, in 1879 German mathematician Gottlob Frege (1848–1925) put forth his Begriffsschrift,5 which established a formal system for logic and mathematics. The pioneering work of Boole and Frege on formal logic laid essential groundwork that enabled subsequent developments in computation and computer science.
The Digital Computer Era
In 1936, mathematician Alan Turing (Figure 1.3) published his landmark paper “On Computable Numbers,”6 conceptually outlining a hypothetical universal machine capable of computing any solvable mathematical problem encoded symbolically. This theoretical Turing machine established a framework for designing computers using mathematical logic and introduced the foundational notion of an algorithm for programming sequences.
Figure 1-3: Alan Turing.
GL Archive / Alamy Stock Photo
Around the same time, Claude Shannon’s 1937 master’s thesis, A Symbolic Analysis of Relay and Switching Circuits,7 demonstrated Boolean algebra’s applicability for optimizing electric relay arrangements: the core components of electromechanical telephone routing systems. Shannon thus paved the way for applying logical algebra to circuit design.
Concurrently in 1941, German engineer Konrad Zuse developed the world’s first programmable, fully functional computer, the Z3. Built from 2,400 electromechanical relays, the Z3 embodied the theoretical computer models proposed by Turing and leveraged Boolean logic per Shannon’s insights. Zuse’s pioneering creation was destroyed during World War II.
In the aftermath of World War II, mathematician John von Neumann made vital contributions to emerging computer science. He consulted on the ENIAC (Figure 1.4), the pioneering programmable electronic digital computer built for the US Army during the war. In 1945, von Neumann authored a hugely influential report on the proposed EDVAC computer, outlining the “stored-program concept”: separating a computer’s task-specific programming from its general-purpose hardware that sequentially executes instructions. This conceptual distinction enabled the adoption of software programs without reconfiguring physical components, thereby allowing a single machine to readily perform different sequences of operations. Von Neumann’s architectural vision profoundly shaped modern computing as the standard von Neumann architecture.
Figure 1-4: ENIAC in building 328 at the Ballistic Research Laboratory (BRL).
Ballistic Research Laboratory, 1947–1955, US Army.
Following the first functioning computer constructions, Alan Turing reflected on the capabilities afforded by these theoretically “universal machines.” In a 1948 report, he argued that one such general-purpose device should be sufficient to carry out any computable task, rather than needing infinite specialized machines. Developing this thread further in his landmark 1951 paper “Computing Machinery and Intelligence,”8 Turing considered whether machines might mimic human capacities. To examine this, he proposed what later became known as the Turing test, an “imitation game” evaluating whether people could distinguish a concealed computer from a human respondent based solely on typed conversation.
In the conventional form of the Turing test, there are three participants: a human interrogator, interacting through written questions and answers with two players: a computer and a person. The interrogator needs to understand which one is the human solely based on the text-based responses to his or her inquiries. Removing other perceptual cues forces the player to rely entirely on the linguistic content and reasoning within the typed replies when attempting to distinguish human from machine. This restriction highlights Turing’s interest in assessing intelligence manifested in communication; if responses are sufficiently comparable between candidates, it suggests the computer can display capacities approaching human-level understanding and dialogue, at least conversationally. Thus, passing this verbal imitation game constituted Turing’s proposed measure for demonstrated machine intelligence.
Cybernetics and the Beginning of the Robotic Era
The term robot entered the lexicon in 1920 with Czech writer Karel Capek’s play R.U.R. (Rossum’s Universal Robots), which featured artificial workers created to serve humans. The specific concept of robotics as a field of study then emerged in science fiction over the following decades. Notably, the 1942 short story “Runaround” by Isaac Asimov introduced his influential Three Laws of Robotics, ethical constraints programmed into the fictional robots to govern their behavior:
A robot may not injure a human being, or, through inaction, allow a human being to come to harm.
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
These simple yet profound guidelines shaped many subsequent fictional depictions and philosophical discussions around machine intelligence safety and control. Along with introducing seminal ideas, Asimov’s writings disseminated the term robotics into broader usage to describe the engineering discipline focused on constructing automated mechanical beings.
The term cybernetics refers to a theory of control mechanisms applied to regulate complex systems. It originates from the ancient Greek word kybernetikos meaning “skilled in steering,” referring to a ship helmsman’s ability to steer and navigate vessels. The mathematician Norbert Wiener (1894–1964) first coined the term cybernetics in his 1948 book of the same name Cybernetics,9 where he laid the foundations of this new interdisciplinary field. Through his pioneering work, Wiener sought to model the principles behind self-governing behavior, drawing parallels between mechanical, biological, and social systems. Ultimately, he envisioned cybernetics as a way to design autonomous, self-correcting control processes across organizations and machines alike, from servomechanisms to human organizations. The connections Wiener made between self-regulation in mechanical, organic, and social contexts provided an insightful new lens through which complex systems could be analyzed and understood.
The emergence of cybernetics was heavily influenced by concurrent research in neuroscience revealing that the brain operates as an electrical network of neurons that transmit signals. Central to Wiener’s theory of cybernetics were feedback loops: mechanisms that enable systems to self-adjust their behavior. This concept of self-governing control through circular causality was analogous to the impulse propagation in neural circuits. At the same time, Claude Shannon’s 1948 work A Mathematical Theory of Communication10 laid the theoretical foundations for the digitization of analog signals. Additionally in 1943, Warren S. McCulloch (1898–1969) and Walter Pitts (1929–1969) demonstrated how neural networks could implement logical propositional calculus. The confluence of these key discoveries in cybernetics, information theory, and neurocomputation suggested the possibility of constructing an “electronic brain.” This prompted attempts in the 1950s to build the first experimental robots guided entirely by analog circuitry designed to mimic biological neural networks. While primitive, these early successes fed momentum toward developing sophisticated systems that could replicate more complex human decision-making and learning. The interdisciplinary fusion of neuroscience and computational controls remains at the heart of efforts to construct adaptable, self-governing artificial intelligence.
In 1950, British inventor Tony Sale built one of the first human-like robots, nicknamed George, which stood six feet tall and was able to walk and talk. Sale constructed George entirely from analog electronics to carry out conversations and movements activated through pulleys and air pressure. While clunky and primitive by modern standards, robots like George and the analog-based machines developed concurrently represented the pioneering first steps toward life-like intelligent automation. The creation of an anthropomorphic machine, or android, capable of human activities like self-directed motion and dialogue opened the door both technologically and psychologically to the widespread cultural adoption of robots. No longer theoretical, these interactive machines inspired sci-fi visions of the future now seemingly within reach. George embodied cybernetics founder Wiener’s vision of building artificial systems that could emulate and perhaps enhance human capabilities. This new generation of thinking machines brought us measurably closer toward that goal.
In the same pioneering year of 1950, Claude Shannon constructed an electromechanical mouse named Theseus, capable of solving a maze on its own – one of the earliest examples of a learning machine. Shannon built Theseus entirely from relay circuits and switches to enable it to navigate through a maze by trial and error, “learning” the correct route based on past experience. The mouse would traverse the labyrinth, remembering each turn as either a reward or failure depending on whether it led toward an end goal. Over time, favored paths emerged through this basic feedback learning. Though extraordinarily simple by modern AI standards, Theseus embodied one of the founding principles of cybernetics: that complex goal-oriented behavior could arise from interconnected networks and repetition of simple rules.
In line with early cybernetics principles, French inventor Albert Ducrocq (1921–2001) built an electromechanical fox between 1950 and 1953 that demonstrated autonomous goal-directed behavior through sensory-feedback loops. Nicknamed Job, the robot fox navigated unfamiliar environments largely on its own using an array of analog sensors akin to animal senses. Job was equipped with two photoelectric cells mounted in his head that acted as organs of sight, a microphone that constituted his ear, while his touch came from sensors that reacted to contact with obstacles. Contacts placed in the neck gave him a sense of direction. He also had “capacitive flair,” which allowed him to recognize an obstacle from a distance. In addition, Job was able to learn through a “memory” and to express himself via two lamps that lit up on the top of his head.
In 1961, Unimate (Figure 1.5), an industrial robot invented by George Devol in the 1950s, became the first to work in a General Motors assembly line in New Jersey. Its responsibilities included transporting die castings from the assembly line and welding the parts on to cars – a task deemed dangerous for humans. Being adopted by the popular culture, it appeared on The Tonight Show hosted by Johnny Carson, knocking a golf ball into a cup, pouring a beer, conducting an orchestra, and grasping an accordion.
Figure 1-5: Unimate pouring coffee for a human, 1967.
Wikipedia, CC-BY-SA 4.0, https://commons.wikimedia.org/wiki/File:Unimate_pouring_coffee_for_a_woman_at_Biltmore_Hotel,_1967.jpg
In the late 1960s, Shakey the Robot (Figure 1.6) was developed as the first mobile robot that had the capacity to apprehend and to interact with its surroundings. It was developed by a group of engineers at Stanford Research Institute (SRI) supervised by Charles Rosen under supervision and funding of the Defense Advanced Research Projects Agency (DARPA).
Figure 1-6: Shakey the Robot at the Computer History Museum.
Wikipedia, CC-BY-SA 2.0, https://commons.wikimedia.org/wiki/File:Shakey.png, no change.
Birth of AI and Symbolic AI (1955–1985)
As transistor and then integrated circuit technology enabled computers to perform increasingly complex functions, the potential for artificial intelligence vastly expanded. Tasks that previously only the human brain could accomplish, such as playing chess or recognizing images, gradually became possible for AI systems to tackle as well. This interdependent advancement of hardware and software drove rapid progress in the capabilities of artificial intelligence.
In 1950, Claude Shannon published a groundbreaking paper entitled “Programming a Computer for Playing Chess.”11 This article was the first to explore the creation of a computer program capable of playing chess.
However, the field of Artificial Intelligence got officially defined in 1956 with a workshop organized by John McCarthy at the Dartmouth Summer Research Project. The goal of this conference was to investigate ways in which machines could be made to simulate aspects of intelligence. McCarthy coauthored the proposal for the workshop with Marvin Minsky, Nathaniel Rochester, and Claude Shannon. McCarthy was credited for the first use of the term artificial intelligence (see Figure 1.7).
Figure 1-7: Dartmouth workshop commemorative plaque.
Since its inception, AI research has followed two distinct yet competing approaches: the symbolic (or “top-down”) method and the connectionist (or “bottom-up” or subsymbolic) method. The top-down approach aims to mimic intelligence by examining cognition independently of the brain’s biological architecture. It frames thinking in terms of manipulating abstract symbols via rules. On the other hand, the bottom-up methodology focuses on building artificial neural networks that imitate the brain’s interconnected architecture. It gets its “connectionist” designation from the emphasis placed on connections between neuronal units. While differing substantially, both the symbolic and connectionist paradigms strive to unlock the secrets of replicating intelligence artificially. The interplay between these rival approaches has largely defined and driven the evolution of AI research over the decades.
In 1952, computer scientist Arthur Samuel created the first self-learning game-playing computer program – one designed to play checkers. His program pioneered the ability to progressively improve its gameplay by accumulating knowledge independently through experience. This marked a significant milestone in developing artificial intelligence that could teach itself skills rather than relying solely on human-programmed rules. Samuel’s checkers application established foundational concepts for AI agents honing strategies through practice over time.
In 1955, researcher Allen Newell and economist Herbert Simon spearheaded the exploration of heuristic search procedures: efficient methods for discovering solutions within massive, multifaceted problem spaces. They implemented this concept in two pioneering AI systems: Logic Theorist, which generated proofs for mathematical theorems, and the more versatile General Problem Solver. Newell and Simon’s adoption of heuristic search techniques enabled their programs to efficiently navigate complex combinatorial possibilities to identify problem-solving pathways. These two programs marked the first implementations of artificial intelligence within computer systems. By leveraging heuristic search, Newell and Simon set the stage for AI applications to practically address intricate real-world problems.
In 1957, Frank Rosenblatt of Cornell University’s Aeronautical Laboratory pioneered research into artificial neural networks, which he termed perceptrons. Through rigorous computer simulations and mathematical modeling, Rosenblatt made seminal contributions unraveling the capabilities of neural nets and mechanisms of learning. His connectionist approach stressed the paramount role of modifying connections between neurons to enable knowledge acquisition. Via groundbreaking empirical and theoretical work on network topologies, Rosenblatt cemented himself as a forefather of neural networks within the Artificial Intelligence community. His perceptron findings seeded advancement in adaptive systems that paralleled biological neural functioning.
In 1958, John McCarthy pioneered the creation of Lisp, which has endured as the predominant and preferred programming language utilized in Artificial Intelligence research. McCarthy’s development of Lisp, with its specialized features for manipulating symbolic expressions, provided AI scientists with an essential and adaptable tool for implementing programs geared toward intelligence. To this day, Lisp remains a widely embraced high-level language within the AI community, having withstood the test of time as a staple in the field McCarthy helped establish.
In 1959, Arthur Samuel introduced the seminal notion of “machine learning” while discussing innovations in his chess-playing computer program. He imparted the idea that a computer could be coded to independently enhance its gameplay beyond the skills of even its programmer. Samuel envisioned the possibility for machines to autonomously build expertise by accumulating knowledge through experience, thus “learning” on their own in a way once solely attributed to human intelligence. His pioneering perspective established machine learning as a fundamental pursuit within Artificial Intelligence research for decades to follow. By conceiving of adaptive programs that exhibit human-like mastery, Samuel set the trajectory toward the advanced AI systems of today.
In 1965, computer scientist Joseph Weizenbaum created ELIZA (Figure 1.8), an early conversational AI program capable of natural language interaction. Though intended to showcase the superficiality of communication between humans and artificial intelligence, ELIZA’s uncannily human-like responses led many to curiously anthropomorphize its underlying system. Through his pioneering bot, Weizenbaum inadvertently revealed people’s propensity to project human qualities onto AI. In doing so, he surfaced key questions about the fundamental nature of relationships between man and machine that philosophers would grapple with for decades hence. Despite ELIZA’s apparent limitations in logic versus human intellect, its historical impact would underscore artificial intelligence’s broader potential to profoundly shape human perception and society.
Figure 1-8: A conversation with ELIZA.
Source: Wikimedia Commons/Public domain.
However, by the 1970s, AI faced growing criticism and budget cuts. Scientists had underestimated the depth of the challenges involved, fueling unrealistic public optimism that could not realistically be met. As AI failed to deliver on its lofty promises, private and government funding swiftly evaporated. Concurrently, research into simple, single-layer neural networks stagnated due in part to Marvin Minsky’s writings spotlighting perceptual limitations. This two-pronged assault on both the symbolic and connectionist approaches left AI reeling from seemingly insurmountable obstacles on all research fronts. For an era once filled with seemingly boundless potential, the 1970s became a bitter period of retreat as the field confronted the harsh realities of its shortcomings. The inflated aspirations for AI would need to be rigorously tempered before progress could resume on more modest, incremental fronts.
Moreover, the disparity between AI’s theoretical potential and practical results continued widening in the 1980s. Constraints imposed by restricted computing power, combinatorial expansions, and knowledge representation barriers dashed hopes for near-term progress. Despite a bright horizon, this persistent gap triggered greatly reduced interest and financial backing for Artificial Intelligence. Previously staunch supporters like the British government, DARPA, and NRC grew disillusioned by meager returns on investment. With the field unable to deliver on long-running promises, these agencies aborted funding, casting AI into a paralyzing “winter.” This first AI winter was characterized by the mass exodus of scientists due to scarce resources and loss of confidence needed to power human-level intelligence milestones. The period confronted AI with learning to walk again by concentrating on attainable stepping stones before contemplating any rekindled sprint toward human cognition.
Subsymbolic AI Era (1985–2010)
In the 1980s, a type of AI program known as expert systems gained traction by offering focused problem-solving abilities. These programs could field questions and provide solutions within a tightly defined area of expertise using logical rule sets distilled from human specialists. Expert systems found momentum by delivering targeted reasoning skills for particular domains like medicine or engineering without attempting to replicate the full spectrum of human cognition. Their aim was not expansive artificial general intelligence but rather narrowly concentrated artificial niche intelligence. By limiting their capabilities, expert systems attained enough practical utility during the AI winter to regain a foothold for the field. Their success highlighted Artificial Intelligence’s path forward: mastering well-scoped applications before progressing to grander human-mimicking aspirations.
Notable pioneering expert systems like DENDRAL (1965) for deducing chemical structures from spectral analyses and MYCIN (1972) for diagnosing bacterial blood infections proved the viability of concentrated knowledge systems. By focusing their algorithmic mastery on specific scientific tasks rather than boundless intelligence, these programs displayed the near-term potential for expert systems. Their demonstrations of feasibility within limited problem spaces offered a pathway for Artificial Intelligence to rebuild practical value, chipping away at obstacles by domain rather than tackling expansive human cognition all at once. Early expert system success stories thereby reignited pragmatic interest in AI amid the winter by evolving expectations toward achievements in targeted reasoning rather than the elusive quest to mimic mankind’s complete intellectual range.
Additional expert system success came in 1980 when Digital Equipment Corporation deployed XCON, an AI system for automatically configuring computer orders per customer specifications. Encoding 2,500 assembly rules, XCON delivered 95–98% precision across 80,000 configurations at DEC’s Salem plant, saving $25 million.
Subsequently in 1984, a bold symbolic AI endeavor named CYC sought to codify extensive common sense knowledge. Developers believed surpassing a critical threshold of human wisdom within CYC would enable self-propelled extraction of further insights from natural language. Although the initiative to systematize reasoning through preprogrammed facts and heuristics proved too ambitious, CYC represented lingering aspirations to manifest more fully general artificial intelligence. But as expectations reset toward more incremental advances, both XCON and CYC highlighted the promise of narrow yet capable AI versus impractical schemes to replicate multifaceted human cognition outright.
The expert systems movement triggered an explosion of commercial interest across industries. Businesses worldwide raced to develop and install domain-specific AI solutions for operational tasks. This surging adoption fueled the rise of affiliated hardware purveyors like Symbolics and Lisp Machines alongside software experts including IntelliCorp and Aion that consulted on specialized system development. As focused artificial intelligence proved economically viable – despite the overall AI winter – robust ecosystems materialized to meet burgeoning corporate demand. The strategic shift from pursuing expansive human mimicry to practical applications revived investment, underscoring the resilient appetite for capable if narrowly intelligent systems. Expert system implementation may not have realized AI’s full potential, but it offered a path out of the trough by delivering value scattered across niches.
In 1982, John Hopfield revived enthusiasm for neural networks within the AI community. By devising the seminal Hopfield net architecture and mathematical convergence proofs, he substantiated key learning properties applicable to interlocking neuron models. Meanwhile researchers Geoffrey Hinton and David Rumelhart elevated backpropagation techniques aimed at incrementally refining neural processing via error signal feedback. This dual resurrection of connectionist approaches proved timely, as demonstrated by Yann LeCun’s backprop-powered neural system at Bell Labs in 1989, which captured complex handwritten digits. Through concerted advancement of neural network algorithms closely paralleling biological brains, this collective renewal reestablished adaptive networks as a versatile methodology complementary to symbolic methods’ logical symbol manipulation.
However, this renaissance soon sputtered as the complexities of neural network programming overwhelmed practical progress. By the early 1990s, this era came to an end despite innovations like IBM’s Deep Blue defending its 1997 chess crown against Garry Kasparov – an accomplishment foretold by Herbert Simon that was achieved via systematic brute force rather than sophisticated cognition.
It became apparent that even heavily funded projects like Japan’s $850 million Fifth Generation Computer, UK’s £350 million Alvey Initiative, and DARPA’s Strategic Computing Program could only produce narrow expert systems ill-suited to general application. As businesses recognized the crippling constraints of commercialization, AI crashed into a second winter by the mid-1990s. The failed expectations of expert systems and inability to capture flexible human thinking in code left investors weary that AI’s theories might chronically lack real-world substance without a paradigm shift. While core innovations continued advancing, the field hunkered into a maturity phase marked by modest objectives rather than aspirations of replicating multifaceted cognition.
Deep Learning and LLM (2010–Present)
The possibility to use massive volumes of data changed everything and triggered a new boom supported by the Big Data era, the access to large amounts of data, cheaper and faster computers, and progress on machine learning.
