Platform and Model Design for Responsible AI E-Book

Amita Kapoor

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 739

Veröffentlichungsjahr: 2023

Beschreibung

AI algorithms are ubiquitous and used for tasks, from recruiting to deciding who will get a loan. With such widespread use of AI in the decision-making process, it’s necessary to build an explainable, responsible, transparent, and trustworthy AI-enabled system. With Platform and Model Design for Responsible AI, you’ll be able to make existing black box models transparent.
You’ll be able to identify and eliminate bias in your models, deal with uncertainty arising from both data and model limitations, and provide a responsible AI solution. You’ll start by designing ethical models for traditional and deep learning ML models, as well as deploying them in a sustainable production setup. After that, you’ll learn how to set up data pipelines, validate datasets, and set up component microservices in a secure and private way in any cloud-agnostic framework. You’ll then build a fair and private ML model with proper constraints, tune the hyperparameters, and evaluate the model metrics.
By the end of this book, you’ll know the best practices to comply with data privacy and ethics laws, in addition to the techniques needed for data anonymization. You’ll be able to develop models with explainability, store them in feature stores, and handle uncertainty in model predictions.

Leseprobe

Platform and Model Design for Responsible AI

Design and build resilient, private, fair, and transparent machine learning models

Amita Kapoor

Sharmistha Chatterjee

BIRMINGHAM—MUMBAI

Platform and Model Design for Responsible AI

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Associate Group Product Manager: Ali Abidi

Senior Editor: Tiksha Lad

Technical Editor: Devanshi Ayare

Copy Editor: Safis Editing

Language Support Editor: Safis Editing

Project Coordinator: Farheen Fathima

Proofreader: Safis Editing

Indexer: Subalakshmi Govindhan

Production Designer: Arunkumar Govinda Bhat

Marketing Coordinators: Shifa Ansari and Vinishka Kalra

First published: April 2023

Production reference: 1250423

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80323-707-7

www.packtpub.com

To my moral compass, friend, and mentor, Narotam Singh – your guidance, wisdom, and unwavering support have been the beacon of light in the journey of exploring responsible AI. This book is a tribute to the profound impact you have made, not only on this work but also on my life. Thank you for instilling the importance of ethics, integrity, and compassion in the pursuit of shaping a better future for AI and humanity alike.

– Amita Kapoor

This book is dedicated to my mother, Anjali Chatterjee, and my late father, Subhas Chatterjee, who as parents provided me with immense encouragement to pursue a career in science, technology, engineering, and mathematics (STEM) and supported me in all the decisions that made me successful. This book is a reflection of my better half, Abhisek Bakshi, who was the first to make me believe that I could author a book. The tremendous support, encouragement, mentorship, and vision he has laid in front of me in my journey in the field of AI deserves special mention. Your intense support and enlightenment have been immensely helpful and shaped my research in the field of responsible AI. The knowledge and wisdom I have gained in this field could not have been possible without your guidance and your support of equal partnership in our marriage. Thanks for shaping my thoughts and making it possible to accomplish this with two little daughters, Aarya and Adrika, aged 5 and 3. Last but not least, I would like to thank the little ones, for giving me space to complete this feat.

– Sharmistha Chatterjee

Contributors

About the authors

Amita Kapoor is an accomplished AI consultant and educator, with over 25 years of experience. She has received international recognition for her work, including the DAAD fellowship and the Intel Developer Mesh AI Innovator Award. She is a highly respected scholar in her field, with over 100 research papers and several best-selling books on deep learning and AI. After teaching for 25 years at the University of Delhi, Amita took early retirement and turned her focus to democratizing AI education. She currently serves as a member of the Board of Directors for the non-profit Neuromatch Academy, fostering greater accessibility to knowledge and resources in the field. Following her retirement, Amita also founded NePeur, a company that provides data analytics and AI consultancy services. In addition, she shares her expertise with a global audience by teaching online classes on data science and AI at the University of Oxford.

I would like to express my deepest gratitude to a number of people whose support and contributions have been invaluable in the creation of this book. First and foremost, I extend my heartfelt thanks to Ajit Jaokar, whose continuous support and stimulating discussions have been instrumental in shaping the ideas presented in this work.

I am immensely grateful to my co-author, Sharmistha, for her persistence and unwavering dedication. I also want to extend my sincere appreciation to the entire Packt team, who have been integral in bringing this book to life. I would like to thank the reviewers for their critical and constructive suggestions. Special mention goes to Ali, David, Kirti, and Tiksha, whose editorial expertise and hard work have been essential in refining and polishing this manuscript.

Without the collaborative efforts and support of these remarkable individuals, this book would not have been possible. Thank you for being a part of this incredible journey.

Sharmistha Chatterjee is an evangelist in the field of machine learning (ML) and cloud applications, currently working in the BFSI industry at the Commonwealth Bank of Australia in the data and analytics space. She has worked in Fortune 500 companies, as well as in early-stage start-ups. She became an advocate for responsible AI during her tenure at Publicis Sapient, where she led the digital transformation of clients across industry verticals. She is an international speaker at various tech conferences and a 2X Google Developer Expert in ML and Google Cloud. She has won multiple awards and has been listed in 40 under 40 data scientists by Analytics India Magazine (AIM) and 21 tech trailblazers in 2021 by Google. She has been involved in responsible AI initiatives led by Nasscom and as part of their DeepTech Club.

I would like to express my heartfelt gratitude to a number of people whose constant support and mentorship have led to the co-authoring of this book. In the first place, I would like to extend my wholehearted thankfulness to my friend, philosopher, and mentor Dr Sushmita Gupta and her husband, Dr Saket Saurabh, both from the Institute of Mathematical Sciences, Chennai, and Dr Fahad Panolan from IIT Hyderabad, who have instilled an interest in researching and designing fairness algorithms. In addition, I am honored and immensely grateful to my co-author, Dr Amita Kapoor, who has given me an opportunity to collaborate with her, listen to my ideas patiently, give timely feedback, and reshape several ideas presented in the book. I would also like to extend my gratitude to Publicis Sapient and my mentor, Roopa Hungund, for their support in researching and co-authoring this book.

The book is a manifestation of the unwavering support I have received from the entire Packt team, who have worked tirelessly to stitch the pieces together to bring a coherent story to this book. Special mention goes to Ali, David, Kirti, and Tiksha, whose support, editorial expertise, and hard work have been essential in refining and polishing this book.

Without the collaborative efforts, encouragement, and guidance of these remarkable individuals, this book would not have been possible. Thank you so much for being a part of this wonderful journey.

About the reviewers

Usha Rengaraju currently heads the data science research at Exa Protocol and is the world’s first women triple Kaggle Grandmaster. She specializes in deep learning and probabilistic graphical models and was also one of the judges of TigerGraph’s Graph for All Million Dollar Challenge. She was ranked in the top 10 data scientists in India by Analytics India Magazine and in the top 150 AI leaders and influencers by 3AI magazine. She is one of the winners of the ML in Action competition organized by the ML developer programs team at Google, and her team won first place in the WiDS 2022 Datathon organized by Stanford University. She is also the winner of the 2022 Kaggle ML Research Spotlight and the 2023 TensorFlow Community Spotlight.

Jeremy Abel has worked professionally in the AI/ML space for several years within the financial services industry, starting at the Bank of America in capital market analytics, via Wells Fargo in fraud prevention, to Ally Financial, where he currently leads the AI platform and ML engineering teams. He has a passion for solving problems to make room for more problems, believing that AI and ML can be leveraged to solve the problems of today, giving us room to think about the more complex problems of tomorrow. He is a firm believer that the application of AI is key to solving our world’s greatest challenges in a variety of sectors, but to do so effectively, we must approach it ethically and responsibly, starting with open conversation.

Sathyan Sethumadhavan works as an AI/ML strategist/architect with Thoughtworks. His expertise includes assessing enterprises for AI readiness, building AI/ML Centers of Excellence (CoEs) for large-scale enterprises, and building and leading data engineering and data science teams. He has led several large-scale AI platform implementations, building digital public goods and transformations for India’s public sector, using an on-premises and open source stack setup. He is also a thought leader in AI operationalization subjects and advises companies on increasing ROI, using value engineering frameworks, AI-powered decision factories (active learning and reinforcement learning), analytics-driven innovation, data as a product, data mesh/fabrics, MLOps, ML engineering, and ModelOps.

Table of Contents

Prefacexix

Part 1: Risk Assessment Machine Learning Frameworks in a Global Landscape

1

Risks and Attacks on ML Models3

Technical requirements3

Discovering risk elements4

Strategy risk5

Financial risk6

Technical risk6

People and processes risk7

Trust and explainability risk7

Compliance and regulatory risk8

Exploring risk mitigation strategies with vision, strategy, planning, and metrics8

Defining a structured risk identification process9

Enterprise-wide controls9

Micro-risk management and the reinforcement of controls9

Assessing potential impact and loss due to attacks23

Discovering different types of attacks25

Data phishing privacy attacks26

Poisoning attacks27

Evasion attacks27

Model stealing/extraction30

Perturbation attacks31

Scaffolding attack34

Model inversion35

Transfer learning attacks37

Summary41

Further reading42

2

The Emergence of Risk-Averse Methodologies and Frameworks45

Technical requirements46

Analyzing the threat matrix and defense techniques46

Researching and planning during the system and model design/architecture phase46

Model training and development56

ML model live in production60

Anonymization and data encryption65

Data masking66

Data swapping67

Data perturbation67

Data generalization67

K-anonymity68

L-diversity69

T-closeness70

Pseudonymization71

Homomorphic encryption73

Secure Multi-Party Computation (MPC/SMPC)76

Differential Privacy (DP)78

Sensitivity81

Properties of DP81

Hybrid privacy methods and models81

Adversarial risk mitigation frameworks83

Model robustness88

Summary93

Further reading94

3

Regulations and Policies Surrounding Trustworthy AI97

Regulations and enforcements under different authorities97

Regulations in the European Union98

Propositions/acts passed by other countries99

Special regulations for children and minority groups109

Promoting equality for minority groups110

Educational initiatives110

International AI initiatives and cooperative actions114

Next steps for trustworthy AI116

Proposed solutions and improvement areas120

Summary121

Further reading122

Part 2: Building Blocks and Patterns for a Next-Generation AI Ecosystem

4

Privacy Management in Big Data and Model Design Pipelines127

Technical requirements127

Designing privacy-proven pipelines128

Big data pipelines128

Architecting model design pipelines131

Incremental/continual ML training and retraining134

Scaling defense pipelines138

Enabling differential privacy in scalable architectures139

Designing secure microservices144

Vault145

Cloud security architecture151

Developing in a sandbox environment152

Managing secrets in cloud orchestration services156

Monitoring and threat detection156

Summary158

Further reading159

5

ML Pipeline, Model Evaluation, and Handling Uncertainty161

Technical requirements161

Understanding different components of ML pipelines162

ML tasks and algorithms163

Uncertainty in ML166

Types of uncertainty167

Quantifying uncertainty169

Uncertainty in regression tasks170

Uncertainty in classification tasks179

Tools for benchmarking and quantifying uncertainty181

The Uncertainty Baselines library182

Keras-Uncertainty182

Robustness metrics182

Summary183

References183

6

Hyperparameter Tuning, MLOps, and AutoML185

Technical requirements186

Introduction to AutoML186

Introducing H2O AutoML189

Understanding Amazon SageMaker Autopilot193

The need for MLOps194

TFX – a scalable end-to-end platform for AI/ML workflows199

Understanding Kubeflow200

Katib for hyperparameter tuning203

Vertex AI205

Datasets206

Training and experiments in Vertex AI208

Vertex AI Workbench211

Summary212

Further reading213

Part 3: Design Patterns for Model Optimization and Life Cycle Management

7

Fairness Notions and Fair Data Generation217

Technical requirements217

Understanding the impact of data on fairness218

Real-world bias examples218

Causes of bias221

Defining fairness223

Types of fairness based on statistical metrics224

Types of fairness based on the metrics of predicted outcomes228

Types of fairness based on similarity-based measures233

Types of fairness based on causal reasoning235

The role of data audits and quality checks in fairness236

Assessing fairness237

Linear regression239

The variance inflation factor240

Mutual information241

Significance tests241

Evaluating group fairness242

Evaluating counterfactual fairness246

Best practices249

Fair synthetic datasets250

MOSTLY AI’s self-supervised fair synthetic data generator250

A GAN-based fair synthetic data generator257

Summary262

Further reading262

8

Fairness in Model Optimization265

Technical requirements266

The notion of fairness in ML266

Unfairness mitigation methods268

In-processing methods268

Explicit unfairness mitigation270

Fairness constraints for a classification task270

Fairness constraints for a regression task276

Fairness constraints for a clustering task278

Fairness constraints for a reinforcement learning task280

Fairness constraints for recommendation systems282

Challenges of fairness285

Missing sensitive attributes285

Multiple sensitive attributes285

Choice of fairness measurements285

Individual versus group fairness trade-off285

Interpretation and fairness286

Fairness versus model performance286

Limited datasets286

Summary287

Further reading287

9

Model Explainability289

Technical requirements290

Introduction to Explainable AI290

Scope of XAI293

Challenges in XAI293

Explain Like I’m Five (ELI5)295

LIME297

SHAP298

Understanding churn modeling using XAI techniques299

Building a model300

Using ELI5 to understand classifier models304

Hands-on with LIME306

SHAP in action307

CausalNex310

DoWhy for causal inference311

DoWhy in action312

AI Explainability 360 for interpreting models316

Summary317

References318

10

Ethics and Model Governance319

Technical requirements319

Model Risk Management (MRM)320

Types of model inventory management320

Cost savings with MRM321

A transformative journey with MRM324

Model risk tiering325

Model risk calibration330

Model version control332

ModelDB332

Weights & Biases336

Further reading345

Part 4: Implementing an Organization Strategy, Best Practices, and Use Cases

11

The Ethics of Model Adaptability349

Technical requirements349

Adaptability framework for data and model drift350

Statistical methods354

Statistical process control362

Understanding model explainability during concept drift/calibration374

Explainability and calibration379

Challenges with calibration and fairness381

Summary382

Further reading382

12

Building Sustainable Enterprise-Grade AI Platforms383

Technical requirements383

The key to sustainable enterprise-grade AI platforms384

Sustainable solutions with AI as an organizational roadmap384

Organizational standards for sustainable frameworks385

Sustainability practices and metrics across different cloud platforms385

Emission metrics on Google Cloud386

Best practices and strategies for carbon-free energy386

The energy efficiency of data centers387

Carbon emission trackers388

The FL carbon calculator388

Centralized learning carbon emissions calculator390

Adopting sustainable model training and deployment with FL391

CO2e emission metrics391

Comparing emission factors – centralized learning versus FL392

Illustrating how FL works better than centralized learning393

The CO2 footprint of FL394

How to compensate for equivalent CO2e emissions396

Design patterns of FL-based model training396

Sustainability in model deployments400

Design patterns of FL-based model deployments402

Summary409

Further reading409

13

Sustainable Model Life Cycle Management, Feature Stores, and Model Calibration411

Sustainable model development practices412

Organizational standards for sustainable, trustworthy frameworks412

Explainability, privacy, and sustainability in feature stores416

Feature store components and functionalities416

Feature stores for FL419

Exploring model calibration422

Determining whether a model is well calibrated423

Calibration techniques424

Model calibration using scikit-learn425

Building sustainable, adaptable systems430

Concept drift-aware federated averaging (CDA-FedAvg)433

Summary435

Further reading436

14

Industry-Wide Use Cases437

Technical requirements437

Building ethical AI solutions across industries438

Biased chatbots438

Ethics in XR/AR/VR439

Use cases in retail440

Privacy in the retail industry440

Fairness in the retail industry441

Interpretability – the role of counterfactuals (CFs)443

Supply chain use cases452

Use cases in BFSI456

Deepfakes460

Use cases in healthcare461

Healthcare system architecture using Google Cloud462

Survival analysis for Responsible AI healthcare applications463

Summary466

Further reading467

Index469

Other Books You May Enjoy490

Preface

Artificial intelligence (AI) has come a long way since its inception, transforming from a futuristic concept into a ubiquitous technology that permeates every aspect of our lives. From healthcare and finance to decision-making processes in both the public and private sectors, AI systems have become integral to our daily existence. As AI-powered applications such as ChatGPT become essential tools for individuals and businesses alike, it is of utmost importance that we address the ethical, social, and technical challenges that accompany this progress.

The motivation behind this book is rooted in our belief that now, more than ever, we must lay the groundwork for a future where AI serves as a force for good. As AI continues to shape our world, this book seeks to provide AI engineers, business leaders, policymakers, and other stakeholders with comprehensive guidance on the development and implementation of responsible, trustworthy AI systems.

In this comprehensive book, we will explore various facets of Responsible AI, including the vulnerabilities of Machine Learning (ML) models, susceptibility to adversarial attacks, and the importance of robust security measures. We will delve into risk-averse methodologies that prioritize safety and reliability, minimizing potential harm and unintended consequences. The book examines policy frameworks and strategies adopted by various countries to ensure ethical AI development and deployment, as well as the crucial aspects of data privacy, with techniques and best practices to protect user information and maintain trust in AI systems. Additionally, we will cover approaches to AI model evaluation, uncertainty, and validation; the roles of MLOps and AutoML in fostering efficient, scalable, and responsible AI practices in enterprise settings; and the importance of fairness in AI, addressing challenges in data collection, preprocessing, and model optimization to reduce biases and ensure equitable outcomes. We will also discuss the need for transparency and explainability in AI systems, ethical governance, and oversight, and cover techniques to build adaptable, calibrated AI models that can respond effectively to changing environments and requirements. Moreover, we will delve into the concept of sustainable feature stores to promote efficiency and consistency in the development of responsible AI models and present real-world case studies and applications, demonstrating the impact and benefits of responsible AI across various industries.

This book aims to serve as a comprehensive resource for those seeking to harness the power of AI while addressing the critical ethical and social challenges it presents. We hope this book inspires you to join the movement toward responsible AI and apply its principles and practices in your own professional and personal endeavors.

Who this book is for

This book is for experienced ML professionals looking to understand the risks and data leakages of ML models and frameworks, incorporate fairness by design in both models and platforms, and learn how to develop and use reusable components to reduce effort and cost when setting up and maintaining an AI ecosystem.

What this book covers

Chapter 1, Risks and Attacks on ML Models, presents a detailed overview of key terms related to different types of attacks possible on ML models, creating a basic understanding of how ML attacks are designed by attackers. In this chapter, you will get familiar with the attacks, both direct and indirect, that compromise the privacy of a system. In this context, this chapter highlights losses incurred by organizations due to the loss of sensitive information and how individuals remain vulnerable to losing confidential information into the hands of adversaries.

Chapter 2, The Emergence of Risk-Averse Methodologies and Frameworks, presents an overall detailed overview of risk assessment frameworks, tools, and methodologies that can be directly applied to evaluate model risk. In this chapter, you will get familiar with the tools included in data platforms and model design techniques that will help to reduce the risk at scale. The primary objective of this chapter is to create awareness of data anonymization and validation techniques, in addition to the introduction of different terms and measures related to privacy.

Chapter 3, Regulations and Policies Surrounding Trustworthy AI, introduces different laws being passed across nations to protect and prevent the loss of sensitive information of customers. You will get to know the formation of different ethics expert groups, government initiatives, and policies being drafted to ensure the ethics and compliance of all AI solutions.

Chapter 4, Privacy Management in Big Data and Model Design Pipelines, presents a detailed overview of different components associated with a big data system, which serves as a building block atop which we can effectively deploy AI models. This chapter brings into the picture how compliance-related issues can be handled at a component level in a microservice-based architecture so that there is no information leakage. In this chapter, you get familiar with different security principles needed in individual microservices, as well as security measures that need to be incorporated in the cloud when deploying ML models at scale.

Chapter 5, ML Pipeline, Model Evaluation, and Handling Uncertainty, introduces the AI/ML workflow. The chapter then delves into different ML algorithms used for classification, regression, generation, and reinforcement learning. The chapter also discusses issues related to the reliability and trustworthiness of these algorithms. We start by introducing the various components of an ML pipeline. The chapter then briefly explores the important AI/ML algorithms for the tasks of classification, regression, and clustering. Further, we discuss various types of uncertainties, their causes, and the techniques to quantify uncertainty.

Chapter 6, Hyperparameter Tuning, MLOPs, and AutoML, continues from the previous chapter and explains the need for continuous training in an ML pipeline. Building an ML model is an iterative process, and the presence of so many models, each with a large number of hyperparameters, complicates things for beginners. This chapter provides a glimpse into the present AutoML options for your ML workflow. It expands on the situations where no-code/low-code solutions are useful. It explores the solutions provided by major cloud providers in terms of ease, features, and model explainability. Additionally, the chapter also covers orchestration tools, such as Kubeflow and Vertex AI, to manage the continuous training and deployment of your ML models.

Chapter 7, Fairness Notions and Fair Data Generation, presents problems pertaining to unfair data collection for different types of data, ontologies, vocabularies, and so on, due to the lack of standardization. The primary objective of this chapter is to stress the importance of the quality of data, as biased datasets can introduce hidden biases in ML models. This chapter focuses on the guiding principles for better data collection, management, and stewardship that need to be practiced globally. You will further see how evaluation strategies initial steps can help to build unbiased datasets, enabling new AI analytics and digital transformation journeys for ML-based predictions.

Chapter 8, Fairness in Model Optimization, presents different optimization constraints and techniques that are essential to optimize and obtain fair ML models. The focus of this chapter is to enlighten you with different, new customized optimizers, unveiled by research, that can serve to build supervised, unsupervised, and semi-supervised fair ML models. The chapter, in a broader sense, prepares you with the foundational steps to create and define model constraints that can be used by different optimizers during the training process. You will also gain an understanding of how to evaluate such constraint-based models with proper metrics and the extra training overheads incurred during the optimization techniques, which will enable the models to design their own algorithms.

Chapter 9, Model Explainability, introduces you to different methods that can be used to unravel the mystery of black boxes in ML models. We will talk about the need to be able to explain a model prediction. This chapter covers various algorithms and techniques, such as SHAP and LIME, to add an explainability component to existing models. We will explore the libraries, such as DoWhy and CausalNex, to see the explainability features available to an end user. We will also delve into the explainability features provided by Vertex AI, SageMaker, and H2O.ai.

Chapter 10, Ethics and Model Governance, emphasizes the ethical governance processes that need to be established with models in production, for quick identification of all risks related to the development and deployment of a model. This chapter also covers best practices for monitoring all models, including those in an inventory. You will get more insights into the practical nuances of risks that emerge in different phases of a model life cycle and how these risks can be mitigated when models reside in the inventory. Here, you will also understand the different risk classification procedures and how they can help minimize the business loss resulting from low-performance models. Further, you will also get detailed insights into how to establish proper governance in data aggregation, iterative rounds of model training, and the hyperparameter tuning process.

Chapter 11, The Ethics of Model Adaptability, focuses on establishing ethical governance processes for models in production, with the aim of quickly detecting any signs of model failure or bias in output predictions. By reading this chapter, you will gain a deeper understanding of the practical details involved in monitoring the performance of models and contextual model predictions, by reviewing the data constantly and benchmarking against the past in order to draft proper actionable short-term and long-term plans. Further, you will also get a detailed understanding of the conditions leading to model retraining and the importance of having a perfectly calibrated model. This chapter also highlights the trade-offs associated with fairness and model calibration.

Chapter 12, Building Sustainable Enterprise-Grade AI Platforms, focuses on how organizational goals, initiatives, and support from leadership can enable us to build sustainable ethical AI platforms. The goal of this chapter is to stress the importance of organizations contextualizing and linking ethical AI principles to reflect the local values, human rights, social norms, and behaviors of the community in which the solutions operate. In this context, the chapter highlights the impact of large-scale AI solutions on the environment and the right procedures that need to be incorporated for model training and deployment, using federated learning. This chapter further delves into important concepts that strongly emphasize the need to stay socially responsible, as well as being able to design software, models, and platforms.

Chapter 13, Sustainable Model Life Cycle Management, Feature Stores, and Model Calibration, explores the best practices that need to be followed during the model development life cycle, which can lead to the creation of sustainable feature stores. In this chapter, we will highlight the importance of implementing privacy so that reusing stores and collaboration among teams are maximized, without compromising security and privacy aspects. This chapter further provides a deep dive into different model calibration techniques, which are essential in building scalable sustainable ML platforms. Here, you will also understand how to design adaptable feature stores and how best we can incorporate monitoring and governance in federated learning.

Chapter 14, Industry-Wide Use Cases, presents a detailed overview of the different use cases across various industries. The primary aim of this is to inform readers coming from different industry domains on how ethics and compliance can be integrated into their systems, in order to build a fair and equitable AI system and win the confidence and trust of end users. You will also get a chance to apply algorithms and tools studied in previous chapters to different business problems. Further, you will gain an understanding of how ethical design patterns can be reused across different industry domains.

To get the most out of this book

Each chapter has different requirements, which have been specified in their respective chapters.

You should have basic knowledge of ML, Python, scikit-learn, PyTorch, and TensorFlow to better understand the concepts of this book.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Platform-and-Model-Design-for-Responsible-AI. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Atlas offers great flexibility in dynamically creating classifications, such as PII, EXPIRES_ON, DATA_QUALITY, and SENSITIVE, with support for the expiry_date attribute in the EXPIRES_ON classification.”

A block of code is set as follows:

model.compile(optimizer='rmsprop', loss=aleatoric_loss, metrics=['mae'])

Any command-line input or output is written as follows:

roc_auc_score(y_test, y_pred_uncal) >>> 0. 9185432154389126

Bold: Indicates a new term, an important word, or words that you see on screen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Moreover, we can see the sequential security controls that we can follow to enhance our security stack by going to RBAC | Policy Management | Discovery | Settings | Real-Time Controls.

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Platform and Model Design for Responsible AI, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere? Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application. 

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781803237077

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Risk Assessment Machine Learning Frameworks in a Global Landscape

This part provides a detailed introduction to the risks, threats, and challenges that machine learning models in production are vulnerable to. In this part, you will learn about different types of attacks that can be carried out by adversaries and the importance of protecting your models from such attacks. This part also covers the guidelines and standards set by different committees across the world, to facilitate various actions and initiatives at both a national and organizational level.

This part is made up of the following chapters:

Chapter 1, Risks and Attacks on ML ModelsChapter 2, The Emergence of Risk-Averse Methodologies and FrameworksChapter 3, Regulations and Policies Surrounding Trustworthy AI

1

Risks and Attacks on ML Models

This chapter gives a detailed overview of defining and evaluating a Machine Learning (ML) risk framework from the instant an organization plans to embark on AI digital transformation. Risks may come in different stages, such as when the strategic or financial planning kicks in or during several of the execution phases. Risks start surfacing with the onset of technical implementations and continue up to testing phases when the AI use case is served to customers. Risk quantification can be attained through different metrics, which can certify the system behavior (amount of robustness and resiliency) against risks. In the process of understanding risk evaluation techniques, you will also get a thorough understanding of attacks and threats to ML models. In this context, you will discover different components of the system having security or privacy bottlenecks that pose external threats and make the model open to vulnerabilities. You will get to know the financial losses and business impacts when models deployed in production are not risk and threat resilient.

In this chapter, these topics will be covered in the following sections:

Discovering risk elementsExploring risk mitigation strategies with vision, strategy, planning, and metricsAssessing potential impact and loss due to attacksDiscovering different types of attacks

Further, with the use of Adversarial Robustness Toolbox (ART) and AIJack, we will see how to design attacks for ML models.

Technical requirements

This chapter requires you to have Python 3.8 along with some necessary Python packages, as follows. The commands to install ART and AIJack are also listed here:

Keras 2.7.0, TensorFlow 2.7.0pip install adversarial-robustness-toolboxpip install git+https://github.com/Koukyosyumei/AIJack

Discovering risk elements

With rapid digitization and AI adoption, more and more organizations are becoming aware of the unintended consequences of malicious AI adoption practices. These can impact not only the organization’s reputation and long-term business outcomes but also the business’ customers and society at large. Here, let us look at the different risk elements involved in an AI digitization journey that CXOs, leadership teams, and technical and operational teams should be aware of. The purpose of these associated teams is one and the same: to avoid any of their systems getting compromised, or any security/privacy violations that could yield discrimination, accidents, the manipulation of political systems, or the loss of human life.

Figure 1.1 – A diagram showing the AI risk framework

There are three principal elements that govern the risk framework:

Planning and execution: This phase ideally covers all stages in product development, that is, the conceptualization of the AI use case, financial planning, execution, including the technical execution, and the design and release of the final product/solution from an initial Minimum Viable Product (MVP).People and processes: This is the most crucial factor as far as delivery timelines are concerned with respect to an MVP or a final product/solution. Leadership should have a clear vision and guidelines put in place so that research, technical, QA, and other operational teams find it easy to execute data and ML processes following defined protocols and standards.Acceptance: This phase involves several rounds of audits and confirmations to validate all steps of technical model design and deployment. This process adheres to extra confirmatory guidelines and laws in place to cautiously review and explain AI/ML model outcomes with due respect to user fairness and privacy to protect users’ confidential information.

Let’s drill down into the components of each of these elements.

Strategy risk

On the strategic front, there should be a prior Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis done on business use cases requiring digital AI transformations. The CXOs and leadership team must identify the right business use case after doing an impact versus effort analysis and formulate the guidelines and a list of coherent actions needed for execution. The absence of this might set infeasible initiatives that are not aligned with the organization’s business goals, causing financial loss and solutions failing. Figure 1.2 illustrates how a specific industry (say, retail) can classify different use cases based on a value-effort framework.

Figure 1.2 – A value-effort framework

If the guidelines and actions are not set properly, then AI systems can harm individuals, society, and organizations. The following are some examples:

AI-powered autonomous vehicles can often malfunction, which can lead to injury or death.Over-reliance on inadequate equipment and insufficient monitoring mean predictive maintenance tasks can lead to worker injury.ML models misdiagnose medical conditions.Political disruption by manipulating national institutional processes (for example, elections or appointments) by misrepresenting information.Data breaches can expose confidential military locations or technical secrets.Infrastructure disruption or misuse by intelligent systems (for example, GPS routing cars through different streets often increases traffic flow in residential areas).

Financial risk

The executive team should understand the finances involved in sponsoring an AI development project right from its inception to all stages of its development. Financial planning should not only consider the cost involved in hiring and retaining top talent but also the costs associated with infrastructure (cloud, containers, GPUs, and so on), data governance, and management tools. In addition, the financial roadmap should also specify the compliance necessary in big data and model deployment management as the risks and penalties can be huge in case of any violations.

Technical risk

The risk associated on the technical front can manifest from the point when the data is ingested into the system. Data quality and the suitability of representation formats can seriously violate regulations (Derisking machine learning and artificial intelligence: https://www.mckinsey.com/business-functions/risk-and-resilience/our-insights/derisking-machine-learning-and-artificial-intelligence). Along with a skilled data science and big data team, what is needed is the availability and awareness of modern tools and practices that can detect and alert issues related to data or model quality and drifts and take timely remedial action.

Figure 1.3 – A diagram showing risk management controls

Figure 1.3 illustrates different risk elements that can cause security breaches or theft of confidential information. The different components (data aggregation, preprocessing, model development, deployment, and model serving) of a real-time AI pipeline must be properly designed, monitored (for AI drift, bias, changes in the characteristics of the retraining population, circuit breakers, and fallback options), and audited before running it in production.

Along with this, risk assessment also includes how AI/ML models are identified, classified, and inventoried, with due consideration of how they are trained (for example, considering data type, vendor/open source libraries/code, third-party/vendor code updates and maintenance practices, and online retraining) and served to customers.

People and processes risk

The foremost objective of leadership and executive teams is to foster innovation and encourage an open culture where teams can collaborate, innovate, and thrive. When technical teams are proactive in bringing in automations in MLOps pipelines, many problems can be foreseen, and prompt measures can be taken to bridge the gaps through knowledge-sharing sessions.

Trust and explainability risk

Businesses remain reluctant to adopt AI-powered applications when the results of the model cannot be explained. Some of the unexplainable results can be attributed to the poor performance of the model for a selected customer segment or during a specific period (for example, many business predictions were affected by the outbreak of COVID-19). The opaqueness of the model – a lack of explanation of the results – causes fear when businesses or customers find there is a lack of incentive alignment or severe disruption to people’s workflows or daily routines. ML models answering questions about the behavior of the model raises stakeholder confidence. In addition to deploying an optimized model that can give the right predictions with minimal delay, the model should also be able to explain the factors that affect the decisions it makes. However, it’s up to the ML/AI practitioners to use their judgment and analysis to apply the right ML models and explainability tools to derive the factors contributing to the model’s behavior. Now, let us see – with an example – how explainability can aid in studying medical images.

Deep Neural Networks (DNNs) may be computationally hard to explain, but significant research is taking place into the explainability of DNNs as well. One such example involves Explainable Artificial Intelligence (XAI), used on pretrained deep learning neural networks (AlexNet, SqueezeNet, ResNet50, and VGG16), which has been successful in explaining critical regions that are affected by Barrett’s esophagus using related data by comparing classification rates. The comparative results can detect early stages of cancer and distinguish Barrett’s esophagus (https://www.sciencedirect.com/science/article/pii/S0010482521003723) from adenocarcinoma. However, it remains up to the data scientist to decide how best to explain the use of their models, by selecting the right data and number of data points, based on the type of the problem.

Compliance and regulatory risk

There are different privacy laws and regulations that have been set forth by different nations and governing agencies that impose penalties on organizations in case of violations. Some of the most common privacy rules include the European Union’s General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). The financial and healthcare sectors have already seen laws formulated to prevent bias and allow fair treatment. Adhering to compliance necessitates extra planning for risk management through audits and human monitoring.

Apart from country-specific regulatory laws and guidance, regulators will likely rely on existing guidance in SR 11-7/OCC 2011-12 to assess the risks of AI/ML applications.

Ethical risk

AI/ML models should go through proper validations and A/B testing to verify their compliance and fairness across different sections of the population, including people of varying genders and diverse racial and ethical backgrounds. For example, credit scoring and insurance models have historically been biased against racial minorities and discrimination-based lending decisions have resulted in litigation.

To make AI/ML models ethical, legal, and risk-free, it is inevitable for any organization and the executive team to have to ascertain the impact of the AI solution and service being rolled out in the market. This includes the inclusion of highly competent AI ethics personnel in the process who have regulatory oversight, and ensuring adherence to protocols and controls for risk mitigation to make sure the entire AI solution is robust and less attractive to attackers.

Such practices can not only add extra layers of security to anonymize individual identity but also remove any bias present in legacy systems. Now let us see what kinds of enterprise-grade initiatives are essential for inclusion in the AI development process.

Exploring risk mitigation strategies with vision, strategy, planning, and metrics

After seeing the elements of risk in different stages of the AI transformation journey, now let us walk through the different enterprise risk mitigation plans, measures, and metrics. In later chapters, we will not only discover risks related to ML model design, development, and deployment but also get to know how policies put in place by executive leadership teams are important in designing systems that are compliant with country-specific regulatory laws. Timely review, awareness, and support in the risk identification process can save organizations from unexpected financial losses.

Defining a structured risk identification process

The long-term mission and short-term goals can only be achieved when business leaders, IT, security, and risk management teams align to evaluate a company’s existing risks, and whether they are affecting the upcoming AI-driven analytics solution. Such an effort, led by one of the largest European bank's COOs, helped to identify biased product recommendations. If left unchecked, it could have led to financial loss, regulatory fines, and disgrace, impacting the organization’s reputation and causing a loss of customers and a backlash.

This effort may vary from industry to industry. For example, the food and beverage industry needs to concentrate on risks related to contaminated products, while the healthcare industry needs to pay special attention to refrain from the misdiagnosis of patients and protect their sensitive health data.

Enterprise-wide controls

Effective controls and techniques are structured around the incorporation of strong policies, worker training, contingency plans, and the redefinition of business rules and objectives that can be put into practice. These policies translate to specified standards and guidelines requiring human intervention as and when needed. For example, the European bank had to adopt flexibility in deciding how to handle specific customer cases when the customer’s financial or physical health was impacted: https://www.mckinsey.com/business-functions/mckinsey-analytics/our-insights/confronting-the-risks-of-artificial-intelligence. In such cases, relationship managers had to intervene to offer suitable recommendations to help them to move on with the death/loss of a family member. Similarly, the healthcare industry needs the intervention of doctors and healthcare experts to adopt different active learning strategies to learn about rare diseases and their symptoms. Control measures necessitate the application of different open source or custom-built tools that can mitigate the risks of SaaS-based platforms and services, protect groups from potential discrimination, and ensure compliance with GDPR.

Micro-risk management and the reinforcement of controls

The tools and techniques put into practice will vary based on the phase of the ML life cycle. Attacks and threats are much too specific to input data, feature engineering, model training, deployment, and the way the model is served to its customers. Hence it is essential to design and evaluate any ML model against a threat matrix (more details on threat matrices will be discussed in Chapter 2). The most important factors that must be taken into consideration are the model's objective, optimization function, mode of learning (centralized versus federated), human-to-machine (or machine-to-machine) interaction, environmental factors (for designing policies and rewards in the case of reinforcement learning), feedback, retraining, and deployment. These factors, along with the model design and its explainability, will push organizations to go for a more transparent and explainable ML model and remove ML models that are overly complex, opaque, and unexplainable. The threat matrix can safeguard ML models in deployment by not only evaluating model performance but also testing models for adversarial attacks and other external factors that cause ML models to drift.

You need to apply a varying mix of risk control measures and risk mitigation strategies and reinforce them based on the outcome of the threat matrix. Along the journey of the AI transformation process, this will not only alleviate risks and reduce unseen costs but also make the system robust and transparent to counteract every possible risk. With such principles put into place, organizations can not only prevent ethical, business, reputation, and regulatory issues but also serve their customers and society with fair, equal, and impartial treatment.

Figure 1.4 – A diagram showing enhancements and mitigations in current risk management settings

A number of new elements related to ethics are needed in current AI/ML risk frameworks, which can help to ascertain risk performance and alleviate risk:

InterpretabilityEthical AI validation toolsModel privacyModel compressionBiasFeature engineeringSustainable model trainingPrivacy-related pre-/post-processing techniquesFairness constraintsHyperparametersModel storage and versioningEpsilonTotal and fairness lossCloud/data center sustainabilityFeature storesAttacks and threatsDriftDynamic model calibrationA review of the pipeline design and architectureModel risk scoringData/model lineage

While we will study each of these components in later chapters, let us introduce the concepts here and understand why each of these components serves as an important unit for responsible/ethical model design and how they fit into the larger ML ecosystem.

To further illustrate, let us first consider the primary risk areas of AI ethics (the regulatory and model explainability risks) in Figure 1.5 by breaking down Figure 1.4. The following figure illustrates risk assessment methods and techniques to explain model outcomes.

Figure 1.5 – Risk assessment through regulatory assessment and model explainability

We see both global and local surrogate models play an important role in interpretability. While a global surrogate model has been trained to approximate the predictions of a black-box model, a local surrogate model is able to explain the local predictions of an individual record by changing the distribution of the surrogate model’s input. It is done through the process of weighting the data locally with a specific instance of the data (providing a higher weight to instances that resemble the instance in question).

Ethical AI validation tools

These tools, either open source, through public APIs, or provided by different cloud providers (Google Cloud, Azure, or AWS), provide ways to validate the incoming data against different discriminatory sections of the population. Moreover, these tools also assist in discovering the protected data fields and data quality issues. Once the data is profiled with such tools, notification services and dashboards can be built in to detect data issues with the incoming data stream from individual data sources.

Model interpretability

ML models, especially neural networks, are often called black boxes as the outcomes cannot be directly linked to the model architecture and explained. Businesses often roll out ML models in production that can not only recommend or predict customer demand but also substantiate the model’s decision with facts (single-feature or multiple-feature interactions). Despite the black-box nature of ML models, there are different open source interpretability tools available that can significantly explain the model outcome, such as, for example, why a loan application has been denied to a customer or why an individual of a certain age group and demographic is vulnerable to a certain disease:

Linear coefficients help to explain monotonic models (linear regression models) and justify the dependency of selected features and the results of the output.Nonlinear and monotonic models (for example, gradient-boosting models with a monotonic constraint) help with selecting the right feature set among many present features for prediction by evaluating the positive or negative relationship with the dependent variable.

Nonlinear and nonmonotonic (for example, unconstrained deep learning models) methodologies such as local interpretable model-agnostic explanations or Shapley (an explainability Python library) serve as important tools for helping models with local interpretability. Neural networks have two broad primary categories for explaining ML models:

Saliency methods/saliency maps (SMs)Feature Attribution (FA)

Saliency Maps are only effective at conveying information related to weights being activated on specified inputs or different portions of an image being selected by a Convolutional Neural Network (CNN). While saliency maps cannot convey information related to feature importance, FA methods aim to fit structural models on data subsets to evaluate the degree/power/impact each variable has on the output variable.

Discriminative DNNs are able to provide model explainability and explain the most important features by considering the model’s input gradients, meaning the gradients of the output logits with regard to the inputs. Certain SM-based interpretability techniques (gradient, SmoothGrad, and GradCAM) are effective interpretability methods that are still under research. For example, the gradient method is able to detect the most important pixels in an image by applying a backward pass through the network. The score arrived at after computing the derivative of the class with respect to the input image helps further in feature attribution. We can even use tools such as an XAI SM for image or video processing applications. Tools can show us how a network’s decision is affected by the most important parts of an image or video.

Model privacy

With laws such as GDPR, CCPA, and policies introduced by different legislative bodies, ML models have absorbed the principle of privacy by design to gain user trust by incorporating privacy-preserving techniques. The objective behind said standards and the ML model redesign has primarily been to prevent information leaking from systems by building AI solutions and systems with the following characteristics:

Proactive and preventive instead of reactive and remedialIn-built privacy as the default settingPrivacy embedded into the designFully functional – no trade-offs on functionalityML model life cycle security, privacy, and end-to-end protectionVisibility and transparencyUser-centric with respect for user privacy

To encompass privacy at the model level, researchers and data scientists use a few principal units or essential building blocks that should have enough security measures built in to prevent the loss of sensitive and private information. These building units are as follows:

Model training data privacy: The data pipeline for the ML training data ingestion unit should have sufficient security measures built in. Any adversary attempting to attack the system should not be able to reverse-engineer the training data.Model input privacy: The security and privacy measures should ensure any input data going for model training cannot be seen by anyone, including the data scientist who is creating the model.Model output privacy: The security and privacy measures should ensure that the model output is not visible to anyone except the recipient user whose data is being predicted.Model storage and access privacy: The model must be stored securely with defined access rights to only eligible data science professionals.

Figure 1.6 illustrates different stages of model training and improvement where model privacy must be ensured to safeguard training data, model inputs, model weights, and the product, which is the ML model output.

Figure 1.6 – A diagram showing privacy in ML models

Model compression

AI ethics, standards, and guidelines have propelled researchers and data science professionals to look for ways to run and deploy these ML models on low-power and resource-constrained devices without sacrificing model accuracy. Here, model compression is essential as compressed models with the same functionality are best for devices that have limited memory. From the standpoint of AI ethics, we must leverage ML technology for the benefit of humankind. Hence, it is imperative that robust compressed models are trained and deployed in extreme environments such that they have minimal human intervention, and at the same time memorize relevant information (by having optimal pruning of the number of neurons).

For example, one technique is to build robust compressed models using noise-induced perturbations. Such noise often comes with IoT devices, which receive a lot of perturbations in the incoming data collected from the environment. Research results demonstrate that on-manifold adversarial training, which takes into consideration real-world noisy data, is able to yield highly compressed models and higher-accuracy models than off-manifold adversarial training, which incorporates noise from external attackers. Figure 1.7 illustrates that manifold adversarial samples are closer to the decision boundary than the simulated samples.

Figure 1.7 – A diagram of simulated and on-manifold adversarial samples

Sustainable model training

Low-powered devices depend on renewable energy resources for their own energy generation and local model training in federated learning ecosystems. There are different strategies by which devices can participate in the model training process and send updates to the central server. The main objective of devices taking part in the training process intermittently is to use the available energy efficiently in a sustainable fashion so that the devices do not run out of power and remain in the system till the global model converges. Sustainable model training sets guidelines and effective strategies to maximize power utilization for the benefit of the environment.

Bias

ML models are subjected to different kinds of bias, both from the data and the model. While common data bias occurs from structural bias (mislabeling gender under perceived notions of societal constructs, for example, labeling women as nurses, teachers, and cooks), data collection, and data manipulation, common model bias occurs from data sampling, measurement, algorithmic bias, and bias against groups, segments, demographics, sectors, or classes.

Random Forest (RF) algorithms work on the principle of randomization in the two-phase process of bagging samples and feature selection. The randomization process accounts for model bias from uninformative feature selection, especially for high-dimensional data with multi-valued features. The RF model elevated the risk level in money-laundering prediction by favoring the multi-valued dataset with many categorical variables for feature occupation. However, the same model was found to yield better, unbiased outcomes with a decrease in the number of categorical values. More advanced models built on top of RF, known as xRF, can select more relevant features using statistical assessments such as the p-value. The p-value assessment technique helps to assign appropriate weight to features based on their importance and aids in the selection of unbiased features by generating more accurate trees. This is an example of a feature weighting sampling technique used for dimensionality reduction.

Feature engineering

This has become increasingly complex to understand for black-box models such as neural networks when compared to traditional ML models. For example, a CNN needs proper knowledge and application of filters to remove unwanted attributes. Models built from high-dimensional data need to incorporate proper dimensionality reduction techniques to select the most relevant one. Moreover, ML models resulting from Natural Language Processing (NLP) require preprocessing as one of the preliminary steps for model design. There are several commercial and open source libraries available that aid in new, complex feature creation, but they can also yield overfitted ML models. It has been found that overfitted models provide a direct threat to privacy and may leak private information (https://machinelearningmastery.com/data-leakage-machine-learning/). Hence, model risk mitigation mechanisms must employ individual feature assessment to confirm included features’ impact (mathematical transformation and decision criteria) on the business rationale. The role of feature creation can be best understood in a specific credit modeling use case by banks where the ML model can predict defaulters based on the engineered feature of debt-to-income ratio.

Privacy-related pre-/post-processing techniques

Data anonymization requires the addition of noise in some form (Gaussian/Laplace distribution) that can either be initiated prior to the model training process (K-anonymity, Differential Privacy (DP)) or post model convergence (bolt-on DP).

Fairness constraints

ML models can be trained to yield desirable outcomes through different constraints. Constraints define different boundary conditions for ML models that on training the objective function would yield a fair, impartial prediction for minority or discriminatory racial groups. Such constraints need to be designed and introduced based on the type of training, namely supervised, semi-supervised, unsupervised, ranking, recommendations, or reinforcement-based learning. Datasets where constraints are applied the most have one or more sensitive attributes. Along with constraints, model validators should be entrusted to ensure a sound selection of parameters using randomized or grid search algorithms.

Model storage and versioning

One important component of ethical AI systems is to endow production systems with the capability to reproduce data and model results, in the absence of which it becomes immensely difficult to diagnose failures and take immediate remedial action. Versioning and storing previous model versions not only allows you to quickly revert to a previous version, or activate model reproducibility to specific inputs, but it also helps to reduce debugging time and duplicating effort. Different tools and best practice mechanisms aid in model reproducibility by abstracting computational graphs and archiving data at every step of the ML engine.

Epsilon (ε)

This is a metric used in DP solutions that is responsible for providing application-level privacy. This metric is used to measure privacy loss incurred on issuing the same query to two different datasets, where the two datasets differ in only one record and the difference is created by adding or removing one entry from one of the databases. We will discuss DP more in Chapter 2. This metric reveals the privacy risk imposed when it is computed on the private sensitive information of the previously mentioned datasets. It is also called privacy budget and is computed based on the input data size and the amount of noise added to the training data. The smaller the value, the better the privacy protection.

Cloud/data center sustainability

With growing concerns about climate change and sustainability issues, the major cloud providers (Google, Amazon, and Microsoft) have started energy efficiency efforts to foster greener cloud-based products. The launch of carbon footprint reporting has enabled users to measure, track, and report on the carbon emissions associated with the cloud. To encourage businesses to have a minimal impact on the environment, all ML deployments should treat sustainability as a risk or compliance to be measured and managed. This propels data science and cloud teams to consider the deployment of ML pipelines and feature stores in sustainable data centers.

Feature stores

Feature stores allow feature reuse, thus saving on extra storage and cloud costs. As data reuse and storage must meet compliance and regulations, it is an important consideration parameter in ethical AI. Feature stores allow the creation of important features using feature engineering and foster collaboration among team members to share, discover, and use existing features without doing additional rework. Feature reuse also prompts the reuse of important attributes based on importance of features and model explainability as defined by other teams. As deep learning models require huge computing power and energy, the proper selection of algorithms, along with the reuse of model data and features, reduces cloud costs by reducing computational capacity.

Attacks and threats

A risk framework designed for production-grade enterprise AI solutions should be integrated with an attack testing framework (third-party and open source), to ascertain the model risk from external adversaries. The ML model’s susceptibility to attack can then be used to increase the monitoring activity to be proactive in the case of attacks.

Drift

Data and model monitoring