MLOps with Red Hat OpenShift E-Book

Ross Brigoli

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 211

Veröffentlichungsjahr: 2024

Beschreibung

MLOps with OpenShift offers practical insights for implementing MLOps workflows on the dynamic OpenShift platform. As organizations worldwide seek to harness the power of machine learning operations, this book lays the foundation for your MLOps success. Starting with an exploration of key MLOps concepts, including data preparation, model training, and deployment, you’ll prepare to unleash OpenShift capabilities, kicking off with a primer on containers, pods, operators, and more.
With the groundwork in place, you’ll be guided to MLOps workflows, uncovering the applications of popular machine learning frameworks for training and testing models on the platform.
As you advance through the chapters, you’ll focus on the open-source data science and machine learning platform, Red Hat OpenShift Data Science, and its partner components, such as Pachyderm and Intel OpenVino, to understand their role in building and managing data pipelines, as well as deploying and monitoring machine learning models.
Armed with this comprehensive knowledge, you’ll be able to implement MLOps workflows on the OpenShift platform proficiently.

Leseprobe

MLOps with Red Hat OpenShift

A cloud-native approach to machine learning operations

Ross Brigoli

Faisal Masood

MLOps with Red Hat OpenShift

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Niranjan Naikwadi

Publishing Product Manager: Sanjana Gupta

Book Project Manager: Hemangi Lotlikar

Senior Editor: Sushma Reddy

Technical Editor: Rahul Limbachiya

Copy Editor: Safis Editing

Proofreader: Safis Editing

Indexer: Manju Arasan

Production Designer: Prafulla Nikalje

DevRel Marketing Coordinator: Vinishka Kalra

First published: January 2024

Production reference: 1240124

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-80512-023-0

www.packtpub.com

“To my partner in life, Hazel, for the love and support in everything that I do. To Yleana, my little genius, may these pages serve as an inspiration for your own intellectual explorations someday.”

— Ross Brigoli

“To my late mother, who has supported me during my ups and downs and encouraged me to explore new horizons.”

— Faisal Masood

Contributors

About the authors

Ross Brigoli is a consulting architect at Red Hat, the largest open source software company in the world, delivering innovative solutions to its customers. He has professional experience spanning more than two decades, marked by expertise in software engineering and architecture, solution design, data engineering, machine learning, DevOps, and MLOps. Before Red Hat, Ross was an associate director at Credit Agricole Corporate and Investment Bank, leading the architecture of a big data platform. Ross, along with Faisal Masood, co-authored the book Machine Learning on Kubernetes.

Faisal Masood is a cloud transformation architect at AWS. Faisal’s focus is on assisting customers in refining and executing strategic business goals. His main interests are evolutionary architectures, software development, the ML life cycle, CD, and IaC. Faisal has over two decades of experience in software architecture and development.

About the reviewers

Baltazar Chua is a seasoned IT professional with over 20 years of diverse industry experience, including finance and telecommunications. Currently serving as a technical architect for one of Europe's leading French banks, Baltazar excels in building and designing distributed systems. His passion lies in crafting innovative solutions, showcasing a commitment to lifelong learning. Beyond his professional endeavors, Baltazar is an enthusiastic contributor to open source projects within the Cloud Native Computing Foundation and other cloud-native initiatives. In his free time, he actively supports cutting-edge solutions that drive the industry forward.

Amreth Chandrasehar is an engineering leader in cloud, AI/ML engineering, observability, and SRE. Over the last few years, Amreth has played a key role in cloud migration, generative AI, AIOps, observability, and ML adoption at various organizations. Amreth is also co-creator of Conducktor Platform, serving T-Mobile’s 100+ million customers, and a tech/customer advisory board member at various companies on observability. Amreth has also co-created and open sourced Kardio.io, a service health dashboard tool. Amreth has been invited to speak at several key conferences and has won several awards.

I would like to thank my wife, Ashwinya, and my son, Athvik, for the patience and support they provided during my review of this book.

Rudrendu Kumar Paul, an AI expert and applied ML industry professional with over a decade of experience across diverse industries, excels in leading data science and engineering teams. His rich background spans multiple Fortune-50 companies in industrial applications, e-commerce, supply chain, and high-tech industries. Proficient in managing end-to-end AI and machine learning processes, he navigates experimentation, advanced analytics, and deploying models in production. Rudrendu holds an MBA, an MS in data science from Boston University, and a bachelor's in electrical engineering.

Table of Contents

Part 1: Introduction

1

Introduction to MLOps and OpenShift

What is MLOps?

Introduction to OpenShift

OpenShift features

Understanding operators

Understanding how OpenShift supports MLOps

Red Hat OpenShift Data Science (RHODS)

The advantages of the cloud

ROSA

Summary

References

Part 2: Provisioning and Configuration

2

Provisioning an MLOps Platform in the Cloud

Technical requirements

Installing OpenShift on AWS

Preparing AWS accounts and service quotas

Preparing AWS for ROSA provisioning

Installing ROSA

Adding a new machine pool to the cluster

Installing Red Hat ODS

Installing partner software on RedHat ODS

Installing Pachyderm

Summary

3

Building Machine LearningModels with OpenShift

Technical requirements

Using Jupyter Notebooks in OpenShift

Provisioning an S3 store

Using ML frameworks in OpenShift

Using GPU acceleration for model training

Enabling GPU support

Building custom notebooks

Creating a custom notebook image

Importing notebook images

Summary

Part 3: Operating ML Workloads

4

Managing a Model Training Workflow

Technical requirements

Configuring Pachyderm

Versioning your data with Pachyderm

Training a model using Red Hat ODS

Building a model training pipeline

Installing Red Hat OpenShift Pipelines

Attaching a pipeline server to your project

Building a basic data science pipeline

Summary

5

Deploying ML Models as a Service

Packaging and deploying models as a service

Saving and uploading models to S3

Updating the pipeline via model upload to S3

Creating a model server for Seldon

Deploying and accessing your model

Autoscaling the deployed models

Releasing new versions of the model

Automating the model deployment process

Rolling back model deployments

Canary model deployment

Securing model endpoints

Summary

6

Operating ML Workloads

Monitoring ML models

Installing and configuring Prometheus and Grafana

Logging inference calls

Optimizing cost

Summary

References

7

Building a Face Detector Using the Red Hat ML Platform

Architecting a human face detector system

Training a model for face detection

Deploying the model

Validating the deployed model

Installing Redis on Red Hat OpenShift

Building and deploying the inferencing application

Bringing it all together

Optimizing cost for your ML platform

Machine management in OpenShift

Spot Instances

Summary

Index

Other Books You May Enjoy

Preface

MLOps, or Machine Learning Operations, is all about streamlining and harmonizing the intricate dance between developing and deploying machine learning models. It’s like the conductor orchestrating a symphony, ensuring a seamless flow from the creative realm of data science to the robust reality of IT operations.

This book introduces a practical approach to implementing MLOps on the Red Hat OpenShift platform. It starts by presenting key MLOps concepts such as data preparation, model training, and packaging and deployment automation. An overview of OpenShift’s fundamental building blocks—deployments, pods, and operators—is then provided. Once the basics are covered, the book delves into platform provisioning and deepens our exploration of MLOps workflows.

Throughout the book, Red Hat OpenShift Data Science (RHODS), a data science platform designed to run on OpenShift, is utilized. You will experience creating ML projects, notebooks, and training and deployment pipelines using RHODS. The book also covers the use of partner software components that complement the RHODS platform, including Pachyderm and Intel OpenVino.

By the book’s end, you will gain a solid understanding of MLOps concepts, best practices, and the skills needed to implement MLOps workflows with Red Hat OpenShift Data Science on the Red Hat OpenShift platform.

Who this book is for

This book is for MLOps engineers, DevOps engineers, IT architects, and data scientists who want to gain an understanding of MLOps concepts and are interested in learning the Red Hat OpenShift Data Science platform. A basic understanding of OpenShift or Kubernetes would help in better understanding the inner workings of the exercises presented in this book. A basic knowledge of data science or machine learning and Python coding skills will also help you perform the data science parts of the exercises smoothly.

What this book covers

Chapter 1, Introduction to MLOps and OpenShift, starts with a brief introduction to MLOps and the basics of Red Hat OpenShift. The chapter then discusses how OpenShift enables machine learning projects and how Red Hat OpenShift Data Science and partner software products comprise a complete MLOPS platform.

Chapter 2, Provisioning an MLOps Platform in the Cloud, will walk you through provisioning Red Hat OpenShift, Red Hat OpenShift Data Science, and Pachyderm on the AWS cloud. The chapter contains step-by-step instructions on how to provision the base MLOps platform.

Chapter 3, Building Machine Learning Models with OpenShift, starts with the initial configurations of the platform components to prepare for model building. The chapter walks you through the configuration steps and ends with an introduction to the data science projects, workbenches, and the Jupyter Notebook.

Chapter 4, Managing a Model Training Workflow, digs deeper into the platform configuration covering OpenShift Pipelines for building model training pipelines and using Pachyderm for data versioning. By the end of the chapter, you will have built an ML model using a training pipeline you created.

Chapter 5, Deploying ML Models as a Service, introduces the model serving component of the platform. The chapter will walk you through how to enhance further the pipeline to automate the deployment of ML models.

Chapter 6, Operating ML Workloads, talks about the operational aspects of MLOps. The chapter focuses on logging and monitoring the deployed ML models and briefly discusses strategies for optimizing operational costs.

Chapter 7, Building a Face Detector Using the Red Hat ML Platform, walks you through the process of building a new AI-enabled application from end to end. The chapter helps you practice the knowledge and skills you gained in the previous chapters. The chapter also introduces Intel OpenVino as another option for model serving. By the end of this chapter, you will have built an AI-enabled web application running on OpenShift and used all of the Red Hat OpenShift Data Science features.

To get the most out of this book

You will need a basic knowledge of Kubernetes or OpenShift and basic Python coding skills on Jupyter Notebooks. Most activities are done using the web-based graphical user of Red Hat OpenShift and Red Hat OpenShift Data Science. However, specific steps require running Linux commands and interacting with the OpenShift API. Lastly, we recommend that you perform the exercises in this book to get a hands-on experience of the platform.

Software/hardware covered in the book

Operating system requirements

AWS CLI (aws)

Windows, macOS, or Linux

Red Hat OpenShift Client (oc)

Windows, macOS, or Linux

The software listed above must be installed on your local machine. These are used to interact with the platform from your client computer. The rest of the interaction with the platform is through the OpenShift web console and the Red Hat OpenShift Data Science web console.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/MLOps-with-Red-Hat-OpenShift

If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Create a user named admin.”

A block of code is set as follows:

storage: backend: MINIO minio: bucket: pachyderm

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

curl -O -L https://mirror.openshift.com/pub/openshift-v4/client/rosa/latest/rosa-linux.tar.gz tar -xvzf rosa-linux.tar echo PATH=$PATH:/home/cloudshell-user >> ~/.bashrc

Any command-line input or output is written as follows:

echo <the rendered yaml string> | oc apply -f-

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Click on the Increase service quotas button if applicable to your cluster.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read MLOps with Red Hat OpenShift, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your e-book purchase not compatible with the device of your choice?

Don’t worry!, Now with every Packt book, you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the following link:

https://packt.link/free-ebook/9781805120230

Submit your proof of purchase.That’s it! We’ll send your free PDF and other benefits to your email directly.

Part 1: Introduction

This part covers the basic concepts of MLOps and an introduction to Red Hat OpenShift.

This part has the following chapters:

Chapter 1, Introduction to MLOps and OpenShift

1

Introduction to MLOps and OpenShift

If you have chosen to read this book, chances are that you have a background in the machine learning (ML) domain. The primary purpose of this book is to show you how Red Hat OpenShift provides the basis for developing, deploying, and monitoring your models in production. In addition, you will learn about different components of the OpenShift ecosystem and how you can weave them together to build a path toward automating the life cycle of your ML project. You will also learn how to leverage Red Hat OpenShift Data Science and its partner components.

Finally, you will see how the approaches presented in this book can help your organization scale its ML initiatives through MLOps practices.

This first chapter focuses on giving you the basic definitions of the concepts and the technologies involved in the Red Hat OpenShift ecosystem for machine learning.

This chapter will cover the following topics:

What is machine learning operations (MLOps)?Introduction to OpenShiftHow OpenShift enables you to implement MLOpsThe advantages of the cloud

Let’s start by defining MLOps.

What is MLOps?

MLOps is a set of practices that aims to streamline the process of deploying and maintaining ML models in production environments. It involves integrating ML workflows with DevOps practices to ensure that ML models are tested, deployed, and monitored in a reliable and scalable manner.

MLOps involves collaboration between data scientists, ML engineers, software developers, and operations teams to create a continuous integration and delivery pipeline for ML models. This pipeline includes steps such as data preparation, model training, model testing, model deployment, and monitoring. MLOps also involves the use of automation, testing, and monitoring tools to ensure that models remain accurate, stable, and scalable over time.

Some common practices in MLOps include version control for models and data, experiment tracking, model validation, continuous integration and delivery, containerization of models, and monitoring of model performance and data drift.

So, what problems does MLOps aim to solve? ML is still an emerging domain, and as per a Gartner study, 53% of projects fail to move from prototypes to production. The primary reasons include the model development life cycle, operationalization of models, and overall workflow governance. MLOps takes learnings from proven software engineering practices, such as DevOps, and applies them to ML projects.

MLOps is an emerging domain that takes advantage of the maturity of existing software development processes – in other words, DevOps combined with data engineering and ML disciplines. MLOps can be simply defined as an engineering practice of applying DevOps to ML projects.

Let’s look at how these disciplines form the foundation of MLOps.

First, unlike traditional programming, where your only input is the code written by the developers, in ML, your input is both code and data. While the application package is the output of the application build process in software engineering, in ML, the written code is only used to facilitate the training of a known algorithm using a training dataset. The behavior of the resulting ML model is highly dependent on the training dataset, the algorithm, and the code used to perform the training. As a result, even if you do not change your code and use a different dataset, the resulting ML model may perform differently. This adds another layer of complexity as this requires not only versioning the code that facilitates model training but also versioning the training dataset for a repeatable outcome. Versioning data requires a different approach compared to versioning code. This is because we generally don’t want to store multiple versions of the entire dataset inside a Git repository. One method is to take the hash of the data and then apply Git-like practices to keep a history of changes. This allows us to avoid storing multiple versions of the entire dataset inside the Git repository. You will see how Pachyderm, a component available on the OpenShift platform, provides such capabilities.

Secondly, an ML project involves more personas than a traditional software project. You have data scientists, ML engineers, and data engineers collaborating with software engineers, business analysts or subject matter experts, and platform and operations teams. Sometimes, these personas are very diverse. For example, a data scientist may not wholly understand the production deployment process of a model. Similarly, the operations team may not fully understand what a model is. Later in this book, you will see how the OpenShift platform provides a way to automate the process of packaging and deploying models for inference with tools such as Intel OpenVino.

Third, unlike traditional software development, where you only need to develop one working code set, in ML, a data scientist or ML engineer may use multiple ML algorithms and generate multiple resulting ML models. In other words, they perform experiments on different types of algorithms and different ways of training, and they use different sets of dataset features. Before selecting a model fit for production, your team needs to compare different dimensions of the model before choosing one. MLOps tackles this additional dimension of the workflow by providing tools to version and compare model performances. MLflow is one such tool that allows to manage experiments and it runs on OpenShift. By using MLflow, we can demonstrate that we can always extend the toolset to fit our needs.

Training an ML algorithm to produce an ML model can take hours or days, sometimes weeks, especially when you use complex deep learning (DL) algorithms. This kind of training process may require a different set of hardware, such as GPUs and memory-optimized machines. You will see how OpenShift and the cloud assist in automating the availability to optimize cost and training time. In addition, OpenShift’s capability for scaling resources during inference helps with meeting production SLAs.

Lastly, because ML models’ performances rely on the dataset used during training, if this dataset no longer represents the real-world situation, the model’s effectiveness may degrade. This will result in inaccurate responses and reduced value for the user. This is called model drift. Early drift detection keeps the model relevant and functional, allowing it to continually deliver value for the business. OpenShift provides a way to not only capture traditional metrics, such as response times, but can also be extended to detect model drifts.

Because of the complexity ML adds compared to traditional programming, the need to address these complexities led to the emergence of MLOps. As a result, Red Hat has packaged a set of components that provides MLOps capabilities for the OpenShift platform. It is called Red Hat OpenShift Data Science or RHODS (pronounced “rhodes”).

This book covers the elements of the RHODS offering and partner components on the OpenShift platform for a complete MLOps solution.

Let’s start by refreshing our understanding of the OpenShift platform.

Introduction to OpenShift

Although this book is not about operationalizing the OpenShift platform, a basic introduction to the platform is helpful. In this section, you will learn about the core concepts of Kubernetes and OpenShift.

OpenShift is a complete application platform based on Kubernetes. It is also categorized as Enterprise Kubernetes. Kubernetes provides a solid foundation for container hosting and orchestration. Moreover, Kubernetes provides core functionalities, such as cluster-state management, where a reconcile loop makes sure that the cluster state and the desired state are in sync. Kubernetes also includes a set of APIs to interact with the cluster. Kubernetes is a great platform, but in reality, applications need much more than just the core services provided by Kubernetes.

Assume that you want to deploy a Python application on a Kubernetes cluster. Let’s assess what is required. First, you need to package your application as a container image. Secondly, you must store this container image in an image registry. Your application may need a backend service such as a cache or a database. You may need to call other services from your application. You will need to monitor your application and may be required to generate alerts of critical events. All these things are needed to make your application ready for production. A complete application platform, such as OpenShift, answers all these concerns. OpenShift enhances the Kubernetes platform to enhance the support of application life cycles, operations, and security. OpenShift is available on all the major clouds and on-premises, via which you can consistently deploy your workloads across multiple cloud providers and hybrid environments.

OpenShift has all the functionalities of a vanilla Kubernetes platform with additions such as developer-centric toolsets that make it easier for developers and ML engineers to build, deploy, and manage containerized applications.

OpenShift is available in several different editions. OpenShift Container Platform is the on-premises version of OpenShift that can be installed on either bare-metal or virtualized infrastructure. OpenShift Dedicated is a fully managed OpenShift cluster managed by Red Hat. You can gain access to it through a Red Hat subscription. Red Hat OpenShift on AWS (ROSA) is an AWS offering of OpenShift. It is fully managed jointly by AWS and Red Hat. Azure Red Hat OpenShift (ARO) is an Azure Cloud offering of OpenShift. It is fully managed jointly by Microsoft and Red Hat. OpenShift Kubernetes Engine is an OpenShift offering on the Google Cloud platform. There are other flavors of OpenShift and different types of OpenShift subscriptions. Among them are Red Hat OpenShift on IBM Cloud, OpenShift Platform Plus, which includes Red Hat ACM, Advanced Cluster Security (ACS), Data Foundation, and Red Hat Quay. Single-Node OpenShift (SNO) is also available for deployments to the edge. Each edition is designed to meet the specific needs of different types of users, from individual developers to large enterprise organizations. One of the strengths of OpenShift is that regardless of where and which infrastructure OpenShift is running, the developer experience remains consistent, making it easy to move your workloads across different infrastructures without impacting the platform’s end users.

Overall, OpenShift is a powerful platform for running and managing modern containerized applications in both on-premises and cloud environments. It is an enterprise-grade Kubernetes distribution that’s loaded with many additional features that focus on improving the developer experience.

Kubernetes resources

Now that