The MLOps Book Guide to AI Operations

As you dive into the world of MLOps, you'll want to have a solid understanding of AI operations. MLOps is all about bridging the gap between machine learning and operations, making it possible to deploy and manage AI models in production.

AI operations involve a range of tasks, including model serving, model monitoring, and model maintenance. These tasks are crucial to ensuring that AI models perform well and continue to improve over time.

In this guide, we'll explore the key concepts and best practices for AI operations, drawing on real-world examples and expert insights.

Our guiding principles are the foundation of successful machine learning implementation. They ensure that we stay focused on what matters most.

First and foremost, we take a data-centric approach, which is a key aspect of our "Lakehouse AI" philosophy. This unifies data and AI at both the governance and model/pipeline layers.

Here are the guiding principles in a nutshell:

By keeping these principles top of mind, you'll be well on your way to successful machine learning implementation.

MLOps is more than just a buzzword, it's a game-changer for businesses looking to harness the power of machine learning.

In May 2020, a survey of 330 data scientists and machine learning professionals found that half were focused on developing models for production use, and over 40% were deploying models to production. This highlights the importance of getting models into production quickly.

The reality is that only a model running in production can bring value. Models have zero ROI until they can be used. This means that time to market should be the number one metric to look at and optimize for any commercial ML project.

MLOps can help you achieve this by automating retraining of models and monitoring models in production. However, this is still a relatively nascent practice, and most respondents in the survey were not yet prioritizing these tasks.

By implementing MLOps architectures, you can accelerate the time to production for ML-powered applications, reduce the risk of poor performance and non-compliance, and reduce long-term maintenance burdens on Data Science and ML teams.

Implementing MLOps principles in your project can be a daunting task, but it's essential for achieving stable performance and long-term efficiency in ML systems.

Our guiding principles for MLOps are straightforward and effective. We take a data-centric approach to machine learning, which is at the heart of our "Lakehouse AI" philosophy. This unifies data and AI at both the governance and model/pipeline layers.

To achieve our goals, we always keep our business objectives in mind. This ensures that our MLOps implementation is aligned with the needs of our organization. By doing so, we can focus on delivering value to our customers and stakeholders.

Implementing MLOps in a modular fashion is another key principle of ours. This approach allows us to break down complex tasks into manageable components, making it easier to maintain and update our systems over time.

Here are our guiding principles in a concise format:

By following these principles, we can create MLOps architectures that accelerate time to production, reduce the risk of poor performance and non-compliance, and reduce long-term maintenance burdens on Data Science and ML teams.

Implementing MLOps principles into your project is a crucial step in ensuring its success. It's too early for MLOps is a common argument we hear, but ignoring it can lead to issues down the line.

To start, you need to adopt tools and infrastructure that align with your MLOps process. There are over 300 tools in the MLOps space, but finding the right one is easier when you have a clear understanding of your process and requirements.

Here are four principles to consider: collaboration, reproducibility, continuity, and testing and monitoring. MLOps encourages collaboration by moving away from local environments and towards shared infrastructure. This helps team members work together more effectively.

To implement these principles, consider the following ideas:

When choosing tools, keep in mind that proprietary and open-source solutions may have different adoption times and requirements. It's essential to start infrastructure work as early as possible to avoid issues with model documentation and release management.

Here are some key considerations when choosing your MLOps tooling:

Machine learning is a subset of artificial intelligence that enables systems to learn from data and improve their performance over time.

It's based on the idea that algorithms can be trained on data to make predictions or decisions, rather than being explicitly programmed.

Machine learning models can be categorized into supervised, unsupervised, and reinforcement learning, which is a key concept to understand when working with MLOps.

In supervised learning, the model is trained on labeled data to learn the relationship between inputs and outputs.

Unsupervised learning, on the other hand, involves training the model on unlabeled data to identify patterns or structure.

Reinforcement learning is a type of learning where the model learns through trial and error by interacting with an environment and receiving feedback in the form of rewards or penalties.

Machine learning is a subset of artificial intelligence that enables machines to learn from data without being explicitly programmed.

Machine learning algorithms can be broadly classified into supervised, unsupervised, and reinforcement learning. Supervised learning involves training a model on labeled data, whereas unsupervised learning involves finding patterns in unlabeled data.

The most common supervised learning algorithm is linear regression, which is used for predicting continuous outcomes. In the context of a house price prediction model, linear regression can be used to identify the relationship between house prices and features such as number of bedrooms and square footage.

Unsupervised learning algorithms, on the other hand, are used for clustering and dimensionality reduction. K-means clustering is a popular unsupervised learning algorithm that groups similar data points into clusters.

The choice of algorithm depends on the type of problem and the characteristics of the data. For example, if the data is high-dimensional and noisy, dimensionality reduction techniques such as PCA or t-SNE may be necessary to improve model performance.

Machine learning models can be trained on a variety of data sources, including images, text, and sensor data. In the context of image classification, convolutional neural networks (CNNs) are commonly used to classify images into different categories.

In addition to algorithm selection, model evaluation is also crucial in machine learning. Metrics such as accuracy, precision, and recall are commonly used to evaluate model performance.

LLMs are developed incrementally, starting from existing models and ending with custom models fine-tuned on curated data.

This incremental development process is a significant departure from traditional machine learning development.

Many LLMs take general queries and instructions as input, which can contain carefully engineered "prompts" to elicit the desired responses.

Prompt engineering is now a crucial part of developing many AI applications.

LLMs can be given prompts with examples or context, making it valuable to use tools such as vector databases to search for relevant context.

This is especially important when augmenting LLM queries with context.

LLMs are often used via paid APIs, which requires a centralized system for API governance of rate limits, permissions, quota allocation, and cost attribution.

This is essential to manage the use of these APIs effectively.

LLMs are very large deep learning models, often ranging from gigabytes to hundreds of gigabytes in size.

This size requirement necessitates the use of specialized hardware, such as GPUs and fast storage.

To optimize performance, specialized techniques for reducing model size and computation have become more important.

LLMs are hard to evaluate via traditional ML metrics since there is often no single "right" answer.

To address this, human feedback should be incorporated directly into the MLOps process, including testing, monitoring, and capturing for use in future fine-tuning.

This ensures that LLMs are developed and refined with real-world relevance.

Here's a summary of the key properties of LLMs and their implications for MLOps:

LLM-Powered Applications are the future of machine learning, and for good reason. They offer a range of benefits, including the ability to leverage your own data to gain a competitive edge.

One key component of LLM-powered applications is prompt engineering, which involves crafting specific prompts to elicit the desired response from the model. While many prompts are specific to individual LLM models, some tips apply more generally.

Retrieval augmented generation (RAG) is another common type of LLM application, which combines retrieval and generation to produce high-quality results. This workflow is typical in many applications, and its benefits are well-documented.

Vector databases play a crucial role in RAG workflows, with vector indexes, libraries, and databases each serving a unique purpose. For instance, vector indexes are ideal for fast lookup, while vector libraries are better suited for complex computations.

Fine-tuning LLMs is a powerful technique that involves adjusting the model's parameters to suit a specific task. However, it requires careful consideration of the trade-offs between scalability and resource efficiency.

Pre-training is another option, which involves training the model on a large dataset before fine-tuning it for a specific task. This approach can be time-consuming, but it offers a high degree of flexibility.

Here's a brief overview of the continuum from simple to complex for using your data to gain a competitive edge with LLMs:

Managing cost and performance trade-offs is also crucial, especially for inference. By using techniques such as model pruning and knowledge distillation, you can significantly reduce costs while maintaining performance.

Production and deployment are crucial steps in the machine learning lifecycle. Deep learning in production takes a hands-on approach to learn MLOps by doing, starting with a vanilla deep learning model and working towards building a scalable web application.

The book "Deep learning in production" by Sergios Karagianakos covers the design phase, emphasizing best practices for writing maintainable deep learning code such as OOP, unit testing, and debugging.

Chapter 5 focuses on building efficient data pipelines, while Chapter 6 deals with model training in the cloud and distributed training techniques.

To serve and deploy models, tools such as Flask, uWSGI, Nginx, and Docker are used.

Here are some key product features that improve MLOps architecture:

Model Serving provides a production-ready, serverless solution to simplify real-time model deployment. It reduces operational costs and streamlines the ML lifecycle.

Using model aliases for tracking champion vs. challenger models is a technique mentioned in the article.

The book "Deep learning in production" by Sergios Karagianakos covers a wide range of tools and technologies for MLOps.

Flask, uWSGI, Nginx, and Docker are some of the tools emphasized in the book for serving and deployment techniques.

These tools are used to build scalable web applications, and the book provides code snippets and visualizations to help readers understand how they work.

Deep learning frameworks are the backbone of modern AI development, and understanding the key players is crucial for any project.

TensorFlow is an open-source framework developed by Google, which has gained massive popularity due to its ease of use and extensive community support.

Keras is a high-level neural networks API that can run on top of TensorFlow, making it a popular choice for rapid prototyping and development.

PyTorch is another popular framework, known for its dynamic computation graph and rapid prototyping capabilities, making it a favorite among researchers and developers.

Caffe is a deep learning framework that was initially developed at the University of California, Berkeley, and is known for its speed and efficiency.

MXNet is a flexible and scalable framework that supports both CPU and GPU acceleration, making it a popular choice for large-scale deep learning projects.

Theano is a Python library that allows developers to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays, making it a popular choice for research and development.

All of these frameworks have their own strengths and weaknesses, and the choice of which one to use depends on the specific needs of the project.

For example, TensorFlow is a good choice for large-scale production environments, while PyTorch is better suited for rapid prototyping and development.

Databricks Asset Bundles are a key component of MLOps Stacks, allowing you to define infrastructure-as-code for data, analytics, and ML.

Databricks Asset Bundles enable you to validate, deploy, and run Databricks workflows such as Databricks jobs and Delta Live Tables.

They also help manage ML assets like MLflow models and experiments, making it easier to manage and track your machine learning projects.

With Databricks Asset Bundles, you can manage and deploy ML assets in a more streamlined and efficient way.

Unity Catalog is a game-changer for managing data and AI assets.

It centralizes access control, auditing, lineage, and data discovery capabilities across Databricks workspaces, making it easier to manage complex projects.

This means that ML teams can have more efficient access and scalable processes, especially for lineage, discovery, and collaboration.

For administrators, Unity Catalog provides simpler governance at the project or workflow level.

A given catalog in Unity Catalog contains schemas, which in turn may contain tables, volumes, functions, models, and other assets.

Models can have multiple versions and can be tagged with aliases.

This flexibility allows Unity Catalog to be tailored to any organization's existing practices.

The reference architectures provided in the updated eBook offer a clear and structured approach to managing ML pipelines.

The eBook includes a multi-environment view that shows how development, staging, and production environments interact. This high-level view is essential for understanding the flow of data and ML assets across different stages.

Here are the four reference architectures provided in the eBook:

The multi-environment view is particularly useful as it illustrates the three stages of ML pipeline development: initial development, testing in staging, and deployment to production.

Implementing MLOps principles requires a collaborative effort from the team. Collaboratively agreeing on your MLOps process is a great starting point, as it helps establish a common understanding of how to work together.

A few sound principles that the whole team agrees on are a great foundation for implementing MLOps. These principles should be obvious to everyone involved, but it's surprising how often they're not. In fact, most people get excited about tools and technologies, but it's essential to first agree on why you'd want to implement them.

It's also crucial to adopt tools and infrastructure to implement the MLOps process. Your process and requirements should lead your tooling selection, not the other way around. With over 300 tools available, finding the correct tooling will be much easier if you've agreed with your team on how you will work.

Here are some ideas on how to implement the four principles practically with MLOps tooling:

Implementing an MLOps process requires recognizing the key people and roles in your organization. Most machine learning projects are multidisciplinary and cross-organizational development efforts that involve a range of stakeholders.

In reality, a single project can span from business and operations to IT, covering data storage, data security, access control, resource management, high availability, integrations to existing business applications, testing, retraining, and more. This is a deep rabbit hole that many machine learning projects end up being.

To properly implement an MLOps process, you'll have to identify the key stakeholders in your organization. These roles are not necessarily one per person, but rather a single person can cover multiple roles, especially in smaller organizations.

Here are some of the key roles you'll need to identify:

You'll need to talk to business owners to understand the regulatory requirements and IT to grant access and provision cloud machines. This will help you gather your specific requirements and use your organization's resources effectively.

It's essential to agree on your MLOps process with your team before implementing it. This may seem obvious, but it's a crucial step that's often overlooked.

Collaboration is key in MLOps, and it's hard to achieve when team members have different ideas about how to work together. By agreeing on a process, you can turn tacit knowledge into code, making machine learning truly collaborative.

To ensure everyone is on the same page, it's a good idea to establish a few sound principles that the whole team agrees on. These principles should be based on the MLOps process, such as collaboration, reproducibility, continuity, and testing and monitoring.

In fact, having a shared understanding of the MLOps process can save time and effort in the long run. It's like doing it right from the start, rather than trying to fix it later on.

Here are some key principles to consider when agreeing on your MLOps process:

By agreeing on these principles, you can set your team up for success and make the most of your MLOps journey.

LLMops is a crucial aspect of building and deploying Large Language Models (LLMs). It's essentially MLOps for LLMs, and many best practices translate to other Generative AI models as well.

The eBook highlights major changes introduced by LLMs, which require a different approach to development and deployment. These changes include the need for more efficient and scalable infrastructure.

Key components of LLM-powered applications are discussed in the eBook, providing detailed best practices for developers. Reference architectures for common Retrieval-Augmented Generation (RAG) applications are also provided.

Developers can leverage these reference architectures to build more effective and efficient LLM-powered applications. This requires a deep understanding of the key components involved.