Tutorials & Notebooks

Fine-tuning

Learn how to fine-tune machine learning and AI models for your specific use cases. This section covers best practices, step-by-step guides, and practical examples to help you adapt pre-trained models to your data and tasks, improving performance and achieving better results with custom fine-tuning workflows.

Frameworks & Pipelines

Explore leading frameworks and pipelines for building, training, and deploying machine learning and AI models. This section provides overviews, best practices, and hands-on guides for integrating tools like Metaflow, MLflow, LangChain, and LlamaIndex into your AI/ML workflows, enabling efficient experiment tracking, workflow automation, and scalable model management.

GPU/TPU

Discover how to leverage GPUs and TPUs to accelerate machine learning and AI workloads. This section covers setup guides, best practices, and practical examples for utilizing GPU and TPU resources, enabling faster training, efficient inference, and scalable deployment of advanced models.

Job Schedulers

Learn how to efficiently manage and automate machine learning and AI workloads with job schedulers. This section covers popular job scheduling tools, configuration tips, and practical examples to help you orchestrate complex workflows, optimize resource utilization, and streamline large-scale model training and deployment.

Storage

Providing persistent and high-performance storage solutions for AI/ML workloads running on Google Kubernetes Engine (GKE).

Workflow orchestration

Workflow orchestration in the ai-on-gke project involves managing and automating the execution of complex, multi-step processes, primarily for AI/ML workloads on Google Kubernetes Engine (GKE).

Inference servers

Deploying and managing servers dedicated to performing inference tasks for machine learning models.

Security

Continue reading:

Checkpoints

Overviews how to convert your inference checkpoint for various model servers

Agent ADK using GKE Autopilot Cluster with Llama and vLLM

This tutorial demonstrates how to deploy the Llama-3.1-8B-Instruct model on Google Kubernetes Engine (GKE) and vLLM for efficient inference. Additionally, it shows how to integrate an ADK agent to interact with the model, supporting both basic chat completions and tool usage. The setup leverages a GKE Autopilot cluster to handle the computational requirements.

ADK VertexAI Example

This tutorial guides you through deploying a containerized agent built with the [Google Agent Development Kit (ADK)](https://google.github.io/adk-docs/) to [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/docs/concepts/kubernetes-engine-overview). The agent uses [VertexAI](https://cloud.google.com/vertex-ai/docs) to access LLMs. GKE provides a managed environment for deploying, managing, and scaling your containerized applications using Google infrastructure.

NVIDIA NeMo

NVIDIA NeMo™ is an end-to-end platform for development of custom generative AI models anywhere. NVIDIA NeMo framework is designed for enterprise development, it utilizes NVIDIA's state-of-the-art technology to facilitate a complete workflow from automated distributed data processing to training of large-scale bespoke models using sophisticated 3D parallelism techniques, and finally, deployment using retrieval-augmented generation for large-scale inference on an infrastructure of your choice, be it on-premises or in the cloud.

Llamaindex

This tutorial will guide you through creating a robust Retrieval-Augmented Generation (RAG) system using LlamaIndex and deploying it on Google Kubernetes Engine (GKE).

vLLM GPU/TPU Fungibility

This tutorial shows you who to serve a large language model (LLM) using both Tensor Processing Units (TPUs) and GPUs on Google Kubernetes Engine (GKE) using the same deployment with [vLLM](https://github.com/vllm-project/vllm)