Creating Inference Checkpoints
Overviews how to convert your inference checkpoint for various model servers
Tags: Inference Servers
Securing Application with Identity Aware Proxy
Overviews how to secure application endpoints with Identity Aware Proxy (IAP)
Inference servers
Deploying and managing servers dedicated to performing inference tasks for machine learning models.
Enable Model Armor for vLLM deployment with Inference Gateway
Overviews how to set up Inference Gateway with Model Armor to secure interaction with LLM models
Security
vLLM GPU/TPU Fungibility
This tutorial shows you who to serve a large language model (LLM) using both Tensor Processing Units (TPUs) and GPUs on Google Kubernetes Engine (GKE) using the same deployment with vLLM