Creating Inference Checkpoints
Overviews how to convert your inference checkpoint for various model servers
Tag: Inference Servers
Inference servers
Deploying and managing servers dedicated to performing inference tasks for machine learning models.
vLLM GPU/TPU Fungibility
This tutorial shows you who to serve a large language model (LLM) using both Tensor Processing Units (TPUs) and GPUs on Google Kubernetes Engine (GKE) using the same deployment with vLLM