Security

Securing Application with Identity Aware Proxy

Overviews how to secure application endpoints with Identity Aware Proxy (IAP)

Enable Model Armor for vLLM deployment with Inference Gateway

Overviews how to set up Inference Gateway with Model Armor to secure interaction with LLM models

Continue reading:

vLLM GPU/TPU Fungibility

This tutorial shows you who to serve a large language model (LLM) using both Tensor Processing Units (TPUs) and GPUs on Google Kubernetes Engine (GKE) using the same deployment with [vLLM](https://github.com/vllm-project/vllm)

Inference servers

Deploying and managing servers dedicated to performing inference tasks for machine learning models.

Checkpoints

Overviews how to convert your inference checkpoint for various model servers

Identity Aware Proxy

Overviews how to secure application endpoints with Identity Aware Proxy (IAP)

Model Armor

Overviews how to set up Inference Gateway with Model Armor to secure interaction with LLM models