TGI: inference with quantisation built in
HuggingFace Text Generation Inference — vLLM alternative
- Video memory
- 12 GB
- Setup
- ~6 min
- Access
- port 8080
- Billing
- hourly, in BRL
TGI is Hugging Face's inference server, with token streaming and quantisation out of the box. It is the natural pick if you already live inside their ecosystem and want the same behaviour in production.
TGI is HuggingFace's production LLM server. Great support for quantized models (GPTQ, AWQ, EETQ), native streaming, Prometheus metrics. Runs Qwen2.5-3B by default.
What it is for
- Serve a quantised model on a smaller card and cut cost
- Response streaming for chat that feels instant
- Direct integration with models from the Hub
- Built-in metrics to watch load and latency
How to deploy
- Create your account and add balance (card or Pix, no subscription).
- In the console, pick the TGI (HuggingFace) template and a machine — the console hides the ones that do not meet the requirement.
- In about 6 minutes the setup finishes and the access address shows up in the panel, on port 8080.
Done? Just destroy the machine and billing stops with it. No contract, no minimum commitment.
FAQ
TGI or vLLM?
vLLM usually gets more throughput under batching and offers an OpenAI-compatible API; TGI integrates better with the Hugging Face ecosystem and ships quantisation ready.
Do I need a Hub token?
Only for gated models. Open models download directly.
Which GPU should I pick?
12 GB runs mid-sized quantised models. For larger models without quantisation, go for 40 GB or more.