🖥️Inference & Deployment

Learn how to save your finetuned model so you can run it in your favorite inference engine.

You can also run your fine-tuned models by using Unsloth's 2x faster inference.

llama.cpp - Saving to GGUF

Ollama

vLLM

SGLang

Unsloth Inference

Troubleshooting

vLLM Engine Arguments

LoRA Hotswapping