🛠️Unsloth Requirements
Here are Unsloth's requirements including system and GPU VRAM requirements.
System Requirements
Operating System: Works on Linux and Windows via WSL
Supports NVIDIA GPUs since 2018+. Minimum CUDA Capability 7.0 (V100, T4, Titan V, RTX 20, 30, 40x, A100, H100, L40 etc) Check your GPU! GTX 1070, 1080 works, but is slow.
Your device must have
xformers
,torch
,BitsandBytes
andtriton
support.Unsloth only works if you have a NVIDIA GPU. Make sure you also have disk space to train & save your model
Fine-tuning VRAM requirements:
How much GPU memory do I need for LLM fine-tuning using Unsloth?
Check this table for VRAM requirements sorted by model parameters and fine-tuning method. QLoRA uses 4-bit, LoRA uses 16-bit.
3B
2 GB
7 GB
7B
4.5 GB
16 GB
8B
5 GB
19 GB
9B
5.5 GB
21 GB
11B
6.5 GB
26 GB
14B
8.5 GB
33 GB
27B
16GB
64GB
32B
19 GB
76 GB
40B
24GB
96GB
70B
41 GB
164 GB
81B
48GB
192GB
405B
237 GB
950 GB
Last updated