Unsloth Requirements
Here are Unsloth's requirements including system and GPU VRAM requirements.
Last updated
Was this helpful?
Here are Unsloth's requirements including system and GPU VRAM requirements.
Last updated
Was this helpful?
Operating System: Works on Linux and Windows.
Supports NVIDIA GPUs since 2018+. Minimum CUDA Capability 7.0 (V100, T4, Titan V, RTX 20, 30, 40x, A100, H100, L40 etc) GTX 1070, 1080 works, but is slow.
If you have different versions of torch, transformers etc., pip install unsloth
will automatically install all the latest versions of those libraries so you don't need to worry about version compatibility.
Your device must have xformers
, torch
, BitsandBytes
and triton
support.
Unsloth only works if you have a NVIDIA GPU. Make sure you also have disk space to train & save your model
How much GPU memory do I need for LLM fine-tuning using Unsloth?
Check this table for VRAM requirements sorted by model parameters and fine-tuning method. QLoRA uses 4-bit, LoRA uses 16-bit. Keep in mind that sometimes it may require more VRAM so these numbers are the absolute minimum:
3B
3.5 GB
8 GB
7B
5 GB
19 GB
8B
6 GB
22 GB
9B
6.5 GB
24 GB
11B
7.5 GB
29 GB
14B
8.5 GB
33 GB
27B
22GB
64GB
32B
26 GB
76 GB
40B
30GB
96GB
70B
41 GB
164 GB
81B
48GB
192GB
90B
53GB
212GB
405B
237 GB
950 GB