Reddit pageDiscordNewsletter

How to Train LLMs with Unsloth & Docker

Learn how to fine-tune LLMs or do Reinforcement Learning (RL) with Unsloth's Docker image.

Local training can be complex due to dependency hell or breaking environments. Unsloth’s Docker image can bypass these issues. No setup is needed: pull and run the image and start training.

Why Use Unsloth & Docker?

Unsloth’s Docker image is stable, up-to-date and works in supported setups like Windows.

  • Fully contained dependencies keep your system clean. Runs safely without root.

  • Use locally or on any platform with pre-installed notebooks.

You can now use our main Docker image unsloth/unsloth for Blackwell and 50-series GPUs - no separate image needed.

⚡ Step-by-Step Tutorial

1

Install Docker and NVIDIA Container Toolkit.

Install Docker via Linux or Desktop (other). Then install NVIDIA Container Toolkit:

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.8-1
sudo apt-get update && sudo apt-get install -y \
  nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
2

Run the container.

unsloth/unsloth is Unsloth's only Docker image. For Blackwell and 50-series GPUs, use this same image - no separate image needed. If using DGX Spark, you'll need to follow our DGX guide.

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth
3

Access Jupyter Lab

Go to http://localhost:8888 and open Unsloth.

Access the unsloth-notebooks tabs to see Unsloth notebooks.

4

Start training with Unsloth

If you're new, follow our step-by-step Fine-tuning Guide, RL Guide or just save/copy any of our premade notebooks.

📂 Container Structure

  • /workspace/work/ — Your mounted work directory

  • /workspace/unsloth-notebooks/ — Example fine-tuning notebooks

  • /home/unsloth/ — User home directory

📖 Usage Example

Full Example

docker run -d -e JUPYTER_PORT=8000 \
  -e JUPYTER_PASSWORD="mypassword" \
  -e "SSH_KEY=$(cat ~/.ssh/container_key.pub)" \
  -e USER_PASSWORD="unsloth2024" \
  -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

Setting up SSH Key

If you don't have an SSH key pair:

# Generate new key pair
ssh-keygen -t rsa -b 4096 -f ~/.ssh/container_key

# Use the public key in docker run
-e "SSH_KEY=$(cat ~/.ssh/container_key.pub)"

# Connect via SSH
ssh -i ~/.ssh/container_key -p 2222 unsloth@localhost

⚙️ Advanced Settings

Variable
Description
Default

JUPYTER_PASSWORD

Jupyter Lab password

unsloth

JUPYTER_PORT

Jupyter Lab port inside container

8888

SSH_KEY

SSH public key for authentication

None

USER_PASSWORD

Password for unsloth user (sudo)

unsloth

-p <host_port>:<container_port>

  • Jupyter Lab: -p 8000:8888

  • SSH access: -p 2222:22

Important: Use volume mounts to preserve your work between container runs.

-v <local_folder>:<container_folder>
docker run -d -e JUPYTER_PORT=8000 \
  -e JUPYTER_PASSWORD="mypassword" \
  -e "SSH_KEY=$(cat ~/.ssh/container_key.pub)" \
  -e USER_PASSWORD="unsloth2024" \
  -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

🔒 Security Notes

  • Container runs as non-root unsloth user by default

  • Use USER_PASSWORD for sudo operations inside container

  • SSH access requires public key authentication

PreviousVision Reinforcement Learning (VLM RL)NextFine-tuning LLMs with NVIDIA DGX Spark and Unsloth

Last updated

Was this helpful?