OpenAI gpt-oss & all model types now supported!
Reddit pageDiscordNewsletter

How to Train LLMs with Unsloth & Docker

Learn how to fine-tune LLMs or do Reinforcement Learning (RL) with Unsloth's Docker image.

Local training can be complex due to dependency hell or breaking environments. Unsloth’s Docker image can bypass these issues. No setup is needed: pull and run the image and start training.

Why Use Unsloth & Docker?

Unsloth’s Docker image is stable, frequently updated and works in any supported setup.

  • Fully contained dependencies keep your system clean. Runs safely without root.

  • Use locally or on any platform with pre-installed notebooks.

⚡ Step-by-Step Tutorial

1

Install Docker and NVIDIA Container Toolkit.

Install Docker via Linux or Desktop (other). Then install NVIDIA Container Toolkit:

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.8-1
sudo apt-get update && sudo apt-get install -y \
  nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
2

Run the container.

unsloth/unsloth is our main Docker image. If using Blackwell or 50 series GPUs, use our Blackwell Docker.

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth
3

Access Jupyter Lab

Go to http://localhost:8888 and start training with Unsloth.

Access the unsloth-notebooks tabs to see Unsloth notebooks.

4

Start training with Unsloth

If you're new, follow our step-by-step Fine-tuning Guide, RL Guide or just save/copy any of our premade notebooks.

📂 Container Structure

  • /workspace/work/ — Your mounted work directory

  • /workspace/unsloth-notebooks/ — Example fine-tuning notebooks

  • /home/unsloth/ — User home directory

📖 Usage Example

Full Example

docker run -d -e JUPYTER_PORT=8000 \
  -e JUPYTER_PASSWORD="mypassword" \
  -e "SSH_KEY=$(cat ~/.ssh/container_key.pub)" \
  -e USER_PASSWORD="unsloth2024" \
  -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

Setting up SSH Key

If you don't have an SSH key pair:

# Generate new key pair
ssh-keygen -t rsa -b 4096 -f ~/.ssh/container_key

# Use the public key in docker run
-e "SSH_KEY=$(cat ~/.ssh/container_key.pub)"

# Connect via SSH
ssh -i ~/.ssh/container_key -p 2222 unsloth@localhost

⚙️ Advanced Settings

# Generate SSH key pair
ssh-keygen -t rsa -b 4096 -f ~/.ssh/container_key

# Connect to container
ssh -i ~/.ssh/container_key -p 2222 unsloth@localhost
Variable
Description
Default

JUPYTER_PASSWORD

Jupyter Lab password

unsloth

JUPYTER_PORT

Jupyter Lab port inside container

8888

SSH_KEY

SSH public key for authentication

None

USER_PASSWORD

Password for unsloth user (sudo)

unsloth

-p <host_port>:<container_port>

  • Jupyter Lab: -p 8000:8888

  • SSH access: -p 2222:22

Important: Use volume mounts to preserve your work between container runs.

-v <local_folder>:<container_folder>
docker run -d -e JUPYTER_PORT=8000 \
  -e JUPYTER_PASSWORD="mypassword" \
  -e "SSH_KEY=$(cat ~/.ssh/container_key.pub)" \
  -e USER_PASSWORD="unsloth2024" \
  -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

🔒 Security Notes

  • Container runs as non-root unsloth user by default

  • Use USER_PASSWORD for sudo operations inside container

  • SSH access requires public key authentication

PreviousUnsloth Dynamic GGUFs on Aider PolyglotNextgpt-oss: How to Run & Fine-tune

Last updated

Was this helpful?