How to Use the Docker Model Runner on Windows: A Complete Guide

Are you looking to run AI models locally on your Windows machine? With Docker’s latest feature  Docker Model Runner  you can now easily download and run popular AI models without complex setups. In this blog, we’ll explain what Docker Model Runner is and guide you step-by-step on how to use it on Windows.

What is Docker Model Runner?

Docker Model Runner is a new feature introduced by Docker that lets you run pre-trained AI models directly on your local machine using Docker commands. It's designed to make working with machine learning models as simple as pulling and running a container. Instead of manually setting up Python environments or dependencies, Docker handles everything in the background.

System Requirements:

• Docker Desktop version 4.41 or later on Windows
• Docker Hub account to access and pull AI models
• Minimum 4GB RAM (recommended based on personal experience)

Step-by-Step Guide to Use Docker Model Runner on Windows:

1. Check Your Docker Version

• Open Docker Desktop.
• Click on “Check for updates” to ensure you’re using version 4.41 or newer.
• Alternatively, open Command Prompt and run the following:

docker model status

If you see an “unknown command” error, it means your Docker version is outdated. Update Docker Desktop and restart it.

2. Log in to Docker Hub

• After the restart, Docker Desktop might prompt you to restart again. Once it's fully loaded, log in with your Docker Hub account.

3. Verify Model Runner Installation

• Run this command in Command Prompt:

docker model status

If everything is set up correctly, you’ll see a response like “status: success.”

Running AI Models with Docker Model Runner

You can browse available AI models at hub.docker.com/ai. For example, you can choose a Hugging Face model or another based on your system’s performance.

Download a Model

To download an AI model, use:

docker model pull ai/<model-name>

Replace <model-name> with the actual model name (e.g., llama2 or mistral).

List All Downloaded Models

After pulling, you can list the models on your system:

docker model list

Two Ways to Run a Model

1. One-Time Prompt Mode

This mode is for simple, single-prompt interactions.

docker run model ai/<model-name> "Your prompt goes here"

2. Interactive Chat Mode

Use this for multi-turn conversations, similar to chatting with ChatGPT.

docker run model ai/<model-name>

You can type multiple messages and get responses 

• To exit the session, type: /bye and hit enter.

Remove a Model

If you want to free up space or remove a model, use:

docker model rm <model-name>

Summary

Docker Model Runner makes it incredibly easy to experiment with AI models right from your Windows desktop. Whether you’re a developer, data scientist, or AI enthusiast, this tool saves you from messy setups and helps you focus on what matters most: building and exploring with AI.

Want to learn more? Visit the official Docker documentation: https://docs.docker.com/model-runner