Install Docker Desktop, run the ODS installer, and open your own local AI interface. Follow the Windows or Apple Silicon Mac setup, verify a model response, and manage your server.
Examples from the pinned ODS model-selection table. NVIDIA rows refer to GPU VRAM; Apple Silicon rows refer to unified memory. ODS makes the actual selection during installation.
| Machine | Example automatic model tier |
|---|---|
| NVIDIA GPU · 8 GB | Qwen3.5 9B class |
| NVIDIA GPU · 12 GB | Phi-4 14B class |
| NVIDIA GPU · 24–32 GB | Qwen3.5 27B class |
| NVIDIA GPU · 48 GB | DeepSeek R1 Distill 70B class |
| Apple Silicon · 16 GB | Qwen3.5 9B class |
| Apple Silicon · 32 GB | Phi-4 14B class |
| Apple Silicon · 48 GB | Qwen3.5 27B class |
| Apple Silicon · 64 GB+ | Qwen3.6 35B-A3B class |
| CPU only | Choose a smaller local model; responses can be slower |
These are catalog examples, not speed benchmarks or guaranteed choices. Free memory, context and optional services affect fit. Start with short chats and reduce model size or context if memory is tight. General AMD/Intel acceleration is not implied; see ODS’s support matrix for backend-specific options.
ODS (Osmantic Deployment System) brings model inference, browser chat and optional AI services together on your computer. Install Docker Desktop, follow the commands for your operating system, then send a real message to your local model. This walkthrough uses the fixed v3.0.0 source release.
Install the software for your operating system, then follow the steps to start your local server.
ODS selects the model during installation based on detected hardware and available memory. The table shows publisher catalog examples; you do not need to choose a model package before installing.
Choose where you want to run it. The instructions below update to match your selection.
Use a normal PowerShell window for ODS. Docker Desktop must be running with its WSL2 backend. NVIDIA acceleration needs a compatible Windows driver; CPU-only can use a smaller model.
Install Docker Desktop for Windows and use the WSL2 backend. Follow Docker’s current Windows requirements, including hardware virtualization. Open Docker Desktop and wait until its engine is running before continuing.
Download Docker Desktop Windows requirements and WSL2 setupInstall Git for Windows if you do not already have it. Close and reopen PowerShell afterward so it can find Git. Run the ODS installer as your normal Windows user, not as Administrator.
Install Git for Windows NVIDIA drivers for GPU accelerationgit --versionThese commands clone v3.0.0 into a separate source folder, then start its official installer. ODS detects your hardware, configures services and downloads a suitable model. Let the installer finish; the initial model download can take time. The execution-policy change applies only to this PowerShell session.
Review the v3.0.0 release Official Windows quickstartSet-Location $env:USERPROFILE
git clone --depth 1 --branch v3.0.0 https://github.com/Osmantic/ODS.git ods-v3-source
Set-Location ods-v3-source
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1Open http://localhost:3000 in your browser. Complete any first-run setup shown by the interface and select the local model downloaded during installation. Keep Docker Desktop running. This browser address points to your own computer.
In the browser chat, send: “Write a two-sentence explanation of what Docker does.” Wait for a coherent response, then ask a follow-up. Confirm a local model is selected rather than an optional cloud provider. A model response is the success check; the dashboard loading alone is not enough.
Start with chat before enabling more services. ODS can also provide model management, a local API, agents, n8n workflows, document search/RAG, SearXNG search, voice and ComfyUI image generation. Optional services use extra memory and some features need additional downloads.
ODS normally installs its working stack under your Windows user folder in ods. This is separate from the ods-v3-source checkout. Run the status command from that installed folder to inspect the services.
Set-Location (Join-Path $env:USERPROFILE 'ods')
.\ods.ps1 statusWhen you want to shut down the local stack, run this from the installed ods folder. This stops its services; use the next step when you want to bring them back.
.\ods.ps1 stopOpen Docker Desktop first. In a new PowerShell window, return to the installed folder and start ODS. When the services are ready, reopen http://localhost:3000 and send another message.
Set-Location (Join-Path $env:USERPROFILE 'ods')
.\ods.ps1 startOpen the browser interface, select the installed local model, and ask: “Write a two-sentence explanation of what Docker does.” Confirm you receive an answer and can continue the conversation without selecting a hosted model provider.
http://localhost:3000Opens on your machineThe project describes V3 as public testing while wider hardware validation continues. This guide pins the source tag v3.0.0; container images and downloaded models can change independently. Keep the default localhost-only access for your first setup. Local inference runs on your computer, while optional web search, cloud APIs and initial downloads still use the network.
Open Docker Desktop and wait for its engine to start, then retry the installer from ods-v3-source. On Windows, check that Docker uses its WSL2 backend and follow the linked Docker requirements.
Install Git using the download link in the Windows instructions. Close PowerShell, open a fresh normal PowerShell window, and run git --version before repeating the clone command.
In Terminal run xcode-select --install, finish the Apple Command Line Tools installation, then retry git --version and the ODS commands.
If you already cloned this release, open the existing ods-v3-source folder and continue with its installer instead of cloning over it. For a separate clean checkout, choose a new source folder name. Keep it separate from the installed ods folder.
Close other GPU-heavy apps and disable optional services you do not need. Choose a smaller model in ODS and lower the context setting. The catalog table is a starting reference, not a promise that every workload fits.
Stop the other application using that port, or follow the pinned ODS quickstart to change the web interface port in its configuration. Use the resulting address shown by ODS instead of assuming localhost:3000.
Wait for the initial model download to finish. Confirm a local model is selected and use the platform’s status command to check the inference service. Inspect the ODS logs before restarting the stack.
Official software, installation instructions, and support resources behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.