Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Package sizes are weight estimates; context and runtime add memory. Exact minimum memory is not established. Ollama context defaults are 4k under 24 GB VRAM, 32k for 24-48 GB, and 256k at 48 GB+.
| Machine | Qwen3.5 · 4B (smaller Ollama package) | Qwen3.5 · 9B (official Ollama package) | Qwen3.5 · 9B abliterated (community GGUF) |
|---|---|---|---|
| NVIDIA GPU, 8 GB VRAM | Fits 3.4 GB weights; Ollama NVIDIA route; start 4k context, raise only with headroom. | 6.6 GB weights may exceed 8 GB VRAM; use 4B or CPU offload; Ollama NVIDIA; start 4k. | 5.2 GB text / 6.1 GB vision; 8 GB VRAM tight; Ollama NVIDIA; start 4k; use 4B if needed. |
| NVIDIA GPU, 12 GB VRAM | Fits 3.4 GB weights with headroom; Ollama NVIDIA; start 4k, raise with VRAM headroom. | 6.6 GB weights fit 12 GB with modest context; Ollama NVIDIA; start 4k, raise cautiously. | 5.2 GB text / 6.1 GB vision fits 12 GB; Ollama NVIDIA; start 4k, raise with headroom. |
| NVIDIA GPU, 24 GB VRAM | Fits 3.4 GB weights; Ollama NVIDIA; start 32k per Ollama 24-48 GB default, raise with headroom. | 6.6 GB weights fit 24 GB; Ollama NVIDIA; start 32k per 24-48 GB default, raise cautiously. | 5.2/6.1 GB fits 24 GB; Ollama NVIDIA; start 32k per 24-48 GB default, monitor VRAM. |
| AMD Radeon GPU, 8 GB VRAM | Fits 3.4 GB; Ollama AMD ROCm/Vulkan route conditional on driver; start 4k. | 6.6 GB may exceed 8 GB; use 4B or CPU offload; Ollama AMD route conditional; start 4k. | 5.2/6.1 GB tight on 8 GB; Ollama AMD route conditional; start 4k; use 4B if needed. |
| AMD Radeon GPU, 16 GB VRAM | Fits 3.4 GB; Ollama AMD ROCm/Vulkan route conditional on driver; start 4k, raise with headroom. | 6.6 GB fits 16 GB with modest context; Ollama AMD route conditional; start 4k, raise cautiously. | 5.2/6.1 GB fits 16 GB; Ollama AMD route conditional; start 4k, raise with headroom. |
| Intel Arc GPU, 8 GB VRAM | 3.4 GB weights fit 8 GB; Ollama Vulkan route for Intel GPU is driver-conditional; start 4k. | 6.6 GB may exceed 8 GB Arc; try 4B; Ollama Vulkan route conditional; start 4k. | 5.2/6.1 GB tight on 8 GB; Ollama Vulkan route conditional; start 4k; vision may need 4B. |
| Intel Arc GPU, 12 GB VRAM | 3.4 GB weights fit 12 GB; Ollama Vulkan route driver-conditional; start 4k, raise with headroom. | 6.6 GB fits 12 GB with modest context; Ollama Vulkan route conditional; start 4k. | 5.2/6.1 GB fits 12 GB; Ollama Vulkan route conditional; start 4k, raise with headroom. |
| Apple Silicon, 16 GB unified memory | 3.4 GB weights fit 16 GB unified; Ollama Metal route; start 4k, raise with headroom. | 6.6 GB fits 16 GB unified; Ollama Metal route; start 4k; 256K is publisher max. | 5.2/6.1 GB GGUF fits 16 GB; Ollama Metal route; start 4k; MLX not selected. |
| Apple Silicon, 24 GB unified memory | 3.4 GB weights fit 24 GB unified; Ollama Metal route; start 32k per 24-48 GB default. | 6.6 GB fits 24 GB unified; Ollama Metal route; start 32k per 24-48 GB default. | 5.2/6.1 GB fits 24 GB unified; Ollama Metal route; start 32k per 24-48 GB default. |
| CPU only, 16 GB system RAM | 3.4 GB weights fit 16 GB RAM; Ollama CPU route slower; start 4k. | 6.6 GB weights fit 16 GB RAM; Ollama CPU route slower; start 4k. | 5.2/6.1 GB fits 16 GB RAM; Ollama CPU route slower; start 4k. |
| CPU only, 32 GB system RAM | 3.4 GB weights fit 32 GB RAM; Ollama CPU route slower; start 4k, raise with headroom. | 6.6 GB weights fit 32 GB RAM; Ollama CPU route slower; start 4k, raise with headroom. | 5.2/6.1 GB fits 32 GB RAM; Ollama CPU route slower; start 4k, raise with headroom. |
Model maximum context is 262,144 tokens; runtime context is set separately and depends on available memory. Package sizes are weights only; context and runtime add overhead. Exact minimum memory is not established.
Starting context values follow Ollama documented defaults where available: 4k under 24 GB VRAM, 32k for 24-48 GB, and 256k at 48 GB+. Raise only with headroom. Publisher maximum is 262,144 tokens, not a runtime guarantee.
| Machine | Qwen3.5 · 4B (smaller Ollama package) | Qwen3.5 · 9B (official Ollama package) | Qwen3.5 · 9B abliterated (community GGUF) |
|---|---|---|---|
| NVIDIA GPU, 8 GB VRAM | 4096; Ollama default under 24 GB VRAM; raise only with VRAM headroom. | 4096; 6.6 GB weights may exceed 8 GB; use 4B or CPU offload. | 4096; 5.2/6.1 GB weights leave little headroom on 8 GB; use 4B if needed. |
| NVIDIA GPU, 12 GB VRAM | 4096; Ollama default under 24 GB VRAM; raise with headroom. | 4096; 6.6 GB weights fit; raise cautiously with headroom. | 4096; 5.2/6.1 GB weights fit; raise with headroom. |
| NVIDIA GPU, 24 GB VRAM | 32768; Ollama default for 24-48 GB VRAM; raise with headroom. | 32768; Ollama default for 24-48 GB VRAM; raise cautiously. | 32768; Ollama default for 24-48 GB VRAM; monitor VRAM. |
| AMD Radeon GPU, 8 GB VRAM | 4096; Ollama default under 24 GB VRAM; AMD driver route conditional. | 4096; 6.6 GB may exceed 8 GB; use 4B or CPU offload; AMD route conditional. | 4096; 5.2/6.1 GB tight on 8 GB; AMD driver route conditional. |
| AMD Radeon GPU, 16 GB VRAM | 4096; Ollama default under 24 GB VRAM; raise with headroom; AMD route conditional. | 4096; 6.6 GB fits; raise cautiously; AMD route conditional. | 4096; 5.2/6.1 GB fits; raise with headroom; AMD route conditional. |
| Intel Arc GPU, 8 GB VRAM | 4096; Ollama default under 24 GB VRAM; Vulkan route driver-conditional. | 4096; 6.6 GB may exceed 8 GB; try 4B; Vulkan route conditional. | 4096; 5.2/6.1 GB tight on 8 GB; Vulkan route conditional. |
| Intel Arc GPU, 12 GB VRAM | 4096; Ollama default under 24 GB VRAM; raise with headroom; Vulkan conditional. | 4096; 6.6 GB fits; raise cautiously; Vulkan conditional. | 4096; 5.2/6.1 GB fits; raise with headroom; Vulkan conditional. |
| Apple Silicon, 16 GB unified memory | 4096; Ollama default under 24 GB unified; raise with headroom. | 4096; 6.6 GB fits; raise with headroom; 256K is publisher max. | 4096; 5.2/6.1 GB GGUF fits; raise with headroom. |
| Apple Silicon, 24 GB unified memory | 32768; Ollama default for 24-48 GB unified; raise with headroom. | 32768; Ollama default for 24-48 GB unified; raise cautiously. | 32768; Ollama default for 24-48 GB unified; monitor memory. |
| CPU only, 16 GB system RAM | 4096; Ollama default; CPU route slower; raise with RAM headroom. | 4096; 6.6 GB fits; CPU route slower; raise with RAM headroom. | 4096; 5.2/6.1 GB fits; CPU route slower; raise with RAM headroom. |
| CPU only, 32 GB system RAM | 4096; Ollama default; CPU route slower; raise with RAM headroom. | 4096; 6.6 GB fits; CPU route slower; raise with RAM headroom. | 4096; 5.2/6.1 GB fits; CPU route slower; raise with RAM headroom. |
Context memory grows with token count. Ollama defaults are documented by VRAM tier, but exact runtime context should be verified with ollama ps. Publisher maximum is not a runtime guarantee.
Run a private local chat with Qwen3.5 · 9B through Ollama, send a first prompt, and confirm the selected model responds. Includes a smaller 4B family package and a community uncensored GGUF option for memory or content needs.
Select the weights you want. The instructions, downloads and settings change with your choice.
The publisher has not provided a complete memory target for this task. Check its hardware guidance before starting.
Ollama managed package. CPU fallback works but is slower. GPU acceleration depends on documented NVIDIA/AMD drivers. Exact minimum memory is not established; longer context increases memory use.
Choose your computer first, then its GPU path. The instructions below update to match your selection.
Install Ollama from the official macOS download; requires macOS Sonoma 14 or newer. Apple M series supports CPU/GPU; Intel Macs are CPU-only per Ollama docs. Exact package: qwen3.5:4b.
Ollama's documented macOS requirements are macOS Sonoma (v14) or newer, with an Apple M series chip for CPU and GPU acceleration, or x86 (CPU only). Confirm both before installing. If the version is older than 14, update macOS first.
Ollama macOS requirements Download Ollama for macOSsw_vers
uname -mDownload the official dm and drag Ollama into /Applications. On first launch the app verifies the ollama CLI is present in PATH and, if it is not, offers to create a link in /usr/local/bin. Accept that prompt, then open Terminal. Ollama requires macOS 14 Sonoma or later. This installer keeps everything under your user account; no model data leaves the machine.
Ollama macOS download Ollama macOS install docsopen "https://ollama.com/download/mac"
ollama --versionThe macOS app starts a background server on http://localhost:11434. Keep exactly that one server running; do not start another ollama serve on the same port. Create a dedicated working folder so no other files are touched, then confirm the local API answers.
Ollama CLI reference Ollama FAQmkdir -p "$HOME/YouRunAI-Qwen35-4B"
cd "$HOME/YouRunAI-Qwen35-4B"
curl -s http://localhost:11434/api/tagsThis tab uses the smaller family package qwen3.5:4b, listed by Ollama at 3.4 GB with a 256K context window. The pull is one-time; the weights then stay local in Ollama's managed model store. Confirm the tag appears with the same name after the pull.
ollama pull qwen3.5:4b
ollama lsStart the interactive chat with the exact evidenced command. Ollama's documented default runtime context is about 4k tokens under 24 GiB VRAM/unified memory, and the 256K figure on the model page is the package maximum, not a runtime guarantee. At the chat prompt type /set parameter num_ctx 4096 for a 16 GB Mac, or /set parameter num_ctx 32768 if you have roughly 24-48 GB of unified memory. Setting a larger context increases memory use, so lower it again if the machine pages. Leave this Terminal open: the model stays loaded while the session runs. At the Ollama chat prompt (not in the shell), enter these lines in order: /set parameter num_ctx 4096. If a later shell command is shown, run it after exiting the chat or from another terminal.
Ollama context length docs Ollama FAQ (num_ctx) Qwen3.5 on Ollamaollama run qwen3.5:4bType the prompt at the >>> prompt in the running session. Thinking mode is on by default, so thinking text may appear before the final answer. The expected final answer is LOCAL-OK. The reply arriving in this Terminal, with no connection error, is the first local model response.
Ollama CLI referenceReply with exactly LOCAL-OK.Ask a question with one verifiable answer. The expected final numeric answer is 391, printed alone. If the reply is anything else, ask again or use /set parameter num_ctx 4096 to free memory before retrying.
Qwen3.5 on OllamaWhat is 17 x 23? Answer only the number.The session is a reusable conversation: type a new prompt at the >>> prompt to continue with the history kept in context, or paste a multi-line prompt between triple quotes. Everything in this session is local, and /bye leaves the chat. The model stays resident until it is explicitly stopped or the keep-alive expires. At the Ollama chat prompt (not in the shell), enter these lines in order: /bye. If a later shell command is shown, run it after exiting the chat or from another terminal.
Ollama CLI reference (multi-line input)Explain in two sentences what you just verified with the arithmetic answer.Open a second Terminal window and leave the chat session in the first one. ollama ps reports the allocated CONTEXT and the PROCESSOR split, so you can see whether the model is fully on the Apple GPU or partly offloaded to CPU. 262144 is the model maximum from the publisher config, not the runtime default. To set the server-wide default instead, quit the Ollama app first so the port is free, then start the server with OLLAMA_CONTEXT_LENGTH in the foreground.
Ollama context length docs Ollama GPU/hardware docsollama ps
OLLAMA_CONTEXT_LENGTH=32768 ollama serveUse ollama stop to release the model from memory while keeping the 3.4 GB package and the working folder intact. Do not remove the Ollama model store unless you intend to re-download the package.
Ollama CLI referenceollama stop qwen3.5:4b
ollama lsThe first local response appears without a connection error. Ask: `Reply with exactly LOCAL-OK.` Then ask: `What is 17 x 23? Answer only the number.` Expected final answers are LOCAL-OK and 391.
https://ollama.com/library/qwen3.5Opens on your machineUse the exact package identifiers and platform routes listed here; follow the linked runtime documentation for installation and context settings.
Confirm Ollama is running, network access exists, and free disk space covers the first download. Use the exact package identifier qwen3.5:9b for the official 9B package, qwen3.5:4b for the smaller fallback, or lukey03/qwen3.5-9b-abliterated for the uncensored text package.
Close other GPU-heavy apps, keep the conversation short, lower OLLAMA_CONTEXT_LENGTH, or switch to the linked Qwen3.5 4B package. Do not treat package size as a measured speed guarantee.
Use the vision package identifier lukey03/qwen3.5-9b-abliterated-vision (about 6.1 GB Q4_K_M); the text-only package has no vision encoder.
Verify the ROCm v7/HIP7-capable driver stack or Vulkan-capable Radeon driver. On mixed iGPU/dGPU systems, ensure the discrete GPU is selected.
Original instructions, model files, and compatibility notes behind this setup.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.
Turn text prompts into local videos with the Wan2.1 T2V 1.3B model, complete pipeline files and a ready-to-load ComfyUI workflow.