Ask Qwen3.5 9B about a short local document and check the answer against source text. Attach a text file or text-based PDF in the Ollama app on Windows or Mac, or paste a short text excerpt into its Linux chat.
Capacity estimates for short text documents at 4K context; model downloads do not include all runtime memory.
| Machine | Qwen3.5 9B (Ollama) | Qwen3.5 9B MLX (Ollama, Apple Silicon) | Qwen3.5-9B-abliterated GGUF (Ollama) |
|---|---|---|---|
| NVIDIA GPU · 8–15 GB VRAM | 6.6 GB weights leave limited headroom; CPU offload may be needed. Prefer 16 GB VRAM. | Selected MLX route uses Apple Silicon; choose the publisher GGUF package on this machine. | 5.6 GB weights leave limited headroom; CPU offload may be needed. Prefer 16 GB VRAM. |
| NVIDIA GPU · 16 GB+ VRAM | Estimated room for 6.6 GB weights and short documents at 4K context. | Selected MLX route uses Apple Silicon; choose the publisher GGUF package on this machine. | Estimated room for 5.6 GB text weights and short documents at 4K context. |
| AMD GPU · 16 GB+ VRAM | Estimated fit at 4K with supported ROCm/Vulkan driver. Confirm acceleration. | Selected MLX route uses Apple Silicon; choose the publisher GGUF package on this machine. | Estimated fit at 4K with supported ROCm/Vulkan driver. Confirm acceleration. |
| Intel Arc GPU · 12 GB+ VRAM | Estimated fit at 4K with supported Vulkan driver/backend; otherwise CPU uses system RAM. | Selected MLX route uses Apple Silicon; choose the publisher GGUF package on this machine. | Estimated fit at 4K with supported Vulkan driver/backend; otherwise CPU uses system RAM. |
| Apple Silicon Mac · 16–31 GB unified memory | 6.6 GB download; estimated minimum at 4K. Close other memory-heavy apps. | 8.9 GB download; tight starting estimate at 4K. Prefer 32 GB unified memory. | 5.6 GB text download; estimated minimum at 4K. Close other memory-heavy apps. |
| Apple Silicon Mac · 32 GB+ unified memory | Recommended capacity estimate at 4K; context and macOS share memory. | Recommended capacity estimate at 4K for the selected 8.9 GB MLX package. | Recommended capacity estimate at 4K; context and macOS share memory. |
| CPU-only PC · 32 GB+ system RAM | Estimated fit at 4K on CPU; responses may be slow. | Selected MLX route needs Apple Silicon; use the publisher GGUF package for CPU operation. | Estimated fit at 4K on CPU; responses may be slow. |
These are estimates, not measured performance. Scanned pages, vision input and large documents are outside this short-text check. More context requires additional memory.
Start every selected package at 4K for a short document and two questions. Input, conversation and output share the context.
| Machine | Qwen3.5 9B (Ollama) | Qwen3.5 9B MLX (Ollama, Apple Silicon) | Qwen3.5-9B-abliterated GGUF (Ollama) |
|---|---|---|---|
| NVIDIA GPU · 8–15 GB VRAM | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | Selected MLX route needs Apple Silicon; use another listed package. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
| NVIDIA GPU · 16 GB+ VRAM | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | Selected MLX route needs Apple Silicon; use another listed package. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
| AMD GPU · 16 GB+ VRAM | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | Selected MLX route needs Apple Silicon; use another listed package. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
| Intel Arc GPU · 12 GB+ VRAM | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | Selected MLX route needs Apple Silicon; use another listed package. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
| Apple Silicon Mac · 16–31 GB unified memory | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
| Apple Silicon Mac · 32 GB+ unified memory | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
| CPU-only PC · 32 GB+ system RAM | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. | Selected MLX route needs Apple Silicon; use another listed package. | 4096 tokens start; try 8192 only after short-text answers work with memory to spare. |
A 4K setting is not suitable for an entire long PDF. Use a smaller excerpt or increase context with memory headroom, then recheck source coverage.
Ask Qwen3.5 9B about a short local document and check the answer against source text. Attach a text file or text-based PDF in the Ollama app on Windows or Mac, or paste a short text excerpt into its Linux chat.
Select the weights you want. The instructions, downloads and settings change with your choice.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
6.6 GB download. Start at 4K context. Estimate 8 GB GPU memory with limited headroom, 16 GB recommended; 16 GB Mac unified memory minimum, 32 GB recommended. CPU-only estimate: 32 GB system RAM. GPU offload estimate: at least 16 GB system RAM. Longer documents need more memory. No speed benchmark.
Choose where you want to run it. The instructions below update to match your selection.
Apple Silicon uses GPU acceleration; Intel Macs use the CPU.
Already have the current Ollama app? Open it and skip installation. Otherwise download the Mac app, drag it into Applications and open it on macOS 14 or newer. The selected package downloads 6.6 GB; leave additional disk space for Ollama and your files. Use the memory estimate above and keep Ollama open.
Download Ollama for Mac Installation requirementsStart a new chat in Ollama. Open the model picker, search for qwen3.5:9b and select that exact name, including the tag after the colon. Allow the 6.6 GB download to finish when first used. Confirm the selected model remains qwen3.5:9b before sending a document. The app manages the downloaded model files.
Exact selected package Ollama desktop chatIn Ollama Settings, move the context-length slider to 4K (4096 tokens). Start with a short text document, leaving room for your question and answer. This guide's memory estimates use 4K. Longer context consumes more memory; try 8K only after the initial chat works with memory to spare.
Ollama context settingsOpen TextEdit and choose Format > Make Plain Text. Enter this text and save it as cedar-note.txt in Documents: “Project Cedar review is on 14 October 2026. The owner is Mira Chen. The review room is Birch.” Drag cedar-note.txt into the selected qwen3.5:9b chat and wait for the attachment to appear. Begin with this tiny text file before using your own short text or text-based PDF. A scanned PDF may need text extraction first.
Supported text and PDF attachmentsWith cedar-note.txt attached, type this prompt in the chat box and send it: “Using only cedar-note.txt, who owns Project Cedar and when is its review? Quote the sentence supporting each answer. If the document does not say, answer “not stated”.” Read the final answer and compare the quoted text with the saved file. It should name Mira Chen and 14 October 2026. A plausible answer without matching source text has not passed this check.
In the same chat, send the follow-up: “What is the review budget? Use only this document and quote its evidence, or answer “not stated”.” The document gives no budget, so check that the answer says “not stated”. Copy the checked answers into a new plain-text TextEdit document and save cedar-answers.txt in Documents beside cedar-note.txt. Keep the source file available when checking answers about your own documents.
Reopen this chat from the app's conversation list to continue with the attached document. Start a new chat for a different document or another model. Wait for the active reply before closing the app. Quit Ollama from its menu-bar icon to stop the local service; reopen it from Applications next time. The model stays downloaded.
The selected model should answer Mira Chen and 14 October 2026 with quotes matching cedar-note.txt, then say “not stated” for the budget follow-up. Save the checked response as cedar-answers.txt beside the source. Repeat with short excerpts from your own documents and inspect every cited passage.
This guide asks questions about short text in one conversation. It does not create a searchable document library. Windows and Mac use Ollama file attachments; Linux uses pasted text. Model accuracy and memory estimates have not been benchmarked by YouRunAI.
Update the Ollama desktop app and try cedar-note.txt first. For a PDF, check that text can be selected in a PDF viewer; scans may need text extraction. On Linux, paste the readable text at the interactive prompt instead of entering a filename.
Compare the exact words with the source. Ask the model to use only the document and say “not stated” when it lacks evidence. Reduce the input to the relevant excerpt and begin a new conversation if it remains unreliable.
Start at 4K with a short excerpt and close other heavy apps. Longer files may exceed the available context; divide them into smaller sections or try 8K only with free memory. On an 8 GB GPU, CPU offload may make replies slower.
Install the current Ollama release and check the full tag in the package link. The selected qwen3.5:9b-mlx route requires Apple Silicon; choose qwen3.5:9b on Windows, Linux or Intel Mac. For the community package, include the lukey03/ prefix.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.