Build a local AI agent team with Qwen3 8B, Ollama and CrewAI. The example assigns reader, writer and reviewer roles to compare two product briefs, with source reads, task handoffs and a final Markdown file you can inspect.
Estimated for two short documents, 16K context and one model loaded at a time. These are capacity starting points, not speed measurements.
| Machine | Qwen3 8B · Q4 | Qwen3 8B · Q8 | Qwen3 8B · Uncensored Q4 |
|---|---|---|---|
| NVIDIA · 8 GB VRAM + 16 GB RAM | Runs with some RAM offload | Prefer Q4; Q8 needs 24 GB system RAM | Runs with some RAM offload |
| NVIDIA · 12 GB VRAM + 24 GB RAM | Recommended Q4 starting point | Use some RAM offload; 16 GB GPU preferred | Recommended Q4 starting point |
| NVIDIA · 32 GB VRAM + 64 GB RAM | Comfortable at the supplied settings | Comfortable at the supplied settings | Comfortable at the supplied settings |
| AMD · 16 GB VRAM + 32 GB RAM | Fits; supported ROCm/Vulkan driver required | Fits; supported ROCm/Vulkan driver required | Fits; supported ROCm/Vulkan driver required |
| Intel · 16 GB VRAM + 32 GB RAM | Fits; supported Vulkan backend required | Fits; supported Vulkan backend required | Fits; supported Vulkan backend required |
| Apple Silicon · 16 GB unified memory | Tight; 24 GB gives more room | Choose Q4; Q8 starts at 24 GB | Tight; 24 GB gives more room |
| Apple Silicon · 32 GB unified memory | Recommended with room for tools and OS | Recommended with room for tools and OS | Recommended with room for tools and OS |
| CPU only · 32 GB system RAM | Runs more slowly; sequential roles limit memory | Runs more slowly; sequential roles limit memory | Runs more slowly; sequential roles limit memory |
These are estimates including model/runtime/context allowance. Do not multiply memory by three: these roles reuse one model sequentially. Concurrent crews or larger documents require a new capacity check.
A 16K task includes role instructions, tool results and replies. The sample files are intentionally short.
| Machine | Qwen3 8B · Q4 | Qwen3 8B · Q8 | Qwen3 8B · Uncensored Q4 |
|---|---|---|---|
| Q4 · 12 GB GPU / 24 GB Mac or system RAM | Start at 16,384 tokens | Use 16 GB GPU / 32 GB Mac for Q8 | Start at 16,384 tokens |
| Q8 · 16 GB GPU / 32 GB Mac or system RAM | Start at 16,384 tokens; keep each input brief short | Start at 16,384 tokens; keep each input brief short | Start at 16,384 tokens; keep each input brief short |
The script and saved model both use 16384 context and 2048 output tokens per response. Summarize or retrieve large documents before sending them to the crew.
Set up collaborating local AI agents with Qwen3 8B and CrewAI. The example uses a reader, writer and reviewer: a file-reading tool supplies facts, task context passes the work forward, and the reviewer checks a final brief you can open and audit.
Select the weights you want. The instructions, downloads and settings change with your choice.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Start at 16,384 tokens per role for the two short supplied documents. One model is reused sequentially. Longer input and output consume more context and memory; keep documents short or add retrieval before expanding the task.
Choose where you want to run it. The instructions below update to match your selection.
Download the native Ollama installer, run it, and launch the tray app. Keep it running. Open a new PowerShell window for this guide. Install the current NVIDIA driver, supported AMD ROCm/Vulkan driver, or Intel Vulkan driver for your exact card. CPU inference is also available. These instructions do not require WSL. If it is already installed and running, skip installation.
Download Ollama for Windows Ollama GPU and driver supportDownload the ZIP below and extract it with File Explorer → Extract All. Move the extracted yourunai-team-qwen3-8b-q4 folder into your home folder. The starter contains both source briefs, tools.py, crew.py and the 16K model preset. No code needs to be copied from this page. Open those files in a text editor whenever you want to inspect the roles or tools.
Download prepared team projectIn PowerShell, run these commands from the extracted folder. They download the exact selected package once and create its 16K team preset using the supplied Modelfile. All three roles share this one local model sequentially.
Exact package Ollama ModelfileSet-Location "$HOME\yourunai-team-qwen3-8b-q4"
ollama pull qwen3:8b-q4_K_M
ollama create yourunai-team-qwen3-8b-q4-16k -f ModelfileInstall uv using its linked official instructions, then open a new PowerShell window. Return to the extracted folder with the commands below. This installs Python 3.12 and the pinned CrewAI dependencies only for this project; there is no system Python setup or environment activation step.
Install uv for your operating system CrewAI installation and dependencies Pinned CrewAI packageSet-Location "$HOME\yourunai-team-qwen3-8b-q4"
uv python install 3.12
uv venv --python 3.12
uv pip install "crewai[litellm]==1.15.22"The prepared task is to compare Alpha and Beta and write a cited recommendation for someone who needs JSON export and an audit log. The supplied crew.py defines three real roles. Source reader uses read_product_brief to read the two supplied documents. Comparison writer receives those facts. Fact reviewer receives both handoffs, checks the original files and can delegate a clarification to Source reader. tools.py restricts reads to alpha.md and beta.md and logs them. The same local model handles the roles sequentially; no cloud API key, external search, telemetry or memory embedding service is used.
Sequential task handoffs Local Ollama connection Task outputs and context CrewAI custom toolsIn PowerShell, run the script from this project folder. Watch the named roles and read_product_brief actions. The first role writes facts.md, the second receives those facts and writes draft.md, and the reviewer writes brief.md. Each role has bounded iterations; Ctrl+C stops the run. Do not accept a run that reports missing source reads or output files.
.\.venv\Scripts\python.exe crew.pyOpen output/brief.md and compare it with both inputs. Verify Alpha is $199/90 days/30 hours/CSV; Beta is $349/180 days/12 hours/CSV+JSON with an audit log. Check filename citations and the Beta recommendation for the stated needs. Inspect output/facts.md, output/draft.md and output/crew-log.json to see the actual handoffs. tool-reads.log must list both source files. A file existing does not prove its facts are correct. Open these files from the output folder in your normal text editor.
Open docs/beta.md in a text editor and change $349 to $399. Run the command below from the same project directory in PowerShell. It starts a fresh crew and overwrites only the demo output files/logs. Confirm the final brief now uses $399. Keep inputs short at 16K; larger document sets need retrieval or deliberate chunking, not unlimited prompt growth.
.\.venv\Scripts\python.exe crew.pyPress Ctrl+C for a running crew, then unload the selected model in PowerShell with the command below. Next time, launch Ollama, return to this project folder and run the same explicit Python path with crew.py. The inputs, preset and environment persist. This is a sequential team on one computer, not a distributed multi-machine cluster.
ollama stop yourunai-team-qwen3-8b-q4-16kBoth filenames appear in tool-reads.log. facts.md feeds draft.md, which feeds the final brief.md. Compare the final table to the supplied documents and verify the rerun reflects your edited price. A successful exit alone does not certify the content.
The guide uses CrewAI 1.15.22 and LiteLLM with a local Ollama endpoint. Source-checked and statically reviewed; the model/framework combination has not been physically executed by YouRunAI. Uncensored Q4 is a community derivative and can behave differently.
Confirm every agent uses the provided llm object and MODEL begins ollama_chat/. Keep memory=False and planning=False. Install crewai[litellm] in the local environment and run the explicit environment Python path, not another system Python.
Stop with Ctrl+C, use the official Q4 package first, and inspect crew-log.json. Keep the two supplied documents short. Confirm current Ollama is running, the exact 16K preset is selected and LiteLLM is installed. A community derivative may need a different model choice rather than repeated retries.
Use Q4, stop other loaded models and close large apps. An 8 GB GPU may use system-RAM offload; check ollama ps. Keep one crew at a time. For Q8 use at least 24 GB system/unified memory, preferably 32 GB.
Use the provided Python 3.12 environment. If uv reports a native build failure, install Microsoft Visual Studio Build Tools with Desktop development with C++, open a new PowerShell and repeat only uv pip install. Do not change the model or remove required dependencies.
Compare it with docs/alpha.md and docs/beta.md, inspect source reads and intermediate facts/draft files, then rerun after correcting the task or source. Do not accept a generated report solely because all files exist.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.