Install the command-line tools
If Git is not installed, run this in Terminal and finish the macOS installer.
xcode-select --installCoding / Intermediate
Build a private coding assistant with Bonsai 2
Use Bonsai 2 as a local coding assistant with the exact package and operating-system commands from its supported runtime, then check its output with a small executable task.
Verified sourceSource checked 9/24/2026
LOCAL
Before you begin
- Difficulty
- Intermediate
- Software
- Git, PrismML Bonsai Demo
Hardware
Bonsai 2 is unusually small for a 27B model. Compare its two package formats against your machine:
| Machine | PTQ1_0 · 5.95 GB | PQ2_0 · 7.21 GB |
|---|---|---|
| 8 GB Apple Silicon Mac | Runs, but tight | Not recommended |
| 16 GB Apple Silicon Mac | Good | Good |
| 6 GB GPU | Runs with CPU/RAM offload | Heavy offload |
| 8 GB GPU | Good at short context | Tight |
| 10–12 GB GPU | Excellent | Recommended |
| 16 GB+ GPU | Excellent | Excellent |
| Intel Arc / Vulkan | Conditional GPU route; validate driver | No native Vulkan kernels; use CPU |
| CPU + 16 GB RAM | Runs | Runs, less headroom |
| CPU + 32 GB RAM | Comfortable | Comfortable |
GPU memory rows assume a backend that supports the format: NVIDIA CUDA or AMD HIP for both, Intel Vulkan for PTQ1_0 only. These are setup-guide estimates, not YouRunAI speed tests. CRACK variants use the matching format. Start at 4K–8K context on smaller machines.
LOCAL CODING / MAC · WINDOWS · LINUXSOURCE-LINKED SETUP
Your local coding assistant, step by step.
Choose an official or third-party Bonsai 2 package. Start its matching local server, then give it a small coding task and verify the result before applying it to your project.
Choose a package Run its commands Check the result
01
BEFORE YOU BEGIN
Choose your package
Select the weights you want. The download and launch commands change with your choice.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
CURRENT SETUP
View weight source PQ2_0 · official
8 GB GPU is tight; 10–12 GB GPU recommended. An 8 GB Mac is not recommended. Start at 8K context.
02
THE WORKFLOW
Set up PQ2_0 on your machine
Pick your operating system and GPU path. Every command below is for the selected package and runtime.
Get PrismML’s Bonsai runtime
Clone the maintained demo and enter its folder before running the following commands.
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demoInstall the runtime and official model
PrismML’s setup installs a compatible runtime and downloads the official PQ2_0 weights plus vision projector.
BONSAI_SKIP_MLX=1 BONSAI_OPENWEBUI=0 BONSAI_CODE_INTERPRETER=0 ./setup.shStart the selected local chat server
BONSAI_GGUF points to Ternary-Bonsai-2-27B-PQ2_0.gguf, so the launcher cannot silently use another installed package. Start with 8K context.
BONSAI_CTX=8192 BONSAI_GGUF="models/bonsai2-gguf/27B/Ternary-Bonsai-2-27B-PQ2_0.gguf" BONSAI_MMPROJ="models/bonsai2-gguf/27B/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf" ./scripts/start_llama_server.shCheck a small coding task
In the local chat, ask Bonsai to write a single-file HTML page with a button that updates a counter. Save the answer, open it in a browser, click the button, and inspect the code before using it in a larger project.
03
SUCCESS CHECK · PQ2_0
Run the generated code
The counter page opens in your browser, the button changes the number, and you can explain the change Bonsai proposed. Keep code review and execution under your control.
This is a coding workflow through local chat. It does not give the model shell access or claim agent-style autonomous coding. The package fit is estimated; code quality is not a YouRunAI benchmark.
When it doesn’t go to plan
The server says invalid ggml type or gives unusable output
Use the runtime installed by PrismML Bonsai Demo, not stock llama.cpp or Ollama for these packed files.
The machine slows down or runs out of memory
Close other GPU-heavy applications and lower BONSAI_CTX to 4096. On a 6 GB GPU, use partial CPU/RAM offload; the package file size is not total runtime memory.
Windows starts a different model
Use the package-specific Windows launch command, which names the exact GGUF file. The default PowerShell launcher searches a shared folder and can select a different installed file.
Windows selected the wrong GPU backend
Check nvidia-smi for NVIDIA, hipcc or HIP_PATH for AMD, or vulkaninfo for Intel. Rerun PrismML setup.ps1, then confirm the expected bin\cuda, bin\hip, or bin\vulkan server exists before launching.
Intel Vulkan cannot load PQ2_0 or gives incorrect output
Use the official PTQ1_0 choice with the Windows · Intel Arc tab. PrismML has no native Vulkan kernels for PQ2_0. If PTQ1_0 still fails on your driver, use the CPU path and check PrismML’s backend support notes.
The generated page does not work as requested
Ask the model to explain its assumptions and correct the specific error. Run the result in a disposable test file and review the changed code before using it in a real project.
REFERENCE LIBRARY
8 SOURCESSources and files
Original instructions, model files, and compatibility notes behind this setup.
01PrismML Bonsai Demo and runtimegithub.com02Official PQ2_0 and PTQ1_0 fileshuggingface.co03CRACK PQ2_0 files (third party)huggingface.co04CRACK PTQ1_0 files (third party)huggingface.co05PrismML backend and format supportgithub.com06PrismML community measurementsgithub.com07PrismML launcher optionsgithub.com08Publisher model behavior and coding claimsgithub.com
The model behind this workflow
Bonsai 2 · 27BWill it run on your machine?
Save your machine to see a personalized rating and its reasoning.
Add my hardwareKeep exploring
Text
BeginnerSet up a private AI chat assistant
Install a local runtime, run Qwen3.5 9B, confirm responses, and know when to choose the smaller 4B package.
SOFTWAREOllama
Coding
IntermediateBuild a private local coding assistant
Connect an open coding-capable model in Ollama to Cline, run a small repository task, and review the result locally.
SOFTWAREOllama + Cline
Images
IntermediateGenerate images on your own GPU
Use the official FLUX.2 Klein 4B ComfyUI template with exact model files, a first prompt, and an output check.
SOFTWAREComfyUI