Workflows/Run a private reasoning assistant with Bonsai 2 · 27B

Text / WORKFLOW

Run a private reasoning assistant with Bonsai 2 · 27B

Select a source-backed Bonsai 2 · 27B package, start the PrismML llama.cpp fork, then verify a multi-step reasoning answer locally.

EstimatedSource checked 9/28/2026

LOCALRENTED GPU

Freshness 82 out of 100. Release age: 52 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,710,588 downloads in 30 days; 2,952 lifetime likes. Checked Nov 7, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

A new release starts at 100. The score halves every 180 days: 100 × 2^(−age in days / 180). When the release date is unavailable, a repository date is a fallback—not proof of when the model was released. Missing dates stay unscored.

Check the date source

Popularity · adoption

Downloads over the past 30 days carry 80% of the score; lifetime likes carry 20%. Both use a logarithmic scale, capped at 10 million downloads and 100,000 likes. If likes are unavailable, downloads carry all the weight. Missing download statistics stay unscored.

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 8, 2026. Statistics checked Nov 7, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

Before you begin

Hardware

Machine	PTQ1_0	PQ2_0	CRACK PTQ1_0	CRACK PQ2_0
NVIDIA GPU 8 GB VRAM	CUDA; 8 GB VRAM works at short context.	8 GB is tight at short context; 10–12 GB recommended.	Community PTQ1_0; same 5.95 GB fit, validate weights.	Community PQ2_0; 8 GB tight, 10–12 GB recommended.
NVIDIA GPU 10–12 GB VRAM	CUDA; comfortable at short context.	CUDA; recommended tier for 7.21 GB PQ2_0.	Community PTQ1_0; same fit, validate weights.	Community PQ2_0; recommended tier, validate weights.
NVIDIA GPU 16 GB VRAM	CUDA; ample VRAM for package plus context.	CUDA; ample headroom for longer context.	Community PTQ1_0; ample VRAM, validate weights.	Community PQ2_0; ample headroom, validate weights.
AMD Radeon GPU 16 GB VRAM (ROCm/HIP)	ROCm/HIP via PrismML fork; validate on your Radeon GPU.	ROCm/HIP via PrismML fork; validate on your Radeon GPU.	Community PTQ1_0; ROCm/HIP route needs your driver validation.	Community PQ2_0; ROCm/HIP route needs your driver validation.
Intel Arc GPU 12 GB VRAM (Vulkan)	Vulkan PTQ1_0 supported; some drivers fall back.	No native Vulkan PQ2_0 kernels; use PTQ1_0 or CPU.	Community PTQ1_0; Vulkan route same as official PTQ1_0.	No native Vulkan PQ2_0 kernels; use PTQ1_0 or CPU.
Apple Silicon 16 GB unified memory	Metal; 16 GB unified memory recommended minimum.	Metal; 16 GB unified memory recommended for 7.21 GB pack.	Community PTQ1_0; Metal route, validate weights.	Community PQ2_0; Metal route, validate weights.
Apple Silicon 8 GB unified memory	Not recommended; 16 GB unified memory is practical minimum.	Not recommended; 16 GB unified memory is practical minimum.	Not recommended; 16 GB unified memory is practical minimum.	Not recommended; 16 GB unified memory is practical minimum.
CPU system RAM 16 GB	CPU-only; 16 GB system RAM can run, 32 GB comfortable.	CPU-only; 16 GB RAM can run, 32 GB comfortable.	Community PTQ1_0; CPU-only same RAM guidance.	Community PQ2_0; CPU-only same RAM guidance.
CPU system RAM 32 GB	CPU-only; 32 GB RAM gives comfortable headroom.	CPU-only; 32 GB RAM gives comfortable headroom.	Community PTQ1_0; CPU-only same RAM guidance.	Community PQ2_0; CPU-only same RAM guidance.

Memory and context figures are setup estimates, not measured benchmarks. Use the PrismML llama.cpp fork for packed Bonsai 2 weights.

Context window

Machine	PTQ1_0	PQ2_0	CRACK PTQ1_0	CRACK PQ2_0
NVIDIA GPU 8 GB VRAM	8192 tokens; short-context start, reduce if OOM.	4096 tokens; 8 GB is tight, 10–12 GB recommended.	8192 tokens; community weights, short-context start.	4096 tokens; community PQ2_0, 8 GB tight.
NVIDIA GPU 10–12 GB VRAM	16384 tokens; short-context start, headroom varies.	16384 tokens; recommended tier, reduce if OOM.	16384 tokens; community weights, same headroom caveat.	16384 tokens; community weights, same headroom caveat.
NVIDIA GPU 16 GB VRAM	32768 tokens; ample VRAM, reduce for longer prompts.	32768 tokens; ample VRAM, reduce for longer prompts.	32768 tokens; community weights, same headroom caveat.	32768 tokens; community weights, same headroom caveat.
AMD Radeon GPU 16 GB VRAM (ROCm/HIP)	16384 tokens; ROCm/HIP starting point, validate driver.	16384 tokens; ROCm/HIP starting point, validate driver.	16384 tokens; community weights, validate ROCm/HIP.	16384 tokens; community weights, validate ROCm/HIP.
Intel Arc GPU 12 GB VRAM (Vulkan)	8192 tokens; Vulkan PTQ1_0 start, reduce if fallback slows.	Unsupported Vulkan PQ2_0; use PTQ1_0 or CPU.	8192 tokens; community PTQ1_0, same Vulkan caveat.	Unsupported Vulkan PQ2_0; use PTQ1_0 or CPU.
Apple Silicon 16 GB unified memory	16384 tokens; 16 GB unified memory start, reduce if swap.	16384 tokens; 16 GB unified memory start, reduce if swap.	16384 tokens; community weights, same unified-memory caveat.	16384 tokens; community weights, same unified-memory caveat.
Apple Silicon 8 GB unified memory	Not recommended; use 16 GB Apple Silicon row.	Not recommended; use 16 GB Apple Silicon row.	Not recommended; use 16 GB Apple Silicon row.	Not recommended; use 16 GB Apple Silicon row.
CPU system RAM 16 GB	8192 tokens; 16 GB RAM start, 32 GB more comfortable.	8192 tokens; 16 GB RAM start, 32 GB more comfortable.	8192 tokens; community weights, 16 GB RAM start.	8192 tokens; community weights, 16 GB RAM start.
CPU system RAM 32 GB	32768 tokens; 32 GB RAM gives comfortable headroom.	32768 tokens; 32 GB RAM gives comfortable headroom.	32768 tokens; community weights, same RAM headroom.	32768 tokens; community weights, same RAM headroom.

LOCAL REASONING / MAC · WINDOWS · LINUXSOURCE-LINKED SETUP

Private reasoning, with a check you can trust.

Choose an official or source-linked Bonsai 2 · 27B package, select your operating-system route, and confirm a multi-step reasoning answer locally.

Choose a package Run its commands Check the result

BEFORE YOU BEGIN

Choose your package

Select the weights you want. The instructions, downloads and settings change with your choice.

Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.

CURRENT SETUP

PTQ1_0 · official

Estimated from 5.95 GB package size; 6 GB GPU needs CPU/RAM offload and 16 GB Apple Silicon is the practical minimum. These are setup estimates, not benchmarks.

Freshness 82 out of 100. Release age: 52 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,710,588 downloads in 30 days; 2,952 lifetime likes. Checked Nov 7, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

Check the date source

Popularity · adoption

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 8, 2026. Statistics checked Nov 7, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

View weight source

THE WORKFLOW

Set up PTQ1_0 on your machine

Choose your computer first, then its GPU path. The instructions below update to match your selection.

CHOOSE YOUR SETUPMac

Metal via PrismML llama.cpp fork; 16 GB unified memory recommended. 8 GB is not recommended because macOS and KV cache leave no headroom.

Install the command-line tools

If Git is not installed, run this in Terminal and finish the macOS installer.

Git downloads

xcode-select --install

Clone the PrismML Bonsai demo runtime

In Terminal create the working folder, clone the publisher demo, and enter its directory.

PrismML Bonsai demo PrismML llama.cpp fork

mkdir -p "$HOME/YouRunAI-Ternary-Bonsai-2-27B"
cd "$HOME/YouRunAI-Ternary-Bonsai-2-27B"
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo

Install the PrismML runtime without default weights

The publisher setup installs the compatible llama.cpp fork binary. Skip the default PQ2_0 download, MLX and Open WebUI because the selected PTQ1_0 package is downloaded separately.

PrismML environment variable reference PrismML backend support

BONSAI_SKIP_GGUF=1 BONSAI_SKIP_MLX=1 BONSAI_OPENWEBUI=0 ./setup.sh

Download the selected PTQ1_0 package and vision projector

Fetch Ternary-Bonsai-2-27B-PTQ1_0.gguf (5.95 GB) and the optional Q8_0 projector into a dedicated subfolder so other installed Bonsai packages are untouched.

Put these files in ${HOME}/YouRunAI-Ternary-Bonsai-2-27B/Bonsai-demo/models/yourunai/official-ptq1/2 required files

models/yourunai/official-ptq1/

Ternary-Bonsai-2-27B-PTQ1_0.ggufOfficial smaller PTQ1_0 package (5.95 GB)Get file Ternary-Bonsai-2-27B-mmproj-Q8_0.ggufOptional vision projector (0.63 GB)Get file

Ternary-Bonsai-2-27B-gguf on Hugging Face

mkdir -p "models/yourunai/official-ptq1"
curl -fL --progress-bar "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/resolve/main/Ternary-Bonsai-2-27B-PTQ1_0.gguf?download=true" -o "models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf"
curl -fL --progress-bar "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/resolve/main/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf?download=true" -o "models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf"

Start the local Metal chat server with the selected package

BONSAI_GGUF pins the launcher to Ternary-Bonsai-2-27B-PTQ1_0.gguf. BONSAI_CTX=16384 matches the 16 GB Apple Silicon row in the context matrix; lower it to 8192 or 4096 if macOS starts swapping. The model reports a 262144-token maximum, but the running Metal context is whatever you pass here. Keep this Terminal window open; open a second Terminal for client requests.

llama.cpp server guide

BONSAI_CTX=16384 BONSAI_GGUF="models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf" BONSAI_MMPROJ="models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf" ./scripts/start_llama_server.sh
open http://localhost:8080

Smoke-test a multi-step reasoning answer

From the second Terminal, send a known-answer multi-step problem through the OpenAI-compatible route and confirm the final number matches your own calculation.

curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"Ternary-Bonsai-2-27B-PTQ1_0.gguf","messages":[{"role":"user","content":"A train travels 60 miles in 1.5 hours, then 90 miles in 2.5 hours. What is its average speed for the whole trip? Show the steps."}],"max_tokens":1024,"temperature":1.0,"top_p":0.95,"top_k":20}'

Continue the conversation in the browser chat

Open the built-in llama.cpp web UI at http://localhost:8080. Type a follow-up prompt, press Send, and wait for the reply that uses the earlier turn. If memory is tight, stop the server with Ctrl+C and restart it with a smaller BONSAI_CTX such as 8192 or 4096.

open http://localhost:8080

SUCCESS CHECK · PTQ1_0

Verify the reasoning result

The selected package responds in local chat and gives the correct answer to your known multi-step problem. Check its result independently; this setup is not a model-quality benchmark.

Reasoning quality varies by prompt and package. Memory guidance is a short-context setup estimate, not a speed claim. Use the PrismML runtime for Bonsai 2 packed weights.

Back to setup steps

When it doesn’t go to plan

The server reports an invalid GGML type or gives unusable output.

Confirm you are using the PrismML llama.cpp fork, not stock llama.cpp or an Ollama tag. These packed Bonsai 2 files need the fork's ternary kernels and Hadamard transforms.

The machine slows down or runs out of memory.

Close other GPU-heavy applications and reduce the context setting. On a 6 GB GPU, use partial CPU/RAM offload; package file size is not total runtime memory.

Windows starts a different model.

Launch with the package-specific GGUF path instead of relying on the shared default, which can select another installed file.

Intel Vulkan cannot load PQ2_0 or gives incorrect output.

Choose a PTQ1_0 package instead. PrismML has no native Vulkan kernels for PQ2_0. If PTQ1_0 fails on your driver, use the CPU path and check PrismML's backend support notes.

REFERENCE LIBRARY

Sources and files

Original instructions, model files, and compatibility notes behind this setup.

8 SOURCES

01PrismML Bonsai Demo and runtimegithub.com 02Official PQ2_0 and PTQ1_0 fileshuggingface.co 03CRACK PQ2_0 files (third party)huggingface.co 04CRACK PTQ1_0 files (third party)huggingface.co 05PrismML backend and format supportgithub.com 06PrismML community measurementsgithub.com 07PrismML launcher optionsgithub.com 08PrismML llama.cpp forkgithub.com

The model behind this workflow

Bonsai 2 · 27B

Keep exploring

Text

SETUP GUIDE

Qwen3.5 · 9B private chat setup

Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.

Freshness 38 out of 100. Release age: 254 days old, Feb 27, 2026. Measures age, not quality.

Build a private coding assistant with Qwen3.8 27B

Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.

Freshness 69 out of 100. Repository age: 95 days old, Aug 5, 2026. Measures age, not quality.

Arena Open Source Web coding: rank 12 of 62. Ranked model: Qwen3.8-27B. Original model ranking, not a benchmark of this package. Export dated Nov 6, 2026. Source checked Nov 7, 2026. Human preference ranking, not hardware suitability.+3

SOFTWAREOllama + Visual Studio Code + Cline + Python

Create images locally with FLUX.2 Klein

Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.

Freshness 32 out of 100. Release age: 297 days old, Jan 15, 2026. Measures age, not quality.

SOFTWAREComfyUI

LOCALRENTED GPU

Verified source