Workflows/Set up private chat with Bonsai 2 · 27B

Text / WORKFLOW

Set up private chat with Bonsai 2 · 27B

Run Bonsai 2 · 27B locally for private chat using PrismML's llama.cpp fork. Choose an official ternary package or a community CRACK uncensored build, then follow the route for your operating system and accelerator backend.

EstimatedSource checked 9/29/2026

LOCALRENTED GPU

Freshness 82 out of 100. Release age: 52 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,710,588 downloads in 30 days; 2,952 lifetime likes. Checked Nov 7, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

A new release starts at 100. The score halves every 180 days: 100 × 2^(−age in days / 180). When the release date is unavailable, a repository date is a fallback—not proof of when the model was released. Missing dates stay unscored.

Check the date source

Popularity · adoption

Downloads over the past 30 days carry 80% of the score; lifetime likes carry 20%. Both use a logarithmic scale, capped at 10 million downloads and 100,000 likes. If likes are unavailable, downloads carry all the weight. Missing download statistics stay unscored.

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 8, 2026. Statistics checked Nov 7, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

Before you begin

Hardware

Machine	PTQ1_0	PQ2_0	CRACK PTQ1_0	CRACK PQ2_0
Apple Silicon Mac · 8 GB unified memory	Tight at 8 GB; use Metal with short context and expect offload overhead.	Not recommended at 8 GB; choose official-ptq1 for this tier.	Same 5.95 GB footprint; tight at 8 GB, short context.	Not recommended at 8 GB; use crack-ptq1 instead.
Apple Silicon Mac · 16 GB unified memory	Comfortable at 16 GB; Metal supports PTQ1_0 with room for context.	Supported at 16 GB; Metal supports PQ2_0, start short context.	Community 5.95 GB pack; 16 GB unified memory comfortable.	Community 7.21 GB pack; 16 GB recommended, start short context.
NVIDIA GPU · 6 GB VRAM	6 GB needs CPU/RAM offload; CUDA supports PTQ1_0.	Requires 8 GB minimum; use official-ptq1 on 6 GB.	6 GB needs CPU/RAM offload; CUDA supports PTQ1_0.	Requires 8 GB minimum; use crack-ptq1 on 6 GB.
NVIDIA GPU · 12 GB VRAM	Comfortable at 12 GB; CUDA supports PTQ1_0 with headroom.	Comfortable at 12 GB; CUDA supports PQ2_0.	Comfortable at 12 GB; CUDA supports the PTQ1_0 footprint.	Comfortable at 12 GB; CUDA supports the PQ2_0 footprint.
AMD GPU · 8 GB VRAM	8 GB is tight; ROCm/HIP supports PTQ1_0, use short context.	8 GB is tight for PQ2_0; 10–12 GB recommended, or use PTQ1_0.	8 GB tight; ROCm/HIP supports PTQ1_0, use short context.	8 GB tight for PQ2_0; 10–12 GB recommended, or use PTQ1_0.
AMD GPU · 16 GB VRAM	Comfortable at 16 GB; ROCm/HIP supports PTQ1_0.	Comfortable at 16 GB; ROCm/HIP supports PQ2_0.	Comfortable at 16 GB; ROCm/HIP supports PTQ1_0.	Comfortable at 16 GB; ROCm/HIP supports PQ2_0.
Intel Arc GPU · 8 GB VRAM	Vulkan supports PTQ1_0; validate driver, 8 GB is tight for context.	No native Vulkan PQ2_0 kernels; use official-ptq1 or CPU.	Vulkan supports PTQ1_0; validate driver, 8 GB tight for context.	No native Vulkan PQ2_0 kernels; use crack-ptq1 or CPU.
CPU · 16 GB system RAM	CPU supported; 16 GB RAM is tight, close other apps, short context.	CPU supported; 16 GB RAM tight for 7.21 GB weights plus overhead.	CPU supported; 16 GB RAM tight, short context.	CPU supported; 16 GB RAM tight for 7.21 GB weights plus overhead.
CPU · 32 GB system RAM	CPU supported; 32 GB RAM comfortable for 5.95 GB weights.	CPU supported; 32 GB RAM comfortable for 7.21 GB weights.	CPU supported; 32 GB RAM comfortable for 5.95 GB weights.	CPU supported; 32 GB RAM comfortable for 7.21 GB weights.

Context window

Machine	PTQ1_0	PQ2_0	CRACK PTQ1_0	CRACK PQ2_0
Apple Silicon Mac · 8 GB unified memory	4096 tokens starting point; 5.95 GB weights leave little headroom at 8 GB.	Not recommended at 8 GB; use official-ptq1.	4096 tokens starting point; 5.95 GB weights leave little headroom at 8 GB.	Not recommended at 8 GB; use crack-ptq1.
Apple Silicon Mac · 16 GB unified memory	8192 tokens starting point; 5.95 GB weights plus KV/runtime overhead.	8192 tokens starting point; 7.21 GB weights plus KV/runtime overhead.	8192 tokens starting point; 5.95 GB weights plus KV/runtime overhead.	8192 tokens starting point; 7.21 GB weights plus KV/runtime overhead.
NVIDIA GPU · 6 GB VRAM	4096 tokens starting point; 6 GB VRAM needs CPU/RAM offload.	Requires 8 GB minimum; use official-ptq1 at 4096 tokens.	4096 tokens starting point; 6 GB VRAM needs CPU/RAM offload.	Requires 8 GB minimum; use crack-ptq1 at 4096 tokens.
NVIDIA GPU · 12 GB VRAM	8192 tokens starting point; 5.95 GB weights leave headroom at 12 GB.	8192 tokens starting point; 7.21 GB weights leave headroom at 12 GB.	8192 tokens starting point; 5.95 GB weights leave headroom at 12 GB.	8192 tokens starting point; 7.21 GB weights leave headroom at 12 GB.
AMD GPU · 8 GB VRAM	4096 tokens starting point; 8 GB VRAM is tight, expect offload.	8 GB tight; 10–12 GB recommended, or use official-ptq1 at 4096.	4096 tokens starting point; 8 GB VRAM is tight, expect offload.	8 GB tight; 10–12 GB recommended, or use crack-ptq1 at 4096.
AMD GPU · 16 GB VRAM	8192 tokens starting point; 5.95 GB weights leave headroom at 16 GB.	8192 tokens starting point; 7.21 GB weights leave headroom at 16 GB.	8192 tokens starting point; 5.95 GB weights leave headroom at 16 GB.	8192 tokens starting point; 7.21 GB weights leave headroom at 16 GB.
Intel Arc GPU · 8 GB VRAM	4096 tokens starting point; Vulkan PTQ1_0 supported, 8 GB tight.	No native Vulkan PQ2_0; use official-ptq1 at 4096 tokens or CPU.	4096 tokens starting point; Vulkan PTQ1_0 supported, 8 GB tight.	No native Vulkan PQ2_0; use crack-ptq1 at 4096 tokens or CPU.
CPU · 16 GB system RAM	4096 tokens starting point; 5.95 GB weights plus runtime overhead.	4096 tokens starting point; 7.21 GB weights plus runtime overhead.	4096 tokens starting point; 5.95 GB weights plus runtime overhead.	4096 tokens starting point; 7.21 GB weights plus runtime overhead.
CPU · 32 GB system RAM	8192 tokens starting point; 32 GB RAM gives more KV headroom.	8192 tokens starting point; 32 GB RAM gives more KV headroom.	8192 tokens starting point; 32 GB RAM gives more KV headroom.	8192 tokens starting point; 32 GB RAM gives more KV headroom.

LOCAL CHAT / MAC · WINDOWS · LINUXSOURCE-LINKED SETUP

Private chat with Bonsai 2 · 27B

Run a private local chat with Bonsai 2 · 27B. Pick the package that fits your memory and preferences, launch the PrismML fork, send a first prompt, and confirm the selected model responds.

Choose a package Run its commands Check the result

BEFORE YOU BEGIN

Choose your package

Select the weights you want. The instructions, downloads and settings change with your choice.

Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.

CURRENT SETUP

PTQ1_0 · official

Estimated from the package size and supplied setup guide: 6 GB GPU needs CPU/RAM offload; 8 GB GPU works best at short context. An 8 GB Apple Silicon Mac is tight. Start at a modest context such as 4K.

Freshness 82 out of 100. Release age: 52 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,710,588 downloads in 30 days; 2,952 lifetime likes. Checked Nov 7, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

Check the date source

Popularity · adoption

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 8, 2026. Statistics checked Nov 7, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

View weight source

THE WORKFLOW

Set up PTQ1_0 on your machine

Choose your computer first, then its GPU path. The instructions below update to match your selection.

CHOOSE YOUR SETUPMac

Use the PrismML fork's Metal build. 8 GB unified memory is tight; 16 GB is more comfortable.

Install the command-line tools

Open Terminal. If Git and the Apple developer command-line tools are not installed, run this command and complete the macOS installer that appears.

Git downloads for macOS

xcode-select --install

Create the working folder and get the PrismML runtime

Work inside a fresh folder in your home directory so the repository stays isolated. Clone the publisher demo (which carries the PrismML llama.cpp fork build scripts) and enter it. These commands do not delete anything.

PrismML Bonsai demo (fork build scripts and launchers) PrismML llama.cpp fork (prism branch)

cd ~
mkdir -p YouRunAI-Bonsai2
cd YouRunAI-Bonsai2
git clone https://github.com/PrismML-Eng/Bonsai-demo.git
cd Bonsai-demo

Install the runtime without default weights or the web UI

setup.sh builds/installs the fork's compatible llama.cpp binary. BONSAI_SKIP_GGUF=1 skips the demo's default PQ2_0 download so it cannot silently substitute a different package; BONSAI_SKIP_MLX=1 keeps this on the llama.cpp route; BONSAI_OPENWEBUI=0 and BONSAI_CODE_INTERPRETER=0 avoid extra components. The script runs setup, not a piped remote shell.

Bonsai demo environment variables reference PrismML fork backend support (Metal PTQ1_0 supported)

BONSAI_SKIP_GGUF=1 BONSAI_SKIP_MLX=1 BONSAI_OPENWEBUI=0 BONSAI_CODE_INTERPRETER=0 ./setup.sh

Download the exact PTQ1_0 package and its vision projector

The published repository exposes Ternary-Bonsai-2-27B-PTQ1_0.gguf (5.95 GB, dense-trit PTQ1_0) and the optional Q8_0 vision projector (0.63 GB). Download both into their own folder so the filename the launcher sees is unambiguous. PrismML's fork is required: upstream llama.cpp reports unknown types for PQ2_0/PTQ1_0.

Put these files in ~/YouRunAI-Bonsai2/Bonsai-demo/models/yourunai/official-ptq1/2 required files

models/yourunai/official-ptq1/

Ternary-Bonsai-2-27B-PTQ1_0.ggufSelected dense-trit PTQ1_0 language model (5.95 GB), resident in unified memory.Get file Ternary-Bonsai-2-27B-mmproj-Q8_0.ggufOptional Q8_0 vision projector (0.63 GB), loaded only for image input.Get file

Publisher repository: Ternary-Bonsai-2-27B-gguf

cd ~/YouRunAI-Bonsai2/Bonsai-demo
mkdir -p "models/yourunai/official-ptq1"
curl -fL --progress-bar "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/resolve/b072e1d3b35a0a630cece372c2127528e0994386/Ternary-Bonsai-2-27B-PTQ1_0.gguf?download=true" -o "models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf"
curl -fL --progress-bar "https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/resolve/b072e1d3b35a0a630cece372c2127528e0994386/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf?download=true" -o "models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf"

Start the local chat server on the selected package

BONSAI_GGUF pins the launcher to Ternary-Bonsai-2-27B-PTQ1_0.gguf, so it cannot fall back to another installed pack. BONSAI_CTX=4096 sets the active runtime context with documented headroom for the Metal build and KV cache on 8–16 GB unified memory (the publisher's 262,144-token limit is the model ceiling, not the active context; raise BONSAI_CTX in steps such as 8192 only while memory stays free). Leave this Terminal window open — the server process must keep running.

Bonsai demo start script reference (BONSAI_CTX and launchers)

cd ~/YouRunAI-Bonsai2/Bonsai-demo
BONSAI_CTX=4096 BONSAI_GGUF="models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf" BONSAI_MMPROJ="models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf" ./scripts/start_llama_server.sh

Confirm the server is the selected package

Open a second Terminal window and leave the server running in the first. The models endpoint reports the loaded model path; it must name Ternary-Bonsai-2-27B-PTQ1_0.gguf.

curl -s http://127.0.0.1:8080/v1/models
curl -s http://127.0.0.1:8080/health

Smoke-test the endpoint from the second Terminal

Send a short generation request to the local OpenAI-compatible route /v1/chat/completions. This is only the smoke test; the real conversation happens in the browser UI next.

curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{"messages":[{"role":"user","content":"What is 17 x 23? Show the arithmetic, then give the final number."}],"max_tokens":512,"temperature":1.0,"top_p":0.95,"top_k":20}'

Have a reusable chat in the browser UI

In the second Terminal, open the server's built-in chat page. In the message box type a known-answer question — for example, "What is 17 × 23? Show the arithmetic, then give the final number." — and press send. llama.cpp's web UI runs a real input-to-inference loop against the loaded PTQ1_0 weights. Verify the reply states 391 and matches the arithmetic shown, and confirm the assistant keeps accepting further turns in the same conversation. The model card's reasoning template defaults to xhigh effort, so allow it time to think before the final answer appears.

Publisher model card (reasoning defaults and context)

open http://127.0.0.1:8080

Troubleshoot Metal builds, memory pressure and context size

If the server exits during load on an M5-class Mac, set GGML_METAL_TENSOR_DISABLE=1 before relaunching. If generation is slow or the process is killed, lower BONSAI_CTX (for example to 2048) and restart; the same launcher honours the smaller value. An 8 GB Mac is tight for this package — close other memory-heavy applications, and prefer the 4096 starting context. Never point this launch at a PQ2_0 or Q2_0 file: this route runs the official PTQ1_0 pack only.

Backend/format support table (Metal PTQ1_0 supported)

cd ~/YouRunAI-Bonsai2/Bonsai-demo
GGML_METAL_TENSOR_DISABLE=1 BONSAI_CTX=2048 BONSAI_GGUF="models/yourunai/official-ptq1/Ternary-Bonsai-2-27B-PTQ1_0.gguf" ./scripts/start_llama_server.sh

SUCCESS CHECK · PTQ1_0

Send your first message

Open the local chat page in your browser. Confirm the server names the package you selected, then ask a short question and check that it responds.

Bonsai 2 requires the PrismML llama.cpp fork; stock llama.cpp and Ollama are not valid substitutes for these packed GGUF files. Memory guidance is estimated from package sizes and the supplied setup guide, not a YouRunAI benchmark. Longer context and image input need extra headroom.

Back to setup steps

When it doesn’t go to plan

The server says invalid ggml type or gives unusable output

Use the PrismML llama.cpp fork, not stock llama.cpp or Ollama, for these packed files.

The machine slows down or runs out of memory

Close other GPU-heavy applications and reduce the context size. On a 6 GB GPU, use partial CPU/RAM offload; the package file size is not total runtime memory.

Windows starts a different model

Use the package-specific launch path that names the exact GGUF file. The default PowerShell launcher searches a shared folder and can select a different installed model.

Windows selected the wrong GPU backend

Confirm your GPU vendor and that the PrismML build has the matching backend folder before launching. Use NVIDIA for CUDA, AMD for HIP/ROCm, and Intel Arc for Vulkan where that route exists.

Intel Vulkan cannot load PQ2_0 or gives incorrect output

Choose the PTQ1_0 package for Intel Arc or use the CPU route. PrismML has no native Vulkan kernels for PQ2_0. If PTQ1_0 still fails on your driver, use CPU and check the backend support notes.

REFERENCE LIBRARY

Sources and files

Original instructions, model files, and compatibility notes behind this setup.

8 SOURCES

01PrismML Bonsai Demo and runtimegithub.com 02Official PQ2_0 and PTQ1_0 fileshuggingface.co 03CRACK PQ2_0 files (third party)huggingface.co 04CRACK PTQ1_0 files (third party)huggingface.co 05PrismML backend and format supportgithub.com 06PrismML community measurementsgithub.com 07PrismML launcher optionsgithub.com 08PrismML llama.cpp forkgithub.com

The model behind this workflow

Bonsai 2 · 27B

Keep exploring

Text

SETUP GUIDE

Qwen3.5 · 9B private chat setup

Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.

Freshness 38 out of 100. Release age: 254 days old, Feb 27, 2026. Measures age, not quality.

Build a private coding assistant with Qwen3.8 27B

Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.

Freshness 69 out of 100. Repository age: 95 days old, Aug 5, 2026. Measures age, not quality.

Arena Open Source Web coding: rank 12 of 62. Ranked model: Qwen3.8-27B. Original model ranking, not a benchmark of this package. Export dated Nov 6, 2026. Source checked Nov 7, 2026. Human preference ranking, not hardware suitability.+3

SOFTWAREOllama + Visual Studio Code + Cline + Python

Create images locally with FLUX.2 Klein

Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.

Freshness 32 out of 100. Release age: 297 days old, Jan 15, 2026. Measures age, not quality.

SOFTWAREComfyUI

LOCALRENTED GPU

Verified source