Workflows/Build a local tool-using agent with Bonsai 2 · 27B

Agents / WORKFLOW

Build a local tool-using agent with Bonsai 2 · 27B

Source-backed metadata for a local agent setup using Bonsai 2 · 27B with PrismML’s llama.cpp fork and Hermes Agent. Choose the smallest official PTQ1_0 pack, a community abliterated PTQ1_0 alternative, or the official PQ2_0 pack used in PrismML’s agent demo.

EstimatedSource checked 9/29/2026

LOCALRENTED GPU

Freshness 82 out of 100. Release age: 53 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,688,388 downloads in 30 days; 2,960 lifetime likes. Checked Nov 8, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

A new release starts at 100. The score halves every 180 days: 100 × 2^(−age in days / 180). When the release date is unavailable, a repository date is a fallback—not proof of when the model was released. Missing dates stay unscored.

Check the date source

Popularity · adoption

Downloads over the past 30 days carry 80% of the score; lifetime likes carry 20%. Both use a logarithmic scale, capped at 10 million downloads and 100,000 likes. If likes are unavailable, downloads carry all the weight. Missing download statistics stay unscored.

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 9, 2026. Statistics checked Nov 8, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

Before you begin

Hardware

Weight sizes and backend routes come from publisher/variant sources. Context values are starting estimates, not measured benchmarks; agent tool traces need extra memory.

Machine	Bonsai 2 PTQ1_0	Bonsai 2 PQ2_0	Bonsai 2 Abliterated PTQ1_0
NVIDIA GPU 24 GB VRAM (RTX 4090)	CUDA route; 5.95 GB weights plus estimated KV overhead; 64K start. PrismML fork required.	CUDA route; 7.21 GB weights plus estimated KV overhead; 64K start. No Vulkan path.	CUDA route; same 5.95 GB PTQ1_0 size; community pack, validate and keep thinking off.
AMD GPU 24 GB VRAM (RX 7900 XTX)	ROCm/HIP route; 5.95 GB weights plus estimated KV overhead; 64K start.	ROCm/HIP route; 7.21 GB weights plus estimated KV overhead; 64K start. No Vulkan path.	ROCm/HIP route; community PTQ1_0, validate agent behavior and keep thinking off.
Intel Arc GPU 12 GB VRAM (B580)	Vulkan PTQ1_0 route; 5.95 GB weights plus estimated KV overhead; 32K start; driver caveats.	No native Vulkan PQ2_0 kernels; use CPU fallback or official PTQ1_0 for Arc Vulkan.	Vulkan PTQ1_0 route; community pack, validate and keep thinking off; driver caveats.
Apple Silicon Mac 32 GB unified memory	Metal route; 5.95 GB weights plus estimated unified-memory overhead; 64K start.	Metal route; 7.21 GB weights plus estimated unified-memory overhead; 64K start.	Metal route; community PTQ1_0, validate agent behavior and keep thinking off.
CPU-only x86 system 32 GB RAM	CPU route; 5.95 GB weights plus estimated RAM overhead; 64K start, but slower.	CPU route; 7.21 GB weights plus estimated RAM overhead; 64K start, expect slow generation.	CPU route; community PTQ1_0, validate and keep thinking off; expect slower generation.

Context window

Recommended starting context values are estimates for agent use; allow headroom for tool traces and long outputs.

Machine	Bonsai 2 PTQ1_0	Bonsai 2 PQ2_0	Bonsai 2 Abliterated PTQ1_0
NVIDIA GPU 24 GB VRAM (RTX 4090)	65536; allow estimated KV/context headroom for agent traces.	65536; allow estimated KV/context headroom for agent traces.	65536; allow estimated KV/context headroom; validate agent behavior.
AMD GPU 24 GB VRAM (RX 7900 XTX)	65536; allow estimated KV/context headroom for agent traces.	65536; allow estimated KV/context headroom for agent traces.	65536; allow estimated KV/context headroom; validate agent behavior.
Intel Arc GPU 12 GB VRAM (B580)	32768; allow estimated KV/context headroom; driver caveats apply.	Unsupported via Arc Vulkan; use CPU fallback or official PTQ1_0.	32768; allow estimated KV/context headroom; validate agent behavior.
Apple Silicon Mac 32 GB unified memory	65536; allow estimated unified-memory headroom for agent traces.	65536; allow estimated unified-memory headroom for agent traces.	65536; allow estimated unified-memory headroom; validate behavior.
CPU-only x86 system 32 GB RAM	65536; allow estimated RAM headroom; CPU generation is slower.	65536; allow estimated RAM headroom; CPU generation is slower.	65536; allow estimated RAM headroom; validate behavior; CPU is slower.

LOCAL AGENT / WINDOWS · MAC · LINUXSOURCE-LINKED SETUP

Build a local tool-using agent with Bonsai 2 · 27B

Connect Bonsai 2 · 27B to Hermes Agent through PrismML’s llama.cpp fork and verify a complete tool round-trip. Choose the smallest official PTQ1_0 pack, a community abliterated PTQ1_0 alternative, or the official PQ2_0 pack used in PrismML’s agent demo.

Choose a package Run its commands Check the result

BEFORE YOU BEGIN

Choose your package

Select the weights you want. The instructions, downloads and settings change with your choice.

The publisher has not provided a complete memory target for this task. Check its hardware guidance before starting.

CURRENT SETUP

Bonsai 2 PTQ1_0 · official

Language model alone is 5.95 GB. Add memory for the agent context and tool traces. Requires the PrismML llama.cpp fork.

Freshness 82 out of 100. Release age: 53 days old, Sep 17, 2026. Measures age, not quality.

Popularity 85 out of 100. 1,688,388 downloads in 30 days; 2,960 lifetime likes. Checked Nov 8, 2026. Measures adoption, not quality.

What do these scores mean?

Two ways to compare models. Neither score measures answer quality, benchmark performance, or whether a model fits your machine.

Freshness · release age

Check the date source

Popularity · adoption

round(80 × min(1, log10(1 + downloads) / 7) + 20 × min(1, log10(1 + likes) / 5))

Check the statistics source

Scores as of Nov 9, 2026. Statistics checked Nov 8, 2026. Statistics are marked for refresh after 72 hours.Full scoring method

View weight source

THE WORKFLOW

Set up Bonsai 2 PTQ1_0 on your machine

Choose where you want to run it. The instructions below update to match your selection.

CHOOSE YOUR SETUPMac

Use the publisher macOS build and Metal backend; agent workloads can need substantial unified memory.

Prerequisites

Apple Silicon Mac (M-series recommended); verify Metal support. Open Terminal.

Bonsai 2 · 27B model card

Install runtime

Use PrismML llama.cpp fork; build with Metal. Clone fork and compile. Keep Terminal open.

PrismML llama.cpp fork

cd ~/YouRunAI-Bonsai-PTQ1_0-Agent
git clone -b prism https://github.com/PrismML-Eng/llama.cpp.git
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build -j$(sysctl -n hw.ncpu)
ls build/bin/llama-cli build/bin/llama-server

Download exact PTQ1_0 weight and tokenizer

Download only the official PTQ1_0 GGUF for Bonsai 2 · 27B. Destination folder: models.

Put these files in ~/YouRunAI-Bonsai-PTQ1_0-Agent/models/1 required file

models/

Ternary-Bonsai-2-27B-PTQ1_0.ggufLanguage model weightsGet file

PTQ1_0 GGUF

mkdir -p models
cd models
curl -L -o Ternary-Bonsai-2-27B-PTQ1_0.gguf https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/resolve/b072e1d3b35a0a630cece372c2127528e0994386/Ternary-Bonsai-2-27B-PTQ1_0.gguf
ls -lh Ternary-Bonsai-2-27B-PTQ1_0.gguf

Launch llama-server with Metal

Start PrismML llama-server with PTQ1_0, full GPU offload, and starting context 65536 for agent traces. Keep this server running. Open a second Terminal for client.

cd ~/YouRunAI-Bonsai-PTQ1_0-Agent/llama.cpp
./build/bin/llama-server -m ../models/Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 -c 65536 --host 127.0.0.1 --port 8080 --jinja --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.05

Install and configure Hermes Agent client

Install Hermes native macOS app. Configure custom endpoint to local llama-server. Use exact model ID from llama-server log. Set context to match active 65536.

Hermes installation Hermes FAQ Hermes local Ollama setup

cd ~/YouRunAI-Bonsai-PTQ1_0-Agent
hermes model
hermes --version

Run bounded agent task

In Hermes, use a disposable workspace. Ask agent to create a file and run a command. Do not disable approval controls. Confirm tool call appears and result is returned.

mkdir -p workspace
cd workspace
hermes

Inspect tool round-trip and result

After agent completes, inspect created file and llama-server log. Verify the final answer reflects the tool result. Lower context if memory pressure occurs: stop server, relaunch with -c 32768 or -c 16384.

ls -la
cat created-file.txt
tail -n 40 ../llama.cpp/server.log

SUCCESS CHECK · Bonsai 2 PTQ1_0

Complete a tool round-trip

Hermes connects to the local OpenAI-compatible endpoint, the model returns a tool call, the client sends the tool result back, and the final answer reflects that result. Check the llama-server log for the tool-call exchange.

Do not disable approval controls just because a publisher demo does. The agent can run model-selected commands as your user; use a disposable workspace. The community abliterated PTQ1_0 pack is not publisher-supported and has reported thinking-mode issues; keep thinking off for agent use.

Back to setup steps

When it doesn’t go to plan

The agent cannot connect to the model server

Confirm the PrismML llama-server is running on the documented local port and that Hermes’ Custom Endpoint points to the /v1 URL with the exact served model ID and supported context.

The server fails to load the selected GGUF

Use the PrismML llama.cpp fork, not stock llama.cpp, and load the exact PTQ1_0 or PQ2_0 file. PQ2_0 also lacks native Vulkan kernels, so use CUDA, ROCm/HIP, Metal, or CPU for that pack.

The machine runs out of memory during a long agent session

Reduce the agent context size or switch to the smaller PTQ1_0 package. Agent traces and browser tools can need substantially more memory than short chat.

REFERENCE LIBRARY

Sources and files

Original instructions, model files, and compatibility notes behind this setup.

10 SOURCES

01PrismML model card and package fileshuggingface.co 02Bonsai demo runtime and scriptsgithub.com 03Bonsai and Hermes agent walkthroughgithub.com 04Bonsai tool-calling and MCP guidegithub.com 05PrismML llama.cpp forkgithub.com 06Bonsai backend supportraw.githubusercontent.com 07Hermes Windows native installationhermes-agent.nousresearch.com 08Hermes installation guidehermes-agent.nousresearch.com 09Hermes local model setuphermes-agent.nousresearch.com 10BoldingBuilds abliterated PTQ1_0 packagehuggingface.co

The model behind this workflow

Bonsai 2 · 27B

Keep exploring

Text

SETUP GUIDE

Qwen3.5 · 9B private chat setup

Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.

Freshness 37 out of 100. Release age: 255 days old, Feb 27, 2026. Measures age, not quality.

Build a private coding assistant with Qwen3.8 27B

Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.

Freshness 69 out of 100. Repository age: 96 days old, Aug 5, 2026. Measures age, not quality.

Arena Open Source Web coding: rank 12 of 62. Ranked model: Qwen3.8-27B. Original model ranking, not a benchmark of this package. Export dated Nov 6, 2026. Source checked Nov 8, 2026. Human preference ranking, not hardware suitability.+3

SOFTWAREOllama + Visual Studio Code + Cline + Python

Create images locally with FLUX.2 Klein

Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.

Freshness 32 out of 100. Release age: 298 days old, Jan 15, 2026. Measures age, not quality.

SOFTWAREComfyUI

LOCALRENTED GPU

Verified source