Run the publisher’s uncensored Gemma 4 E4B Q4 GGUF with llama.cpp, open the local browser chat, and check a real response.
Estimated capacity for the 5 GB Q4_K_M file plus runtime and a 4096-token context; these are not speed benchmarks.
| Machine | Gemma 4 E4B Uncensored · Q4_K_M |
|---|---|
| NVIDIA GPU · 8 GB VRAM | Possible with limited headroom; start at 4096 tokens. |
| AMD Radeon · 8 GB VRAM | Possible if your llama.cpp HIP or Vulkan backend works; start at 4096 tokens. |
| Intel Arc · 8 GB VRAM | Possible if your llama.cpp SYCL or Vulkan backend works; start at 4096 tokens. |
| Apple Silicon · 16 GB unified memory | Recommended starting point for local chat at 4096 tokens. |
| CPU only · 16 GB system RAM | Should fit in RAM at 4096 tokens; expect slower replies. |
Weight size alone is not the full memory requirement. Backend compatibility and context length can change the result.
Starting context for the exact Q4_K_M package; increase only if memory remains available.
| Machine | Gemma 4 E4B Uncensored · Q4_K_M |
|---|---|
| 8 GB GPU VRAM | Start at 4096 tokens; lower to 2048 if loading fails. |
| 16 GB unified memory or system RAM | Start at 4096 tokens; try 8192 only after a successful first chat. |
Install llama.cpp, load the exact publisher Q4_K_M GGUF, and talk to it through a local browser page. This is the shortest verified path for this package on Mac, Windows and Linux.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
The publisher has not provided a complete memory target for this task. Check its hardware guidance before starting.
Start with 4096 tokens on an 8 GB GPU or 16 GB unified-memory Mac. Longer context consumes more VRAM or RAM; lower it to 2048 if the first load fails.
Choose where you want to run it. The instructions below update to match your selection.
Use Terminal. The official llama.cpp package runs this publisher GGUF with Metal acceleration on Apple Silicon.
Open Terminal and install the current llama.cpp CLI through Homebrew. If Homebrew is absent, install it from the linked official site first.
Homebrew llama.cpp installer llama.cpp releasesbrew install llama.cpp
llama-server --versionIn Terminal, run the publisher-documented Hugging Face package through llama.cpp. The first run downloads the approximately 5 GB Q4_K_M GGUF. Keep this Terminal open while chatting. Start with a 4096-token context; raising it uses more unified memory.
Exact publisher package and run commandllama-server -hf HauhauCS/Gemma-4-E4B-Uncensored-HauhauCS-Aggressive:Q4_K_M -c 4096 --host 127.0.0.1 --port 8080Open http://127.0.0.1:8080 in a browser on the same computer. In the browser chat UI, type the message "What is 17 x 23? Answer with only the number" and confirm the reply is 391. Then send the follow-up question "Which two numbers did I multiply?" and check that the same chat remembers 17 and 23. You can keep sending messages in this local session.
If loading fails, close other memory-heavy apps and start again with -c 2048. When finished, press Ctrl+C in Terminal to stop the local server.
The browser page at http://127.0.0.1:8080 answers 17 × 23 with 391, remembers the follow-up, and the llama.cpp process is running on your machine.
This community variant is separately published from the official Gemma model. Its “uncensored” label comes from its publisher; this guide verifies the download and run path, not every claimed behavior. Text chat does not need a vision projector.
Open a new PowerShell or Terminal window, then check the installer output and the llama.cpp installation instructions.
Close other apps and rerun with -c 2048. Confirm the exact Q4_K_M package was selected before trying a larger quant.
Keep the server terminal running and open http://127.0.0.1:8080 on the same machine; check whether another program already uses port 8080.
Verify that your llama.cpp build and GPU driver support the appropriate CUDA, HIP, Vulkan, SYCL or Metal backend; CPU-only mode is slower.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.