Import the exact intermediate MiniCPM5 2B SFT weights into Ollama, connect VS Code/Cline, make a small local code change, and inspect the test result.
Estimated memory starting points for the exact publisher SFT BF16 package at 4K context. This is not the final post-trained MiniCPM5 2B package.
| Machine | MiniCPM5 2B SFT · publisher BF16 |
|---|---|
| NVIDIA GPU · 8 GB | Estimated short-context BF16 starting point; inspect actual offload and memory use. |
| NVIDIA GPU · 12 GB | Estimated short-context fit with additional headroom. |
| AMD Radeon · 8 GB | Estimated fit if Ollama supports your driver/backend; inspect GPU placement. |
| Intel Arc · 8 GB | Estimated fit with supported Ollama Vulkan backend; inspect GPU placement. |
| Apple Silicon Mac · 16 GB unified | Estimated 4K-context starting point with limited headroom. |
| Apple Silicon Mac · 24 GB unified | Estimated 4K-context fit with more headroom. |
| CPU-only · 16 GB RAM | Can load in RAM at short context; expect slower replies than a GPU. |
The Ollama import route is based on the publisher LlamaForCausalLM architecture and Ollama Safetensors documentation; it is source-verified, not physically benchmarked.
Import the publisher’s intermediate BF16 SFT weights into Ollama, connect VS Code/Cline and verify one small code change. Windows, Mac and Linux each have a complete local route.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Start with 4K context. BF16 weights occupy about 5 GB; Ollama runtime and the 4K KV cache need more. 8 GB GPU is a conservative starting estimate; a 12 GB GPU or 24 GB Mac provides more headroom. Increasing context raises memory use.
Choose where you want to run it. The instructions below update to match your selection.
Use the exact publisher SFT Safetensors and local Ollama on Windows. The final RL/OPD model is a separate checkpoint.
Install Ollama, VS Code and the Cline extension for windows. Start Ollama, then open a fresh PowerShell in an empty working folder. Install Python 3 first if your shell does not have py.
Ollama download VS Code download Cline extension Python downloadIn PowerShell run these commands in the empty folder. They create a virtual environment, download the publisher's BF16 SFT checkpoint with its config/tokenizer, and write a Modelfile pointing at that exact local directory. This checkpoint predates the publisher's final RL/OPD model.
Exact SFT weight source Hugging Face CLI download guidepy -m venv .venv
.\.venv\Scripts\python -m pip install -U huggingface_hub
.\.venv\Scripts\hf download openbmb/MiniCPM5-2B-SFT --local-dir MiniCPM5-2B-SFT
Set-Content -Path Modelfile -Value 'FROM ./MiniCPM5-2B-SFT'In the same PowerShell folder, import the Safetensors directory into Ollama and start this exact local model. Ollama documents direct Safetensors import from a directory.
Ollama Safetensors importollama create minicpm5-sft -f Modelfile
ollama run minicpm5-sftIn VS Code open a small local test repository. In Cline choose OpenAI-compatible provider, Base URL http://127.0.0.1:11434/v1, exact model ID minicpm5-sft, and starting context 4K. Keep automatic command approval off and review every proposed edit.
Ollama OpenAI-compatible endpointAsk Cline: “Create calculator.py with an add(a,b) function and one unit test that checks add(2,3)==5. Show the diff, run the test, and tell me the outcome.” Inspect the file and diff in VS Code, then run the test yourself before accepting a change.
If the output or tool use is weak, compare the publisher final MiniCPM5 2B release. This SFT checkpoint is useful for comparison and experimentation, but is not the publisher final post-trained package.
Publisher final modelCline proposes calculator.py and one test in your local workspace, you inspect the diff, and the add(2,3)==5 test passes when run locally.
The source model is an intermediate SFT checkpoint before the final publisher RL/OPD stage. Do not present its behavior as the final MiniCPM5 2B. The exact Ollama import has not been physically benchmarked by YouRunAI.
Use the exact publisher SFT directory, confirm all Safetensors/config/tokenizer files downloaded, create the Modelfile in its parent folder and repeat ollama create. The architecture is LlamaForCausalLM.
Confirm Ollama is open, http://127.0.0.1:11434/v1/models responds and the selected model ID is minicpm5-sft with a 4K context limit.
Inspect and correct the diff; this is the intermediate SFT checkpoint before publisher RL/OPD post-training. For routine work prefer the final MiniCPM5 2B package linked above.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.