Run the openbmb/MiniCPM5-2B-SFT model locally with llama.cpp, expose it through llama-server's OpenAI-compatible chat completions endpoint, and verify a complete request/response round-trip against the model. This setup targets Windows, macOS, and Linux and uses the publisher-documented server launch command and client request so you can build local agent or tool-using workflows on top of a working endpoint.
Run the openbmb/MiniCPM5-2B-SFT model locally with llama.cpp, expose it through llama-server's OpenAI-compatible chat completions endpoint, and verify a complete request/response round-trip against the model. This setup targets Windows, macOS, and Linux and uses the publisher-documented server launch command and client request so you can build local agent or tool-using workflows on top of a working endpoint.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
The publisher has not provided a complete memory target for this task. Check its hardware guidance before starting.
This setup uses llama.cpp. The exact checkpoint format, context, and runtime overhead determine its memory need; a weight-file size alone is not a GPU recommendation.
Pick your operating system. Every command below is for the selected package and runtime.
Clone the official llama.cpp repository and build it (or download a release build) so that the llama-server executable is available on your system. The build produces the llama-server binary you will use in the next step. Ensure the binary is on your PATH, or note its absolute path so you can run it directly.
Put the MiniCPM5-2B-SFT GGUF weights file named MiniCPM5-2B-F16.gguf into a directory you will launch from. This is the exact filename the server command expects, so keep the name unchanged.
Start llama-server on port 8080 with the documented MiniCPM5-2B-SFT command. This loads the F16 GGUF, sets the alias to MiniCPM5-2B, offloads 99 layers to the GPU (ignored on CPU-only runs), sets an 8192-token context, and enables the Jinja chat template used for chat completions.
llama-server -m MiniCPM5-2B-F16.gguf -a MiniCPM5-2B --port 8080 -ngl 99 -c 8192 --jinjaIn a second terminal, POST a JSON chat completion request to the local endpoint. The model name must match the alias MiniCPM5-2B used at launch, and the request includes the publisher-recommended sampling parameters for this model.
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "MiniCPM5-2B",
"messages": [{"role": "user", "content": "1+1=?"}],
"temperature": 1.0, "top_p": 0.95, "min_p": 0.0, "max_tokens": 256
}'The curl command must print a JSON body containing a choices array whose first element has a message.content string with the model's answer to the arithmetic question. Seeing only an HTTP 200 or an empty choices list means the round-trip did not complete and the endpoint is not yet ready for agent use.
Give the agent a small bounded task. Confirm the tool request, its result, and the final response all complete through the local endpoint.
This is a source-linked setup, not a YouRunAI hardware test. Confirm your exact runtime version, package, and output before relying on it.
Lower min_p in the request body to 0.0 as recommended by the publisher; if repetition continues, also add "repetition_penalty": 1.05 to the request.
Make sure the model value in the chat completion request exactly matches the alias MiniCPM5-2B passed to llama-server with the -a flag, and that llama-server is listening on port 8080 as shown in the launch command.
Run the server from the directory that contains MiniCPM5-2B-F16.gguf, or supply the full path to that file in place of the relative name in the command.
Start llama-server first in its own terminal and wait for it to report that it is listening, then run the curl command from a second terminal so the endpoint is reachable on localhost:8080.
Original instructions, model files, and compatibility notes behind this setup.
Save your machine to see a personalized rating and its reasoning.
Add my hardwareInstall a local runtime, run Qwen3.5 9B, confirm responses, and know when to choose the smaller 4B package.
Connect an open coding-capable model in Ollama to Cline, run a small repository task, and review the result locally.
Use the official FLUX.2 Klein 4B ComfyUI template with exact model files, a first prompt, and an output check.