Set up MiniCPM5-1B on SGLang for Linux and confirm tool calling works end-to-end by sending a chat completion request that returns a model response. SGLang is the recommended backend for tool/function calling with this model.
Set up MiniCPM5-1B on SGLang for Linux and confirm tool calling works end-to-end by sending a chat completion request that returns a model response. SGLang is the recommended backend for tool/function calling with this model.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
The publisher has not provided a complete memory target for this task. Check its hardware guidance before starting.
This setup uses SGLang. The exact checkpoint format, context, and runtime overhead determine its memory need; a weight-file size alone is not a GPU recommendation.
Pick your operating system. Every command below is for the selected package and runtime.
Use a Linux host with a working Python and pip installation. The model is a dense 1B Transformer intended for local, resource-constrained deployment, and SGLang is the recommended backend for its tool calling. Install the SGLang package with SRT support.
Install the SGLang runtime including the SRT components at the documented minimum version.
pip install "sglang[srt]>=0.5.12"Start the SGLang OpenAI-compatible server for the exact model identifier openbmb/MiniCPM5-1B on port 30000. Leave this running in the foreground or in its own terminal.
python -m sglang.launch_server --model-path openbmb/MiniCPM5-1B --port 30000From another terminal, send an OpenAI-compatible chat completion to the running server. This produces an actual model response and confirms the server is serving the model. Inspect the returned JSON: it must contain choices with a message and generated content from openbmb/MiniCPM5-1B. curl http://localhost:30000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "openbmb/MiniCPM5-1B", "messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}], "max_tokens": 128, "temperature": 0.7 }'
curl http://localhost:30000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "openbmb/MiniCPM5-1B",
"messages": [{"role": "user", "content": "Who are you? Please briefly introduce yourself."}],
"max_tokens": 128,
"temperature": 0.7
}'Once the basic response check passes, keep the same SGLang server and send a tool/function-calling style request to the OpenAI-compatible endpoint, including a tools array with one simple function and a user message that requires it. Confirm the model reply contains a tool call (for example, a tool_calls entry naming your function and arguments) rather than only plain text. This exercises the model's tool-use behavior, the task MiniCPM5-1B is highlighted for, without needing any external tool server.
If the server fails to start, verify the SGLang version installed is at least 0.5.12, that the port 30000 is free, and that your GPU driver and Python environment are functional. If model loading fails, confirm network access to the Hugging Face model repository so openbmb/MiniCPM5-1B can be downloaded, or pre-download the repository and point --model-path at the local directory.
If you get empty or repetitive generations, retry with the sampling values commonly recommended for MiniCPM5: temperature=1.0, top_p=0.95, and min_p=0.0; if repetition persists, add repetition_penalty=1.05. Note that support for these sampling parameters varies by inference framework.
Give the agent a small bounded task. Confirm the tool request, its result, and the final response all complete through the local endpoint.
This is a source-linked setup, not a YouRunAI hardware test. Confirm your exact runtime version, package, and output before relying on it.
Confirm SGLang is installed at >=0.5.12 with SRT support, that no other process uses port 30000, and that the host has a compatible GPU and Python environment.
Ensure the machine can reach the Hugging Face repository for openbmb/MiniCPM5-1B, or pre-download the repository and pass its local directory to --model-path.
Retry with temperature=1.0, top_p=0.95, min_p=0.0; if repetition continues, add repetition_penalty=1.05. Parameter support varies by inference framework.
Use the SGLang server on port 30000 (recommended for tool calling), keep the thinking mode settings aligned with the documented chat template behavior, and check that the request includes a valid tools array and a user message that clearly requires the tool.
Original instructions, model files, and compatibility notes behind this setup.
Save your machine to see a personalized rating and its reasoning.
Add my hardwareInstall a local runtime, run Qwen3.5 9B, confirm responses, and know when to choose the smaller 4B package.
Connect an open coding-capable model in Ollama to Cline, run a small repository task, and review the result locally.
Use the official FLUX.2 Klein 4B ComfyUI template with exact model files, a first prompt, and an output check.