Serve GLM-5.3 Flash · FP8 privately, connect VS Code and Cline, then review and test a real code change.
Complete checkpoint plus runtime and KV-cache capacity, not active parameters or a speed score.
| Machine | GLM-5.3 Flash · FP8 |
|---|---|
| 4× GB200 · at least 768 GB aggregate GPU memory | Documented starting class; begin at 16,384 tokens and confirm free memory. |
| Single RTX 5090 · 32 GB VRAM | Does not hold this full checkpoint in the documented server route; choose 4× GB200. |
| AMD GPU server | The NVIDIA command is unsupported on AMD; use the publisher’s separate 4× MI355X ROCm recipe. |
| Intel GPU server | No documented Intel serving route for this checkpoint; use the listed NVIDIA topology. |
| Apple Silicon Mac · 128 GB unified memory | No publisher native route for this package; use Mac as a client to 4× GB200. |
| CPU only · 128 GB RAM | No practical publisher CPU route; use 4× GB200 for this setup. |
The 4×MI355X AMD path needs the separate ROCm recipe; the NVIDIA command here is not interchangeable.
Run the exact checkpoint on a Linux GPU host, connect VS Code + Cline from your computer, and verify an approved code edit and test.
This setup uses the package documented for this task. Review its source and supported platforms before starting.
Save your machine in My Hardware to get an automatic starting choice. You can always choose any package yourself.
Start at 16,384 tokens on the listed GPU topology. Longer context raises KV-cache memory; lower context before changing model or renting more GPUs.
Choose your computer first, then its GPU path. The instructions below update to match your selection.
Windows: The publisher server recipe is a Linux accelerator deployment; use Windows as a private client over an SSH tunnel, not as an undocumented native inference host. Compatibility source
Mac: This publisher server recipe requires the listed Linux accelerator topology; use macOS as a private client or choose a different model for local Mac inference. Compatibility source
Use your own Linux server with 4× GB200 or compatible accelerator memory. Confirm the driver/runtime and storage before downloading the full GLM-5.3 Flash · FP8 checkpoint; active-parameter count is not a memory requirement.
Model weights and requirementsIn Linux Terminal, install Docker and pull the current vLLM CUDA nightly image required by the GLM-5.3 Flash recipe. A plain pip install is not the documented route for its sparse-MLA integration.
Publisher serving recipe Install Docker EngineIn Linux Terminal, select the publisher checkpoint and launch with the model-specific parser, GPU topology and a conservative 16,384-token context. Keep this terminal running; the model download can be large. Do not expose unauthenticated port 8000 to the public internet.
Exact official model repositoryIn a second Linux Terminal, check the model list at http://127.0.0.1:8000/v1/models. It must return the exact model ID used in the next step. For a remote client, tunnel port 8000 over SSH and keep the public port closed.
API response formatIn a second Linux Terminal, run the small read-only check against zai-org/GLM-5.3-Flash. This only proves that the server parser returns a tool call; it is not the agent setup or its final result.
Download the tool-call check Publisher tool-calling documentationInstall VS Code on your own Windows, Mac or Linux computer, then install the Cline extension from its official Marketplace listing. Keep approval required for file writes and terminal commands.
Download VS Code Install Cline extensionThrough an SSH tunnel, select Cline’s OpenAI-compatible provider with Base URL http://127.0.0.1:8000/v1, exact Model ID zai-org/GLM-5.3-Flash, and no more than 16,384 context. Check that the endpoint lists this model before asking it to edit files.
Cline provider settingsOpen a disposable repository in VS Code. Ask Cline to add a small button and a test for it. Review each proposed file write and terminal command before approval; inspect the diff, run the test, and open the app. A file-reading note summary does not count as a coding result.
Cline approval and provider setupKeep the tested code change or discard the disposable repository, then stop the GPU server or rental. If Cline cannot use its tools with this exact model/runtime, mark the route unverified instead of presenting it as working.
Cline made an approved change in the disposable repository; you inspected the diff, ran its test, and confirmed the feature works. A tool-call probe alone does not satisfy this outcome.
Publisher-source-backed setup, not a YouRunAI hands-on benchmark of this model in Hermes or Cline. Verify tool compatibility on the exact runtime before relying on it. Use the vLLM CUDA nightly image required by the linked recipe. For AMD, use its separate ROCm command rather than the NVIDIA tab.
Confirm every listed GPU is visible, check full weight plus KV-cache memory, start at 16,384 tokens, then use the publisher-specific container or build before renting a larger topology.
Check the exact parser flags and vLLM compatibility in the linked publisher recipe. Plain chat is not a passing agent result; keep the model off agent recommendations until a tool call works.
Keep the service bound to a private network and use an SSH tunnel to port 8000. Check /v1/models through the tunnel; do not open an unauthenticated public endpoint.
Original instructions, model files, and compatibility notes behind this setup.
Source-backed package and platform-route plan for private local chat with Qwen3.5 · 9B. Preserves the official Ollama 9B package, a smaller 4B family fallback, and a clearly labeled community uncensored GGUF option.
Run Qwen3.8 27B with Ollama, connect it to Cline in VS Code, and make one reviewable code change in a tiny local project.
Generate images from text prompts in ComfyUI with the fast distilled or more adjustable base 4B model. Save your results and reuse the visual workflow.