Qwen3.8-Flash-Next · Strata
Run Qwen3.8-Flash-Next through Strata, an inference runtime that shares model experts across GPU memory, system RAM, and storage. Start with its reduced-expert Coder derivative for a low-VRAM coding assistant, or choose a fuller model pack when you have more RAM.
What do you want to do?
Choose a task. Get the software, files, settings, and steps to run it.
Open a guide for its software, hardware requirements, and setup steps.
At a glance
- Parameters
- 125B base · ~6B active; Coder removes half the experts
- Architecture
- Mixture of experts; Strata GPU expert cache + RAM/SSD
- Context length
- 262,144 native · begin at 8K–16K on constrained hardware
- License
- Qwen Community License 1.0 (base); MIT (Strata runtime); derivative terms apply
- Software
- Strata
Will it run on your machine?
Save your machine to see a personalized rating and its reasoning.
Add my hardwareModel loaded. Remote chat and coding still need validation.
Coder IQ1_M reportedly loaded, answered a local /health request, and displayed Chat / Monitor / About through Jupyter. Submitting browser chat then returned a Strata host/origin 403. The prescribed origin fix, a successful SSH tunnel, and a complete Cline task have not been confirmed.
October 9, 2026 · Reported by the user in supplied installation notes. YouRunAI has not independently reproduced this deployment or completed the remaining tests.
Read the reported resultsOne model family. A different way to run it.
Qwen3.8-Flash-Next is the underlying model. Strata is the inference runtime. Coder is a derivative that retains 256 of the original 512 experts; it trades broader model capability for a smaller expert working set.
The original open-weight base and its capabilities.
Moves expert weights between your GPU, RAM, and SSD.
Pick the right memory and capability tradeoff.
- Base model
- Qwen3.8-Flash-Next · open weights
- Parameter accounting
- 125B base, ~6B active per token; the publisher also lists 51B n-gram embeddings and 4B MTP parameters separately.
- Memory floor
- 12 GB supported GPU recommended; 32 GB system RAM for Coder. 8 GB VRAM is experimental and slow.
- Operating systems
- Windows 10/11 · Ubuntu 22.04/24.04 · compatible Linux GPU Pod
- Disk planning
- Coder: 58.4 GB pack + about 6 GB draft files. Allow 120 GB SSD for downloads, cache, and working room; larger full packs and converted files need more.
- Interfaces
- Browser chat · OpenAI-compatible Chat Completions and Responses · Anthropic-compatible Messages
- Deployment evidence
- User-reported Coder load, local health, and dashboard access. Browser chat returned 403; origin fix and Cline task not confirmed.
What would you like to do?
Choose Coder or a full pack, start Strata, connect Cline, and review a small code change.
Use local browser chatUse Windows or Linux locally; remote browser access has separate proxy and origin checks.
Try a rented GPUFollow the reported deployment and the remaining browser/API checks in the RunPod tab.
Compare the evidenceSee publisher measurements separately from the partial RunPod report and tuning results.
Start with Coder when RAM is tight.
Coder is the first choice for a 32 GB RAM coding test. Full packs retain the complete expert set and need more system memory.
| Package | System RAM | Download | Choose it for |
|---|---|---|---|
| Coder · reduced experts | 32 GB | 58.4 GB | First coding test on a 12–16 GB GPU family coder / IQ1_M; 256 of 512 experts retained and a ~23.4 GB expert arena in installer metadata. General/multilingual quality can differ. Low-RAM mode can read experts from SSD. |
| Full Q2_0 | 48 GB | 66.4 GB | Speed-oriented full-model testing Do not assume a 32 GB RAM + small-GPU machine fits. Some CPU optimizations create about 40 GB of extra files. |
| Full IQ2_XS | 48–64 GB | 68.0 GB | Balanced general-purpose full-model setup The full-family default is not the reduced-expert Coder pack. |
| Full IQ3_XXS | 64 GB | 75.8 GB | More precision with more available RAM Installer recommends at least 60 GB RAM; 64 GB is the practical configuration tier. |
| Full IQ3_S | 64 GB+ | 83.6 GB | Higher precision among these standard packs Installer recommends at least 62 GB RAM. Leave system headroom; publisher output speed is lower. |
Download sizes are Strata installer estimates for a selected pack and exclude roughly 6 GB of draft-model files. Compression choice, created optimization files, context, and GPU cache allocation change storage and memory use. These are not speed guarantees.
Package documentationCheck both VRAM and RAM.
These are starting configurations for an offloading runtime. The model does not live entirely inside a 12–16 GB GPU.
Coder load, local health, and the dashboard were reported successful. Browser chat and Cline remain unvalidated. Start at 8K–16K with images off; the actual context in the reported run was not confirmed. This Pod is slightly below the documented 32 GB RAM class.
| Machine | GPU VRAM | System RAM | Starting point |
|---|---|---|---|
| NVIDIA · supported consumer GPU | 12 GB | 32 GB | Coder: Candidate with low-RAM mode Full: Choose more RAM first Context: 8K–16K Estimated from runtime documentation |
| RunPod · RTX 2000 Ada · 6 vCPU | 16 GB | ~31 GB | Coder: User reported engine load, local health, and dashboard access Full: No full-pack result reported on this Pod Context: 8K–16K suggested; observed setting not confirmed Partial deployment; browser chat and Cline not validated |
| NVIDIA · supported consumer GPU | 16 GB | 64 GB | Coder: Documented memory class Full: IQ2_XS candidate Context: 16K–32K Estimated from runtime documentation |
| NVIDIA · supported consumer GPU | 24 GB | 32 GB | Coder: Candidate; check available RAM Full: Conditional; inspect cache and RAM allocation Context: 16K–32K Estimated; runtime allocation must be checked |
| NVIDIA · RTX 5090 | 32 GB | 32 GB | Coder: Candidate; check available RAM Full: Q2_0 is conditional on offload and available RAM Context: 16K–32K Estimated; not a speed proxy for the lower-end Pod |
| NVIDIA · supported consumer GPU | 12–24 GB | 64 GB | Coder: Documented memory class Full: Recommended class for IQ2_XS Context: 32K Selected cards have publisher measurements |
| AMD · supported Radeon GPU | 12–16 GB | 32 GB | Coder: Conditional on supported card and backend Full: Choose more RAM first Context: 8K–16K Estimated; check the installer’s AMD support |
| Intel GPU · experimental source build | Backend dependent | Package requirement still applies | Coder: Not a standard prebuilt setup in this guide Full: Validate the experimental backend before selecting a pack Context: Validate on your build Experimental source-build route; no fit claim |
| Apple Silicon | Unified memory | Shared pool | Coder: No official native Strata inference route Full: Use Mac as a client to a remote Strata host Context: Remote server setting No Mac inference support claimed |
VRAM and system RAM are separate requirements. These context values are conservative starting points, not tested maxima. Longer contexts use additional cache memory and can reduce the experts kept on the GPU. A fast local NVMe SSD and 64 GB RAM may behave very differently from a low-RAM cloud Pod.
Performance: keep the evidence separate.
Strata publisher results: RTX 5070 12 GB, Ryzen 5 7600, and 64 GB system RAM. Output figures are short-chat generation rates; prompt figures use a 32K prompt. These are not the RTX 2000 Ada / ~31 GB RAM RunPod results and were not measured by YouRunAI. Neither column is total coding-task time.
| Package | Short-chat output tokens / second | 32K prompt processing tokens / second |
|---|---|---|
| Full Q2_0 | 94 | 2,650 |
| Full IQ2_XS | 79 | 2,090 |
| Full IQ3_XXS | 62 | 1,750 |
| Full IQ3_S | 53 | 1,620 |
| Coder | 55 | 2,180 |
What the RunPod report actually shows
RunPod RTX 2000 Ada · 16 GB VRAM · ~31 GB system RAM · 6 vCPU
Reported by the user in supplied installation notes. YouRunAI has not independently reproduced this deployment or completed the remaining tests.
| Check | Reported result | What it establishes |
|---|---|---|
| Coder expert loading | 23.42 GiB reported in RAM | The engine reached expert loading; this is not a full-model-in-VRAM measurement. |
| GPU expert cache | 4,733 experts · 9.01 GiB VRAM | The runtime populated its expert cache; total system memory remains separate. |
| PCIe-share calibration | 19.0 tokens/s median · share 1 | A tuning-stage candidate result, not sustained serving throughput. |
| Draft-floor calibration | 22.6 tokens/s median · floor 0.7 | Another tuning-stage candidate result, not a completed chat benchmark. |
| Local GET /health | Reported successful | The local process responded; remote browser/API access requires separate checks. |
| Jupyter-proxied dashboard | Chat / Monitor / About displayed | The dashboard loaded through authenticated Jupyter after proxy setup and restart. |
| Browser chat submission | 403 host/origin rejection | The UI loaded, but its chat request was rejected. Fix success after restart is unconfirmed. |
| Cline tool-using task | Not confirmed | No completed agent-task or external Cline performance claim can be made. |
19.0 and 22.6 tokens/s came from different calibration phases. They must not be quoted as final sustained output speed. Calibration was interrupted; completion and saved tuning were not confirmed. Obtain three identical warmed serving runs and a real Cline task before upgrading this evidence.
Browser chat: local and remote access differ.
Follow the Windows or Linux route below, then open http://127.0.0.1:8080 on that computer. Start with a short text prompt. Check Strata’s Monitor before increasing context or connecting an agent.
On the reported 8888-only Pod, the dashboard loaded through Jupyter at /proxy/8080/ after installing its proxy extension and restarting Jupyter. Chat then returned a separate Strata 403. The workflow covers that exact sequence and the unconfirmed origin repair; dashboard access alone is not a successful chat test.
Windows
Native PowerShell + START-HERE.bat. Select Coder and confirm a compatible NVIDIA or AMD driver. The copyable quick-start commands are in “Ways to run it” below.
Windows installation sourceUbuntu Linux
Terminal + setup.sh. Use your normal user with sudo locally; the workflow gives separate root-container commands for RunPod. Confirm the GPU backend before launch.
Linux installation sourceOne server, several compatible APIs.
Read the actual model ID from /v1/models and keep Cline’s Base URL ending in /v1, without /chat/completions. Match its context to the active server setting. Browser dashboard access and an external coding-agent API are separate connections.
Same-machine browser + Cline
http://127.0.0.1:8080/v1Use / for browser chat and /v1 for Cline. Use the Strata key if configured; a local placeholder only applies to an unauthenticated localhost server.
RunPod Jupyter browser dashboard
https://POD_ID-8888.proxy.runpod.net/proxy/8080/Browser route only: Jupyter Server Proxy, a Jupyter-only restart, and private token/cookie login are needed. Dashboard access was reported, but browser chat still returned a Strata 403. Do not paste this UI URL into Cline.
Cline → existing Pod via SSH
http://127.0.0.1:18080/v1Set up the SSH key and use the exact host, port, and user from RunPod Connect. Forward desktop port 18080 to Pod localhost:8080 and keep the tunnel open. This connection was not demonstrated in the reported deployment.
Cline → new Pod with exposed 8080
https://POD_ID-8080.proxy.runpod.net/v1Only works when HTTP 8080 is actually exposed and Strata has the correct bind plus a long API key. Do not redeploy the existing 8888-only Pod without backing up its files.
Replace POD_ID and every other uppercase placeholder with your own values. The SSH hostname comes from RunPod Connect, not the Jupyter web address. No successful external Cline connection was demonstrated in the supplied report.
| Method | Endpoint | What it does |
|---|---|---|
| POST | /v1/chat/completions | Chat, streaming, and tool calls |
| POST | /v1/responses | Responses-compatible clients; confirm client behavior with a short task |
| POST | /v1/messages | Anthropic-compatible client adapter |
| GET | /v1/models | Read the actual served model ID before configuring Cline |
| GET | /health | Check the local server; this does not prove browser chat or a remote coding client works |
| GET | /v1/status | Read runtime status; /status is also documented for the reviewed version |
| GET | /metrics | Inspect runtime measurements |
A running local server does not create an external HTTP link. The reported Pod exposed Jupyter 8888 only.
Its dashboard proxy requires Jupyter login. A browser cookie does not automatically authenticate Cline.
API keys and browser host/origin rules are checked separately. A loaded dashboard can still return 403 when chat submits.
Why the dashboard can load while chat returns 403
For Strata’s own UI behind the Jupyter proxy, the upstream trusted_origins setting accepts the exact HTTPS origin, for example https://POD_ID-8888.proxy.runpod.net, in the actual model config. allowed_hosts uses a hostname; cors_origins has a different role for browser API calls. Back up the config, use only the exact trusted origin, and restart Strata only. The supplied report did not confirm that this repair succeeded. The workflow gives the full recovery sequence.
Open the complete RunPod troubleshooting pathHow the offloading works
- 01
A sparse model activates selected experts for each token. This reduces computation, but all required expert weights still need to be stored somewhere.
- 02
Strata keeps a working set of experts on the GPU and uses system RAM or disk-backed storage for the rest. Its draft-token decoding and prompt processing depend on the selected pack and runtime settings.
- 03
GPU capacity, RAM bandwidth, CPU, SSD reads, prompt length, context, and compression all affect the result. A large model can be runnable without being fast on every low-VRAM machine.
What to record when testing your own machine
Record the Strata commit, family and pack, GPU, CPU, system RAM, disk, actual context/KV/reasoning settings, prompt/output length, cold/warmed state, time to first token, and three serving prefill/decode measurements. Then record Cline model ID, real tool calls, reviewed changes, tests, retries, total task time, the access path used, and unresolved origin/auth/streaming errors. Keep calibration candidates separate.
Exact source revision used for this guideBest for
- Run a local coding assistant in VS Code and Cline with an OpenAI-compatible model endpoint.
- Try large sparse-model inference on a supported 12–16 GB GPU when your system RAM and SSD can support expert offload.
- Use Strata’s browser chat on a Windows or Linux computer with local model files.
- Compare cold and warmed prompt processing, output speed, and complete agent-task time on your own hardware.
Tradeoffs
- Strata is a runtime route for Qwen3.8-Flash-Next. Coder is a reduced-expert IQ1_M-labelled pack with 256 of 512 experts retained, not the full base checkpoint or an ordinary one-bit full-model quantization.
- A 16 GB GPU does not hold all model weights. Low-RAM mode can keep remaining experts in system RAM after GPU caching, or read them from SSD when memory is tighter. The actual allocation and I/O pattern determine performance.
- Intel GPU support is an experimental source-build path in the reviewed runtime. It is not the standard prebuilt installation documented in this guide.
- Apple Silicon has no official supported Strata inference installation path in the reviewed version. A Mac can connect as a client to a remote server.
- The runtime uses a single active generation request by default. Multiple coding agents can queue and increase response latency.
- OpenAI-compatible API syntax does not guarantee every Cline tool or streaming feature. Complete a small task before relying on a long agent session.
- The reported RunPod deployment loaded the model and dashboard, but browser chat returned an origin/host 403. Successful application of the origin fix and a complete Cline task have not been confirmed.
- A Jupyter dashboard URL has its own login and proxy path; it is not automatically a valid Cline Base URL. Use a configured SSH tunnel or directly exposed, authenticated Strata API.
- Calibration is optional. Its 15–30 minute estimate begins after downloads and can take longer on six vCPUs. Calibration candidate rates are not sustained serving throughput.
- A rented GPU is remote infrastructure. Protect the public endpoint with a key, and do not treat it as on-device privacy.
Ways to run it
Strata · Windows · NVIDIA or AMD
Run in PowerShell. Choose Coder IQ1_M, start with 8K–16K context, and turn images off for a 32 GB RAM first test. Check the required GPU driver. Browser UI: http://127.0.0.1:8080. These are source-documented instructions, not a completed Windows hardware test.
git clone https://github.com/Niko1221/Strata.git
cd Strata
git checkout fb58e0dbc8399662c0e47c76578c6e878b14f6cf
.\START-HERE.batStrata · Ubuntu Linux · NVIDIA or AMD
Use Ubuntu Terminal. Before running the block, optionally start a separate session with tmux new -s strata. Then select family coder and its IQ1_M pack, 8K–16K context, and images off. tmux keeps a long download or server session alive after the terminal disconnects; use tmux attach -t strata to return. Confirm RAM and the GPU backend. Keep the server local unless you configure authenticated remote access.
sudo apt-get update
sudo apt-get install -y git curl tmux
git clone https://github.com/Niko1221/Strata.git
cd Strata
git checkout fb58e0dbc8399662c0e47c76578c6e878b14f6cf
./setup.shStrata · RunPod · Linux NVIDIA
The user-reported RTX 2000 Ada 16 GB / ~31 GB RAM / 6 vCPU run reached Coder load, local /health, and the Jupyter-proxied dashboard. The Pod exposed only Jupyter HTTP 8888; direct external 8080 was unavailable. Browser chat returned an origin/host 403. Follow the complete workflow for recovery, a Jupyter-only restart, precise trusted-origin settings, and separate SSH-tunnel Cline access at localhost:18080/v1. Those final fixes and a complete agent task remain unconfirmed.
Official instructionsStrata Docker · Linux NVIDIA · optional advanced route
A secondary route for an existing Docker + NVIDIA Container Toolkit host. Follow the pinned installer documentation for image building and GPU access. Bare-metal setup is simpler for the first RunPod PyTorch Pod test.
Official instructionsWhat to expect
Coder has a reported successful model load and dashboard on the 16 GB / ~31 GB RunPod, but this is not a completed remote-chat or coding-agent benchmark. Start with Coder around 32 GB RAM, validate the actual access path, then complete one supervised Cline task. Use more RAM for full packs. The publisher’s RTX 5070 / 64 GB results and the user’s calibration figures describe different machines and different tests.
Explore its uses
Sources and files
Primary sources and files behind this model page.
Keep exploring
Bonsai 2 · 27B
A compressed 27B-class reasoning model with publisher-provided GGUF packs and a dedicated llama.cpp fork for CUDA, Metal, and CPU.
DeepSeek V4.1 Flash
A multimodal reasoning model with a compressed key-value cache, published as open weights.
Qwen3.8
Explore 4 versions of Qwen3.8. Compare model sizes, packages, hardware requirements, and available setups.