Workflows/Run local text completion with AliceAI Foundation 80B A3B Base

Text / Intermediate

Run local text completion with AliceAI Foundation 80B A3B Base

Load the AliceAI Foundation 80B A3B Base weights in Transformers, run the documented generation script on a short Russian prompt, and read the generated continuation to verify the model responds.

EstimatedSource checked 9/25/2026

LOCALRENTED GPU

Before you begin

Difficulty: Intermediate
Software: Python 3 with venv, transformers[sentencepiece]==5.16.1, accelerate==1.14.0, flash-linear-attention==0.5.0, PyTorch with CUDA
Hardware: This setup uses Transformers. The exact checkpoint format, context, and runtime overhead determine its memory need; a weight-file size alone is not a GPU recommendation.

TEXT / TRANSFORMERSSOURCE-LINKED SETUP

Run local text completion with AliceAI Foundation 80B A3B Base, step by step.

Load the AliceAI Foundation 80B A3B Base weights in Transformers, run the documented generation script on a short Russian prompt, and read the generated continuation to verify the model responds.

Choose a package Run its commands Check the result

BEFORE YOU BEGIN

The documented package

This setup uses the package documented for this task. Review its source and supported platforms before starting.

The publisher has not provided a complete memory target for this task. Check its hardware guidance before starting.

CURRENT SETUP

AliceAI Foundation 80B A3B Base · official

This setup uses Transformers. The exact checkpoint format, context, and runtime overhead determine its memory need; a weight-file size alone is not a GPU recommendation.

View weight source

THE WORKFLOW

Set up AliceAI Foundation 80B A3B Base on your machine

Pick your operating system. Every command below is for the selected package and runtime.

The publisher supplies Python code but does not specify an operating system. Linux is a practical starting environment, not a publisher-tested machine. Run the Python snippets in one session, in order.

Create and activate a Python virtual environment

From the directory where you will run the model, create an isolated Python environment so the pinned packages do not disturb your system Python.

python3 -m venv .venv
source .venv/bin/activate

Install the documented runtime dependencies

Install the reference Transformers version, Accelerate, and the flash-linear-attention package required to execute the KDA layers on GPU. The version pins come from the publisher's guide.

pip install \
  transformers[sentencepiece]==5.16.1 \
  accelerate==1.14.0 \
  flash-linear-attention==0.5.0

Confirm PyTorch and CUDA are available

The model loads in bfloat16 and is placed with device_map="auto", so PyTorch must see your GPU(s) before you proceed. Ensure a CUDA-enabled PyTorch build is installed in this environment.

Load the model, tokenizer, and generate a completion

Save the following as a Python file and run it. The tokenizer is loaded with trust_remote_code=True, the model is loaded in bfloat16 and sharded automatically across available devices, and the Russian prompt from the publisher's example is passed through generate. The script then decodes only the newly generated tokens, so the printed text is the model's continuation.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "yandex/AliceAI-Foundation-80B-A3B-Base"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "Есть 256 монет с разным весом, за какое минимальное количество попарных взвешиваний можно найти вторую по весу монету?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output_ids = model.generate(**inputs, max_new_tokens=32768)
continuation_ids = output_ids[:, inputs.input_ids.shape[1] :]
print(tokenizer.decode(continuation_ids[0], skip_special_tokens=True))

Check the generated text

Run the script with `python your_script.py`. In the terminal you should see a non-empty Russian continuation after the coin-weighing prompt — typically a line of reasoning followed by a numeric answer. If the printed continuation is empty, contains only the prompt echoed back, or the process crashes before printing, revisit the previous steps and the troubleshooting notes below.

SUCCESS CHECK · AliceAI Foundation 80B A3B Base

Check the completed prompt

Give the base model a short prompt and inspect the generated continuation. A base checkpoint is not assumed to follow chat instructions.

This is a source-linked setup, not a YouRunAI hardware test. Confirm your exact runtime version, package, and output before relying on it.

Back to setup steps

When it doesn’t go to plan

Load fails complaining that flash-linear-attention or KDA support is missing.

Reinstall the pinned packages from the activation of the same virtual environment, ensuring `flash-linear-attention==0.5.0` is present and importable.

CUDA out-of-memory or device placement errors during loading or generation.

Use the same generation script but keep device_map="auto" so the model is sharded across all visible GPUs; ensure no other large process is occupying the GPUs.

Generation returns empty or truncated output.

Confirm you are decoding only the continuation slice `output_ids[:, inputs.input_ids.shape[1] :]` and that max_new_tokens is set to 32768 as in the documented script.

Import errors for transformers or torch.

Activate the virtual environment created in the first step (`source .venv/bin/activate`) and rerun the documented pip install command.

REFERENCE LIBRARY

Sources and files

Original instructions, model files, and compatibility notes behind this setup.

2 SOURCES

01Publisher model card and weightshuggingface.co 02Publisher run instructionshuggingface.co

The model behind this workflow

AliceAI Foundation 80B A3B Base

MY HARDWARE

Will it run on your machine?

Save your machine to see a personalized rating and its reasoning.

Add my hardware

Keep exploring

Text

Beginner

Set up a private AI chat assistant

Install a local runtime, run Qwen3.5 9B, confirm responses, and know when to choose the smaller 4B package.

Build a private local coding assistant

Connect an open coding-capable model in Ollama to Cline, run a small repository task, and review the result locally.

SOFTWAREOllama + Cline

Generate images on your own GPU

Use the official FLUX.2 Klein 4B ComfyUI template with exact model files, a first prompt, and an output check.

SOFTWAREComfyUI

LOCALRENTED GPU

Verified source