The short answer
Abliteration is a technique that removes a language model's tendency to refuse, by editing its weights directly. It comes from a 2024 finding that, inside many chat models, refusal is driven largely by a single direction in the model's internal activations. Take that direction out and the model mostly stops refusing. Most of what it knows and can do stays the same.
The name blends ablation (removing a component to see what it does) with obliterate.
Where it came from
In 2024, Andy Arditi, Neel Nanda and colleagues published Refusal in Language Models Is Mediated by a Single Direction, presented at NeurIPS 2024. They studied 13 open chat models of up to 72B parameters and found the same pattern in every one:
- Removing one direction from the model's internal activations stopped it from refusing harmful instructions.
- Adding that same direction made it refuse even harmless requests.
The authors noted how fragile current safety fine-tuning is, since even a simple rank-one change to the weights could nearly eliminate refusals. Community builders turned that research result into a repeatable recipe, and "abliterated" models started appearing on Hugging Face soon after.
How abliteration works, step by step
- Collect two prompt sets. One contains requests the model refuses, the other harmless requests it answers.
- Record the model's activations. Run both sets and capture the model's internal state at each layer.
- Find the refusal direction. Subtract the average activation on harmless prompts from the average on refused prompts. The difference points along the direction the model uses to signal "refuse this."
- Pick the best layer's direction. Test the candidates and keep the one that most reliably switches refusal on and off.
- Remove it from the weights. Change the weight matrices so the model can no longer write anything along that direction. This is a permanent edit to the model files, not a filter applied while it runs.
- Quantize and publish. The edited model can be converted to GGUF and run in Ollama, LM Studio or llama.cpp like any other.
No new training data is needed, and it takes a tiny fraction of the compute a fine-tune would. That's why abliterated versions of popular models often appear within hours of a release.
Abliteration vs fine-tuning vs Heretic vs neutral alignment
| Method | How it works | Strengths | Weaknesses |
|---|---|---|---|
| Abliteration | Deletes the refusal direction from the weights | Fast, cheap, keeps the original model's skills | Crude. Can dent quality or leave some refusals behind |
| Filtered fine-tune (Dolphin) | Retrains on data with refusals removed | Coherent behavior, steerable by system prompt | Costly, and lags behind new base-model releases |
| Heretic | Automated search for the abliteration that removes the most refusals while changing the model least | Usually closer to the original model | Newer and less widely reviewed |
| Neutral alignment (Hermes 4) | Trained to follow the operator's system prompt instead of a fixed policy | Predictable and configurable | May still decline without a permissive system prompt |
InsiderLLM reports that Gemma 3 resisted standard abliteration unusually well, which is part of why Heretic-style methods caught on with the Gemma 4 generation.
Does abliteration hurt model quality?
It can. Abliteration changes the weights without retraining, so some unrelated behavior can shift along with refusals. Even huihui-ai, one of the most prolific publishers of abliterated models, describes its own method on its model cards as a "crude, proof-of-concept implementation." In practice:
- Run the abliterated model and the original side by side on your tasks before you switch.
- If quality matters more than a few refusals, try a fine-tuned option like Dolphin, or a Heretic build.
- Use the original model's recommended prompt format and settings. Abliteration doesn't change them.
Does it make a model "unsafe"?
It removes the model's own refusals. That's the whole point. Model cards for abliterated releases warn plainly that safety filtering has been significantly reduced and there are no default safety guarantees. If you deploy one for other people, the guardrails have to come from you: a system prompt, input and output checks, and your own policies. Read Is uncensored AI legal and safe?.
How to spot an abliterated model
Look for "abliterated", "heretic" or "uncensored" in the name, and check the model card for:
- the base model it was made from (the "Model tree" on Hugging Face)
- the method used and any evaluation the maintainer ran
- the license, which is inherited from the base model
Ready to try one? Pick a model from the best uncensored LLMs list and run it with Ollama.
Sources
- Arditi, Obeso, Syed, Paleka, Panickssery, Gurnee and Nanda, Refusal in Language Models Is Mediated by a Single Direction, arXiv:2406.11717, NeurIPS 2024
- The authors' original write-up on LessWrong / GreaterWrong
- InsiderLLM: uncensoring techniques (dataset filtering, abliteration, Heretic)
- Huihui Qwen3.6-27B abliterated model card, for the method description and usage warnings
- Apidog: Hermes 4 neutral alignment