Uncensored Without "Brain Damage": Orthogonal Weight Abliteration & How bf16.ai Verifies Checkpoints

TL;DR · Model Quality & Verification

• Why Uncensored Models Got a Bad Reputation: Naive LoRA fine-tuning on low-quality "uncensored" datasets — compounded by unannounced Q3 / NVFP4 quantization on cheap aggregators — destroys AIME math logic, truncates context above 64K, and breaks nested JSON tool_calls.
• Rank-1 Orthogonal Projection Preserves Intelligence: Proper weight abliteration removes only the 1D refusal mediator vector r̂ from residual-stream matrices (W' = W - r̂(r̂ᵀW)), preserving (d_model - 1) dimensions of coding and reasoning fidelity.
• Database-Enforced Verification Gate on bf16.ai: An endpoint can only go active on bf16.ai after it passes a CSPRNG-seeded 6-vector verification run — and pre-first-token upstream faults automatically refund 100% of reserved credits.

1. The Dirty Secret of Open-Weight Inference Markets: Silent Quantization & Truncation

Ask ten Staff engineers why they hesitated to put "uncensored" open-weight models into production agent loops a year ago, and nine will give the same answer: "It stopped lecturing me, sure — right before it hallucinated invalid JSON and forgot half my system prompt."

That degradation — widely nicknamed "brain damage" in open-source ML circles — comes from two distinct engineering shortcuts that look identical from the outside of an opaque API:

1. Destructive Full-Layer Fine-Tuning: Training a frontier checkpoint on small, noisy synthetic Q&A datasets overwrites attention and MLP weights responsible for symbolic deduction and strict schema formatting.
2. Undisclosed Sub-FP8 Quantization & Context Clipping: Many bargain inference endpoints advertise a flagship 1M-context model ID while secretly serving an uncalibrated 3-bit/4-bit (Q3_K_M / NVFP4) quant with a silently capped 64K KV cache. Casual chat looks fine; deterministic temperature=0 math and 200K-token passkey retrieval collapse immediately:

[bf16-verifier] Probing candidate upstream endpoint: "provider-x/flagship-abliterated-1m"
[bf16-verifier] V1 Tokenizer BPE boundary ............ PASS
[bf16-verifier] V5 Dual-use low-refusal fidelity ..... PASS
[bf16-verifier] V2 Symbolic Math Canary (temp=0) ..... FAIL (drift detected: NVFP4/Q3 quant!)
[bf16-verifier] V3 Passkey NIAH @ 196,608 tokens ..... FAIL (silent middle-out truncation >64K)
[bf16-verifier] V4 Parallel JSON Tool Grammar ........ FAIL (unescaped trailing comma)
[bf16-verifier] Verdict: FAIL (3 vectors below their floor). Endpoint REJECTED by DB CHECK constraint.

2. Why Rank-1 Orthogonal Abliteration Keeps Coding & Tool Calling Sharp

Modern interpretability research demonstrated that refusal behavior in instruction-tuned Transformers is mediated primarily by a single one-dimensional direction r̂ ∈ ℝ^(d_model) in the residual stream. When a prompt contains superficial triggers (like <input type="password"> or a SQL injection diff), projections onto r̂ spike and steer downstream layers into pre-baked safety boilerplate.

Instead of retraining billions of parameters on noisy text, orthogonal weight abliteration computes r̂ via contrastive activation differences between harmless and over-refused developer prompts, then projects r̂ out of the residual-writing matrices (o_proj and down_proj) in float32 before casting back to checkpoint precision. Because a rank-1 projection removes only 1 dimension out of 4,096–7,168 hidden dimensions, multi-step coding, parallel JSON tool calling, and 1M-token RoPE attention remain virtually untouched.

Even with clean weights, bf16.ai never trusts upstream provider labels on faith. Every candidate endpoint must pass our 6-Vector Behavioral Verification Gate — symbolic math, 200K+ NIAH passkey, parallel JSON grammar, and low-refusal probes — before PostgreSQL CHECK constraints even allow status = 'active'.

3. Inside the Math & Querying the Catalog on bf16.ai

For ML engineers curious about the linear algebra under the hood, here is the exact rank-1 orthogonal projection applied to residual-writing weight tensors so refusal mediation vanishes without perturbing orthogonal subspaces:

import torch

def orthogonalize_refusal_direction(weight: torch.Tensor, refusal_dir: torch.Tensor) -> torch.Tensor:
    """
    Rank-1 orthogonal projection in bfloat16/float32 (Arditi et al. formulation):
    Removes ONLY the 1D refusal mediator subspace r_hat from residual-writing matrix W,
    leaving the (d_model - 1)-dimensional reasoning & tool-calling subspace untouched.
    """
    r_hat = refusal_dir / torch.linalg.norm(refusal_dir)
    r_hat = r_hat.to(dtype=torch.float32, device=weight.device)
    w_fp32 = weight.to(torch.float32)
    # W' = W - (W @ r_hat)[:, None] * r_hat[None, :]  (for output-feature projection)
    projection = torch.outer(r_hat, r_hat @ w_fp32)
    return (w_fp32 - projection).to(weight.dtype)

Each model page at bf16.ai/models/<slug> lists the model's context window, pricing, and Zero-Prompt Retention (ZDR) terms — and you can query the same catalog programmatically via GET /v1/models:

# Inspect context windows & pricing before routing agent jobs
curl -sS https://api.bf16.ai/v1/models \
  -H "Authorization: Bearer $BF16_API_KEY" | jq '.data[] | {
    id,
    context_window,
    pricing_usd_per_1m
  }'

# Route via bf16/auto with a deterministic output price ceiling ($1.50 / 1M max)
curl -sS https://api.bf16.ai/v1/chat/completions \
  -H "Authorization: Bearer $BF16_API_KEY" \
  -H "X-BF16-Max-Price-Per-M-Output: 1.50" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bf16/auto:coding",
    "temperature": 0,
    "messages": [{"role": "user", "content": "Emit a strict JSON schema patch for session fixation."}]
  }'

4. All 3 Uncensored Models on bf16.ai: Clean Catalog, Official Pricing

Rather than cluttering our catalog with dozens of unverified experimental merges, bf16.ai curates three production-grade 1M-context abliterated models — each held to the same 6-vector gate and priced at or below official vendor standard rates:

Model ID	Retail Price (In / Cache / Out per 1M)	Primary Engineering Strength
deepseek/deepseek-v4.1-flash-abliterated	$0.30 / $0.05 / $1.20	Ultra-fast 24/7 MausBot & CLI loops
zai-org/glm-5.3-flash-abliterated	$0.24 / $0.06 / $1.00	Multimodal Vision + Text browser QA & ETL
zai-org/glm-5.3-abliterated	$1.40 / $0.30 / $4.40	Frontier SAST patching & complex code diffs

5. Ledger & Routing Guarantees Engineers Can Audit Yourself

1. Zero Charge on Pre-First-Token Upstream Errors: bf16.ai uses a three-phase double-entry nano-USD ledger (Hold -> Stream -> Settle). If an upstream replica errors or times out before emitting the first content or tool token, 100% of your credit hold is released immediately ($0.00 billed).
2. Run Your Own temperature=0 Canary Prompts Anytime: Don't take our verification gate on faith — paste your gnarliest TypeScript AST transform or 200K-token log needle into bf16.ai/playground using the $1.00 free signup trial.
3. Pin Price Ceilings via X-BF16-Max-Price-Per-M-Output: Combine bf16/auto smart routing with explicit unit-cost headers so failover never surprises your bill.
4. Clean, Zero-Noise Model List: GET /v1/models returns only active checkpoints — zero synthetic test stubs, zero shadow-quantized bait-and-switch endpoints.