Posts from 2026

Your AI Coding Agent Reads 166 Tokens for Every 1 It Writes. Here's Why That's a Problem.

For every 1 token your AI writes, it reads 166. That 165:1 ratio explains why AI coding is expensive, slow, and hitting limits constantly. Here's the data.

Why You Get Banned From Claude for 5 Hours (And Why It's Not Really About Your Message Count)

Claude locks you out for 5 hours and you barely sent any messages. The real reason: your AI agent silently consumed 100,000+ tokens reading files you didn't ask it to read.

Claude Code Keeps Compacting and Losing My Work. Here's What's Actually Happening.

Claude Code compacts your session and suddenly forgets which files it modified, what errors it found, and what it was working on. Here's the technical explanation of why — and how to prevent it.

Claude Max at $200/Month and Your Limit Still Drains in 90 Minutes. Here's Why.

Even at $200/month, Claude Max 20x users report their session draining from 21% to 100% on a single prompt. The problem isn't the plan — it's what's consuming your tokens.

Why Your $20/Month Claude Pro Plan Runs Out in 3 Prompts (And What's Actually Eating Your Tokens)

Your Claude Pro subscription burns through its 5-hour session limit in minutes. Here's the technical reason: your AI agent wastes 70% of your tokens reading files instead of answering your question.

"Context Left Until Auto-Compact: 0%" — What This Warning Means and Why Your AI Just Forgot Everything

When Claude Code shows 'context left until auto-compact: 0%', it's about to summarize everything and throw away the details. Here's exactly what gets lost and why it matters.

Context Rot: Why Your AI Gets Worse the Longer You Talk to It

Research tested 18 frontier models. Every single one gets worse as input length increases. This phenomenon — context rot — is why your AI coding assistant degrades mid-session.

What Happens When Your AI's Context Window Fills Up (The Technical Explanation)

Every AI coding assistant has a fixed context window — the maximum information it can hold at once. Here's what happens step by step when that window fills up, and why bigger windows don't fix the problem.

End-to-End System Evaluation: The Stress Test of GraphRAG

Individual layers may pass, but systems often fail at the seams. This blog details how to conduct holistic 'System-in-the-Loop' tests, measuring how retrieval noise compounds into generation errors across 25+ repositories. We provide a blueprint for evaluating the full journey from a vague natural language query to a multi-repo pull request.

Why "Context Graph" Has Become the Most Misunderstood Term in AI Engineering

A technical deep-dive into LLM context management, the computational limits of context windows, and why every AI tool's 'context graph' solution might just be clever marketing around semantic search and RAG.

Every AI Coding Tool Has the Same Problem. None of Them Will Tell You.

Claude Code, Cursor, Copilot, Codex — they all share the same fundamental flaw: no persistent memory, brute-force file reading, and context that fills up and gets thrown away. Here's what none of them admit.

Evaluating Generation and Grounding in Multi-Repo Systems

Retrieving nodes is only half the battle; the LLM must synthesize code that adheres to cross-repo constraints. This post explores measuring faithfulness, checking execution-level correctness against internal SDKs, and using LLM-as-a-Judge to verify that generated code respects the security and type contracts of separate repositories.

I Tracked 100 Million Tokens of Claude Code Usage. 99.4% Were Wasted on Reading.

A developer tracked every token Claude Code consumed for a month. The result: 99.4% were input tokens. For every 1 token written, 166 were consumed reading. Here's what that means for your bill.

Why Your AI Gets Dumber the Longer You Talk to It

You're not imagining it. Research tested 18 frontier models and every single one degrades as the conversation gets longer. Here's the science behind it.

Benchmarking Context Windows: NIAH, RULER, MRCR, and What They Actually Measure

Needle in a Haystack tests one thing. RULER tests another. MRCR tests yet another. Here's what each benchmark actually measures and what it misses.

The Cost Curve: How Context Length Affects Your API Bill (With Formulas)

A 20-turn conversation costs quadratically more than you'd expect. Here are the exact formulas for multi-turn conversation costs across providers.

Context Window vs. Training Data: Two Completely Different Things

Training data is everything the model learned. The context window is what it's thinking about right now. Here's why the distinction matters.

What Is a Context Window? The Simple Explanation Everyone Gets Wrong

The context window is your AI's short-term memory. It's not storage — it's a fixed-size desk. Everything the AI is thinking about right now must fit on this desk.

The Future: Will Context Windows Grow Forever? (Ring Attention, SSMs, Retrieval-Augmented Everything)

Three competing paradigms: grow context via hardware, replace attention with O(n) alternatives like Mamba, or build external memory systems. Which will win?

Flash Attention: How One Paper Made Long Context Possible

Flash Attention never materializes the full n×n attention matrix. Instead, it computes in tiles using fast GPU SRAM. Here's how it works and why it's 2-4× faster.

The Quadratic Memory Wall: A Precise Analysis of GPU Memory Requirements per Context Length

Total GPU memory = model weights + KV cache + activations + workspace. Here's the exact formula to compute maximum context length for any GPU configuration.

Infini-Attention and Compressive Memory: Unbounded Context with Bounded Memory

Google's Infini-Attention combines standard attention with a compressive memory that persists across segments — enabling theoretically infinite context at O(1) memory.

The Information-Theoretic Limits of Context Windows

There are fundamental limits on how much information a fixed-width attention mechanism can extract from n tokens. Here's the math from Shannon's channel capacity to attention bounds.

The KV Cache: Why Your AI Needs So Much GPU Memory

During inference, the model stores Key and Value vectors for every token. This KV cache is often the biggest memory consumer. Here's the math behind it.

The 'Lost in the Middle' Problem: Why Your AI Ignores Information in the Center of Long Prompts

Research proved that LLMs attend strongly to the beginning and end of context but poorly to the middle. This is a mathematical property, not a bug.

KV Cache Memory Math: Calculating Exactly How Much VRAM You Need

The exact formula for KV cache memory and worked examples for every major model architecture. Calculate your GPU requirements precisely.

Multi-Query Attention and Grouped-Query Attention: Reducing KV Cache by 8× at the Architecture Level

Standard multi-head attention uses separate K and V for each head. MQA and GQA share them — reducing KV cache dramatically with minimal quality loss.

Position Encodings: How AI Knows Where Words Are (And Why It Breaks at Long Lengths)

Transformers have no inherent notion of order. RoPE encodes relative positions via rotation matrices. Here's the full math from sinusoidal to NTK-aware scaling.

Prefill vs. Decode: Why Long Context Makes Your AI Slow Before It Even Starts Talking

Before the AI generates its first word, it must process EVERY token in the context. Here's why time-to-first-token increases with context length.

Prompt Caching: How to Pay for Context Once Instead of Every Turn

Prompt caching saves 90% on repeated context. For coding agents carrying 20-40K tokens of system prompts, the savings are enormous.

The O(n²) Problem: Why Doubling Your Context Window Quadruples the Cost

Self-attention computes all pairwise interactions between tokens. For n tokens, that's n² computations. Here's the full mathematical derivation.

The Rate-Distortion Theory of Context Compression

Context compaction is a lossy compression problem. Rate-distortion theory gives the theoretical lower bound on how much conversation history can be compressed.

Ring Attention and Sequence Parallelism: Distributing Context Across GPUs

When a single GPU can't hold the KV cache, you distribute the sequence across multiple GPUs. Here's how ring attention enables million-token contexts.

Scaling Laws for Context Length: How Performance Changes with Window Size

Similar to Kaplan's scaling laws for model size, there are scaling laws for context length. Performance doesn't scale linearly — here's the math.

Rotary Position Embeddings: The Full Mathematical Derivation

RoPE is used by virtually every modern LLM. Here's the complete derivation from first principles, proof of the relative position property, and NTK-aware scaling.

Softmax Attention and the Dilution Problem: The Math Behind Context Rot

As context grows, softmax normalizes attention weights so each relevant token gets less attention. This mathematical property is why AI accuracy drops with length.

Small vs. Large Context Windows: What's the Actual Difference for Users?

4K vs 32K vs 128K vs 200K vs 1M — what do these numbers actually mean for your experience? Bigger isn't always better.

Sparse Attention Architectures: Breaking O(n²) with O(n√n), O(n·log n), and O(n)

Multiple research approaches attack the quadratic bottleneck: Longformer, Reformer, Linformer, and linear attention. Here's the math behind each one.

Tokens Explained: Why Your AI Counts Words Differently Than You Do

AI doesn't read words — it reads tokens. Learn how tokenization works, why code costs more than prose, and how to calculate your token usage.

How the Transformer Attention Mechanism Actually Works (No Math Required)

The attention mechanism is the beating heart of every LLM. Here's how it decides which parts of your conversation matter most — explained with analogies before equations.

What Happens Inside the Context Window When You Upload a File to Claude

When you upload a PDF or code file, it gets converted to tokens and placed directly into the context window — consuming the same limited space as your messages.

Why Does My AI Forget Everything Between Sessions?

AI models are stateless. Each conversation is a fresh start. There's no database storing what you said yesterday. Here's why — and what it means for you.

Why RAG Exists: Context Windows Are Too Small for Your Data

If context windows were infinite, free, and didn't degrade, we wouldn't need RAG. Here's why retrieval-augmented generation exists and when to use it.

Claude Is Rate-Limiting Everyone. Here's Why Good Context Beats a Smarter Model.

Anthropic just throttled Claude Opus and Sonnet during peak hours. Developers are canceling subscriptions and looking for alternatives. Here's the argument: a good open-source model with great context beats a frontier model that won't let you use it.