The journal

How the context layer is built.

Benchmarks, retrieval architecture and the engineering behind indexing a codebase once and serving it to every tool that asks.

All 86 Customers 20 Engineering 66

31–40 of 66 · page 4 of 7

Mar 31, 2026 MQA GQA multi-query attention

Multi-Query Attention and Grouped-Query Attention: Reducing KV Cache by 8× at the Architecture Level

Standard multi-head attention uses separate K and V for each head. MQA and GQA share them — reducing KV cache dramatically with minimal quality loss.

Mar 31, 2026 position encoding RoPE rotary embeddings

Position Encodings: How AI Knows Where Words Are (And Why It Breaks at Long Lengths)

Transformers have no inherent notion of order. RoPE encodes relative positions via rotation matrices. Here's the full math from sinusoidal to NTK-aware scaling.

Mar 31, 2026 prefill decode TTFT

Prefill vs. Decode: Why Long Context Makes Your AI Slow Before It Even Starts Talking

Before the AI generates its first word, it must process EVERY token in the context. Here's why time-to-first-token increases with context length.

Mar 31, 2026 prompt caching API optimization cache hit

Prompt Caching: How to Pay for Context Once Instead of Every Turn

Prompt caching saves 90% on repeated context. For coding agents carrying 20-40K tokens of system prompts, the savings are enormous.

Mar 31, 2026 O(n²) complexity self-attention computational cost

The O(n²) Problem: Why Doubling Your Context Window Quadruples the Cost

Self-attention computes all pairwise interactions between tokens. For n tokens, that's n² computations. Here's the full mathematical derivation.

Mar 31, 2026 rate-distortion context compression Shannon

The Rate-Distortion Theory of Context Compression

Context compaction is a lossy compression problem. Rate-distortion theory gives the theoretical lower bound on how much conversation history can be compressed.

Mar 31, 2026 ring attention sequence parallelism distributed inference

Ring Attention and Sequence Parallelism: Distributing Context Across GPUs

When a single GPU can't hold the KV cache, you distribute the sequence across multiple GPUs. Here's how ring attention enables million-token contexts.

Mar 31, 2026 RoPE rotary position embeddings mathematical derivation

Rotary Position Embeddings: The Full Mathematical Derivation

RoPE is used by virtually every modern LLM. Here's the complete derivation from first principles, proof of the relative position property, and NTK-aware scaling.

Mar 31, 2026 scaling laws context length Chinchilla

Scaling Laws for Context Length: How Performance Changes with Window Size

Similar to Kaplan's scaling laws for model size, there are scaling laws for context length. Performance doesn't scale linearly — here's the math.

Mar 31, 2026 context window size model comparison AI performance

Small vs. Large Context Windows: What's the Actual Difference for Users?

4K vs 32K vs 128K vs 200K vs 1M — what do these numbers actually mean for your experience? Bigger isn't always better.