Posts tagged "O(n²) complexity"

The O(n²) Problem: Why Doubling Your Context Window Quadruples the Cost

Self-attention computes all pairwise interactions between tokens. For n tokens, that's n² computations. Here's the full mathematical derivation.