ByteBell - AI for Engineering TeamsSkip to main content

Posts tagged with "sparse attention"

1 post found

Featured image for article: Sparse Attention Architectures: Breaking O(n²) with O(n√n), O(n·log n), and O(n)
Mar 31, 2026 sparse attention Longformer Linformer linear attention Reformer efficient transformers

Sparse Attention Architectures: Breaking O(n²) with O(n√n), O(n·log n), and O(n)

Multiple research approaches attack the quadratic bottleneck: Longformer, Reformer, Linformer, and linear attention. Here's the math behind each one.