Why Context Is The New Moat: How Our Stack Delivers Under 4% Hallucination

TLDR : Big foundation models are great for speed and general facts. They are not built to solve your organization’s knowledge problem. Our receipts first, version aware stack retrieves the smallest correct context, verifies it, and refuses to guess. The result is under 4% hallucination on real engineering work. For background on why retrieval reduces hallucinations, see Retrieval Augmented Generation from 2020 and follow on work. ( arXiv )

---

The uncomfortable truth about foundation models

Foundation model companies optimize for serving massive user bases with minimal private context. That creates three limits that more parameters or bigger windows do not erase. Studies show that as context grows very large, models struggle to reliably use information away from the beginning and end of the prompt, a pattern called lost in the middle. ( arXiv )

1. They cannot carry your real context window

Vendors now advertise 200,000 tokens and beyond. Anthropic documents 200K for Claude 2.1 and explains that special prompting is needed to use very long context effectively. Recent reporting highlights pushes to 1 million tokens. Independent evaluations still find degraded recall as input length grows. ( Anthropic )

Our stack avoids dumping entire repos into a single prompt. We do four things.

1. Build a permission aware knowledge graph of code, docs, commits, issues, and discussions
2. Retrieve only minimal high signal chunks for the current question
3. Verify those chunks across multiple authoritative sources
4. Return answers with exact file path, line, branch, and release

This design aligns with peer reviewed findings that retrieval augmented generation improves factual grounding on knowledge intensive tasks. ( arXiv )

2. They choose speed over accuracy

Mass market assistants must favor latency. That tradeoff is fine for general facts. It breaks for system behavior where wrong answers cause outages or security bugs. Multiple empirical studies show non trivial hallucination rates for general assistants, including in high stakes domains like law and medicine. Some clinical workflows can be pushed near 1 to 4% with strict prompts and verification, which is the direction our stack takes by design. ( Stanford HAI )

We accept 2 to 4 seconds typical latency to deliver under 4% hallucination, zero unverifiable claims, and version correct results including time travel answers like how did auth work in release 2.3. The core idea matches the literature consensus that grounding plus verification reduces hallucination risk. ( Frontiers )

3. Their search only sees what public search sees

Your real knowledge lives in GitHub, internal docs, Slack, Discord, forums, research PDFs, governance proposals, and sometimes on chain data. Retrieval augmented systems were created exactly to bridge that gap by pulling from live external sources and citing them. ( arXiv )

We ingest these sources and keep them fresh so new changes are searchable within minutes. Freshness and receipts reduce guessing, which is a primary cause of hallucinations in large models. ( Frontiers )

---

Why Web3 is the hardest test

Web3 demands cross domain context. EVM internals and Solidity. Consensus and finality. Cryptography including SNARKs, STARKs, and KZG commitments. ZK research that ships quickly from preprints to production. Public references below show how fast these areas move and why long static training sets lag reality. ( arXiv )

We leaned into this problem.

• Substrate aware parsing for pallets and runtimes
• On chain context binding to runtime versions and blocks
• Multi repo relationship mapping across standards and implementations
• ZK and FHE awareness that links theory papers to working code

Surveys and empirical work on hallucinations reinforce the need for grounded retrieval and conservative answers when uncertainty is high. ( arXiv )

---

How our stack drives under 4% hallucination

The ingredients are simple. The discipline is the moat.

1. Receipts first retrieval

Every answer cites file, line, commit, branch, and release. No proof means no answer. This mirrors research that source citation and retrieval reduce fabrication. ( TIME )

What happens on a query

• We normalize intent and identify entities like service names and modules
• We fan out to code, docs, and discussion indices with structure aware chunking
• We gather candidates and attach receipts for each candidate span

2. Structure aware chunking

We do not split by blind token counts. This was the hardest part to come up with chunking strategies for different data types and to use different models to deliver it.

• Code chunks align to functions and classes and keep imports and signatures intact
• Docs chunks follow headings and lists to preserve meaning
• Discussion chunks follow thread turns to keep causality
• PDFs use layout aware extraction so formulas and callouts survive OCR

Aligned chunks raise precision and reduce the need for model interpolation. Academic and industry reports show that longer raw prompts without structure produce recall drops, while targeted retrieval improves use of long inputs. ( arXiv )

3. Cross source verification

Before we answer, we check agreement.

• Code outweighs docs when both exist
• Docs outweigh forum posts
• Forum posts outweigh chat logs
• Multiple agreeing sources raise confidence
• Conflicts trigger a refusal with receipts for both sides

Agreement scoring plus source quality weighting reduces confident wrong answers, which recent surveys identify as a key safety goal. ( Frontiers )

4. Version and time travel

Every node in the graph stores valid from, valid until, and version tags. When you ask about release 2.3 or a block height, retrieval filters spans to that time. This avoids blended answers from different eras, a common failure mode in ungrounded assistants. RAG style retrieval explicitly supports time scoped knowledge when indexes track freshness. ( arXiv )

5. Conservative confidence thresholds

Each candidate span carries semantic similarity, source weight, cross source agreement, and version fit. If the final confidence clears our fuzzy threshold we answer with receipts. When it does not, we first expand and correct the query using edit distance based fuzzy matching and query expansion so that misspellings or partial terms still retrieve the closest high confidence context. Only when those steps cannot raise confidence do we say I do not know, and we return the best receipts so the user can continue the search. This balances usability for new developers with safety guidance on calibrated uncertainty and selective prediction. ( arXiv )

6. Real time ingestion

We keep knowledge fresh without re indexing the world.

• Webhooks and scheduled pulls detect changes
• Only changed spans are re embedded
• The graph updates relationships incrementally
• End to end freshness target is under 5 minutes

Fresh sources reduce guessing. Surveys emphasize that stale training data increases hallucination risk and that retrieval from current sources mitigates it. ( Frontiers )

7. Workflow native surfaces

Answers appear where engineers work. IDE through MCP. Slack and CLI. Browser extension. The same receipts first policy applies everywhere so people can verify without breaking flow. Practitioners note that grounded answers with receipts build trust, while unguided chat increases subtle errors. ( TIME )

---

Results you can feel in daily work

What this looks like on a normal day

• You paste a stack trace and ask what changed in auth between 2.2 and 2.3 You get a 2 to 4 second answer with the exact diff, the PR link, the commit id, and a three line fix tied to file and line

• You ask how a Substrate nomination pool calculates rewards on a specific runtime version You get a precise description with the Rust function span, a tutorial that explains it, and the forum thread that clarified an edge case

• You ask whether an EIP impacts gas in your codebase You get links to the EIP, the client code, and the lines in your repo that call the affected opcodes

Each answer carries receipts you can open and verify. That is how error rates drop. Independent research in medicine shows that with strict workflows, hallucination rates can approach one to 2%, which is the bar we target. ( Nature )

---

Why models alone will not get you there

Bigger models will get faster and better at general facts. They still do not know your code, your decisions, your history, or your permissions. Without a receipts first context layer, they must guess. Guessing is what creates hallucination. The RAG literature and long context evaluations converge on this point. ( arXiv )

Our stack changes the objective. Retrieve the smallest correct context. Verify it. Refuse to answer if confidence is low. Then let any strong model generate with receipts attached. This is how you keep hallucinations under control even as prompts and corpora grow. ( Frontiers )

---

Try it on public deployments

These are community instances you can test now.

• ZK ecosystem: https://zcash.bytebell.ai
• Ethereum ecosystem: https://ethereum.bytebell.ai

Ask questions you care about. Look for the receipts. Compare with a raw chat model. Notice the difference in specificity, version awareness, and willingness to refuse. Background on why this works comes from the original RAG paper and follow ups on long context degradation. ( arXiv )

---

Reference list

• Liu et al. Lost in the Middle, 2023. ( arXiv )
• Anthropic. Long context prompting for Claude 2.1 and context guidance, 2023. ( Anthropic )
• The Verge coverage of 1 million token context windows, 2025. ( The Verge )
• Databricks blog on long context RAG performance, 2024. ( Databricks )
• Lewis et al. Retrieval Augmented Generation, 2020 NeurIPS. ( arXiv )
• JMIR study on hallucination and reference accuracy for GPT 3.5, GPT 4, and Bard, 2024. ( PMC )
• Nature npj Digital Medicine framework with approximately 1.47% hallucination in a controlled clinical workflow, 2025. ( Nature )
• Recent survey on hallucinations and mitigation strategies, 2025. ( Frontiers )

If you want, I can also annotate any specific sentences with additional primary sources, for example EVM opcode references or ZK proof primers, and fold those citations in line the same way.

#Your team does not need a bigger model. You need receipts.