← All posts

Agent Skills Tell the Model How to Work. Nothing in That Loop Checks the Result

Skills are the best thing to happen to agent workflows in the last year, and we write them ourselves. This is not an argument against using them. It is an argument about what category of thing a skill is, because a large number of teams are now treating a code review skill as the answer to whether AI generated code is safe to merge, and a skill structurally cannot be that.

Agent Skills Tell the Model How to Work. Nothing in That Loop Checks the Result

Agent Skills Tell the Model How to Work. Nothing in That Loop Checks the Result

Skills are the best thing to happen to agent workflows in the last year, and the format won quickly for good reasons. A SKILL.md file in a directory, loaded on demand when the agent decides it is relevant, portable across Claude Code, Cursor, Gemini CLI, Codex CLI and the rest. Bundled reference files cost zero tokens until something reads them. A script that runs consumes only its output rather than its source.

We write skills ourselves and recommend them. This is not an argument against using them.

It is an argument about what category of thing a skill is, because a large number of teams are now treating a code review skill as the answer to whether AI generated code is safe to merge, and a skill structurally cannot be that.

A skill is an instruction, and an instruction is not a check

The distinction carries the whole piece, so it is worth stating plainly.

A skill tells the model how to approach a task. Review this diff for security issues. Follow the naming conventions this team uses. Run the linter before committing. That is process guidance, and where the model complies it improves the output.

Verification is a different operation entirely. It compares a produced artifact against a stored statement of what was supposed to happen, and returns a judgement. Nothing in a skill file holds that statement, because a skill describes the method rather than the system, and there is no record inside it of what your billing service already promises about what counts as an active subscription.

So a review skill can tell a model to look for problems. It cannot tell the model what a problem is in your codebase, and the model ends up inferring that from whatever files it managed to read during the session.

Three failure modes people are hitting right now

The first failure mode is triggering. A skill only helps if it loads, and the description field is what decides that. Too vague and the model never matches it to the request. Too narrow and it fails to fire on the general version of the same task. The description field has a hard character limit and skill descriptions can be silently dropped from the token budget before the model ever reads them, which produces the particular frustration of a skill that works when you test it manually and gets ignored in real sessions.

The second is compliance decay. Instructions are suggestions rather than rules, and adherence falls as a session runs. One 2026 study across 4,416 trials at six conversation depths found compliance with an agent’s own explicit constraints dropping from 73% at turn 5 to 33% at turn 16 with no memory mitigation applied. A verification step that fires two thirds of the time is not a gate.

The third is the budget. Every skill description sits in context at startup, and every MCP tool definition sits there alongside it. Install enough of them and you have spent thousands of tokens on instructions before the model has read a line of your code, while the added bulk itself degrades attention on everything else. The thing meant to make the agent more reliable becomes a source of context rot.

The gap the research keeps finding

The pattern behind all three is that skills operate on the model’s process while the actual problem sits in the codebase.

In the MSR 2026 dataset of agentic pull requests rejected by maintainers, only about 36% reflected a clear failure in the agent’s code. Another 31% were rejected because the change violated a workflow constraint or a decision the project had already taken, and that context does not live in the repository.

No skill file contains that. The information needed to judge those rejections is what the surrounding code already promised, and it has to be derived from the code before any instruction can make use of it.

The security numbers point the same way. Veracode’s 2026 GenAI Code Security Report found roughly 44% of AI code generation tasks introduced a risky security vulnerability, with the average security pass rate across models sitting near 56% and barely moved from the previous report. That pass rate has stayed flat across testing cycles while coding benchmarks improved, and larger models did not outperform smaller ones. Better instructions have not moved that number, because the model is not failing at instruction following. It is failing at knowing what the surrounding system requires.

What we put underneath the skill

ByteBell is a verification layer generated on top of a codebase of any size below 10 million files, and it is the artifact a review skill needs in order to be a check rather than a suggestion.

Cheap open source models read every file once at indexing time. An agent detects the boundaries of each language element, stores them separately, then analyses each one in the context of its file, its module, its repository and across every repository. What gets stored for each file is what it does, why it exists in business terms, what users lose if it fails, the capabilities it provides, its edge cases, the contracts it provides, its preconditions, and the specific places where the behaviour is genuinely undecided. Every claim carries the file and the line it came from.

The verification step then has something to verify against. We trace what a change touches, generate tests for it, run them, and review each modified file against what that file promised at the indexed base rather than against what the diff says it changed. Collateral damage is checked across the other repositories reading the same helper before anyone approves. A full pass came to 87,001 tokens at $0.11.

That is deterministic in the way that matters. The layer either holds a promise the change contradicts, or it does not, and the answer does not depend on whether an instruction loaded at turn 3 was still being followed at turn 16.

It also fixes the budget problem rather than adding to it

The layer is served over MCP, so an agent pulls the relevant slice at the moment it needs it instead of carrying an architecture description in context for the whole session. A slice comes back as a dozen facts with a file and a line attached rather than as coherent prose about the codebase, which is a meaningfully better shape for retrieval.

Accuracy of code holds above 93% while token spend drops by roughly 80%. On a traced run, tracing a change across 32 impacted files in 4 repositories took 4 minutes 9 seconds and cost 0.89,againstroughly0.89, against roughly30 for the same work by brute force. Indexing runs about $13 per 1,000 files, once, with only what changes re-derived on each commit.

What skills are still the right tool for

Everything about how work gets done. House style, the order of operations in a release, which package manager to use, how to structure a commit message, what a good test looks like on this team. Skills change agent behaviour measurably on all of that, and a repository without them is worse off.

The two of them compose directly. A review skill that reads the verification layer through MCP is the combination worth building, because the skill supplies the method and the layer supplies the ground truth.

The honest limitation

A derived layer is an inference about what code means rather than a statement of intent from the person who wrote it, so it can be wrong where a hand written instruction would have been exactly right. Every claim carries a file and a line for that reason, and the index re-derives on every commit, though there is a window after a push where a claim can lag the branch.

We also do not cover every language. Mojo is not indexed today.

This is what ByteBell builds

Open your code review skill and find the sentence that says what your system is supposed to do. If it is not there, the skill is telling a model how to look rather than what to look for, and the model is filling in the difference from whatever it happened to read.

ByteBell is the verifiable context layer for code. We run the LLM compiler pattern, a one time pass where a model reads every file and lowers it into a verifiable code IR capturing purpose, business context and cross repository relationships, on your own infrastructure through Docker, with your source never leaving your environment. Every engineer, on any copilot, queries the same representation through a single MCP url, and every agent edit is checked against it before it lands, using per file SHA-256 diffing so only what actually changed gets examined again. Across 46 Kubernetes ecosystem repositories and 150,000 files we measured roughly 10% higher accuracy at 70% lower cost, on about a fifth of the tokens.

www.bytebell.ai

All posts