CLAUDE.md Is Failing Your Agent. State Of The Art Recommendations.

By Lo Zarantonello · Published February 27, 2026 · 1 min read · Source: Level Up Coding

Member-only story

CLAUDE.md Is Failing Your Agent. State Of The Art Recommendations.

A new study shows context files can actually impair coding agents. The results should change how you work.

Lo Zarantonello4 min read·Just now

Running /init was a good practice. Now forget about it.

A new study from ETH Zurich is questioning the usefulness of context files like AGENTS.md and CLAUDE.md.

They tested three conditions:

no context file at all,
an LLM-generated file (the kind you get from /init),
a human-written file crafted by the actual developers of the repo.

Figure 1:Overview of our evaluation pipeline. We begin with real-world repositories and tasks derived from past pull requests. For each repository state, we generate three settings: \{1}⃝ If a developer-provided context file exists, we include it in the repository. In \tiny{2}⃝, we omit the context file. \tiny{3}⃝ We use the coding agent’s recommended settings to generate the context file. Then we pass the repository and context file to the coding agent and instruct it to autonomously resolv — Overview of the study evaluation pipeline. Credits.

The results weren’t what anyone expected.

TLDR: Context files impair your agent

The Numbers

According to the study, LLM-generated context files like CLAUDE.md and AGENTS.md decreases task success rates by ~3% compared to providing no file at all.

And on top of that, they increased inference cost by over 20%!

Congrats, we just automated paying more to get less.

Humans-written context files

Human-written context files did just slightly better.

This article was originally published on Level Up Coding and is republished here under RSS syndication for informational purposes. All rights and intellectual property remain with the original author. If you are the author and wish to have this article removed, please contact us at [email protected].