Architecture Java ArchUnit Agents Evaluation

Architecture Decision Records in Java: Writing Them, Enforcing Them, and What an Agent Does With Them

November 2, 2026Michał Woliński26 min read

Six months after a decision, the code still shows what the team chose. It no longer shows why, or that anybody chose at all. A new colleague sees LocalDate computed in a Warsaw time zone and "fixes" it to UTC. An AI agent sees the same thing and copies whichever pattern is nearest. An architecture decision record (ADR) is the small tool for this problem: a short file in the repository that says what was decided, why, and what it costs.

This is a tutorial, so it starts from zero: the vocabulary, the formats, when a decision deserves a record, and how a record lives and dies. It then makes records enforceable with ArchUnit on Java 25, and looks honestly at what those rules cannot see.

The last part is an experiment, because I wanted a number instead of an opinion. I gave a coding agent the same tickets in five versions of one repository and graded the code it left behind. Without any record, it broke the team's decisions in 23 of 30 episodes. With records in a folder nothing pointed to, it broke one in 30. The more interesting result is what happened with rules and no records at all, and which decisions it never needed to be told about. It cost $1.60, it has real limits, and my first design measured nothing. All of that is below.

Everything is in the companion repository: the records, the rules, the harness and every run this article quotes. Every code block here is a real file in that repository, and its first line says which one.

The problem: code shows what, not why

Take one line of a shop's code: a count of orders placed on a calendar day, using Europe/Warsaw. Reading it tells you exactly what happens. It does not tell you whether Warsaw is a requirement or an accident, and that is the only thing a person changing it needs to know.

Left panel: code computing placedOn in the Warsaw zone tells you what it does but not why Warsaw rather than UTC, or whether the choice is still in force. Right panel: the record docs/adr/0012 adds that accounting closes in Warsaw, that UTC disagreed around midnight, and that the status is Accepted. A newcomer or an agent in the middle either decides the code looks like a bug, or reads why. — Figure 1. The code answers "what". The person changing it needs "why", and "does this still apply".

Michael Nygard named this problem in the 2011 post that introduced ADRs: "One of the hardest things to track during the life of a project is the motivation behind certain decisions." Without the motivation, he wrote, a newcomer can only accept the decision blindly or reverse it blindly, and "It's better to avoid either blind acceptance or blind reversal." His answer was deliberately small, because "Large documents are never kept up to date" and "Nobody ever reads large documents, either."

The vocabulary

These are the words you will meet in the literature, and in this article. Come back to the table when a later section uses one.

term	what it means
ADR	Architecture decision record: one short file about one decision.
architecturally significant	Nygard's scope for a record: decisions "that affect the structure, non-functional characteristics, dependencies, interfaces, or construction techniques".
decision log	The collection of records of one project, usually a folder such as `docs/adr`.
context	The forces at the time: technical, political, social. Written neutrally, since it is the part that goes stale.
decision	What we will do, in plain active voice.
consequences	What becomes true afterwards: good, bad and neutral.
status	Whether the record is in force: proposed, accepted, rejected, deprecated or superseded.
superseded	Replaced by a newer record. The old file stays, and points to its replacement.
deprecated	No longer applies, with nothing replacing it.
fitness function	An automated check that an architectural property still holds, a term from Building Evolutionary Architectures. An ArchUnit rule is one.
ArchUnit	A Java library that reads compiled classes and lets you assert things about their structure in a unit test.
freezing	ArchUnit's way to accept the violations that exist today and fail only on new ones.

Anatomy of a record

Here is a complete record from the repository. It is short on purpose. Nygard's limit is one or two pages, and most good records are shorter.

<!-- fixture/overlays/folder/docs/adr/0012-the-business-day-is-warsaw.md -->
# 12. The business day is in Europe/Warsaw

Date: 2026-10-05

## Status

Accepted

## Context

Accounting closes the day in Warsaw. An order placed at 23:30 Warsaw time in winter is 22:30 UTC, and the UTC-based numbers disagreed with accounting by every order placed around midnight.

## Decision

Any rule that depends on a calendar day uses the `Europe/Warsaw` time zone. Instants are still stored in UTC.

## Consequences

`OrderReports` predates this decision and is scheduled for migration. It is not a pattern to follow.

The six parts of ADR 12 stacked in a document, each with its purpose: the title has a number that is never reused and a short name; the date tells you when to read the context as of; the status says whether it is in force; the context holds the forces stated neutrally; the decision, highlighted, says what we will do in plain active voice; the consequences list all effects, including the bad ones. — Figure 2. Each part of a record, and what it is for.

Three of these parts do most of the work, and each has a common way to go wrong:

The context has to be true on its date. When someone later wonders whether the decision still holds, they compare today with this paragraph. "Accounting closes the day in Warsaw" can be checked. "Time zones are hard" cannot.
The decision has to be a decision. "We should consider a consistent approach to time" commits nobody to anything. A good test: could a reviewer point at a line of code and say that it breaks this?
The consequences have to include the bad ones. Nygard is explicit that positive, negative and neutral consequences all belong. A record with only upsides is a sales pitch, and readers discount it. ADR 12's bad consequence is a known exception that still lives in the code.

Three formats, pick one

The format matters less than having one. These three cover almost everything you will see:

format	shape	good for
Nygard	Title, context, decision, status, consequences	The default. It is what this article's repository uses.
MADR	Markdown Architectural Decision Records. Context and problem statement, considered options, decision outcome, plus optional parts such as decision drivers, pros and cons, and confirmation	Decisions with real alternatives, when you want the rejected options on the page.
Y-statement	One sentence: "In the context of [...], facing [...], we decided for [...] and neglected [...] to achieve [...], accepting that [...]"	Small decisions, or a first draft that forces you to name the trade-off.

MADR has one optional section I would borrow whichever format you use. Its template calls it Confirmation and asks you to "Describe how the implementation of/compliance with the ADR can/will be confirmed", suggesting "a design/code review or a test with a library such as ArchUnit". Answering that question while you write the record is the best habit in this article.

When a decision deserves a record

If everything gets a record, nobody reads them. The usual criterion is cost of reversal: choosing a database deserves a record, and naming a helper does not. The experiment below convinced me to add a second question, which catches decisions that are cheap to reverse but easy to break by accident.

A decision flow. You just made a decision. If it is expensive to reverse, write a record. If not, ask whether a careful newcomer would do it differently. If yes, write a record. If no, no record is needed, and a code comment or the formatter is enough. After writing a record, add a rule too if a machine can check it. — Figure 3. Two questions decide it. The second one comes from the experiment in this article.

Would a careful newcomer do it differently? "No floating point for money" fails this test: they would not use a double anyway, so the record tells them little. "No BigDecimal either, rates are whole basis points" passes it, because a careful person would reach straight for BigDecimal. That one exists only because your team had two specific bugs. Decisions like it are invisible from outside, and they are where a record pays.

Be careful with this question, because you are bad at answering it about your own team. I was: I filed "no remote calls inside a transaction" under obvious, and the experiment broke it every time. Treat the answer as a guess. When it matters, give a colleague or an agent the ticket with no record, and see what comes back. For a well-known rule that fails the question, you can still add an ArchUnit rule with no record behind it: the experiment shows that a good because can stand alone.

How a record lives and dies

Top: a proposed record is reviewed in a pull request and becomes accepted, or it is rejected. An accepted record later becomes deprecated when it no longer applies, or superseded when a newer record replaces it. Bottom: ADR 3 from March, no floating point with rates as BigDecimal, has the status Superseded by 8. ADR 8 from August, no BigDecimal either with rates as basis points, has the status Accepted and points back to ADR 3. Only the status line of an old record changes. — Figure 4. The statuses, and a real chain from the repository.

The rules are few, and they all protect the same thing: the record is history you can trust.

Numbers are permanent. In Nygard's words: "ADRs will be numbered sequentially and monotonically. Numbers will not be reused."
Accepted records are not edited. If you change your mind you write a new record, and the old one only gets a new status line. "If a decision is reversed, we will keep the old one around, but mark it as superseded."
Records live next to the code, so they are versioned with it, and reviewed in the same pull request as the change they justify. That review is the approval process, and a small team may not need another.
Write it when you decide. A record written afterwards tends to justify the outcome, because the options you rejected have already been forgotten.
In an existing code base, do not backfill everything. Start with your next decision. Write a record for an old one only when people keep asking about it, and say plainly that it records a past choice, since you can no longer list the options honestly.
Revisit by trigger, not by calendar. When a fact in the context stops being true, supersede the record. "Accounting closes in Warsaw" tells you what to watch.

Where the log lives, and who decides

A record should live as close as possible to the code it binds. A decision about one service goes in that service's repository, where its reviewers and its agents will see it. A decision that binds many teams, such as a company-wide choice of message broker, needs a shared log, and each affected repository should link to it from its instructions, or nobody working there will find it.

Records are also not the place to have the argument. A large or contested decision often starts as an RFC or a design document, where people disagree at length. The record is the short, final result that links back to it. Someone also has to be able to say "decided": a record with no owner stays proposed until the code has decided for you. I have no data on any of this. It is what I have seen work.

How records go wrong

failure	what it looks like	fix
Justification, not decision	Written after the fact, with one option on the page	Write it in the pull request that makes the choice
Non-decision	"We should aim for consistency"	Could a reviewer reject a line of code with it?
Edited history	An accepted record silently rewritten	Supersede it, and keep the old file
Stale status	Three records say "accepted" and contradict each other	Update the old record's status in the same change
Noise	A record for every library bump	The two questions in figure 3
The forgotten folder	A good log nobody opens	Point to it from where people and agents start, and enforce what you can
Unmarked exceptions	Old code breaks the record, so the record looks fake	Name the exceptions in its consequences

Enforcing a decision with ArchUnit

A record tells you what was decided. A rule tells you, in CI, that you just broke it. ArchUnit imports compiled classes and checks them from an ordinary test, so the only setup is one test dependency:

<!-- fixture/overlays/enforced/pom.xml -->
<groupId>com.tngtech.archunit</groupId>
<artifactId>archunit-junit5</artifactId>
<version>1.5.1</version>
<scope>test</scope>

Each accepted decision that a machine can check becomes one rule. The rule's name carries the record's number, and because carries the instruction:

// fixture/overlays/enforced/src/test/java/com/wolinski/shop/ArchitectureTest.java
static final ArchRule adr0008_noBigDecimal = noClasses()
        .that().doNotHaveSimpleName("OrderPricing")
        .should().dependOnClassesThat().areAssignableTo(BigDecimal.class)
        .because("docs/adr/0008: rates are whole basis points in a long");

// fixture/overlays/enforced/src/test/java/com/wolinski/shop/ArchitectureTest.java
static final ArchRule adr0011_domainFailuresAreShopProblems = noClasses()
        .that().doNotHaveSimpleName("Money").and().doNotHaveSimpleName("OrderReturns")
        .should().callConstructor(IllegalStateException.class, String.class)
        .orShould().callConstructor(IllegalArgumentException.class, String.class)
        .because("docs/adr/0011: throw platform.ShopProblem with a stable code");

The because text is not decoration. It ends up in the failure message, which is the moment the rule teaches. This is the real message an agent got in the experiment, trimmed to its start:

Architecture Violation [Priority: MEDIUM] - Rule 'no classes that do not have simple name 'Money' and do not have simple name 'OrderReturns' should call constructor IllegalStateException.<init>(String) or should call constructor IllegalArgumentException.<init>(String), because docs/adr/0011: throw platform.ShopProblem with a stable code' was violated (1 times)

That run had no docs folder at all, so the path led nowhere and the sentence after it did all the work. The agent's next step was a one-line reply and a one-line fix: "The architecture test (ADR 0011) requires domain failures to be ShopProblem with a stable code, not IllegalStateException. I'll switch to that."

Let the compiler go first

A rule is a second-best check for anything the compiler could reject. Making a module's internals package-private, using Java modules, or splitting into Maven modules makes ADR 2's violation impossible to write, and no agent or colleague needs to read anything. On Spring, Spring Modulith's ApplicationModules.verify() checks module boundaries and cycles out of the box, and it is implemented with ArchUnit. Other platforms have the same idea: ArchUnitNET for .NET, and Deptrac for PHP.

The fictional ADR 2 gets this wrong, through my own carelessness: it rejects package-private internals because of "Spring's wiring", and that reason does not hold, since Spring wires package-private classes without complaint. I noticed after the paid runs, so the file stays as they saw it. Treat it as a small example of a record whose context nobody checked.

Old code that breaks a new decision

You decide today, and the code base was written yesterday. There are two ways to stop that from turning the build red forever, and they are not equal.

The repository uses the simple one: it names the exceptions, as you can see in doNotHaveSimpleName("OrderReturns") above. The weakness is that a named class is exempt for everything, including what you add to it tomorrow. The repository has a test that proves it: a new Instant.now() added to OrderReports leaves the visible rules green.

For a real project, look first at ArchUnit's FreezingArchRule. In the user guide's words: "On the first run all violations of that rule will be stored as the current state. On consecutive runs only new violations will be reported." That narrows the hole, at the cost of a violation store to keep in the repository. The guide also says line numbers are ignored by default, and I have not tested how that treats a second identical violation in the same method. Two settings matter when you adopt it. freeze.store.default.allowStoreCreation defaults to false, so you have to allow the first run explicitly, and the guide suggests that a CI build should not create a store at all. And freeze.refreeze=true accepts every current violation and reports success, so it belongs on the list of switches to guard. I did not use freezing in the experiment, so all of this rests on the documentation and not on my tests.

What a rule cannot see

Building the experiment's grader showed me this in concrete terms, because I had to catch what my own rules missed.

Three columns. An ArchUnit rule reading bytecode sees dependencies between packages, signatures, fields, annotations and direct method calls, but not a double in a local variable, a call hidden in a helper, or what the code means. A source scan sees text including local variables and anything a regex can match, but not comments versus code, indirection, or meaning. A test at runtime sees what actually happened, including a call made through a helper and the behaviour of the Warsaw day, but only on the paths a test exercises, it needs the API of one feature, and it is slow to write in bulk. The grader in this article uses all three. — Figure 5. Each kind of check is blind somewhere else. The blind spots below have tests in the repository, except where the text says otherwise.

Locals. My first rule for "no floating point" checked fields and return types. A hand-written shortcut, Math.round(total * Double.parseDouble(percent) / 100), passed it, because a local double is not part of a class's structure. I added a rule for the parsing calls, but a double computed without one still passes, and only the grader's source scan finds it.
Indirection. The visible rule for ADR 5 forbids a @Transactional method from directly calling a gateway. Moving the call into a private helper gets past it, while the transaction is just as open. The check that holds is at runtime, in a fake carrier that records whether a transaction was open when it was called.
Meaning. "Any rule that depends on a calendar day uses Warsaw" has no reliable structural form. You could ban ZoneOffset.UTC, but LocalDate.now() or a stored date get around that. It takes a test per feature, so ADR 12 has no rule here, on purpose. Remember this one for the results.
The rule itself. Deleting the test turns CI green. So does an archunit_ignore_patterns.txt file, and, according to the user guide, an @ArchIgnore annotation. Treat rule files like the records they enforce: change them only together with a record, and consider a code-owner rule on the path.

That runtime check held a Spring trap of its own. TransactionSynchronizationManager.isActualTransactionActive() still returns true inside an after-commit callback, when the work is already committed, so the obvious assertion would fail correct code that uses @TransactionalEventListener. There is a test for that too, and the shop's transaction manager exposes its own flag:

// fixture/base/src/main/java/com/wolinski/shop/platform/InMemoryTransactionManager.java
/**
 * True from begin until commit or rollback. Spring's isActualTransactionActive() stays true during
 * after-commit callbacks as well, when the work has already been committed.
 */
public static boolean isOpen() {
    return OPEN.get();

After-commit code holds a second trap, and it caught me. My first reference solution booked the shipment in a @TransactionalEventListener and saved the tracking number right there. Its tests passed. In production it would have lost the number, because Spring's Javadoc warns that "any data access code triggered at this point will still "participate" in the original transaction, but changes will not be committed to the transactional resource." The synchronization Javadoc gives the fix: "Use PROPAGATION_REQUIRES_NEW for any transactional operation that is called from here." This shop's repositories are in memory, so no test here can see the difference. The reference solution now does it properly, by reading the warning and not because a test failed.

ADR 5 has one more cost, which it names but cannot solve. If the process dies between the commit and the call to the carrier, the booking never happens. When that matters, the answer is a transactional outbox, which deserves a record of its own.

The experiment: does an agent follow your decisions?

Teams now point coding agents at their repositories, and "add it to the docs" has become a common answer to "the agent keeps doing X". I wanted to measure that on one concrete setup. Does an agent follow a decision that exists only in a record? Does it find a record nobody tells it about? What does a rule add?

The setup

The subject is a small modular-monolith shop on Java 25, 28 production classes and interfaces, using Spring Framework through the Spring Boot 4.2.0-RC1 parent and ArchUnit 1.5.1. The shop and its history are invented. So are its decisions: basis points and lookup records are not my recommendations, they were chosen to go against what a careful engineer would do by default, and the incidents in their context sections never happened.

The agent gets seven tools: list, read, search, write, edit and delete files, and run_tests. It has no shell. It gets one ticket at a time, and tickets never mention decisions, though each ends with "Make sure the tests pass". The repository's AGENTS.md goes into its system prompt, the way coding tools load such files, so the pointer arm's instruction arrives there: "Architecture decisions are recorded in docs/adr. Read them before changing code, and follow them." Five arms vary only what the repository says about decisions:

Five arms lead into one agent, Haiku 5.5, with read, edit and run_tests tools: none, with no records and no rules; rules, with ArchUnit rules only; folder, with docs/adr and nothing pointing at it; pointer, with docs/adr and an instruction to read them first; enforced, with the pointer and ArchUnit rules. From the agent's output, its src/main goes to a hidden grader that uses rules, a source scan and runtime checks, and also checks that the feature works. Its tests, pom and ignore files are thrown away before grading. — Figure 6. Five versions of one repository, and a grader the agent cannot see or influence.

Grading follows the principle of my article on evaluating agents: grade the world, not the transcript. The grader takes only the agent's src/main, throws away its tests, its pom and any ignore files, and puts its own around it. It runs the agent's own suite once beforehand, only to record whether CI would have been green:

// src/main/java/com/wolinski/adr/Grader.java
Path clean = Files.createTempDirectory(workspace.root.getParent(), "grade-");
Workspace.copyTree(workspace.root.resolve("src/main"), clean.resolve("src/main"));
removeArchUnitConfiguration(clean.resolve("src/main"));
Files.copy(fixture.resolve("grader/pom.xml"), clean.resolve("pom.xml"));
Workspace.copyTree(fixture.resolve("grader/src/test"), clean.resolve("src/test"));

The tickets come in two groups, and that split turned out to be the whole story:

ticket	the decision it tempts	group
`discount`	ADR 8: rates are whole basis points, no `BigDecimal`	team
`nightly-expiry`	ADR 9: recurring work is a `ShopJob`, no `@Scheduled`	team
`latest-for-customer`	ADR 10: a lookup returns a record, not `Optional` or null	team
`no-cancel-confirmed`	ADR 11: failures are `ShopProblem`s	team
`placed-on`	ADR 12: the business day is in Warsaw (no rule exists)	team
`ship-on-confirm`	ADR 5: no remote calls inside a transaction	team
`refund-on-summary`	ADR 2: modules talk through their API	practice
`expire-stale`	ADR 4: time comes from the injected `Clock`	practice
`cancel-mail`	ADR 6: notifications go through events	practice
`last-payment`	ADR 7: API types do not expose internals (no rule exists)	practice
`order-note`	none: the control ticket

The repository also has old code that breaks nearly every decision, as real repositories do, and the records name it as "not a pattern to follow". A class that predates a decision is exempt in the grader only while it stays byte for byte as it was.

Proving the harness before paying for it

Before any paid call, scripted agents with known behaviour had to get the known result. This is the workflow from the same evaluation article, and it found real problems for free, including the local double above:

scripted agent	what it does	what the harness must report
oracle	Solves each ticket within the decisions	Feature works, nothing broken, in every arm
shortcut	Solves each ticket the tempting way	Feature works, exactly the tempted decision broken
null	Does nothing, claims success	Feature fails, nothing broken
tamper	Takes the shortcut, deletes the rules	CI green, decision still broken, deletion reported
evader	Hides the carrier call in a helper	CI green, ADR 5 still broken

My first two designs measured nothing

The first pilot was one repetition of the catalogue I had first written: six tempted decisions such as no floating point for money, an injected Clock, and module boundaries. Even with no records at all, the agent broke only one of six. My reading is that these are simply good engineering, and a capable model does good engineering. Either way, there was nothing to measure.

I suspected the clean code was teaching it, so the second pilot added old code that breaks each decision, right where the agent would look. The result was the same: one of six. Copying bad neighbours was not enough to change what it did.

Only then did I see what was wrong: my decisions were common knowledge. So I wrote a third catalogue of decisions that go against a careful engineer's default, and each has a specific reason that exists only inside the team. The third pilot broke four of six without records. That is the catalogue the main run uses, with the old one kept alongside it for comparison. All three pilots are committed under runs/.

One judgment call is worth flagging, because it favours my thesis. ADR 5 was in the first catalogue as common practice, and the agent broke it in every pilot, so I moved it to the team group after seeing that. If you put it back, the team group is 18 of 25 and common practice is 8 of 25 in the arm with no records. The pattern holds, and the gap is smaller.

The results

The main run used Claude Haiku 5.5 at medium effort, with 11 tickets, 4 arms and 5 repetitions: 220 episodes, in shuffled order, with no plumbing failures. The rules-only arm was a separate follow-up of 30 episodes on the team tickets, added once the main run showed it could not answer that question. The figure counts episodes where the agent broke the decision its ticket tempted.

Bar chart. Team decisions broken: none 23 of 30, rules 5 of 30, folder 1 of 30, pointer 0 of 30, enforced 0 of 30. Common practice decisions broken: none 3 of 20, folder 0 of 20, pointer 0 of 20, enforced 0 of 20. — Figure 7. Episodes that broke the tempted decision. The common-practice numbers include one correction by hand, described below.

arm	team decisions broken	95% interval*	common practice broken
none	23 / 30	59–88%	3 / 20
rules	5 / 30	7–34%	not run
folder	1 / 30	1–17%	0 / 20
pointer	0 / 30	0–11%	0 / 20
enforced	0 / 30	0–11%	0 / 20

*Wilson intervals. They treat episodes as independent, which they are not: outcomes cluster heavily by ticket, as the next table shows. Treat them as a lower bound on the uncertainty. The real sample is six tickets.

By ticket, with no records, the team group is close to all or nothing:

team ticket	none	rules	folder	pointer	enforced
`discount`	5/5	0/5	0/5	0/5	0/5
`latest-for-customer`	5/5	0/5	0/5	0/5	0/5
`placed-on`	5/5	5/5	0/5	0/5	0/5
`ship-on-confirm`	5/5	0/5	0/5	0/5	0/5
`no-cancel-confirmed`	3/5	0/5	1/5	0/5	0/5
`nightly-expiry`	0/5	0/5	0/5	0/5	0/5

"Broken" with no records is not a criticism of the model. Nobody told it. A BigDecimal is the right call for a discount unless you happen to work at this shop. That is the point of the arm: it shows how much of a team's architecture is impossible to guess.

"Common practice" did not predict this cleanly. "Do not call a remote system inside a transaction" is well known, and it was broken five times of five. Maybe the reason is that OrderFacade.confirm is already @Transactional, so the call fits right there. Maybe the idea is known without being the first thing anyone thinks of. I can only say that my intuition about which decisions need writing down was wrong at least once in ten, so use the distinction as a tendency, not a rule.

A visible example can do a record's job. nightly-expiry broke zero of five with no records, because the agent found the ShopJob interface and a job implementing it: all five episodes wrote implements ShopJob. A decision with a visible, working example needed no record. The old @Scheduled method in OrderReports was not copied in any of them, so old code does not always win.

The forgotten folder was not forgotten. I expected an unreferenced folder to be close to useless. The agent read at least one record in 53 of 55 episodes, and the one relevant to the ticket in 49 of 50. The single miss is instructive: in no-cancel-confirmed, repetition 5, it opened no record at all and threw IllegalStateException, explaining that "I followed the same exception style as OrderReturns". That is the one episode where the folder failed, and it is exactly the old code the record warns about. But this is a repository of 28 classes: in 44 of those 55 episodes, the very first tool result already showed docs/adr. I would not assume the same of a repository with thousands of files.

It handled the supersede chain. ADR 3 still tells the reader to use BigDecimal, with a status pointing to ADR 8. In all 15 discount episodes with records, the agent read both and used basis points.

Rules without records

The main run has a blind spot: since the agent reads the records anyway, enforced could not beat pointer. No rule ever went red in the enforced arm. That says nothing about whether rules work, which is why I added the fifth arm: the ArchUnit rules, with no records and nothing in the instructions about them.

Left panel: the five team decisions with a rule. With nothing in the repository, 18 of 25 were broken. With the rules alone, 0 of 25. In 16 of 25 episodes the agent read the rules first, and in the other 9 a rule went red. It followed them every time. Right panel: ADR 12, the one decision with no rule. With the rules alone, 5 of 5 were broken: new code in placedOn copied UTC from the old OrderReports code. CI stayed green, and the numbers will disagree with accounting. — Figure 8. A rule did the record's job where it existed. Where it did not, the old code won.

The two halves are the most useful result of the experiment.

Where a rule existed, it was enough: 0 of 25 broken, against 18 of 25 with nothing. Each of those 25 episodes went one of two ways. In 16, the agent read ArchitectureTest.java before its first test run and followed it. In the other 9, it wrote the default code, a rule went red, and it fixed the code. In none of the 9 did it touch the rule. I did not stage that: the deleting and ignoring paths were open to it, and my own tamper agent had shown they would turn CI green. Nine episodes on one model is not a guarantee, so keep the protection from earlier.

Where no rule existed, nothing stopped it: placed-on broke the Warsaw decision in 5 of 5, with a green build. The reasoning in its summary is the failure mode in one sentence: "That matches how OrderReports.placedToday() already assigns orders to days." It looked for a precedent and found the old code. With the same ticket in the arms with records, it broke nothing: 0 of 15. Old code did not sway it in the second pilot, and it did here. My reading, not a measurement: a precedent wins when its choice looks reasonable. UTC does, and a double for money does not.

So a rule and a record are not substitutes, and this data shows why. A rule is a precise check on what a machine can see, hard to miss by accident, and its because smuggles in a short version of the reasoning. A record covers what no rule can express, and it is the only place the reasoning fully lives.

What it cost, and what I got wrong

All paid calls together were 335 episodes for $1.60, so the money is not the barrier. Records do make each episode bigger, because the agent reads them. Input tokens below include cache reads, which were about 80% of them in every arm, so the cost grows more slowly than the tokens do:

arm	records read per episode	input tokens per episode	cost per episode
none	0	43,244	$0.0035
folder	3.1	58,912	$0.0045
pointer	5.9	81,322	$0.0056
enforced	5.8	90,829	$0.0060

The pointer arm costs about 60% more than no records in this setup, and 25% more than the unreferenced folder, while finding almost the same answer. On the team tickets alone, the rules arm cost $0.0046 per episode and the pointer arm $0.0057, so rules are the cheaper of the two, and they leave the gap you saw on ADR 12. With twelve short records that is small. With two hundred it would not be, and it is a reason to keep records short and well named, so an agent can pick the two that matter from the file names.

Reading the transcripts changed one number and how I read several others, so here is what the raw report does not tell you:

A false alarm in my grader, corrected by hand. The raw report says pointer broke 1 of 20 common-practice decisions. It did not. In cancel-mail, repetition 1, the agent also changed emailOf to follow ADR 10, and correctly updated its caller in OrderReturns. That cost the class its exemption, and its old violations were counted against the agent. The article says 0 of 20, and the committed run file still shows the raw value.
Three more of the same in the rules arm. These did not touch the headline: on discount, integer code added to OrderPricing cost that class its exemption and raised ADR 3, not the tempted ADR 8.
A rule broader than its decision. ADR 11 is about business rules, yet the grader flagged an IllegalArgumentException inside a record's constructor, which is a programming error. ArchUnit cannot tell these apart. It is why the headline counts only the tempted decision, and not "any violation".
A grader bug the null agent found late. Rerunning the scripted agents on the final catalogue, the agent that does nothing was reported as breaking ADR 10, because a test treated the missing method as a violation. I fixed the test and added a check on every ticket. It could not have touched the results: every ADR 10 hit in the paid runs is on a feature that exists, as the run files show.
A shop that breaks its own decisions by accident. RefundNotifications sends mail from a plain @EventListener, inside the transaction. That breaks ADR 5 as written, is not named as an exception, and undercuts ADR 6, whose context complains about a slow mail server: a synchronous event does not fix that. I only noticed in review, and my reference solution for cancel-mail copied the mistake, while the agent used an after-commit listener in 19 of those 20 episodes. The runtime check only watches the carrier, so none of this moved a number. The shop stays as it is, because that is what the paid runs saw. It is also a useful warning: writing a decision does not mean it achieves its purpose, and no rule here would tell you.
A bug in one of my tickets, which became the puzzle below.
Runs I cannot re-grade. The harness keeps transcripts, with long tool output trimmed, and it does not keep the final source tree. I should have saved it, and could not re-score these runs under a corrected grader, which is why the corrections above are by hand.

What this does not show

23 of 30 is not an estimate of anything in nature. I rewrote the catalogue until the arm with no records broke decisions, because an experiment where nothing breaks cannot compare anything. The number shows the arms differ on tickets built to tempt. It says nothing about how often an agent breaks a real team's decisions.
It is one model. I capped spending at $2, which left no room for a second model at this sample size. Other models may read less, or infer more. The arms with records sit at or near zero, and with no weaker model to compare, I cannot say whether a less capable agent would stay there.
It is a tiny repository. 28 classes and twelve records, all easy to find. Discovery is the hardest part in a large repository, and this setup barely tests it. The folder's result may not survive scale.
It has six team tickets, written by the same person who wrote the decisions and the grader. They are a sample of what I could think of, not of real architecture.
The rules arm ran separately, right after the main run and on the same model and settings, so it was not shuffled with the other arms.
It says nothing about people. It does not show that records help human teams, or that they cost humans so little. An agent reads a folder without getting bored, and a person may not.
One session, one ticket. It does not cover decisions eroding across hundreds of changes, which is where rules should matter most, and which this design cannot measure.

A puzzle

You review this pull request. The ticket says:

The admin screen needs the most recent payment of an order. Add lastPayment(String orderId) to PaymentsFacade; the returned value has amount() of type Money and status() of type String.

The repository has this accepted record, and an order may have no payments:

A facade method that may find nothing returns a record named <Thing>Lookup from its api package, with boolean found() and the value, which is null only when found() is false. Facades neither return null nor use Optional.

The agent added LastPaymentLookup(boolean found, LastPayment value). All rules are green, and its tests pass. The admin screen's acceptance test does not compile, because it calls amount() on the result.

Who is wrong: the agent, the ticket or the record? And is there an implementation that satisfies both? Think about it before reading on.

The answer

Yes, there is one, and it is a single record: PaymentLookup(boolean found, Money amount, String status). It has found(), as the decision asks, and amount() and status(), as the ticket asks, with the values null when nothing is found. In the arms with records, this ticket ran 15 times. Four episodes found that shape. Eleven wrapped the value, followed the record, and failed the acceptance test. In the arm with no records, nothing got in its way, and the feature worked all five times.

Nobody is really wrong, and that is why this is a useful failure. This bug was mine: I added ADR 10 after writing this ticket, and never noticed they had collided. The ticket's author did not know the record existed, which is the normal state of a ticket's author. The agent's behaviour is the part I would keep. Every one of the 15 summaries mentioned the decision, and this one states the conflict and hands it back:

"If the admin screen needs the flat form, that would be a change to the ADR, not just the code."

Notice what happened to authority here. A decision file outranked an explicit instruction from the current ticket, in 11 of 15 episodes, and the tests stayed green. That is what you want when the ticket is careless, and not when the ticket is right and the record is stale. Either way, when your records can outrank your tickets, someone has to read the summary. An agent following a decision that no longer applies looks the same as one following a decision that does, which is one more reason to keep the status lines honest.

What I would do on Monday

Start with the decisions an outsider would get wrong: the odd rule with a story behind it. Those disappear first. Do not assume the rest are safe. If a well-known rule matters, a one-line ArchUnit rule costs less than an outage.
Use a short format, and answer the confirmation question in each record: how would we notice this being broken?
Let the compiler reject what it can. Turn most of the rest into a rule, with the record's number in because. In this experiment the failure message alone corrected the code in all 9 episodes where a rule fired, without any record.
Name your old exceptions in the record, and freeze them instead of listing classes, so new code cannot hide behind old code. Both failures quoted in this article name old code as their reason.
For decisions no rule can express, write a behaviour test per feature, like the Warsaw midnight case. Otherwise only the record protects them, and only if somebody reads it.
Replace the precedent, not only the rule. A working example made ShopJob obvious without any record. Migrating the worst old code is also a documentation change.
Point to the log from your agent instructions. It made little difference here, and costs little. In a large repository I expect it to matter more, though that is an expectation and not a measurement.
Protect rule files as you protect records, and read what agents say about conflicts: it is often the first sign that a record or a ticket is out of date.

Run it yourself

The repository needs Docker and bash. ./run.sh test runs the tests and scripted agents inside a pinned Maven and JDK 25 image with no API key, and takes about 15 minutes, since each scripted episode compiles and tests a real Maven project. ./run.sh eval --agent oracle runs the whole catalogue for free. With an API key, ./run.sh eval --agent claude --reps 5 --budget-usd 2 runs your own measurement, and --budget-usd is a hard stop on new requests. Write a decision of your own in fixture/overlays/folder/docs/adr, add a ticket that tempts it, and see whether your team's oddest rule can be guessed.