AI coding agents write much of the code in OAS. They work from instructions kept in the repository. Instructions help, but an instruction file cannot prove, by itself, that the code that comes out respects the rules the system depends on.
When the instruction file became the problem
The project's main instruction file started small. Within about a month it had grown to 901 lines and roughly 71 KB, because every lesson learned was written into the same document. At that size it was hard for a person to find a rule, and hard to know which rules applied to a given change.
In September 2026 we split it in one change. What remains at the root is a short "constitution", 23,598 bytes today — 96% of its cap: what OAS is, the rules that hold everywhere, and a routing table. The table sends each kind of change to the rulebook it owes. There are six domain rulebooks and three instruction files that belong to particular directories. A test now caps the root file at 24 KB and checks that every rulebook the table names actually exists, so the file cannot quietly grow back into one long document.
That made the rules easier to find. It did not make them true of the code.
From advice to checks
Alongside the rulebooks sits a set of architecture tests, 39 of them, that fail when the code breaks a specific rule. For example, a check fails when:
- a second part of the code starts writing to a database table that already has a designated owner;
- application code reads a business, user or role identifier from a form, a query string, a webhook payload or a request header;
- someone adds an API route that is not on the approved list;
- code reaches the database around the one data layer the application is allowed to use.
Each of these checks states what it cannot see. A rule that a check cannot express stays in prose, and a person has to hold it.
The prose explains why a boundary exists. The tests catch the violations they can express. Neither replaces the other.
What counts as evidence
Our checks run locally on the change, and the result is recorded against the pull request's revision. Hosted CI runs on the same pull requests when the month's free Actions allowance permits it, but it is not what decides whether a change merges. The project's own rule is that a skipped check, or a deployment that went green, proves nothing on its own. The evidence is a full run of the checks, quoted with its output. An agent saying it ran the tests is not evidence; the output is.
What stays a human decision
Some decisions are not delegated because the code to implement them can be generated quickly. Product scope, credentials and security exceptions, what gets published, changes that reach production, and any claim made in public are reserved for the person who runs OAS, by written rule.
The point of the checks is not to make the system autonomous. It is to give that person sharper questions to ask, with evidence in front of them.
What we can claim, and what we cannot
The reorganisation and the architecture tests are specific engineering changes, and they are in the code today. They do not show that AI-assisted development is error-free, that agents replace experienced engineers, or that every mistake can be caught by the rules we have. They make some of the mistakes we know about much harder to miss.
If you want to know how we build the system that holds your customer relationships, get in touch.