Writing · attestor

The Model Wrote The Report. Not One Of The Numbers.

Attestor · 21 min read · 4,586 words

A language model wrote a full CSRD sustainability statement — the disclosures, the methodology notes, the prose that explains what each figure means and cites the evidence behind it.

It did not write a single number in that document. Not one digit. It is structurally incapable of it, and the build fails twice over if it tries.

That is not a stylistic preference. Under the CSRD an external auditor reads the finished statement line by line and asks where each figure came from. A plausible number in a regulated filing is worse than no filing at all, because nobody can tell it apart from a real one until somebody asks — and by then it has been signed.

So I built a system on one rule and never moved it: the model writes sentences, code owns every number, and the decision to refuse belongs to neither.

What it is

Attestor is a multi-tenant factory for regulated reports. An advisory firm serves several client companies; each must publish something an auditor will inspect — a CSRD/ESRS sustainability statement, or an EU AI Act Annex IV technical file. Attestor produces those documents end to end: DOCX, XLSX and PPTX, with the lineage of every figure printed inside them.

It runs on AWS Bedrock AgentCore, Knowledge Bases and Guardrails, over an Iceberg lakehouse on Athena, with Cedar for authorization and Terraform for all five infrastructure layers.

Three tenants exercise it. Two are peers in the same vertical — helios and aegis, both under CSRD — which is what makes a leakage test mean anything. The third, lumen, files under a completely different regulation, through the identical code path. Its first engagement is Attestor itself: the platform produces its own EU AI Act technical documentation, from its own repository as the evidence corpus.

The name is the auditor's word. Attestor — one who attests.

Look at the prose underneath it twice. The first pass shows what is there: every sentence carries an evidence marker — [ev:32e43254], [ev:1f58c0dd], [ev:fc15487f] — so each claim names the document it came from. The second pass shows what is not there. The transition plan is described as resting on "four decarbonisation levers", against a pathway "limiting warming to one and a half degrees". Not 4. Not 1.5°C. The model wrote those words that way because writing them as digits would have failed the build, and that is the rule showing up in the grammar of the sentence.

The one line the whole design rests on

The clearest statement of the boundary is one line of a template and the same line in the finished document.

The narrative layer is never handed a figure it may place. It receives a list of placeholder ids and emits {{dp:...}} where a number belongs. The deterministic layer resolves those ids against pinned Iceberg snapshots and substitutes the values afterwards.

Enforcement is not a convention or a code review habit. A draft is checked the moment it comes back from the model: strip the citation markers, strip the placeholders, and if any character that remains is a digit, the draft is refused. Not thresholded, not allowlisted — zero digits in model-authored prose.

This is the part where a rule that sounds absolute usually quietly isn't, so here is the failure I hit. The digit rule originally ran over the raw draft, which meant the placeholder {{dp:ESRS_E1-6_gross_scope_1}} contained the digits 1 and 6, and a model that had done exactly what it was instructed to do was told it had broken the rule. The prompt and the check were in direct contradiction. Stripping identifiers before the scan is what keeps the rule absolute for prose rather than softening it to accommodate them.

Two tenants published. The third did not.

Three tenants, one code path. The two CSRD peers, helios and aegis, share their templates as well — so between those two, the only thing that differs is the data.

Read aegis's four failures, because their shape is the argument:

• ESRS_E1-6_gross_scope_1 — E_OUT_OF_TOLERANCE. The independent cross-check against fuel spend disagrees with the primary resolver by 4.3045% against a declared bound of 0.5000%.
• ESRS_E1-6_gross_scope_3 — E_UPSTREAM_QUARANTINE. 1,284 source rows failed their data contract.
• ESRS_E1-6_total_ghg and ESRS_E1-6_ghg_intensity — both marked inherited from. A derived figure inherits its parent's defect instead of averaging it away. Total emissions cannot be sound when a component of the total is not.

And the reason codes are checkable one level down, without taking the system's word for anything:

The refusal is the feature, and it is the cheapest thing in the repository to get right. Under the CSRD, "not disclosed, and here is the reason" is what the standard asks for when a figure cannot be supported. It is a legal requirement, not a design nicety. A system that always produces a document is not compliant — it is confident.

Which is why the reason codes are a closed vocabulary, split down the middle. Four codes are lawful omissions: not material, phase-in, seriously prejudicial, not applicable. Each names the article of the standard that permits it, prints in the report, and an auditor accepts it as an answer. Seven are internal failures: a crashed resolver, quarantined rows, evidence out of period. Those are our problem, they block the report, and there is no configuration flag to soften that.

The distinction is the whole point. Without it, "our pipeline broke" quietly becomes "the datapoint was not material" — a bug laundered into a regulatory exemption, in a document nobody can un-sign. A test fails if anyone adds a way to blur the two, and a free-text reason is fatal on sight.

Every number carries its proof

Each published figure is emitted with the resolver that produced it, the source tables it was read from with their pinned snapshot ids, and a lineage identifier — and that annex is printed inside the document the auditor receives, not held in a log they will never open.

That snapshot id is not decoration. It is checkable by hand, by somebody who does not trust me:

Re-resolving as of an earlier instant yields identical values and identical lineage hashes. That is what makes a restatement a deliberate, declared event rather than something you discover when two copies of last year's report disagree.

The gate reads the finished file, not the data that went into it

Here is where most provenance stories quietly stop. It is easy to check the values you are about to render. It is harder, and much more useful, to check what the renderer actually produced.

So the provenance gate opens the finished DOCX, XLSX and PPTX — the binaries, as a reader receives them — pulls out every scrap of visible text, and asks where each numeral came from. A digit may legally appear for exactly two reasons: a resolver produced it, or it was in template text a human reviewed in a pull request. The manifest knows the multiset both sources permit. Anything beyond it is a numeral nobody can account for, and the build fails.

Getting the scope of that scan right was more interesting than the rule. Office files ship with slide masters, layouts and theme boilerplate — a slide master carries the literal text of its date placeholder, so an untargeted scan reports numerals like 27 and 13 that no author wrote and no reader will ever see. The obvious fix is an allowlist, and an allowlist is where a gate goes to die. The actual fix is scoping to content rather than chrome: body, headers, footers, notes, slides, speaker notes, worksheet cells and shared strings. A part outside that list cannot put a visible numeral on a page.

An honest limit, stated because it exists: the numeral check is over a multiset, not over positions. A model-authored digit that happens to match a permitted one, in a document with spare count, survives that question. It does not survive the second one — narrative runs are held to the absolute zero-digit rule, in the manifest and again in the file. Both checks exist because neither is sufficient alone.

Every closed door has a key. One door has none.

A control with no override does not prevent the override. It moves it outside the system, into a spreadsheet and a phone call, where it leaves no evidence. That is worse.

So a known gap may still ship — under rules:

1. An override changes what ships, never what is true. It never relabels a defect as compliant. The reason code survives the override, in the record and on the page.
2. It is visible in the artefact, not only in a log. CloudWatch is not where an auditor looks.
3. It expires. On expiry the finding returns and CI goes red again.
4. Severity decides who turns the key — how many approvers, in which roles, for how long.
5. The system may never open a door for itself. No model, no agent, no service principal may request, approve or classify one.

A register is still a place a reader has to go looking. The rule that costs something is the next one.

And then one door has no key at all. E_RESOLVER_ERROR — a crashed resolver — cannot be overridden by anyone, at any severity, ever. The reasoning is short: a crash is an unknown deficiency, so nobody, including the approver, has the information the approval would be about. Signing it off is signing a blank page.

That is the whole article in one frame, and it does not flatter me. The model did try to write a number. It was not stopped by a prompt asking it nicely, or by a reviewer noticing — it was stopped by a check that reads the draft and refuses it, and then by a retry budget that runs out rather than lowering the bar. And because the failure is E_RESOLVER_ERROR, nobody can sign it away: not me, not a second approver, not at any severity.

This is also the run where aegis has five blockers rather than the four in the terminal above. Both counts are correct. Offline the narrative replays a recorded draft; in CI it is written for real, and the datapoint that fails here is one that cannot fail when the draft is a fixture.

Having exactly one unopenable door is what keeps the other five honest. A break-glass that opens everything is a rubber stamp with extra ceremony.

The evidence corpus is untrusted, and I mean that literally

Everything above assumes the evidence is merely wrong sometimes. It is also, in this repository, hostile.

Supplier attestations, invoices, model cards and evaluation reports are content written by other people. Anyone who can put a document into the corpus can try to instruct the model through it. That is not a hypothetical here — it is a real supplier attestation sitting in the repo, indexed and retrievable like any other document.

Now the part that matters more than the detection. Detection is not the defence. Detection is a classifier, a classifier has a false-negative rate, and an attacker gets to iterate against it. The defence is structural, and it is spread across the rest of the system:

• A model cannot produce a figure — a narrative run carrying a digit fails the build.
• A model cannot choose its own tool arguments — tenant, period and scope come from the session, and the session comes from a token whose issuer is bound to the tenant it names.
• A model cannot authorize anything — Cedar decides before execution, and no automated principal may sign an override.
• A model cannot widen its own retrieval — the metadata filter is built from the session, not the conversation.

So an injection that succeeds perfectly — the model believes every word of it — still cannot change a number, call a tool, approve an omission or reach another tenant.

What the scanner adds on top is signal: a supplier trying to instruct the reporting system is a finding worth acting on, whether or not it would have worked.

Scored offline on a labelled corpus on every push: 16/16 poisoned passages flagged, 0/11 benign wrongly flagged. The false-positive number is the one that decides whether a control is still switched on in six months. A detector that flags real supplier letters gets disabled within a week, and then you have neither the control nor the knowledge that you lost it.

One tenant never sees another

Isolation is not a filter in one function. It is a gateway, a runtime and a memory store per tenant surface, a Cedar policy engine whose every policy is scoped to exactly one of those gateways, and a retrieval filter that refuses to run unscoped.

The tool surface declares six tools. An agent can reach five. request_override exists, is forbidden to every agent by an explicit Cedar forbid, and asking for it returns Tool Execution Denied … [Policy evaluation denied due to forbid_override_through_the_agent_helios]. The system may not open a door for itself, and that sentence is enforced by something other than the system's own good intentions.

Retrieval is scoped the same way, and it is worth watching it happen with one variable moved and nothing else.

Note what the first frame hands over: INV-HEL-2026-0009 is the poisoned supplier attestation, returned to helios without hesitation. The metadata filter is not a safety control and is not doing a safety control's job. It selects one tenant's documents, poison included, and leaves the poison to the scanner. Two different jobs, two different mechanisms — and conflating them is how systems end up with neither.

Consoles are where a boundary is shown. The eval is where it is measured: twelve distinct leakage routes run on every push — retrieval filter bypass, session tenant rewritten mid-request, cache-key poisoning, memory namespace collision, gateway tool-argument injection, a replayed token from another tenant's IdP, an override signed for someone else, and five more. They are twelve different routes, not twelve rewordings of one, which is the difference between measuring how well one control works and measuring whether any control is missing. All twelve must fail to leak.

Live, a helios token gets HTTP 403 insufficient_scope at the aegis gateway and Claim 'iss' value mismatch at the aegis runtime.

The system writes its own conformity file

lumen's engagement is Attestor. The platform produces its own EU AI Act Annex IV technical file, from this repository as the evidence corpus, through the same resolver engine, the same gates and the same renderers as a sustainability statement. Different contracts, a different corpus, a different template — identical code path. That is the generalisation claim, and it is the whole reason a third tenant exists.

Two of the figures in that file are not synthetic.

Those two rows are not asserted anywhere. They are read:

Weaken the detector and those rows change, the recorded values stop reproducing, and the Annex IV goes red with the build. The document does not describe the scanner. It is computed from what the scanner did.

Every other value in the lake is synthetic, generated backwards from recorded targets and labelled provenance: synthetic. These two are the exception, and the distinction is recorded in the repository rather than glossed — which is itself the honest version of a conformity claim.

Every gate is attacked on every push

A gate nobody has tried to break is a gate nobody knows works.

make gate-proof copies the repository, plants a real violation in the source, and fails unless the named gate refuses it for the right reason.

Three rules keep it a proof rather than a ritual, and each one exists because the lazy version of it is worthless:

1. Every gate must be green first. A mutation that "fails" a gate that was already failing proves nothing.
2. A non-zero exit is not evidence. The named check must report the failure. Otherwise a typo in the planted diff counts as a passing security control, which is how a test suite quietly becomes theatre.
3. A mutation whose target has moved is reported STALE, not passed. Refactor the code out from under a mutation and it starts silently succeeding at nothing.

The twenty-fifth mutation is stop reading headers and footers. It exists because I found that the test guarding those parts of the provenance gate asserted nothing at all — it was green, it had always been green, and it would have stayed green if I had deleted the code it was testing.

Deployed, verified, destroyed the same day

None of this needs a cloud to be checked. 489 tests, 40 preflight checks and all five claims run on a laptop with no AWS account and no credentials, in about fifteen seconds. Cloud is where proof is captured, not where logic is validated.

But the claims were also checked against a real estate. One dispatch applied four Terraform layers, seeded the lakehouse, built the dbt models, ingested the evidence corpora, stood up two AgentCore gateways and two runtimes, produced the documents, and then checked its own claims — in 21 minutes 47 seconds.

Then the estate was destroyed. It has never stood for a month, and that is structural rather than a habit: the deploy workflow takes a days input with no default, every resource carries an expiry tag that a scheduled reaper enforces, and an AWS Budget attaches a deny policy to the deploy role at the ceiling. Not an email — an action. An alert that arrives while nobody is reading it has never stopped a bill.

What this is worth to a business

The expensive failure in regulated AI is not a wrong answer. It is an unattributable one. A figure that is merely wrong gets corrected. A figure nobody can trace has to be defended, then restated, then explained to a regulator — and every other figure in the document becomes suspect at the same time. Provenance is not a nice-to-have on top of accuracy; it is what makes accuracy checkable by somebody who does not trust you.

Refusing is a product feature with a price tag. A blocked aegis run costs €0.022 and produces nothing. The alternative — a plausible Scope 1 figure that is 4.3% off, published, assured, and found later — costs a restatement, an assurance re-engagement and a paragraph in the press. The whole factory runs at €0.017 per report in model spend, and resolve_datapoint is €0.000000, because no model is involved in producing a figure. The meter attributes the cost of the refusal too, which is the only way anyone finds out that refusing is cheap.

"The AI decides" is not an architecture. The useful question about any AI system in a regulated path is not can the model do the task — it obviously can — but what stands between its output and the person who will act on it. Here that is a closed reason-code vocabulary, a resolver the model cannot reach, a gate that reads the rendered binary, and an override register with named humans and expiry dates. Four deterministic things. The model is a drafting tool.

Cost discipline is a design property, not a rule people follow. The infrastructure would cost roughly $536/month standing, and three interface-endpoint-and-OpenSearch line items are 98% of that. It has never stood for a month. Three real bounded blocks — stand up, capture, destroy — billed $0.62, $3.39 and $4.09, measured from the invoice rather than modelled.

What I'm not claiming

The tenants are invented and nearly every figure in the lake is synthetic — generated backwards from recorded targets, correct for a lake standing in for a client's ERP, and labelled as such. Nothing here has been through an assurance provider.

aegis ships no evidence documents at all, so when it reaches for helios's corpus and gets nothing back, that is the filter working and an empty index at once. The frame shows the boundary; it does not on its own isolate the cause, and the repository says so next to the screenshot.

Its blocker count is four offline and five in CI — both correct, for the reason shown above. Any count is a count in a mode, and quoting only the flattering one would be exactly the sleight of hand this project exists to prevent.

The retrieval bake-off has never run against a live embedding model. There is no load, latency or concurrency testing. The 32 live checks ran once, on one estate, and are reproducible by re-deploying — but this is not a continuously green integration environment.

A platform that documents its own gaps is auditable. One that doesn't is a demo.

The takeaway

The interesting boundary in an AI system is not between what the model gets right and what it gets wrong. It is between what the model is allowed to decide and what it is structurally unable to touch.

Here the model writes every sentence in a regulated filing and cannot write a digit, cannot pick its own tool arguments, cannot widen its own retrieval, and cannot approve its own exception. What is left for it is the part that genuinely needs language: interpreting a standard, finding evidence, explaining a figure it never saw.

And when the evidence does not support a disclosure, the correct output is not a smaller number or a hedged sentence. It is nothing at all, with a reason code and a named human beside it.

Full repo — the contract layer, the five Terraform layers, the eval harnesses, and the screenshots from the live run before it was torn down: https://github.com/theofanis-tsakanikas/attestor

If you run AI anywhere near a regulated output: what is the last thing that stands between a generated document and the person who signs it — a model, a review meeting, or code? I'd like to hear what's actually working.

---

One of a series of write-ups on the projects in my portfolio — each one a reference implementation of the trust layer that makes data and AI safe to ship.