I ran the framework against my own system and published the score.

FintelliGuard · self-assessment · August 2026 · method

Every consultant selling an assessment should be willing to take it first. So here is the Responsible AI Readiness Framework run end to end against FintelliGuard — an enterprise RAG compliance agent on AWS Bedrock, paired with real-time fraud scoring.

It scores 81 out of 100: production-ready, not audit-ready. Three dimensions cost it real points, and the gaps are documented in the repository's own README rather than discovered here. That is the standard — a readiness score is only worth reading if it can come back lower than you wanted.

Revised August 2026 This system scored 84 under the previous six-dimension version of the framework. Adding Risk & human oversight — Article 9, Article 14 and Article 27 of the AI Act, which the framework had been quietly skipping — cost it three points, and the system itself did not change. Better a framework that can lower your score than one that only ever confirms it.

fintelliguard Production-ready

81 % production ready

Band71–90 · Production-ready

WeakestObservability · Risk & oversight

Risk & human oversight

weight 15%67

What earns it

The human is the last tier, in code. Tier 1 scores every transaction, Tier 2 reasons under guardrails, Tier 3 is a human analyst reviewing what was flagged. Releasing a transaction the model flagged is a person's decision, never a generated one.
Escalate, never soften — enforced by the gate. A recommended action more cautious than the model's hint is accepted with a stated reason; anything less cautious is refused outright. Human oversight the system cannot quietly route around. gate_proof.py →
Intended purpose is written down and committed, and the classification — high-risk, financial decisioning affecting access to payment services — is stated rather than avoided.
The analyst gets the score and its top contributing drivers with every verdict, and the gate refuses a verdict that omits them. The confidence behind a decision is never optional.

What costs it

There is a risk classification, not a risk assessment. Two lines saying "high-risk" is not Article 9: nobody is named as affected, no likelihood is estimated, no mitigation is mapped to a harm, and nothing forces a revisit when the system changes. The largest gap in this dimension.
Limitations live in the model card, not in the interface. An analyst reading a verdict is not told what the system is known to be bad at — they would have to go and find the document.
No lawful basis is recorded for the automated decision, and out-of-scope requests have no explicitly defined behaviour. Defensible for a reference build on synthetic data; not defensible the day it scores a real person's transaction.

Data quality & lineage

weight 15%87

What earns it

A medallion pipeline with declared expectations at each hop; rows that fail are quarantined with the rule they violated, not dropped.
Train/serve parity is enforced and proven by test — the same 14 Gold features feed training and serving, so skew cannot appear silently.
Every scored transaction writes a replayable Article 12 decision record, so any output traces back to the exact inputs that produced it.
PII is classified at the data layer, and the decision record refuses to be written if it would carry raw PII.

What costs it

The Gold layer full-recomputes per card rather than using Spark stateful streaming — correct, but it means lineage at scale is untested.
A data-quality metric was once able to only ever read 100%. It was found and fixed, but it is the kind of blind spot that argues for an independent check on the checker.

Guardrails & safety

weight 20%100

What earns it

A five-check deterministic gate — schema · no-PII · grounding · faithfulness · decision — sits between the model and the analyst. Cited articles must exist in the retrieved context by set membership, so a fabricated article appended to a real one is refused.
The agent may escalate a decision but may never soften one. That asymmetry is enforced in code, not requested in a prompt.
Guardrails bound at an immutable version and CI-gated against a labelled set in both error directions, because a guardrail that blocks everything is broken, not safe. The offline red-team score is a regression score rather than a measured block rate, so the number quoted here is the gate proof: 80 planted violations, each refused by the named check for the named reason, on every pull request. gate_proof.py →
The tests are themselves tested. make gate-proof breaks each control on purpose and fails unless the real gate refuses it for the right reason — and reports STALE rather than passing when a mutation's target has moved. gate_proof.py →

What costs it

Nothing at this level of the scale. The honest next step is not a higher score but an independent red team — every probe in that set was written by the same person who wrote the defence.

Observability & drift

weight 15%53

What earns it

A Prometheus exporter emitting the exact metric names the dashboards read, with a one-command local funnel that makes the observability claim reproducible instead of aspirational.
Each decision is traceable end to end: request → features → model → guardrails → output.

What costs it

The scheduled drift monitor is deferred. Drift can be computed; nothing runs it on a schedule and nothing pages anyone when it moves. This is the single largest gap in the system.
No alert routes to a named owner. A dashboard nobody is paged from is decoration.
LLM tracing is not wired into the live deploy.

Governance as code

weight 15%93

What earns it

The EU AI Act Annex IV technical document, the model card and the dataset card are rendered from the source and fail CI when they drift from it. Documentation cannot quietly go stale, because staleness is a build failure.
A model-promotion gate blocks anything below AUC-ROC 0.83 and fraud precision 0.85 — the live run passed at 0.8661 and 0.8699, and the numbers were not moved to fit.
Article 12 record-keeping is produced by the system rather than assembled for an occasion.

What costs it

Access control is least-privilege IAM in Terraform, which is correct, but there is no queryable audit trail of who read what — the governance-as-code work that answers that lives in a different repository, and cross-system claims do not count here.

Tested & reproducible

weight 10%100

What earns it

592 CI-gated cases across 72 files, all credential-free — real local PySpark, real XGBoost and MLflow, real LangGraph, mocked clients only at the cloud boundary. Anyone can run the suite without an AWS account.
Infrastructure is offline-validated per layer, and the entire estate was provisioned on real AWS and Databricks by one CI dispatch in 1h 37m, then destroyed to zero cost. Reproducibility proven by doing it, not by claiming it.
Versions pinned; teardown is a first-class guarded workflow.

What costs it

Nothing measurable at this level. Note that a maximum here means the suite is trustworthy — not that the system is finished.

Self-healing reliability

weight 10%60

What earns it

A remediation layer with deterministic, idempotent handling of known incident classes — endpoint-latency rollback, consumer-lag scaling, bounded pipeline retry — covered by 37 tests against real LangGraph.
Destructive operations are guarded behind typed confirmation, so a teardown cannot happen by accident.

What costs it

That remediation layer is tested-only. It is not wired into the live deploy, so it has never recovered anything in anger. The repository says so plainly rather than implying otherwise — but a control that has not run in production does not score as one that has.
Feature resolution falls back to a bundled snapshot rather than a live online store, which is a reliability dependency the demo does not have to face.

Why publish a score that isn't 100

Because a scoring method nobody can fail is a marketing device, not an assessment. Two of six dimensions came back amber on a system I built deliberately to be defensible — and the way to raise them is obvious and unglamorous: schedule the drift monitor, route an alert to a person, and wire the remediation layer into a real deployment.

That is what the audit produces for a client: not a verdict, but a short list of specific, boring, high-leverage things — ranked, with the evidence for why each one is on the list.

Get this for your system Score yours in three minutes Read the build in full