What building an honesty-first AI consent system taught me about my own defects.

Created for the All Things Agentic Hackathon.

I spent August building Hodi, a consent control plane for creative work used by AI. A creator registers a work’s fingerprint and machine-readable terms. A buyer asks for a license in ordinary language. Gemini translates the request into a typed scope, but a deterministic policy lattice decides permission. If the creator changes their mind, a governed fleet of Google ADK agents runs a revocation cascade, signs the notices with Cloud KMS, and closes the standing offer. Nothing in the system is allowed to assert what it cannot verify.

That last sentence is the whole thesis. So while I built it, I kept a ledger of every place my own implementation violated it.

The ledger contains sixty-six real defects. They sort into nine classes. Four of those classes recurred after being fixed once, which is the actually interesting part, because a bug you fix twice is telling you something a bug you fix once is not.

The title says seven. The ledger says nine, and the count when I first published this piece was fifteen. The count moved because it is now generated from an enumerated ledger instead of typed into documents by hand, and the enumeration kept finding more than the prose had been carrying. A number repeated in seven places and derived in none is not a measurement. The title stays as a record of what I thought the shape was. The number moves, because a build step checks it now.

Here are the three findings that carry the ledger.

1. Comparing two attacker-controlled values is not an authorization boundary

Hodi’s first invariant is that no buyer can read another buyer’s terms. It is the reason the system has four agents instead of one, each under its own service account, separated by conflict of interest: the agent that holds creator identity must never see commercial terms, the agent that negotiates for one buyer must be incapable of reading another’s, and so on. Every inter-agent call crosses a policy gateway that logs every denial as a structured event.

The gateway worked exactly as designed. And the boundary was still breakable.

The licensing endpoint took counterparty_id from the request body and used it as both the database query filter and the session identity the gateway validated that filter against. The gateway was comparing the caller’s claim to itself, and it always agreed. One unauthenticated request with a signature field containing the literal string NOT-A-REAL-SIGNATURE returned another counterparty’s grant, their negotiated scope, and a receipt in their name.

One day later, the same class recurred three lines below the handler I had just fixed. The revocation route took no authenticator at all. Anyone could revoke any published work. And because the grant log is append-only by IAM policy (the deployed identity holds create but not update or delete), those writes could not simply be erased.

The lasting fix was structural, not local. Identity now derives from a verified credential, never from the body. The original exploit is a permanent regression test replayed against the deployed service. And a CI guard enumerates the router’s own routes and fails the build if any mutating method reaches an endpoint that never authenticates. That guard is twenty lines. Both defects would have been caught on first commit.

The lesson: a security invariant is not what your policy engine enforces. It is what your policy engine is handed.

2. Your observability stack needs the same adversarial testing as your product

Hodi runs a live evidence endpoint on Cloud Run that has logged every access since August 6, alongside a robots.txt that points at machine-readable consent terms at /.well-known/hodi.json. The idea: if AI crawlers visit, we can observe whether any of them looks for the terms.

My first published finding was a clean negative result: nothing identifying itself as a crawler had asked. It was a good line, and it was built on two bugs pulling in opposite directions.

First, the false positive. The list of self-originated user agents omitted Google-Cloud-Scheduler, so my own scheduled audit job was being counted as third-party traffic. My infrastructure was manufacturing my evidence.

Then, the false negative. The crawler detector required a word boundary before the string bot, so the commonest crawler naming convention there is (a vendor prefix glued directly onto bot) never matched. The zero was not evidence of absence. It was a regex that could not see.

After both corrections, the audited numbers as of 2026-08-29 read: 6,956 accrued records, of which 6,675 are my own instrumented tooling, 281 are non-self but browser-like and stay labeled unattributed, and 29 match a generic crawler signature. Those 29 visits came from three distinct self-identifying crawler user agents. Twenty-one of those 29 fetched /robots.txt. Exactly one, ever, fetched the consent document named inside it.

That last sentence is the product thesis in one audit: the crawlers ask whether they may enter, and almost none ask what they may do, because for most of the web there is still nowhere to ask it.

Note what the claim is not. A user agent is self-declared, so these records are attributable, not authenticated. No company is named, because the records cannot prove who sent anything, and the point was never who they are. The counts regenerate from source, and the build fails if this essay’s figures drift from the audit file.

The lesson: an evidence pipeline that is not tested adversarially will eventually testify in your favor, and that is exactly when you should not believe it.

3. Generating docs from source prevents drift. It does not make the source true.

Hodi must never claim that a work was inside a model’s training data, because nobody can reliably know that today. The evidence schema has no field for the claim, and a render-time lint rejects overclaim phrasings in generated text.

The README said the lint catches paraphrases. So I measured it. Against a twelve-paraphrase probe set, seeded from phrasings the lint was deliberately not written against, the regex layer rejected four.

The claim came out and the number went in. A Gemini embedding backstop now takes measured coverage to twelve of twelve, and it is admissible for exactly one reason: it can only ever add a refusal. It runs after every regex has declined, so a wrong embedding produces a false refusal, never a false permission. If the embedding model is unreachable, the system falls back to the deterministic four-of-twelve layer instead of silently trusting a model it cannot reach. Both numbers stay published, because one depends on a model and the other does not.

This finding generalizes further than the other two. Everything in this project that could be generated from source is generated from source: the IAM conflict matrix, the metrics in the README, the defect count in this essay. That prevents documentation from drifting away from reality. It does not prevent reality from being read wrongly. The sharpest defect in the ledger was a conflict matrix generated perfectly from a policy module whose enforcement function matched collections by prefix, so the document was accurate about a permission that was not the permission being granted.

The lesson: generation-from-source moves the lie one layer down. You still have to go look.

The meta-pattern

Every one of the nine classes reduces to the same shape: a stated property, a mechanism that does not enforce it, and nothing connecting the two. The property is always written down. The mechanism usually exists. What is missing is the wire that fails loudly when they come apart.

So the fixes that matter are not the sixty-six patches. They are the four guards: the route enumerator that fails CI on an unauthenticated mutating endpoint, the self-traffic classifier that is itself tested, the measured lint coverage that is republished on every build, and the doc-drift check that fails the build when a published number disagrees with its source.

One more, for anyone building on ADK. The agents in Hodi are deterministic BaseAgent subclasses (no LLM inside the authority hops, which is the point), and BaseAgent is a Pydantic model. Shared mutable state passed as a model field gets deep-copied per agent. Every agent mutated its own private copy, the orchestrator saw none of it, nothing errored, and the run reported success while returning nothing. It is now a regression test, and it is the kind of failure this whole essay is about: every component honest, the composition lying.

What the system refuses to say

Hodi will not claim your work was in a model’s training data. It will not claim revocation un-trains a model; it withdraws permission going forward and closes the offer. It will not call ownership verified until it is proven: even my own five registered works display as asserted, because the verifier refuses to upgrade a claim it cannot check.

A registry that flatters you is worthless. The only version worth building is one that is built to be believed.

You can try the whole thing, live, with no account: the platform has a Studio for creators, a Market for buyers, and a three-minute guided walkthrough. The demo video shows the full run. The code, the defect ledger, and every audit behind the numbers in this essay are public at github.com/Jeremiah-Sakuda/Hodi.

Your voice deserves a door. Hodi is the knock.