JJ Signal · Point of view · 6 min read
Guardrails belong in the schema, not in the prompt
An advice system that cannot select a product is safer than one instructed not to.
Published Financial Services & Fintech, Professional Services
Every serious constraint on an AI system in regulated advice should be structural rather than instructional. 'Do not recommend a product' in a system prompt is a request; a schema with no field for a product recommendation is a guarantee — and only one of those survives a model change, a jailbreak or a well-meaning prompt edit six months later.
The evidence
Drawn from our own delivery work. Each claim states its sample and its method, including where the sample is small.
- 01Making an unciteable claim unrenderable removed a whole class of review finding, because the failure could not reach the file.One advice firm, roughly 2,300 client relationships. A suitability line with no source record does not render — it is not flagged for review, it does not exist.
- 02Commitment fields left blank by construction were never populated by an agent across the engagement.Price, scope and date. The count is zero because the fields have no write path from a model, not because the model behaved.
- 03Isolation asserted by tests rather than by policy documents caught a real gap at a second firm.A cross-adviser read returning zero rows is a test that fails loudly. The policy document had said the same thing and had been true for nobody in particular.
What follows from it
Audit your AI guardrails by asking which would survive someone rewriting the system prompt. The ones that would not are documentation, and should be re-implemented as schema, policy or an absent code path.
What we would do
- 01Enumerate every commitment the system must never make, then remove the write path rather than adding an instruction.
- 02Make citation mandatory at the render boundary, so an unsourced claim cannot appear rather than appearing flagged.
- 03Assert tenant isolation with tests that a cross-boundary read returns zero rows, run in CI against real Postgres.
- 04Keep the human gate where the consequence is, and record who passed it.
Get JJ Signal by email
Roughly monthly, and only when there is something measured to report. If a month produces nothing worth your attention, you get nothing.
What you are signing up to