Smuz
Agent reliability

Your agents say
they ran.
Some are lying.

A scheduled agent rarely fails loudly. It keeps reporting success while doing nothing, and nobody notices for weeks. We find those, close them, and leave behind a check that proves each one actually did its job.

$1,000 · one week · free if it finds nothing material

$ runproof check --profile outbound.toml
agent outbound-mailerrun 4f2a912026-08-31 09:00 UTC
fired on schedule09:00:04 · within 60s
leads selected412 of 412
emails actually sent0 · expected ≥ 1
no duplicate sends0 duplicates
daily cap respected0 / 40
owner declarednone
1 FAILED · 4 PASSED · 1 –last successful send: 11 days ago

A worked example of the report. That failing row is the failure we find most often: an agent reporting success while doing nothing.

The failure mode

It never sounds
like a failure.

An agent that is 85% reliable across eight steps completes correctly 27% of the time. None of those failures raise an error.

It said it ran.

The job reported success. The logs are green. It processed zero records, and has done since the API key rotated eleven days ago.

We found out from a customer.

Not from monitoring. From someone downstream asking why they stopped getting the thing.

It worked in dry-run.

Because dry-run never touched the integration that was never created. The mode designed to make it safe is the mode that hid the failure.

It ran twice.

Two deployments, two schedulers, one agent. Duplicate outbound before anyone noticed.

Nobody owns it.

The person who built it changed teams. It's still running. Probably.

What we do

Find it.
Fix it.
Keep watching it.

Three steps, in order. The audit is the front door and everything after it is a consequence of what the audit finds.

01

Audit

$1,000 1–5 agents · $2,000 for 6–15

Read-only. One week. A written report naming every agent that isn't doing what it claims.

  • Per-agent table: fires · produces output · has an owner · has a check
  • Silent no-ops, empty successes, duplicate runs, dry-run masking
  • Prioritised remediation list with our estimate against each line
  • Free if it finds nothing at High or Critical severity
02

Harden

$1,500 per agent · $1,000 from the fourth

We close what the audit found and leave behind a deterministic, zero-LLM check that asserts the agent did its job.

  • Written acceptance criteria per agent, agreed before work starts
  • A runbook and a named owner
  • Checks live in your repo, on your infra
  • We don't rewrite your agents. We make them verifiable.
03

Watch

$400 /mo up to 5 agents · $800 up to 15

The checks run continuously. When one fails, a named human on your side gets the run ID and the failing assertion.

  • Monthly written summary
  • Alerts into Slack, Teams, PagerDuty or email
  • Cancel any time — the checks are yours and keep running without us
  • We alert. We don't take your on-call.
Don't have the agents yet? We build them — two weeks, written acceptance criteria, verification included rather than sold separately.$2,500 / agent
What we need from you

Read-only, and
less than you think.

The usual reason this doesn't happen is that granting a stranger access to a scheduler feels like a bigger decision than the problem. So here is the entire ask, in advance, before you ever talk to us.

Day 1

Access, and one call

You grant read-only access and we spend 45 minutes with whoever built the agents. That call is the highest-value hour of the week — it's where we learn what each job is supposed to do.

Days 2–4

Inspection

We read the scheduler, the logs and the tables the agents write to, and reconstruct what each one actually did over the last 30–90 days. No changes, no execution, nothing of ours running in your production.

Day 5

The document, and a readout

You get the written report, then 30 minutes to walk it with your engineers in the room. We'd rather be argued with than quietly filed.

Four things we ask for

  • 01
    Read access to whatever fires the jobsCron, Airflow, Temporal, GitHub Actions, a queue consumer — whatever it is. An export of run history works just as well if that's easier to grant.
  • 02
    The logs those runs writeWherever they land. If the honest answer is “a file nobody reads”, say so — that's a finding, not an embarrassment.
  • 03
    A read-only database userOr a read replica. We count rows and compare timestamps: the job is to establish that 412 rows were written, not to read what's in them.
  • 04
    One line per agent on what it's supposed to doNot documentation — a sentence each. If nobody in the company can produce that list, that is finding number one and the audit has already paid for itself.

Five we never will

  • Production write access. Not once, not briefly, not “just to test something”.
  • Admin or root on anything.
  • Customer data. Assertions are counts, timestamps and IDs — never contents.
  • Credentials you can't lawfully share. We skip that agent and say so in the report rather than pushing.
  • Anything of ours installed in your stack. During an audit nothing we wrote runs on your infrastructure.

All of it goes in the engagement letter before you grant anything — including the sentence naming exactly what we read and what we don't.

What you get back

A document, not
a dashboard.

One markdown file in your repo, readable in a browser, no login. It has the same three parts every time, so the second audit is comparable to the first.

Part one · every agent, four questions

They are kept separate on purpose. Almost every silent failure we find is an agent that passes one of these and fails the next one along.

Liveness

Did it run?

A perfect agent that isn't firing is a dead agent.

Throughput

Did it do work?

A run that processes zero records exits successfully and looks identical to a healthy one.

Effect

Did the work land?

“The API call returned 200” is not “the thing happened.”

Ownership

Would anyone know if it hadn't?

An unowned failure is an indefinite failure.

Part two · the fleet table

One row per agent, and a verdict you can read across a room. This is the page your VP Eng will actually look at.

AUDIT.md · § 1 Summary
AgentLivenessThroughputEffectOwnershipVerdict
outbound — as audited, 29 AugNot verifiable
outbound — after remediation, 31 AugVerifiable; not yet owned
These two rows are real — our own agent, before and after we audited it. As audited it failed three of the four outright and could only partly answer the fourth.
Part three · every finding, four parts

Each one is graded Critical, High, Medium or Low, indexed in a table at the top, then written out in full below. Our own audit produced nine of them: three critical, three high, three medium.

01

Evidence

The file and line, or the query and the number it returns. Never an adjective, never “appears to”. You can check every finding yourself without taking our word for anything.

02

Failure scenario

The concrete sequence that bites you, written in your terms — “you run three days of dry runs to review the copy, and every one of those leads is now permanently un-emailable.” Not “may lead to data inconsistency.”

03

Why it matters

Or why it doesn't. Some findings come with a recommendation to leave them alone; an internal digest agent does not need the treatment one that emails customers does.

04

Fix, with an estimate

What closing it involves and roughly how long, so the remediation list is something you can price, split between us and your team, or decline line by line.

The report ends with the remediation list, ranked by consequence — money, external communication, client data — with an estimate against each line. That ranking is what lets us recommend hardening three agents instead of ten.

The check

Boring enough
to be trusted.

Every check is a plain assertion about an outcome: it fired at 09:00, it wrote 412 rows, 412 matches the input count, the last write landed. Readable by anyone on your team, and by you, in about a minute.

Pointing it at a new agent means writing a profile, not writing code. It opens your database read-only and it has no dependencies beyond the standard library — which is why you can run it yourself without giving us access to anything.

outbound.toml
[[assert]]
name = "emails actually sent"
kind = "scalar"
query = "SELECT count(*) FROM sends WHERE run_id = :run"
min = 1
catches = "A run that reports success having sent nothing."
catches is required. An assertion that can't name the failure it prevents gets deleted, not explained.

No model in the checker.

You can't verify a nondeterministic system with another nondeterministic system. If the checker can hallucinate, a green check means nothing.

A missing check renders –, never ✓.

An assertion that errored renders ERROR, never a pass. The one thing a verification tool must never do is round up.

Bounds live in the check, not the agent.

A cap that verifies itself against its own configuration cannot detect a misconfigured cap. So the expected value is declared separately, by a human.

We ran this against our own outbound agent first. It found nine problems, three of them critical — including one quietly destroying our own lead list. We wrote all of it down, unedited. You get a report in the same format.

Pricing

A thousand to audit — free if I find nothing. Fifteen hundred an agent to fix what I find. Four hundred a month to keep checking.

Audit
$1,000
one week, read-only

$2,000 for 6–15 agents. Free if it finds nothing at High or Critical severity.

Harden
$1,500
per agent

$1,000 from the fourth agent onward — the harness is built once, so charging full rate for the tenth would be charging for work already done.

Watch
$400
per month, up to 5 agents

$800 up to 15. Priced on fleet size, not on how many were hardened. Cancel any time.

Everything adds.
Nothing is subtracted.

No credits, no bundles, no arithmetic you need a spreadsheet to follow. With ten agents we will usually recommend hardening three and watching all ten — smaller invoice up front, and the right engineering answer.

They haveThey buyUp frontMonthly
3 agents, 1 brokenAudit + harden 1$2,500
5 agents, 2 criticalAudit + harden 2$4,000
10 agents, 3 criticalAudit + harden 3 + Watch$6,500$800
12 agents, wants eyes on themAudit + Watch$2,000$800
Straight answers

What we
won't do.

  • We don't rewrite your agents. We make them verifiable.
  • We're not an observability platform. No dashboard, no seat licences.
  • We don't take production write access during an audit.
  • We don't evaluate model quality. Whether the output is good is your eval problem; whether the job happened is ours.
  • We don't take on-call. We alert a named human on your side.

Those tell you what the model did. They don't assert that the job got done. A trace showing a clean, successful LLM call proves nothing about whether the CRM row was written or whether the run processed 400 records or zero. Two more gaps: they're instrumentation you had to install before the failure, so they can't tell you about the agent nobody instrumented — always the one that breaks — and they're trace-based rather than outcome-based, so a job running perfectly on an empty input set looks identical to a healthy one. We're not replacing them.

The qualifying question

What runs on a schedule
that nobody watches?

If the honest answer is “nothing”, you don't need us and I'll say so on the call. If you can name three, a week of read-only access will tell you which of them have been quietly doing nothing.

Read-only · one week · free if it finds nothing material