How to keep autonomous agents within budget: spend caps, quotas and kill switches

To keep autonomous agents within budget, enforce limits outside the model: a hard spend cap per agent and per domain, a rate quota on tool calls, an approval gate on expensive or irreversible actions, and a kill switch that revokes credentials instead of asking the agent to stop. The Model Context Protocol spec deliberately leaves policy to the host, so the budget has to live in the layer that executes tool calls. Prompts that say "be careful with cost" are not a control.

Key Takeaways

• •Budget limits must be enforced by the runtime that executes tool calls, never by instructions inside the prompt.
• •Three caps cover most failure modes: money (spend per window), volume (calls and bytes scanned per window) and time (maximum runtime per task).
• •Set limits per domain, not globally. A freshness-monitoring domain can run loose while a backfill domain stays on a short leash.
• •A kill switch is only real if it revokes credentials at the connection layer and has been tested.
• •Every cap should degrade to "ask a human" before it degrades to "stop", so useful work is paused rather than lost.

Why agent budgets fail in data platforms

Data work is unusually expensive to get wrong because a single tool call can bill by the byte or the second. An agent that retries a failed query in a loop, widens a backfill window, or resizes a warehouse to "fix" slowness can cost real money before anyone reads a log. In our opinion, the failure is almost never malice or a model bug. It is an unbounded loop with write access and no meter attached.

The Anthropic documentation covers token usage and limits for the model itself. That is only half the bill. The other half is what the agent does downstream: warehouse credits, orchestrator runs, API calls to third-party systems. This guide covers that second half, the actions your agents take on your data stack.

Step 1: Define the budget unit for each domain

Start by deciding what "spend" means for each area an agent works in. Do not use one global dollar figure. Different domains burn different resources, and a single number hides which one is burning.

• •Query and compute domains: warehouse credits or bytes scanned per day.
• •Pipeline domains: orchestrator task runs and backfill row counts per window.
• •Governance and catalog domains: metadata API calls per hour, which is usually a rate limit rather than a money limit.
• •Model usage: tokens per task, tracked separately from infrastructure spend.

Write these down as a short table of domain, unit, window and ceiling. If you cannot name the unit, you cannot meter it.

Step 2: Set hard spend caps at the tool layer

A hard cap is a number the runtime checks before it executes a tool call. If the call would push the window total past the ceiling, the call is rejected and the agent gets a structured error it can reason about. Because every Data Workers agent is an MCP server, the check sits in the framework that wraps every tool, not in each agent's prompt, so no individual agent can opt out.

Use two thresholds. A soft threshold, for example 70% of the window, triggers a notification and tightens the next step's approval requirement. A hard threshold blocks execution. Our Cost Agent reads warehouse usage, so it can feed the actual spend figure into those checks rather than relying on the agent's own estimate.

Here is an illustrative policy shape. It is a pattern to adapt, not a literal config file from our product:

• •domain: backfills; unit: rows written; window: 24h; soft: 70%; hard: 100%; on_hard: require human approval
• •domain: warehouse-sizing; unit: credits; window: 24h; soft: 60%; hard: 90%; on_hard: block and page owner
• •domain: catalog-sync; unit: API calls; window: 1h; soft: 80%; hard: 100%; on_hard: back off and retry later

Step 3: Add rate quotas and loop breakers

Spend caps catch cost after the fact. Rate quotas catch runaway behavior early. Limit calls per minute per tool, limit concurrent runs per agent, and cap retries per task at a small number. The most common budget incident we see described by teams is a retry loop, and a retry cap of three prevents it outright.

Add a loop breaker that fingerprints repeated identical calls. If an agent issues the same query with the same parameters more than a few times in a window, stop it and escalate. Repetition without new information is the signature of an agent that is stuck, not one that is working.

Step 4: Gate expensive and irreversible actions behind approval

Not every action needs a cap; some need a human. Classify tool calls into three tiers: read-only, reversible write, and irreversible or costly write. Read-only runs freely under quota. Reversible writes run under a cap. Anything irreversible or above a cost threshold, such as dropping a column, resizing a warehouse or launching a large backfill, waits for approval.

This is where the per-domain autonomy dial belongs. The Autonomous Data Conductor lets you set how much each domain may do without asking: observe only, propose, act with approval, or act and report. A new domain starts at propose. You turn the dial up only after the audit trail shows the agent's proposals were right. In the open-source edition, agents are read-only by default, so you begin at the safest setting; Pro adds writes and approval workflows.

Step 5: Build a kill switch that actually kills

A kill switch that asks the agent to stop is a suggestion. A real one acts on the layer the agent cannot talk its way around: revoke the credentials or close the MCP connection. Design it with three scopes: one agent, one domain, or everything.

• •Revoke at the connection layer so in-flight calls fail instead of finishing.
• •Make the switch reachable from the terminal. Teams already working in Claude Code or Cursor should be able to trigger it without opening another dashboard.
• •Log who flipped it, when, and what was in flight, so the post-mortem has facts.
• •Test it on a schedule. An untested kill switch is a belief, not a control.

Pair it with an automatic trip. If the hard cap is exceeded twice in a window, or the loop breaker fires, the domain drops back to propose-only until a human re-enables it.

Step 6: Watch the meter and review the audit trail

Budgets drift. A cap that was right in March is wrong after a new product line triples query volume. Review spend per domain weekly, and track how often agents hit the soft threshold. A domain that hits it constantly needs either a higher ceiling or a better-designed agent, and the audit trail will tell you which.

Our Governance Agent maintains tamper-evident audit records of agent actions, which is what makes this review possible. The Observability Agent can flag when a pipeline's cost per run changes, so the signal reaches you before the invoice does. We covered how the swarm shares context across these agents in a separate post on the context layer.

A short checklist before you grant write access

• •Every domain has a named unit, window, soft threshold and hard threshold.
• •Retries are capped and repeated identical calls trip a breaker.
• •Irreversible actions require approval, whatever the spend.
• •The kill switch revokes credentials and has been tested this quarter.
• •Someone owns each domain's budget and receives its alerts.

Frequently Asked Questions

Can I rely on the model to respect a budget stated in the prompt? No. A prompt instruction is advice the model may weigh against its task, and it cannot enforce anything. Enforce limits in the runtime that executes tool calls, where the agent cannot override them.

What should happen when an agent hits its hard cap? In most domains it should stop and ask a human, not fail silently. The agent returns what it completed, what it wanted to do next and the estimated cost, so the approver can raise the ceiling or decline with full context.

How do I set the first cap if I have no usage history? Start low and read-only. Run the domain in observe or propose mode for a couple of weeks, measure what it would have spent, then set the ceiling a comfortable margin above the observed normal. Our opinion is that a cap you have to raise is a far cheaper mistake than one that was never set.

Does a per-domain autonomy dial replace spend caps? No, they work together. The dial decides what an agent may do without asking, and caps bound how much it may do even when allowed. You want both.

Where does the kill switch run if my engineers work in Claude Code or Cursor? It should be reachable as a command from the same terminal, because the person who notices a runaway agent is usually the person already working in that tool. Since each Data Workers agent is an MCP server, revoking its connection also takes it out of the client's reach.