Agent To Agent Communication Data

Agent-to-agent communication is where most data swarms bleed cost and correctness. When one agent talks to another, the channel decides whether you get a deterministic pipeline or a chat room that hallucinates its own facts. Getting the protocol right matters more than picking the right model.

This guide walks through the three common agent-to-agent patterns in data engineering, the failure modes of each, and the state-based protocol Data Workers uses to keep autonomous pipelines reliable at scale.

Three Patterns for Agent-to-Agent Communication

The three dominant patterns are free-form chat, typed message passing, and shared state. Free-form chat is the default in most demos because it is trivial to wire up, but it is also the most expensive and the least reliable. Typed message passing enforces contracts but still replays messages. Shared state decouples communication entirely — agents write to and read from a common store, which makes the system debuggable and cheap to scale.

Why Free-Form Chat Fails in Production

When agents chat in plain English, they invent ambiguity. Agent A says 'the users table looks fine,' which Agent B interprets as 'no action required,' when Agent A actually meant 'schema is fine but row counts dropped fifty percent.' Natural language is lossy, and data work cannot absorb lossiness.

• •Ambiguity — plain English hides the difference between a warning and a success
• •Context drift — long chats fall out of window and agents start making things up
• •Token explosion — every message replays the full history
• •No replay — you cannot re-run a failed decision because there is no typed input
• •No observability — logs are unstructured transcripts, not events
• •Hallucinated facts — agents invent table names and column types that never existed

Typed Message Passing

A better pattern is typed message passing: each handoff is a JSON schema validated object with explicit fields like task_id, status, metrics, and next_step. Validation rejects malformed handoffs before they poison the downstream agent. The tradeoff is you have to design the schemas up front, and any time a sub-agent wants to say something new, you need a schema change.

Shared State Is the Scalable Answer

Data Workers uses shared state as the primary communication channel. Each agent writes structured output to a Postgres or Redis state store — run metadata, quality metrics, incident tickets — and downstream agents query that store. Conversation history is kept only for observability, never as the source of truth. This eliminates context replay and makes every handoff auditable.

Shared state also plays nicely with multi-tenant deployments. Each tenant gets a namespace in the store, and agents cannot accidentally leak data across tenants. See the broader autonomous data engineering design for how the shared state store integrates with the agent runtime.

Reducing Communication Cost

Three rules cut agent-to-agent token cost by 60 to 80 percent. First, never replay conversation history — pass state pointers instead. Second, keep handoffs under 500 tokens by offloading detail to the state store. Third, use cheap models for routing and classification, and only escalate to the expensive model when a reasoning step actually needs it.

Observability and Debuggability

With shared state, you get replay for free. Every agent transition is a row in a ledger, every decision references a state version, and you can re-run any failed step against the exact state that caused the failure. Compare that to a chat transcript where you cannot even tell which agent said what. For the full story on operationalizing agents in data infrastructure, see AI for data infrastructure.

Pick the communication pattern before you pick the framework. If your agents talk in plain English, you are running a chat room, not a pipeline. To see a shared-state swarm in action on a real warehouse, book a demo.

In a recent production incident, a free-form chat multi-agent system reported a database migration 'completed successfully' when in fact the downstream agent had misinterpreted a 'schema validated' message from the upstream agent as 'migration applied.' The two agents used the same English vocabulary to mean different things, and the ambiguity was invisible from the logs because the transcript read like a normal conversation. A typed handoff schema with an explicit 'migration_state' enum would have caught the mistake at the validation step.

The tradeoff with shared state is that you have to design the state schema up front, which feels heavier than just letting agents chat. In practice the schema design takes a few hours and pays for itself within the first week of production. Data Workers ships reference schemas for the most common agent communication patterns (task handoff, incident escalation, approval request, result publish) so teams can adopt the pattern without building schemas from scratch. Picking a schema off the shelf is faster than designing your own free-form chat prompts, and it is orders of magnitude more reliable.

Observability is the killer feature of shared-state communication. When every handoff is a row in a state store, you get replay, audit, and debuggability for free. Want to know why the downstream agent made a particular decision? Query the state it read. Want to reproduce a failure? Re-run the downstream agent against the historical state. None of this is possible with free-form chat because chat history is unstructured and hard to query programmatically. Teams that adopt shared state quickly discover that observability improves debugging speed by an order of magnitude.

The migration path from chat to shared state is usually incremental. Start by adding a state store alongside your existing chat-based system, write to both, and gradually move consumers from reading chat to reading state. Once all consumers are on state, drop the chat channel. This zero-downtime migration works with LangGraph, CrewAI, and most other frameworks. Data Workers' reference implementation shows the migration pattern step by step so teams can adopt it without rewriting their entire agent stack at once.

Free-form chat is the default and the worst choice. Typed messages are better. Shared state is best. Pick the protocol before you pick the framework.