Designing Multi-Agent Systems That Don't Fall Apart in Production

Most multi-agent demos fall apart the moment you put them in front of a real user. The failure mode is rarely the model — it's the plumbing between agents.

The orchestrator pattern

When I built Career Agents, I started with a single free-form conversation where every "agent" was just a different system prompt swapped mid-thread. It worked for the demo video and fell over within a day of real use — context from the resume-tailoring step would silently vanish by the time the interview-prep step ran.

The fix was structural, not prompt-related: one orchestrator agent owns the task graph, and every specialist agent — Resume Agent, Interview Agent, LinkedIn Agent, Job Tracker — receives and returns a typed JSON object instead of prose. The orchestrator validates the shape before handing it to the next agent.

Shared memory, not shared conversation

Instead of one long conversation thread, each agent gets a scoped slice of a shared session-memory object. This keeps individual prompts short (cheaper, faster, less prone to distraction) while still giving every agent the facts it actually needs.

Where human-in-the-loop earns its keep

Any action with real-world consequence — sending a message on the user's behalf, submitting an application — gets a checkpoint. Agents are good at drafting; they're not yet good enough to be trusted with unreviewed high-stakes actions.

What I'd do differently

If I rebuilt this today, I'd introduce evaluation harnesses earlier — a small labeled set of "known good" outputs per agent stage, checked automatically on every prompt change. Right now that verification is still partly manual, and it's the biggest lever left for reliability.