AI • Architecture • Agentic systems
Reference Architecture for AI Agentic Platforms
Agents are not chatbots with extra prompts. In production they are long-running systems: they plan, call tools, write state, and must survive retries, policy, and partial failure. This is the platform shape I use when an iPaaS has to host agent workloads alongside classic integrations.
Five layers, not one mega-prompt
Collapse these layers into a single service and you will debug “the LLM was weird” instead of a failed tool, a stale graph edge, or a stuck workflow.
Why orchestration belongs in the platform
LLM loops are great for local reasoning and terrible as a transaction log. Tokens get lost, processes restart, and a half-applied CRM update is still a half-applied CRM update. Durable workflows give you:
- Explicit steps with retries and compensation (same saga model as iPaaS).
- Human approval as a first-class wait state, not a Slack hack.
- Fan-out to specialist agents without nested prompt spaghetti.
- A searchable execution history for evals and incidents.
| Concern | In the model loop | In Conductor / iPaaS |
|---|---|---|
| Retry a failed ticket create | Hope the next completion agrees | Idempotent task + backoff |
| Pause for a manager | Custom poller | WAIT / human task |
| Audit who changed what | Prompt log | Workflow + CDC trail |
| Cost / latency SLO | Guess | Per-step meters |
Memory is a graph, not a blob
Working on knowledge graphs at Freshworks taught a simple rule: agents retrieve better when entities and relationships are modeled, not when every past chat is stuffed into context.
Store episodic memory as edges (AgentRun —used_tool→ Adapter)
rather than bloated prompt context. Retrieval becomes a bounded graph
walk with ACL filters at each hop.
Deployment topology for agent workloads
Agent GPU pools and Java tool workers rarely share the same scaling profile. Split Kubernetes node pools by task domain.
Safety and cost as platform features
Tool allow-lists per agent role and tenant.
Token and $ caps abort the workflow, not the JVM.
Replay golden traces on every prompt pack change.
High-risk tools always hit an approval task.
This is the same operational culture as messaging SLOs: if you cannot graph it, you cannot ship it. Agent platforms inherit PagerDuty, canaries, and domain isolation from the iPaaS—not a parallel stack.