A modular control plane with clean service boundaries.

The operator owns Kubernetes reconciliation and protected workload controls. The gateway owns request entry. The registry owns metadata. Scheduler, policy, Model Router, Guard, context, and evidence stay separate.

Technical view

AgentFleetGateway gives platform and security teams one Kubernetes-native control plane for policy, approvals, routing, protected model access, and audit-ready visibility.

Desired state Runtime decisions Execution Evidence

Declarative Setup Kubernetes APIs publish the desired agent fleet state.

Declarative input GitOps / Helm / CRDs Agent, AgentRoute, AgentPolicy

Kubernetes reconciliation Operator Workloads, Services, identity, egress, status

Inventory Registry Agents, routes, health, endpoints

Runtime Request Path Every call is resolved, checked, selected, and then invoked.

Caller User application Tasks, intents, capabilities, tenant metadata

Request coordinator Gateway Auth, route resolution, invocation, telemetry

Governance + selection Control services Guardrails, deterministic ranking, and human gates before invocation.

Policy Scheduler Approval

Execution Agent service Kubernetes-hosted or external endpoint

Integrations and Evidence Optional context, protected model access, MCP, A2A, and audit stay governed.

Optional Context Metadata-only or retained history

Model access Model Router OpenAI, Anthropic, Bedrock, comparison, cost

Optional inspection Guard Configured input/output checks and minimized findings

Tool protocol MCP Router Discovery + governed calls

Agent protocol A2A Adapter Cards, import, delegation

Operator evidence Audit, metrics, usage, traces Payload-minimized records for operations and assurance review

GitOps / Helm Agent, AgentRoute, AgentPolicy CRDs Tenant RBAC + policy Registry and health metadata

Clients User app Console agentfleetctl

AgentFleetGateway POST /v1/run request ID, tenant, actor

Gateway Orchestrator normalizes requests coordinates modules sets failure policy

Control services

Registry Scheduler Policy Approval Context

Agent Invoker Kubernetes Service or external endpoint

Agent runtime skills, tools, model requests /invoke, /readyz, /metrics

Governed targets Model Router + Guard MCP tools A2A peers External APIs

AgentFleet Operator watches CRDs and reconciles workloads, Services, protected ServiceAccounts, egress policies, mesh metadata, status, and registry publication

Evidence plane audit events, Prometheus metrics, traces, usage, approvals, model cost, and operational assurance evidence

AgentFleetGateway ecosystem: declarative Kubernetes intent, request-time orchestration, optional protected model access, governed integrations, and evidence collection.

flowchart LR
  User["User app / Console / agentfleetctl"] --> GW["AgentFleetGateway<br/>POST /v1/run"]

  subgraph Decl["Declarative Control Plane"]
    GitOps["GitOps / Helm"]
    CRDs["Agent, AgentRoute, AgentPolicy CRDs"]
    K8s["Kubernetes API"]
    Op["AgentFleet Operator"]
    Svc["Workloads, Services, identity, egress"]
    GitOps --> CRDs --> K8s --> Op --> Svc
  end

  subgraph Runtime["Request-Time Control Plane"]
    GW --> Orch["Gateway Orchestrator<br/>request ID, tenant authz, route coordination"]
    Orch --> Reg["Registry<br/>agents, endpoints, capabilities, health"]
    Orch --> Sch["Scheduler<br/>ranking and selection reasons"]
    Orch --> Pol["Policy<br/>allow, deny, approval-required"]
    Pol --> Appr["Approval Service<br/>human gate, immutable events"]
    Orch --> Ctx["Context<br/>optional metadata/history"]
    Orch --> Invoker["Agent Invoker"]
  end

  subgraph AgentPlane["Agent Execution Plane"]
    Invoker --> Hosted["Kubernetes-hosted Agent Service"]
    Invoker --> External["External / managed agent endpoint"]
    Hosted --> Agent["Agent runtime<br/>skills, tools, models, health"]
    External --> Agent
  end

  subgraph Integrations["Governed Integration Plane"]
    Agent -- "projected workload token" --> ModelRouter["Model Router<br/>TokenReview, Registry binding, policy"]
    ModelRouter --> GuardIn["Guard input inspection<br/>optional binding"]
    GuardIn --> Providers["Approved model providers"]
    Providers --> GuardOut["Guard output inspection<br/>optional binding"]
    GuardOut --> Agent
    Agent --> MCP["MCP Router<br/>tool discovery and governed calls"]
    Agent --> A2A["A2A Adapter<br/>Agent Card, import, delegation"]
    MCP --> Tools["Tool servers / APIs"]
    A2A --> Peer["External A2A agents"]
  end

  subgraph Evidence["Evidence Plane"]
    Audit["Audit events"]
    Metrics["Prometheus metrics"]
    Traces["Traces / request metadata"]
    Usage["Usage, token, cost evidence"]
    Release["Operational assurance evidence"]
  end

  Orch --> Evidence
  ModelRouter --> Evidence
  GuardIn --> Evidence
  GuardOut --> Evidence
  MCP --> Evidence
  A2A --> Evidence
  Op --> Reg
  Op --> Evidence

Mermaid topology view of AgentFleetGateway control planes, execution paths, protected model calls, integrations, and evidence collection.

Control plane#

Gateway, Registry, Scheduler, Policy, Approval, Context, Console, and CLI communicate through explicit APIs rather than sharing database ownership.

Execution plane#

Hosted agents run behind Kubernetes Services. Protected agents use dedicated workload identity and router-only model egress; external agents remain outside that boundary.

Evidence plane#

Audit events, usage records, metrics, traces, approvals, and assurance records stay separate from prompt and payload data wherever possible.

Request-time flow#

User applications call the gateway directly or through optional ingress or mesh infrastructure.
The gateway normalizes the request and resolves candidate agents through registry and route metadata.
Policy evaluates the request, selected agent context, models, tools, approvals, and audit requirements.
Scheduler ranks eligible agents and returns a transparent decision.
The gateway invokes a Kubernetes Service or external endpoint, records optional context metadata, and emits audit evidence.
When a protected managed agent needs a model, it presents its projected token to the Model Router. The router verifies workload identity, applies policy and optional Guard inspection, injects provider credentials, and emits payload-minimized evidence.

Optional protected model path#

The protected profile is deliberately separate from ordinary agent routing. It applies only to AgentFleet-managed agents in explicitly labeled namespaces whose CNI passes the NetworkPolicy enforcement probe. The operator creates per-agent ServiceAccounts and egress policies; the Model Router verifies TokenReview and Registry identity; spec.suspended stops workload, Gateway routing, and model access.

Deterministic Guard input/output inspection is integrated for configured tenant and namespace bindings. Learned classification is not a product default. See Protected model path for the operational boundary and evidence status.

Deployment shape#

AgentFleetGateway ships as modular commands and Kubernetes services: operator, gateway, registry, scheduler, policy, Model Router, Guard, context, approval, MCP router, and Console. Helm profiles decide which modules run in local, enterprise, demo, protected-path, and optional Istio-ready modes.

Mesh posture#

Plain Kubernetes is the baseline. Istio is optional and documented as a sidecar profile, with strict mTLS conformance required before compatibility is claimed.