Troubleshooting

Symptoms you are likely to meet on a first install, what each one actually means, and how to clear it.

Requests are denied#

Response	Meaning	Fix
403 policy_denied missing_policy	Policy is enforcing and no allow rule matched. This is the default state of a fresh enforcing install.	Author `policy.rules`. See Write a policy that allows traffic.
409 no_matching_agent	No healthy and registered or managed agent provides the requested capability. An agent in `Degraded` is deliberately not routed to, and neither is one whose Registry record is `pending` (the default for an agent applied on a profile that does not set `registry.operatorWritesAreApprovals`, until it is approved; the approval service, issue #727, is not built yet).	`kubectl get agents -A`, then inspect the pod behind the degraded agent.
403 policy_denied tenant_denied	The tenant is outside `allowedTenants`.	Add the tenant, or send the request as one already allowed.
409 approval_required	Working as intended: an approval rule matched and the request is waiting for a decision.	Approve or deny it in Console → Approvals.

Pods will not start#

Symptom	Meaning	Fix
CrashLoopBackOff on first install	Services started before PostgreSQL was accepting connections. They retry the connect for up to a minute before exiting.	Usually clears itself. If it persists, the DSN or the database is genuinely wrong — read the pod logs.
connect: connection refused to the database	The database is reachable by DNS but not accepting connections yet, or the port is wrong.	Confirm the database pod is ready and the DSN port matches the Service.
no such host	DNS could not resolve the database host, often a CoreDNS restart mid-install.	Check the hostname in the DSN and that CoreDNS is healthy.
ImagePullBackOff	The cluster cannot pull the images.	Confirm private-registry access and configure `global.imagePullSecrets` when your platform does not provide registry identity.
Migration Job fails	Checksums rejected the recorded history, meaning an applied migration was edited.	Do not edit applied migrations. Restore the database or roll forward with a new one.

The gateway is not ready#

Read /readyz rather than guessing — it names the dependency that is failing:

kubectl port-forward -n agentfleet-system svc/agentfleet-gateway 8080:8080 &
curl -s http://127.0.0.1:8080/readyz

A required check reporting unavailable is what holds the gateway out of service. A check reporting disabled is switched off in your values and is not a fault.

Helm install or upgrade is refused: gateway.config.auth.mode#

The chart refuses to render the Gateway in off (the default), header_dev or oidc_jwt unless it is told the callers are not checked against the Registry. Helm prints an Error: execution error at (agentfleet/templates/gateway-auth-validation.yaml…) line that ends:

gateway.config.auth.mode="off" admits a caller with no Registry caller record: the caller is not checked against the Registry. off and header_dev take the tenant and actor from the request and are for local development only; oidc_jwt takes them from a verified token's claims and may run in production. To run this mode, set gateway.config.auth.allowUnregisteredCallers=true to acknowledge it. gateway.config.auth.mode=entra_app needs no acknowledgement: it requires an approved Registry binding for the caller.

Pick the remedy that matches the install:

Local or evaluation install, or an oidc_jwt install that accepts that callers are not checked against the Registry: add --set gateway.config.auth.allowUnregisteredCallers=true to the command. It changes nothing about how the mode behaves. off and header_dev are for local development only.
Callers that must be registered: choose gateway.config.auth.mode=entra_app with your own directory ID, application role, token audience, issuer and JWKS URL, and the names of the identity-binding and audit-digest Secrets (and, when the chart runs migrations, the identity database's schema-owner DSN Secret, migrations.secrets.identity.existingSecret); the flags are on Install with Helm. A release already in one of the three modes moves to entra_app by adding those flags to the upgrade command.

The local-light, local-durable and demo-observability profiles set the acknowledgement themselves. A value that is not true or false (for example --set-string…=false) is refused with its own message.

Helm upgrade fails#

Upgrading from an older release

If helm upgrade --reuse-values fails on a value that did not exist in the release you are upgrading from, re-render with an explicit values file instead. --reuse-values does not merge new chart defaults into an old release.