Troubleshooting

Problems with signing in, with the edge in front of the console, and with reading the Overview. Each message is quoted from the code, so you can search for it. For other areas see the troubleshooting page of DevEx, DataEx, Intelligence and FORGE.

Console sign-in

“… is not authorized for this console”

Cause. The Google account signed in correctly, but it is not on the console’s allowlist (emailAllowed in auth.go). When neither CONSOLE_ALLOWED_EMAILS nor CONSOLE_ALLOWED_HD is set, every Google sign-in is refused. This is deliberate: an unset allowlist used to admit every Google account on the internet.

Fix. Add the address to CONSOLE_ALLOWED_EMAILS (a list) or set CONSOLE_ALLOWED_HD to your hosted domain. The name is CONSOLE_ALLOWED_EMAILS, not the apps’ AUTH_ALLOWED_EMAILS.

“Sign-in is unavailable: this console cannot open its session store (appfs core-console/run/sessions). Your credentials were not the problem — tell an operator.”

Cause. Sessions are stored server-side in the console’s encrypted appfs root, and it cannot be mounted. At boot the console logs 🛑 CONSOLE SIGN-IN DISABLED — a sign-in method is configured but the session store (appfs core-console/run/sessions) cannot be mounted, so every sign-in will be refused. It does not exit, because the same process also reconciles the fleet.

Fix. Check CORE_ENVELOPE_KEK and the OBJECTSTORE_* or CONSOLE_APPFS_DIR wiring of the core-operator Deployment. The mount is retried on the next sign-in.

“Invalid username or password.”

The password path compares against CONSOLE_ADMIN_USERNAME (default admin) and CONSOLE_ADMIN_PASSWORD. Every failure is recorded in the audit chain as auth.login denied.

“password sign-in is not configured”

A 404 from /auth/password: CONSOLE_ADMIN_PASSWORD is unset, so the password path is off. Use Google sign-in, or set the password.

“Captcha verification failed. Please try again.”

reCAPTCHA is configured and the assessment failed. It runs before the credential check, on both sign-in paths.

“csrf check failed”

A 403 from /auth/callback. The Google Identity Services form post must carry the same g_csrf_token in the cookie and in the body. A request without it is refused. Reload the sign-in page and try again, and check that nothing between you and the console strips cookies.

“Sign-in failed. Try again.” / {"error": "sign-in failed"}

The Google ID token did not verify: a wrong audience (the console’s GOOGLE_CLIENT_ID), a wrong issuer (OIDC_ISSUER, default https://accounts.google.com), or an unverified email.

“sign-out could not revoke your session on the server; it expires on its own — tell an operator”

A 503 from /auth/logout: the session store could not be written. The session still expires after its 12-hour lifetime.

{"error": "unauthorized"} on every request

Your session expired or was revoked (for example with POST /api/sessions/revoke-all). The web app returns you to the sign-in screen on any 401.

Refused writes

“… is not permitted to … on this console (CONSOLE_AGENT_ADMINS)”

The shape is <identity> is not permitted to <action> on this console (<ALLOWLIST>). You are signed in, but a privileged write checks a named allowlist and you are not on it. The name in brackets is the one to change: CONSOLE_AGENT_ADMINS, CORE_CONNECTION_ADMINS, CONSOLE_INSTANCE_ADMINS, CORE_ATLAS_ADMINS, CORE_RUNNER_ADMINS or CONSOLE_SESSION_ADMINS. DataEx › Trust › Policy & ReBAC (GET /api/access) shows which lists are live and whether you are on each.

“this console has no sign-in configured, so … would be unattributable”

The console is running open (neither GOOGLE_CLIENT_ID nor CONSOLE_ADMIN_PASSWORD is set). Reads work, but no privileged write is accepted, because the audit trail could not name anyone. Configure a sign-in method.

“this console has no sign-in configured, so nobody can be shown the audit trail — …”

Same cause, on DevEx › Audit chain. With sign-in configured, reading the audit trail also needs CONSOLE_SESSION_ADMINS, and an unset list means nobody.

The edge (devgateway)

These come from grpc/cmd/devgateway/auth.go, in front of every routed host.

At boot, the edge logs its configuration errors as ERROR: edge auth is ENABLED but …:

• … DEVGW_AUTH_ALLOWED_EMAILS is empty — every sign-in will be REFUSED. Populate edge-auth-secrets/allowed-emails, or set DEVGW_AUTH_ALLOW_ANY_VERIFIED=true to admit any verified Google account on purpose. (An empty allowlist used to admit everyone; it no longer does.)
• … DEVGW_SESSION_KEY is unset or shorter than 16 bytes …. Populate edge-auth-secrets/session-key (identical on every edge pod) and roll the DaemonSet.
• … the session store cannot be mounted …. Set CORE_ENVELOPE_KEK and either OBJECTSTORE_* or a writable DEVGW_APPFS_DIR, then roll the DaemonSet.

The edge stays up in every one of these cases and keeps serving /healthz. It terminates TLS for every routed host, so crash-looping it would turn a sign-in outage into a total one.

Reading the Overview

Navigation