S10: a missing theme falls back silently, and the error arrives after the window closes
- Upgrade
- 26.0.0 → 26.7.1
- Scale
- 2,000,003 users, 2 realms
- Database
- PostgreSQL 16.15, single node, container
- Topology
- single container
- Host
- Hetzner CCX33 (fsn1) — 8 vCPU dedicated, 30 GB RAM, local NVMe
- Condition
- Theme configured, files absent
Summary
The lab realm has named custom themes since the first run — seed-realm.sh's
typical profile sets loginTheme: lab-login, accountTheme: lab-account,
emailTheme: lab-email — and no theme files have ever been mounted. So S10
has silently been in force in every record this lab has produced.
The answer: Keycloak falls back to the built-in theme and serves a working
login page. It logs the problem at ERROR level — but lazily, on the first
request that needs the theme, not at startup. An operator who upgrades, watches
the startup log, sees a clean start and closes the maintenance window has had no
opportunity to see it.
Environment
| Field | Value |
|---|---|
| Keycloak version | 26.7.1 (arrived via 26.0.0 → 26.7.1) |
| Distribution / start mode | quay.io/keycloak/keycloak official image, start w/ external DB |
| Database | PostgreSQL 16.15, single node, container |
| Postgres tuning | stock — shared_buffers 128MB, maintenance_work_mem 64MB, statement_timeout 0 |
| Adverse scenario | S10 |
| Dataset scale | 2,000,003 users, 2 realms |
| Seeding method | direct SQL COPY, over a partialImport realm skeleton |
| Realm theme config | login_theme=lab-login, account_theme=lab-account, email_theme=lab-email, admin_theme=keycloak — read from the realm table |
| Themes mounted | none — /opt/keycloak/themes/ is empty; only fixtures/providers is bind-mounted |
| Topology | single container |
| Host | Hetzner CCX33 (fsn1) — 8 vCPU dedicated, 30 GB RAM, local NVMe |
Procedure
docker compose exec -T db psql -U keycloak -c \
"select name, login_theme, account_theme, email_theme, admin_theme from realm"
curl -sL "http://localhost:8080/realms/lab/protocol/openid-connect/auth\
?client_id=spa-public&response_type=code&scope=openid\
&redirect_uri=https%3A%2F%2Fspa.lab.invalid%2Fcb\
&code_challenge_method=S256&code_challenge=<S256 of a verifier>"
curl -s "http://localhost:8080/realms/lab/account/"
docker compose logs keycloak | grep -i theme
Results
| Surface | HTTP | Body | Theme actually served |
|---|---|---|---|
lab realm login page | 200 | 8,094 bytes, full kc-form-login form, <title>Sign in to Lab realm lab</title> | built-in login/keycloak.v2 |
lab realm account console | 200 | 4,213 bytes | built-in account/keycloak.v3 |
master realm login (no custom theme configured) | 302 → 200 | — | built-in, as expected |
The login page is fully functional. Asset URLs in the served HTML resolve to
resources/he5u5/login/keycloak.v2/… and
resources/he5u5/account/keycloak.v3/… — the built-in themes, not the
configured names.
The logging is real but late
19:03:19,962 ERROR [org.keycloak.theme.DefaultThemeManager] Failed to find ACCOUNT theme lab-account, using built-in themes
19:03:42,949 ERROR [org.keycloak.theme.DefaultThemeManager] Failed to find LOGIN theme lab-login, using built-in themes
Three theme-related lines in the entire container log, and all three carry the timestamp of the request that triggered them, not of startup. The ACCOUNT line appeared when the account console was first fetched; the LOGIN line 23 seconds later, when the login page was first fetched. Before those requests, the log contained nothing about themes at all.
That is the finding. The severity is right — ERROR — but the timing is
wrong for an upgrade: the signal is emitted by the first user, not by the
upgrade.
Outcome
- Reached ready
- Login flow renders — with the wrong branding
- Themes render — no, silently substituted
-
Logged, at
ERROR, but only on first use
Verification
| Claim | Primary source | Checked |
|---|---|---|
| Realm names themes that do not exist | realm table read directly | ☑ |
| Login page still returns 200 and a usable form | curl, 8,094 bytes, kc-form-login present | ☑ |
| Built-in theme is substituted | asset paths login/keycloak.v2, account/keycloak.v3 in the served HTML | ☑ |
| The ERROR is emitted on first request, not at startup | log timestamps vs. request times; zero theme lines before the requests | ☑ |
| Email theme behaviour | not tested — needs an actual send; a broken emailTheme may not be as forgiving | ☐ |
| A partially present theme (dir exists, files missing) | not tested — likely a different and worse failure than a wholly absent one | ☐ |
| Admin theme | not tested — admin_theme was set to the valid built-in keycloak | ☐ |
Publishable extract
If your Keycloak upgrade loses its custom theme, Keycloak will not tell you during the upgrade. It will tell the first person who tries to log in.
We ran a realm configured with
loginTheme: lab-loginand no such theme mounted, on Keycloak 26.7.1 / Postgres 16.15. The login page returned HTTP 200 with a complete, working sign-in form — rendered in the built-in theme. The only signal was one line, atERROR:ERROR [org.keycloak.theme.DefaultThemeManager] Failed to find LOGIN theme lab-login, using built-in themesand it was written at the moment of the first login request, not at startup. Our container log contained no theme-related line at all until somebody asked for a page.
This is a graceful fallback, and as engineering it is the right call — nobody wants an outage because a CSS file is missing. But it means the standard upgrade verification, "the server came up clean, close the window", cannot catch it. Custom themes are mounted from outside the image, so they are exactly the kind of thing an image bump, a chart change, or a moved volume quietly drops.
Add one line to your post-upgrade checks: fetch the login page and confirm the asset URLs contain your theme name, not
keycloak.v2.