26.0.0 → 26.7.1 at 2,000,003 users
- Upgrade
- 26.0.0 → 26.7.1
- Scale
- 2,000,003 users · 2,000,002 credentials · 4,000,001 user attributes · 2 realms · 10 clients · 8 roles
- Database
- PostgreSQL 16.15, single node, container
- Topology
- single container
- Host
- Hetzner CCX33 (fsn1) — 8 vCPU dedicated, 30GB RAM, local NVMe
- Condition
- routine — no adverse condition applied
Environment
| Field | Value |
|---|---|
| Keycloak from → to | 26.0.0 → 26.7.1 |
| Distribution | quay.io/keycloak/keycloak official image |
| Start mode | start --http-enabled=true w/ external DB |
| Database | PostgreSQL 16.15, single node, container |
| Postgres tuning | stock — shared_buffers 128MB, maintenance_work_mem 64MB, statement_timeout 0 |
| Dataset scale | 2,000,003 users · 2,000,002 credentials · 4,000,001 user attributes · 2 realms · 10 clients · 8 roles |
| DB size | 3,512 MB |
| Seeding method | seed-sql — direct SQL, not partialImport. All users share one argon2id hash. See bin/seed-sql.sh |
| Fixture profile | typical (structure) + seed-sql (users) |
| Topology | single container |
| Host | Hetzner CCX33 (fsn1) — 8 vCPU dedicated, 30GB RAM, local NVMe |
| JVM heap | -Xms1g -Xmx4g |
| Adverse scenario | none |
Comparability warning. Per our fixture-comparability rule, this fixture is not
comparable to the 1k and 100k records in every respect: those were seeded through
Keycloak's own write path via partialImport, this one bypasses it. Row counts
and column contents match; provenance does not.
Timings
| Phase | Value |
|---|---|
Quarkus started | 3.32s (self-reported) |
/health/started → 200 | 10.83s |
| Liquibase migration | 18:06:58,654 → 18:07:02,490 = 3.84s |
/health/ready → 200 | 15.60s |
Against every other scale measured
| Users | DB size | Liquibase phase | Time to ready |
|---|---|---|---|
| 1,000 | ~13 MB | 3.97s | 17.0s |
| 100,002 | 264 MB | 3.79s | 15.4s |
| 2,000,003 | 3,512 MB | 3.84s | 15.6s |
Migration duration is flat across a 2,000× range in user count and a 270× range in database size. This is not a weak effect; there is no effect.
Outcome
-
Reached ready in 15.6s · [x] 2,000,003 users intact · [x]
MIGRATION_MODEL→ 26.7.1 -
DATABASECHANGELOG211 rows · [x] login as a SQL-seeded user OK -
IDX_USER_CREATED_TIMESTAMP— absent, silently
Breakage observed
The upgrade completes with a missing index on USER_ENTITY
- Symptom:
IDX_USER_CREATED_TIMESTAMPabsent after a successful upgrade. - Evidence: the two WARN lines above;
pg_indexesreturns 0. - Cause: confirmed —
CustomCreateIndexChangeskips index creation above 300,000 rows. Deliberate Keycloak behaviour. - Fix: create it manually, ideally
CREATE INDEX CONCURRENTLYso production is not locked — Keycloak's plainCREATE INDEXis exactly what it avoided. - Would this hit a customer? Every customer above 300,000 rows in any affected table, on every upgrade that adds an index. Scale-dependent by definition, and it hits the largest installs hardest.
Verification
| Claim | Primary source | Checked |
|---|---|---|
| 2,000,003 users present and migrated | count(*) from Postgres, before and after | ☑ |
| Migration flat 1k → 2M | three instrumented runs, same box, same version pair | ☑ |
USER_ENTITY index skipped at 2M | pg_indexes = 0, plus both WARN lines | ☑ |
| Same index created at 1k and 100k | earlier runs in this series | ☑ |
Threshold is 300,000 for USER_ENTITY too | not bisected — only 100k (created) and 2M (skipped) tested on this table | ☐ |
| Holds for other version pairs | not tested | ☐ |
seed-sql fixture behaves like a partialImport one under migration | not tested — different provenance, see warning above | ☐ |
Publishable extract
We upgraded a two-million-user Keycloak. The schema migration took 3.8 seconds — the same as a thousand-user realm. Here is why that is bad news.
26.0.0 → 26.7.1, official image, Postgres 16.15 at stock settings, Hetzner CCX33 (8 vCPU dedicated, 32 GB, local NVMe), JVM
-Xms1g -Xmx4g.
Users Database Migration Time to serving 1,000 13 MB 3.97s 17.0s 100,002 264 MB 3.79s 15.4s 2,000,003 3,512 MB 3.84s 15.6s A 2,000-fold increase in users changes nothing. We went looking for the reason and found it: Keycloak skips creating indexes on tables larger than 300,000 rows. It logs a WARN and moves on, and
DATABASECHANGELOGrecords the changeset asEXECUTED— so a schema-version check shows everything applied.At two million users, this upgrade quietly declined to create
IDX_USER_CREATED_TIMESTAMPonUSER_ENTITY. That same index is created automatically on a hundred-thousand-user realm. The bigger your Keycloak, the more indexes it does not build.The design choice is defensible — locking
USER_ENTITYon a two-million-row table during startup would be far worse than a late index, and it is why your upgrade window is short. But the consequence is rarely stated: a fast Keycloak upgrade at scale is fast partly because work was deferred onto you, and the only notice is a warning in the noisiest log of your month.After any Keycloak upgrade, list the indexes its changelogs define and diff them against your database. If any are missing, create them yourself with
CREATE INDEX CONCURRENTLY. Then keep doing it, because every future release that indexes a large table will behave the same way.One note on method, because it changes what these numbers mean: the two-million-user realm was seeded with direct SQL rather than through Keycloak's API.
partialImportruns at about 19 users/second — Keycloak 26 hashes every password with argon2id on a single thread — which puts two million users at roughly 29 hours. Direct SQL does it in 167 seconds. Every user shares one password hash, which is fine for migration timing and useless for anything about authentication performance.