The Keycloak Upgrade Ledger

Every upgrade we have rehearsed, with the environment stated and the clock running.

2026-08-26 · run 2026-08-26-runB-optimized-image-100k

Run B at 100k: optimized image, migration timing and heap floor at scale

Upgrade
26.0.0 → 26.7.1
Scale
100,000 users, typical, baseline-26.0.0-typical-100k
Database
PostgreSQL 16, single node, container
Topology
single container
Host
Hetzner CCX33 (fsn1) — 8 vCPU dedicated, 30 GB RAM, local NVMe
Condition
routine — no adverse condition applied

Why this run existed

Previously untested: the stepped ladder and Run B each had only one scale. our lab protocol asks every path for two scales. This is the second scale for Run B, and it clears two ☐ rows in the original Run B record — the heap floor at scale, and the migration timing at scale.

Environment

FieldValue
Keycloak from → to26.0.0 (stock image) → 26.7.1 (pre-built image)
Imagekclab/keycloak-optimized:26.7.1 (same Dockerfile as Run B)
Start modestart --optimized --http-enabled=true
DatabasePostgreSQL 16, single node, container
Postgres tuningstock
Dataset scale100,000 users, typical, baseline-26.0.0-typical-100k
Topologysingle container
HostHetzner CCX33 (fsn1) — 8 vCPU dedicated, 30 GB RAM, local NVMe
JVM heap-Xms1g -Xmx4g for the migration; variable in the heap leg
Compose overridesdocker-compose.yml:docker-compose.optimized.yml:docker-compose.optimized-noprov.yml

Result

Migration timing at 100k

1k (Run B)100k (this run)
upgrade.sh elapsed11s11s
migration_model26.7.126.7.1
Index audit118/118 correct, 0 missing, 0 wrong shape

The migration is flat between 1k and 100k — same 11s. This is the same mechanism the stock-image runs found (index-skip-threshold.md, Run A): the expensive DDL is declined above 300,000 rows, so "time to ready" does not grow with realm size on this pair. The optimized image saves the augmentation step regardless of scale; it does not change the migration.

Heap floor at 100k

Same bisect boundary values as B2, but on the 100k database:

-XmxResult at 100k
96mREADY
64mFAILEDTerminating due to java.lang.OutOfMemoryError: Java heap space

The floor is (64m, 96m] at 100k, unchanged from 1k. Combined with S5's finding that the stock floor is identical at 1k and 2M, this closes the "scale-dependence" question for the optimized floor too — the heap floor is a function of the build/augmentation step, not of dataset size.

Reproduced: the providers bind-mount trap (B3b-i)

The first attempt at this run failed with exit code 2 before any migration:

A provider JAR was updated since the last build, please rebuild for this to be fully utilized.

Cause: bin/build-spi.sh 25.0.6 (run earlier for the KC-SERVICES0047 probe) had left lab-event-listener-built-for-25.0.6.jar in fixtures/providers/, and the base compose bind-mounts that directory over the optimized image's own empty /opt/keycloak/providers. The runtime jar did not match the image's build. Removing the mount (docker-compose.optimized-noprov.yml) makes the same image start. This is a faithful re-occurrence of B3b-i, not a new finding — worth noting because it means any later run that builds a SPI must clear fixtures/providers/ before an optimized-image run, or add the noprov override.

Tooling observation (minor)

upgrade.sh reported from: 26.7.1 on the successful run, because the failed first attempt had already rewritten .env's KC_VERSION to 26.7.1. The actual migration was 26.0.0 → 26.7.1 (the migration log shows the full 26.1.0 → 26.7.0 realm-migration ladder). upgrade.sh's from is read from .env, not from the database's migration_model — a stale .env after a failed attempt makes the label lie. Not load-bearing for this run; noted for the tooling.

Verification

ClaimPrimary sourceChecked
Migration at 100k takes the same 11s as at 1kupgrade.sh elapsed, both runs
Index audit clean at 100kindex-audit.sh output: 118/118
Heap floor (64m, 96m] holds at 100k96m READY, 64m OOM at 100k
The failure mode is the providers shadowing, not the migrationexit 2 before Liquibase; noprov fixes it
Heap floor at 2M under --optimizednot tested — 100k only; S5's stock 2M point + this 100k point imply it, but it is not measured
Migration timing at 2M under --optimizednot tested — 100k only

← All runs