diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,61 @@ # Changelog +## 0.4.0.0 — 2026-08-16 + +### Breaking Changes + +* **The payload of migration `0010` is corrected, which changes its checksum.** + `pg-migrate` keys an applied migration by `(component, migration)` and + verifies the exact SHA-256 of its payload bytes, so a database that already + applied `0010` from 0.3.2.0 or 0.3.2.1 will fail every `up` and `verify` with + a `MigrationChecksumMismatch` until its ledger row is re-baselined. Run + `ledger-fixups/2026-08-16-rebaseline-0010-checksum.sql` against such a + database once, before migrating; it rewrites that one checksum and touches + nothing else. A database that never reached `0010` — every PostgreSQL 17 + upgrade, which is what this release fixes — needs nothing: `0010` is still + pending there and applies from the corrected payload. + + Editing a released payload is normally forbidden by this package, and a + forward migration is the documented remedy. There is none available here: the + withdrawn payload fails at DDL parse time, so no migration ordered after it + can ever run. 0.3.2.0 and 0.3.2.1 are deprecated on Hackage. + +### Fixes + +* Migration `0010` no longer defaults `history_retention_leases.lease_id` to an + unqualified `uuidv7()` (BUG-1). `uuidv7()` is a PostgreSQL 18 builtin; on + PostgreSQL 17 the name comes from the fallback `0001` installs into the + Kiroku schema, reachable only through the `search_path` that `0001` itself + sets. `0010` therefore parsed on a fresh install, where `0001` had just run in + the same session, and failed with SQLSTATE 42883 on every ordinary upgrade of + a database already bootstrapped through `0009`. Reported by Kioku; confirmed + and fixed against PostgreSQL 17.10. + +### New Features + +* `kiroku.uuidv7()` is now the component's version-independent, always + schema-qualified UUIDv7 generator. `0010` publishes it on PostgreSQL 18 as an + alias for the builtin, and PostgreSQL 17 already had it from `0001`'s + fallback. Migrations after `0001` can name one generator that resolves without + any session state on every supported PostgreSQL version. +* Added forward migration `0011`, which converges databases that applied the + withdrawn `0010`: it publishes `kiroku.uuidv7()` where missing and binds + `lease_id`'s stored default to it. It is a no-op on a database that applied + the corrected `0010`. Verified on PostgreSQL 17.10 and 18.4: a converged + database and a fresh install dump identically. + +### Other Changes + +* The test suite covers the upgrade path that this defect broke — applying the + pending tail of the plan in a session that never ran `0001` and cannot reach + the Kiroku schema through `search_path`. The suite connects as role `kiroku`, + whose default `"$user"` `search_path` entry resolves to the Kiroku schema, so + every previous case was masked from exactly this class of failure. The new + case fails on the withdrawn payload against PostgreSQL 17 and passes on the + corrected one; on PostgreSQL 18 the builtin makes both payloads parse, so + PostgreSQL 17 coverage is what guards this. +* The `ledger-fixups/` scripts ship in the source distribution. + ## 0.3.2.1 — 2026-08-15 ### Other Changes diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ `kiroku-store-migrations` owns Kiroku's PostgreSQL schema as one native `pg-migrate` component named `kiroku`. The component embeds an ordered manifest -and ten SQL payloads, so applications can compose it with other libraries +and eleven SQL payloads, so applications can compose it with other libraries without copying Kiroku SQL. The first seven payloads are immutable historical -Codd bytes; `0008`, `0009`, and `0010` are native-only forward migrations. +Codd bytes; `0008` through `0011` are native-only forward migrations. ## Public API @@ -111,8 +111,29 @@ Migration `0010` adds replay-history retention leases, the per-schema coordinator, an indexed active-lease predicate, and statement-level -`DELETE`/`TRUNCATE` guards on the three event-store data tables. +`DELETE`/`TRUNCATE` guards on the three event-store data tables. Migration +`0011` converges databases that applied the withdrawn 0.3.2.x payload of `0010` +(see below). +## The `kiroku.uuidv7()` generator + +`uuidv7()` is a PostgreSQL 18 builtin. On PostgreSQL 17 `0001` installs a +fallback into the Kiroku schema, and it does so under its own +`SET search_path`, so the bare name resolves only in a session that ran `0001`. +Every migration after `0001` runs in whatever session the operator's upgrade +happens to use, so none of them may name it unqualified — that is BUG-1, fixed +in 0.4.0.0. + +`0010` therefore publishes `kiroku.uuidv7()` on every supported major version: +PostgreSQL 17 already has it from `0001`, and PostgreSQL 18 gets a thin alias +for the builtin. **New migrations that need a UUIDv7 value must call +`kiroku.uuidv7()`, never bare `uuidv7()`.** The qualified name resolves without +any session state on every version the component supports. + +The same rule holds for every other object: name it `kiroku.`. Only +`0001` may rely on `search_path`, because it is the only migration guaranteed +to have set it. + ## Recovery Migrations are forward-only. Before a persistent upgrade, take a backup. If an @@ -120,7 +141,14 @@ migration. Do not delete or rewrite an applied `pgmigrate.migrations` row except through the reviewed `pg-migrate` repair workflow. -The historical script under `ledger-fixups/` remains checked in only as source -evidence for databases that previously needed Codd timestamp repair. New native -migrations use component-local numeric identities and do not use timestamped -filenames. +`ledger-fixups/` holds operator scripts that adjust the migration ledger's +bookkeeping without touching your schema. Read the header of a script before +running it; each states exactly which databases need it. + +* `2026-07-05-realign-kiroku-migration-timestamps.sql` is historical, kept as + source evidence for databases that once needed Codd timestamp repair. New + native migrations use component-local numeric identities. +* `2026-08-16-rebaseline-0010-checksum.sql` re-baselines `0010`'s stored + checksum for databases that applied the withdrawn 0.3.2.0/0.3.2.1 payload. + Required before migrating such a database onto 0.4.0.0 or later; see the + changelog. diff --git a/kiroku-store-migrations.cabal b/kiroku-store-migrations.cabal --- a/kiroku-store-migrations.cabal +++ b/kiroku-store-migrations.cabal @@ -1,6 +1,6 @@ cabal-version: 3.0 name: kiroku-store-migrations -version: 0.3.2.1 +version: 0.4.0.0 synopsis: Schema migrations for kiroku-store description: Native pg-migrate component, Codd history mapping, and migration executable @@ -15,6 +15,7 @@ category: Database, Eventing extra-doc-files: CHANGELOG.md extra-source-files: + ledger-fixups/*.sql migrations/*.sql migrations/manifest migrations.lock diff --git a/ledger-fixups/2026-07-05-realign-kiroku-migration-timestamps.sql b/ledger-fixups/2026-07-05-realign-kiroku-migration-timestamps.sql new file mode 100644 --- /dev/null +++ b/ledger-fixups/2026-07-05-realign-kiroku-migration-timestamps.sql @@ -0,0 +1,62 @@ +-- Ledger realignment for the kiroku migration-timestamp rename. +-- +-- The kiroku-store migrations were renamed from hand-assigned sentinel +-- timestamps (…-00-00-00, …-00-00-01, …) to their real UTC authoring times +-- (commits dac1a0b and e1f6c02). codd decides whether a migration is already +-- applied by FILENAME (`SELECT … FROM codd.sql_migrations WHERE name = ?`), +-- so a database that already applied the old names would otherwise treat every +-- renamed file as pending and re-run it. +-- +-- This script rewrites the `name` and `migration_timestamp` columns of the +-- codd ledger from the old identity to the new one, so codd sees the renamed +-- migrations as already applied and skips them. It changes ONLY codd's +-- bookkeeping — never your schema. +-- +-- WHEN TO RUN: once per long-lived database (staging/prod/persistent local), +-- BEFORE the next `codd up` / migrate that carries the renamed files. This +-- includes downstream databases that ran kiroku migrations bundled with a +-- consumer's own (e.g. keiro's combined kiroku<>keiro ledger). Ephemeral / +-- template-per-suite test databases do not need it — they apply from scratch. +-- +-- SAFETY: the remap is 1:1 onto brand-new values, so neither UNIQUE(name) nor +-- UNIQUE(migration_timestamp) can be violated (the new timestamps also do not +-- collide with any keiro rows in a combined ledger); and it is idempotent — a +-- second run matches no rows. Wrapped in a transaction so it is all-or-nothing. +-- +-- LEDGER LOCATION: codd v0.1.8 stores fresh ledgers at +-- `codd.sql_migrations` and auto-renames older `codd_schema` ledgers on first +-- contact. This script detects `codd.sql_migrations` first and falls back to +-- `codd_schema.sql_migrations` for databases that have not yet been touched by +-- a v0.1.8 migrate. + +BEGIN; + +DO $$ +DECLARE + ledger_table regclass; +BEGIN + ledger_table := to_regclass('codd.sql_migrations'); + IF ledger_table IS NULL THEN + ledger_table := to_regclass('codd_schema.sql_migrations'); + END IF; + + IF ledger_table IS NULL THEN + RAISE EXCEPTION 'Could not find codd.sql_migrations or codd_schema.sql_migrations'; + END IF; + + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-05-16-12-17-14-kiroku-bootstrap.sql', '2026-05-16 12:17:14+00', '2026-05-16-00-00-00-kiroku-bootstrap.sql'); + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-05-29-15-26-04-add-subscription-dead-letters.sql', '2026-05-29 15:26:04+00', '2026-05-26-00-00-00-add-subscription-dead-letters.sql'); + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-06-14-13-17-09-notify-trigger-append-guard.sql', '2026-06-14 13:17:09+00', '2026-06-11-00-00-00-notify-trigger-append-guard.sql'); + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-06-14-13-25-40-dead-letters-event-id-index.sql', '2026-06-14 13:25:40+00', '2026-06-11-00-00-01-dead-letters-event-id-index.sql'); + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-06-14-13-54-48-index-hygiene-and-streams-fillfactor.sql', '2026-06-14 13:54:48+00', '2026-06-11-00-00-02-index-hygiene-and-streams-fillfactor.sql'); + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-06-14-14-01-17-stream-name-length-check.sql', '2026-06-14 14:01:17+00', '2026-06-11-00-00-03-stream-name-length-check.sql'); + EXECUTE format('UPDATE %s SET name = %L, migration_timestamp = %L::timestamptz WHERE name = %L', ledger_table, '2026-06-24-09-42-22-stream-truncate-before.sql', '2026-06-24 09:42:22+00', '2026-06-24-00-00-00-stream-truncate-before.sql'); +END $$; + +-- Sanity check: no stale sentinel-named kiroku rows should remain. +-- Expect zero rows. +-- SELECT name FROM codd.sql_migrations +-- WHERE name LIKE '2026-%' AND substr(name, 12, 8) IN +-- ('00-00-00','00-00-01','00-00-02','00-00-03'); + +COMMIT; diff --git a/ledger-fixups/2026-08-16-rebaseline-0010-checksum.sql b/ledger-fixups/2026-08-16-rebaseline-0010-checksum.sql new file mode 100644 --- /dev/null +++ b/ledger-fixups/2026-08-16-rebaseline-0010-checksum.sql @@ -0,0 +1,83 @@ +-- Ledger re-baseline for the corrected payload of migration 0010 (BUG-1). +-- +-- kiroku-store-migrations 0.3.2.0 and 0.3.2.1 shipped a payload of 0010 whose +-- history_retention_leases.lease_id default named uuidv7() unqualified. On +-- PostgreSQL 17 that name resolves only through search_path, which is set by +-- 0001 and by nothing else, so 0010 could not parse in any session that did not +-- also apply 0001 -- that is, in every ordinary upgrade. 0.4.0.0 corrects the +-- payload; there is no forward-migration alternative, because the withdrawn +-- payload fails at DDL parse time and nothing after it can run. +-- +-- pg-migrate keys an applied migration by (component, migration) and verifies +-- the exact SHA-256 of its payload bytes. Correcting the payload therefore +-- changes 0010's checksum, and a database that already applied the withdrawn +-- payload will fail every subsequent `up` and `verify` with a +-- MigrationChecksumMismatch until its stored checksum is re-baselined. This +-- script performs that re-baseline and nothing else. +-- +-- WHEN TO RUN: once per long-lived database that ALREADY APPLIED the withdrawn +-- 0010 -- PostgreSQL 18, or a fresh PostgreSQL 17 install performed by +-- 0.3.2.0/0.3.2.1 -- BEFORE the next migrate carrying 0.4.0.0. A database that +-- never reached 0010 (the PostgreSQL 17 upgrade path this bug broke) does not +-- need it: 0010 is still pending there and applies from the corrected payload. +-- Ephemeral test databases do not need it either; they apply from scratch. +-- +-- WHAT IT DOES NOT DO: it does not touch your schema. The withdrawn and +-- corrected payloads differ in two schema-visible ways -- the corrected one +-- publishes kiroku.uuidv7() on PostgreSQL 18 and binds lease_id's default to +-- it. Forward migration 0011 applies both to an already-0010 database through +-- the ordinary runner, so run this script first and then migrate normally. +-- Verified on PostgreSQL 17.10 and 18.4: a database converged this way dumps +-- byte-for-byte identically to a fresh 0.4.0.0 install. +-- +-- SAFETY: the UPDATE matches only a row still carrying the withdrawn checksum, +-- so it is idempotent -- a second run, or a run against a database that applied +-- the corrected payload, matches no rows. Wrapped in a transaction. +-- +-- LEDGER LOCATION: pg-migrate's default ledger schema is `pgmigrate`. If your +-- application configured a different schema through `ledgerConfig`, change the +-- schema name in the to_regclass call below to match. + +BEGIN; + +DO $$ +DECLARE + ledger_table regclass; + withdrawn_checksum bytea := + decode('257d94b8ea24156af0ee477196c5a6682cf5616cf8dfd0cfc527a49b5e7a97ac', 'hex'); + corrected_checksum bytea := + decode('debc19187d79cd263be66c6c85cd789c1176e508d562a27390095aaa70f650c4', 'hex'); + rebaselined integer; +BEGIN + ledger_table := to_regclass('pgmigrate.migrations'); + + IF ledger_table IS NULL THEN + RAISE EXCEPTION + 'Could not find pgmigrate.migrations; edit this script if the ledger uses a different schema'; + END IF; + + EXECUTE format( + 'UPDATE %s SET checksum = $1 + WHERE component = ''kiroku'' AND migration = ''0010'' AND checksum = $2', + ledger_table + ) + USING corrected_checksum, withdrawn_checksum; + + GET DIAGNOSTICS rebaselined = ROW_COUNT; + + IF rebaselined = 0 THEN + RAISE NOTICE + 'no kiroku/0010 row carried the withdrawn checksum; nothing to re-baseline'; + ELSE + RAISE NOTICE 're-baselined the kiroku/0010 checksum'; + END IF; +END $$; + +-- Sanity check: kiroku/0010 must now carry the corrected checksum, or be +-- absent because this database has not reached 0010 yet. Expect one row with +-- ok = true, or zero rows. +-- SELECT encode(checksum, 'hex') = +-- 'debc19187d79cd263be66c6c85cd789c1176e508d562a27390095aaa70f650c4' AS ok +-- FROM pgmigrate.migrations WHERE component = 'kiroku' AND migration = '0010'; + +COMMIT; diff --git a/migrations/0010.sql b/migrations/0010.sql --- a/migrations/0010.sql +++ b/migrations/0010.sql @@ -1,5 +1,39 @@ -- add replay history retention +-- Publish kiroku.uuidv7() as the component's version-independent, always +-- schema-qualified UUIDv7 generator. +-- +-- PostgreSQL 18 provides pg_catalog.uuidv7(), so 0001 installed no fallback +-- and the name exists only in pg_catalog. PostgreSQL 17 has no builtin, so +-- 0001 installed the fallback into the Kiroku schema. Naming uuidv7() +-- unqualified therefore resolves only through search_path, which no migration +-- after 0001 may depend on. Establishing kiroku.uuidv7() on both versions lets +-- this migration -- and every migration after it -- name one generator that +-- resolves without session state on every PostgreSQL version the component +-- supports. +DO $$ +BEGIN + IF to_regprocedure('kiroku.uuidv7()') IS NOT NULL THEN + -- PostgreSQL 17: 0001's fallback already occupies the name. + RETURN; + END IF; + + IF to_regprocedure('pg_catalog.uuidv7()') IS NULL THEN + RAISE EXCEPTION + 'no uuidv7() generator: neither pg_catalog.uuidv7() nor kiroku.uuidv7() exists'; + END IF; + + -- PostgreSQL 18+: alias the builtin so the qualified name is available. + EXECUTE $fn$ + CREATE FUNCTION kiroku.uuidv7() + RETURNS uuid + LANGUAGE sql + VOLATILE + AS 'SELECT pg_catalog.uuidv7()' + $fn$; +END +$$; + -- A schema-local singleton row serializes lease lifecycle changes with every -- destructive statement without adding work to append or ordinary read paths. CREATE TABLE kiroku.history_retention_coordinator ( @@ -15,7 +49,7 @@ -- set to remain stable. Active state is derived from released_at and database -- time; expiry needs no worker or cleanup mutation. CREATE TABLE kiroku.history_retention_leases ( - lease_id UUID PRIMARY KEY DEFAULT uuidv7(), + lease_id UUID PRIMARY KEY DEFAULT kiroku.uuidv7(), owner TEXT NOT NULL, reason TEXT NOT NULL, protected_through BIGINT NOT NULL, diff --git a/migrations/0011.sql b/migrations/0011.sql new file mode 100644 --- /dev/null +++ b/migrations/0011.sql @@ -0,0 +1,51 @@ +-- converge databases that applied the withdrawn 0.3.2.x payload of 0010 + +-- kiroku-store-migrations 0.3.2.0 and 0.3.2.1 shipped a payload of 0010 whose +-- lease_id default named uuidv7() unqualified. That payload could not parse on +-- PostgreSQL 17 outside the bootstrap session (BUG-1), and where it did apply +-- -- PostgreSQL 18, or a fresh PostgreSQL 17 install -- it left the schema in a +-- state the corrected 0010 does not produce: no kiroku.uuidv7(), and a lease_id +-- default bound to whichever uuidv7() the applying session happened to resolve. +-- +-- The corrected 0010 cannot reach those databases: its ledger row is already +-- present, and re-baselining that row (see ledger-fixups/) restores the +-- checksum without replaying the SQL. This migration is what makes both +-- populations converge, and it is a no-op on any database that applied the +-- corrected 0010. + +-- 1. kiroku.uuidv7() exists on every supported major version. Identical to the +-- header of the corrected 0010, and skipped there because 0010 ran first. +DO $$ +BEGIN + IF to_regprocedure('kiroku.uuidv7()') IS NOT NULL THEN + -- PostgreSQL 17, or a database that applied the corrected 0010. + RETURN; + END IF; + + IF to_regprocedure('pg_catalog.uuidv7()') IS NULL THEN + RAISE EXCEPTION + 'no uuidv7() generator: neither pg_catalog.uuidv7() nor kiroku.uuidv7() exists'; + END IF; + + EXECUTE $fn$ + CREATE FUNCTION kiroku.uuidv7() + RETURNS uuid + LANGUAGE sql + VOLATILE + AS 'SELECT pg_catalog.uuidv7()' + $fn$; +END +$$; + +-- 2. Bind lease_id's default to the qualified generator. A column default is +-- stored as a resolved function OID, so a database that applied the +-- withdrawn payload on PostgreSQL 18 holds pg_catalog.uuidv7() here. Both +-- generators produce the same values; this only makes the stored default +-- match what the corrected 0010 installs, so a dumped schema is identical +-- whichever payload the database applied. Existing lease rows are untouched +-- and no table is rewritten. +ALTER TABLE kiroku.history_retention_leases + ALTER COLUMN lease_id SET DEFAULT kiroku.uuidv7(); + +COMMENT ON SCHEMA kiroku IS + 'Managed by pg-migrate component kiroku through 0011'; diff --git a/migrations/manifest b/migrations/manifest --- a/migrations/manifest +++ b/migrations/manifest @@ -8,3 +8,4 @@ 0008-schema-management-comment.sql 0009.sql 0010.sql +0011.sql diff --git a/test/Main.hs b/test/Main.hs --- a/test/Main.hs +++ b/test/Main.hs @@ -11,6 +11,8 @@ import Data.Foldable (toList) import Data.Int (Int32, Int64) import Data.List (sort) +import Data.List.NonEmpty (NonEmpty ((:|))) +import Data.List.NonEmpty qualified as NonEmpty import Data.Text (Text) import Data.Text qualified as Text import Data.Text.IO qualified as Text.IO @@ -42,7 +44,7 @@ main :: IO () main = hspec $ do describe "native Kiroku migration definition" $ do - it "tracks the ten native files in manifest order" $ do + it "tracks the eleven native files in manifest order" $ do directory <- findMigrationsDirectory manifest <- Text.lines <$> Text.IO.readFile (directory "manifest") manifest `shouldBe` Text.pack <$> nativeMigrationFiles @@ -55,7 +57,7 @@ bytes <- ByteString.readFile (directory nativeName) lookup legacyName lockEntries `shouldBe` Just (checksumText bytes) - it "builds component kiroku and a ten-migration plan" $ do + it "builds component kiroku and an eleven-migration plan" $ do component <- requireRight kirokuMigrations component `seq` pure () plan <- requirePlan @@ -91,7 +93,7 @@ `shouldReturn` "0007-existing.sql\n" describe "fresh native databases" $ do - it "applies all ten, verifies strictly, and reports AlreadyApplied on rerun" $ do + it "applies all eleven, verifies strictly, and reports AlreadyApplied on rerun" $ do plan <- requirePlan result <- withMigratedDatabase plan $ \connection -> do assertSchema connection @@ -273,6 +275,66 @@ facts `shouldBe` (1, 9, True, True, 6) either (expectationFailure . show) pure result + -- BUG-1. Migrations 0002 onward are parsed in sessions that never ran 0001, + -- so a name that only 0001's `SET search_path` makes resolvable parses on a + -- fresh install and fails on every ordinary upgrade. 0.3.2.x's 0010 named + -- uuidv7() that way and could not be applied to a database bootstrapped + -- through 0009 on PostgreSQL 17. These cases apply the tail of the plan in a + -- session that carries none of 0001's state. + describe "upgrades of already-bootstrapped databases" $ do + it "applies the pending tail in a session that never ran the bootstrap" $ do + plan <- requirePlan + throughBootstrap <- planThrough (length nativeMigrationFiles - 2) + withKirokuPg $ \database -> do + let settings = Pg.connectionSettings database + bootstrapped <- + runMigrationPlan defaultRunOptions settings throughBootstrap + >>= requireMigration + reportOutcomes bootstrapped + `shouldBe` replicate (length nativeMigrationFiles - 2) AppliedNow + + -- A separate session, and one that cannot reach the Kiroku + -- schema through search_path. The suite connects as role + -- "kiroku", so the default "$user" entry resolves to the Kiroku + -- schema and would hide exactly the name resolution this case + -- exists to prove -- the same accident that let 0001's + -- SET search_path carry a fresh install through 0.3.2.x's 0010. + withConnection settings $ \upgradeSession -> do + useSession upgradeSession (Session.script "SET search_path TO pg_catalog") + upgraded <- + runMigrationPlanWith defaultRunOptions (providerFor upgradeSession) plan + >>= requireMigration + reportOutcomes upgraded + `shouldBe` replicate (length nativeMigrationFiles - 2) AlreadyApplied + <> replicate 2 AppliedNow + + verified <- verifyMigrationPlan defaultRunOptions settings plan >>= requireMigration + case verified of + VerificationReport verificationIssues applied _ _ -> do + verificationIssues `shouldBe` [] + length applied `shouldBe` length nativeMigrationFiles + withConnection settings assertSchema + + it "resolves the lease default through the qualified component generator" $ do + plan <- requirePlan + result <- withMigratedDatabase plan $ \connection -> do + -- kiroku.uuidv7() is the component's version-independent + -- generator: PostgreSQL 17 gets 0001's fallback, PostgreSQL 18 + -- gets 0010's alias for the builtin. Either way the stored + -- default names it, and no later migration needs search_path to + -- reach a UUIDv7 generator. + generator <- useSession connection (Session.statement () leaseDefaultStatement) + generator `shouldBe` (True, Just "kiroku.uuidv7()") + either (expectationFailure . show) pure result + + it "acquires a lease from a session whose search_path excludes kiroku" $ do + plan <- requirePlan + result <- withMigratedDatabase plan $ \connection -> do + useSession connection (Session.script "SET search_path TO pg_catalog") + version <- useSession connection (Session.statement () insertLeaseStatement) + version `shouldBe` 7 + either (expectationFailure . show) pure result + describe "Codd history import" $ do it "imports a current codd V5 ledger, verifies, and never replays SQL" $ importFixture "codd" @@ -318,14 +380,14 @@ pendingIds <- traverse (requireRight . migrationId "kiroku") - ["0008-schema-management-comment", "0009", "0010"] + ["0008-schema-management-comment", "0009", "0010", "0011"] verifiedBeforeCanary <- verifyMigrationPlan defaultRunOptions settings plan >>= requireMigration case verifiedBeforeCanary of VerificationReport verificationIssues _ _ _ -> verificationIssues `shouldBe` (PendingMigration <$> pendingIds) up <- runMigrationPlan defaultRunOptions settings plan >>= requireMigration - reportOutcomes up `shouldBe` replicate 7 AlreadyApplied <> replicate 3 AppliedNow + reportOutcomes up `shouldBe` replicate 7 AlreadyApplied <> replicate 4 AppliedNow verifiedAfterCanary <- verifyMigrationPlan defaultRunOptions settings plan >>= requireMigration case verifiedAfterCanary of VerificationReport verificationIssues _ _ _ -> @@ -355,8 +417,27 @@ , "0008-schema-management-comment.sql" , "0009.sql" , "0010.sql" + , "0011.sql" ] +{- | The plan truncated to its first @count@ migrations, read from the checked-in +directory. It stands in for a database that a previous release bootstrapped: the +ledger it produces is byte-identical to the one the full plan would have written +for those entries, so the full plan later sees exactly the remainder as pending. +-} +planThrough :: Int -> IO MigrationPlan +planThrough count = do + directory <- findMigrationsDirectory + entries <- traverse (readEntry directory) (take count nativeMigrationFiles) + component <- + requireRight + (migrationComponentFromEmbeddedSql "kiroku" mempty (NonEmpty.fromList entries)) + requireRight (migrationPlan (component :| [])) + where + readEntry directory file = do + bytes <- ByteString.readFile (directory file) + pure (file, bytes) + findMigrationsDirectory :: IO FilePath findMigrationsDirectory = findDirectory ["kiroku-store-migrations/migrations", "migrations"] @@ -472,11 +553,48 @@ (EXISTS (SELECT 1 FROM pg_catalog.pg_indexes WHERE schemaname = 'kiroku' AND indexname = 'ix_dead_letters_event_id')), (EXISTS (SELECT 1 FROM pg_catalog.pg_constraint WHERE conname = 'chk_streams_stream_name_length')), (EXISTS (SELECT 1 FROM pg_catalog.pg_attribute a JOIN pg_catalog.pg_class c ON c.oid = a.attrelid JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace WHERE n.nspname = 'kiroku' AND c.relname = 'streams' AND a.attname = 'truncate_before' AND NOT a.attisdropped)), - (obj_description(to_regnamespace('kiroku'), 'pg_namespace') = 'Managed by pg-migrate component kiroku through 0010') + (obj_description(to_regnamespace('kiroku'), 'pg_namespace') = 'Managed by pg-migrate component kiroku through 0011') ) AS checks(ok) """ Encoders.noParams (Decoders.singleRow (Decoders.column (Decoders.nonNullable Decoders.bool))) + +-- | Whether @kiroku.uuidv7()@ exists, and the stored @lease_id@ default. +leaseDefaultStatement :: Statement () (Bool, Maybe Text) +leaseDefaultStatement = + Statement.preparable + """ + SELECT to_regprocedure('kiroku.uuidv7()') IS NOT NULL, + (SELECT pg_catalog.pg_get_expr(stored.adbin, stored.adrelid) + FROM pg_catalog.pg_attrdef AS stored + JOIN pg_catalog.pg_attribute AS attribute + ON attribute.attrelid = stored.adrelid + AND attribute.attnum = stored.adnum + WHERE stored.adrelid = 'kiroku.history_retention_leases'::regclass + AND attribute.attname = 'lease_id') + """ + Encoders.noParams + ( Decoders.singleRow + ( (,) + <$> Decoders.column (Decoders.nonNullable Decoders.bool) + <*> Decoders.column (Decoders.nullable Decoders.text) + ) + ) + +-- | Insert one lease and report the UUID version its default generated. +insertLeaseStatement :: Statement () Int32 +insertLeaseStatement = + Statement.preparable + """ + INSERT INTO kiroku.history_retention_leases + (owner, reason, protected_through, created_at, renewed_at, expires_at) + VALUES + ('kiroku-store-migrations-test', 'search_path independence', 0, + pg_catalog.now(), pg_catalog.now(), pg_catalog.now() + interval '1 hour') + RETURNING pg_catalog.get_byte(pg_catalog.uuid_send(lease_id), 6) >> 4 + """ + Encoders.noParams + (Decoders.singleRow (Decoders.column (Decoders.nonNullable Decoders.int4))) historyRetentionColumnsStatement :: Statement () [(Text, Text, Text, Bool)] historyRetentionColumnsStatement =