ephemeral-pg 0.3.0.0 → 0.3.1.0

raw patch · 5 files changed

+185/−2 lines, 5 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• CHANGELOG.md +23/−0
• README.md +11/−1
• docs/temporary-roots-and-stale-cleanup.md +127/−0
• ephemeral-pg.cabal +2/−1
• src/EphemeralPg.hs +22/−0

@@ -2,6 +2,29 @@  ## Unreleased +## 0.3.1.0++### Bug Fixes++- Export `CacheKey (..)`, `getCacheKey`, `CowCapability (..)`, and+  `CowMethod (..)` from `EphemeralPg`. `clearCache` takes a `CacheKey` and+  `CacheConfig.cow` holds a `Maybe CowCapability`, but neither type was reachable+  from any exposed module, so `clearCache` could not be called and `cow` could+  not be set to anything but `Nothing`.+- Export `withCachedConfig` from `EphemeralPg`. It was the only bracketing entry+  point that accepts both a `Config` and a `CacheConfig`, so cached callers had no+  way to set `temporaryRoot` without reimplementing `startCached`/`stop` bracketing.++### Other Changes++- Document that the startup sweep is scoped to a single temporary root, and that+  an unset `temporaryRoot` resolves to `$TMPDIR`. Environments that allocate a+  per-session `$TMPDIR` (`nix develop`, `nix-shell`, systemd `PrivateTmp`, some CI+  runners) sweep a fresh empty directory on every run and never reclaim clusters+  abandoned by earlier sessions. See `docs/temporary-roots-and-stale-cleanup.md`.+- Ship `docs/temporary-roots-and-stale-cleanup.md` in the source distribution via+  `extra-doc-files`, and correct the `withCachedConfig` haddock example.+ ## 0.3.0.0  ### Breaking Changes

@@ -516,7 +516,17 @@ ```  Create a custom temporary root before using it. The sweep returns sorted canonical-paths actually removed. Disable automatic sweeping with+paths actually removed.++The sweep only inspects its own temporary root. With `temporaryRoot` unset that+root is `$TMPDIR`, which `nix develop`, `nix-shell`, systemd `PrivateTmp` and some+CI runners make unique per session — each run then sweeps a fresh empty directory+and never reclaims what earlier runs abandoned. Set a stable `temporaryRoot` and+use `withCachedConfig` or `withConfig` (the zero-argument `withCached` and `with`+always resolve to `$TMPDIR`). See+[Temporary roots and stale cleanup](docs/temporary-roots-and-stale-cleanup.md).++Disable automatic sweeping with `config { Pg.sweepStaleOnStart = Last (Just False) }`; explicit sweeps still run, and new temporary instances still hold ownership locks. An absent setting enables automatic sweeping.

@@ -0,0 +1,127 @@+# Temporary roots and stale cleanup++`start` and `startCached` sweep abandoned PostgreSQL clusters before allocating a+new one, so a consumer killed with `SIGKILL` does not leak a postmaster forever.+The sweep is scoped to a single directory: the configured `temporaryRoot`, or the+system temporary directory when that is unset.++That scope is the part worth understanding. If the temporary root changes between+runs, the sweep never sees what earlier runs left behind, and abandoned clusters+accumulate indefinitely while the feature appears to be enabled.++## The default is `$TMPDIR`, which is not always stable++With `temporaryRoot` unset, the root comes from+`System.Directory.getTemporaryDirectory`, which returns `$TMPDIR` on POSIX and+falls back to `/tmp`. Several common environments give each session its own+`$TMPDIR`:++| Environment | `$TMPDIR` |+| --- | --- |+| `nix develop` / `nix-shell` | `/tmp/nix-shell.XXXXXX`, new per shell |+| systemd units with `PrivateTmp=yes` | private `/tmp` per service start |+| Some CI runners and container images | per-job or per-step directory |+| macOS launchd services | per-user `/var/folders/...` |++Under any of these, a test run allocates its clusters inside that session's+directory. When the run is killed, the postmaster survives and keeps that+directory alive. The next run gets a *different* `$TMPDIR`, sweeps an empty new+root, finds nothing, and leaks again.++The orphans are still perfectly reapable — they simply are not being looked at.+Nothing in the safety protocol rejects them.++### What this looks like++Four clusters abandoned by four separate `nix develop` sessions, each in its own+root, still running well over a day later:++```+PID 44741  /tmp/nix-shell.Ex1kKN/ephpg-data--81268893861081c0  port 62643+PID 45523  /tmp/nix-shell.Vwr5fv/ephpg-data--67042a9bec14c9a7  port 62675+PID 48818  /tmp/nix-shell.mofVo9/ephpg-data--6ae20a3606256f38  port 62710+PID 58308  /tmp/nix-shell.pPgbfp/ephpg-data--553fa4c95bdb49fe  port 62872+```++Every one had been reparented to PID 1 and held zero client connections — that is,+they satisfied the identity requirements a sweep imposes on a live untracked+postmaster. They survived only because no later session ever searched those roots.++## Set a stable root++Point `temporaryRoot` at a path that does not change between runs, and create it+before use:++```haskell+import Data.Monoid (Last (..))+import EphemeralPg qualified as Pg+import System.Directory (createDirectoryIfMissing)++testConfig :: IO Pg.Config+testConfig = do+  let root = "/tmp/ephpg-my-project"+  createDirectoryIfMissing True root+  pure Pg.defaultConfig { Pg.temporaryRoot = Last (Just root) }+```++Then use the config-taking entry point rather than the zero-argument convenience+wrapper:++```haskell+config <- testConfig+result <- Pg.withCachedConfig config Pg.defaultCacheConfig $ \db ->+  -- Use the database...+```++`withCached` and `with` take no `Config`, so they always resolve to `$TMPDIR`.+Reach for `withCachedConfig` or `withConfig` as soon as you need a stable root.++### Choosing the path++- **Keep it short.** The socket directory lives under the same root, and+  `validateSocketPath` rejects anything that would push the Unix socket path past+  the platform limit. A deep root fails at startup rather than silently.+- **Keep it per-user on shared machines.** The instance registry is already+  namespaced as `.ephemeral-pg-instances-<uid>`, but the root directory itself is+  not. Include the user name if several accounts share the host.+- **Do not point it at a permanent data directory.** Permanent data, sockets,+  snapshots and caches are excluded from sweeping by design.++## Sharing one root across suites++A shared root is safe for concurrent runs and is the intended arrangement. Live+instances hold exclusive lifetime locks, and a sweep claims only unlocked+candidates without blocking, so a sweep in one suite cannot reap a cluster another+suite is actively using. Multiple packages in the same project can and should+point at the same root — that is what lets a run of one suite clean up after a+killed run of another.++## Verifying it works++Cleanup after an abnormal exit is deferred to the *next* startup, so a sweep is+observable only on a subsequent run. To check the wiring end to end:++1. Start an instance, then `SIGKILL` the consumer process (not the postmaster).+2. Confirm the postmaster survives and its data directory is still present under+   the root.+3. Start a second instance from a **new shell session**.+4. Confirm the abandoned postmaster is gone and the directory was removed.++Step 3 is the one that matters: running it from the same shell passes even with a+per-session `$TMPDIR`, which is exactly the bug this document is about.++`sweepStaleInstances` performs the same sweep on demand and returns the sorted+canonical paths it actually removed, which makes it convenient as an assertion:++```haskell+removed <- Pg.sweepStaleInstances config+```++It honours `temporaryRoot` independently of `sweepStaleOnStart`, so it still works+when automatic sweeping is disabled.++## Related++- [ADR 1: Stale instance ownership and reaping](adr/1-stale-instance-ownership-and-reaping.md)+  — the ownership, identity and signalling protocol.+- `README.md`, "Cleanup after a killed consumer" — the short version.

@@ -1,6 +1,6 @@ cabal-version: 3.0 name: ephemeral-pg-version: 0.3.0.0+version: 0.3.1.0 synopsis: Temporary PostgreSQL databases for testing description:   A modern library for creating temporary PostgreSQL instances for testing.@@ -15,6 +15,7 @@ extra-doc-files:   CHANGELOG.md   README.md+  docs/temporary-roots-and-stale-cleanup.md  source-repository head   type: git

@@ -43,6 +43,7 @@     with,     withConfig,     withCached,+    withCachedConfig,     start,     startCached,     sweepStaleInstances,@@ -59,7 +60,11 @@      -- * Cache Management     CacheConfig (..),+    CacheKey (..),+    CowCapability (..),+    CowMethod (..),     defaultCacheConfig,+    getCacheKey,     clearCache,     clearAllCaches, @@ -103,6 +108,7 @@   ) import EphemeralPg.Internal.Cache   ( CacheConfig (..),+    CacheKey (..),     cleanupRuntimeFiles,     clearAllCaches,     clearCache,@@ -112,6 +118,10 @@     isCached,     restoreFromCache,   )+import EphemeralPg.Internal.CopyOnWrite+  ( CowCapability (..),+    CowMethod (..),+  ) import EphemeralPg.Internal.Directory   ( createTempDataDirectory,     createTempSocketDirectory,@@ -380,6 +390,18 @@ withCached = withCachedConfig defaultConfig defaultCacheConfig  -- | Like 'withCached' but with custom configuration.+--+-- Set 'temporaryRoot' here when the caller runs under a per-session @TMPDIR@+-- (@nix develop@, @nix-shell@, systemd @PrivateTmp@, some CI runners). The+-- startup sweep only inspects its own temporary root, so a per-session root+-- hides clusters abandoned by earlier sessions. See+-- @docs/temporary-roots-and-stale-cleanup.md@.+--+-- @+-- let config = 'defaultConfig' { temporaryRoot = Last (Just root) }+-- 'withCachedConfig' config 'defaultCacheConfig' $ \\db -> do+--   -- Use the database...+-- @ withCachedConfig :: Config -> CacheConfig -> (Database -> IO a) -> IO (Either StartError a) withCachedConfig config cacheConfig action = mask $ \restore -> runStartup $ do   db <- liftE $ startCached config cacheConfig