okf-core 0.1.2.0 → 0.2.0.0

raw patch · 37 files changed

+966/−16 lines, 37 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• CHANGELOG.md +18/−0
• dhall/FrontmatterRules.dhall +6/−0
• dhall/Profile.dhall +24/−0
• dhall/TypeRule.dhall +17/−0
• dhall/defaults/FrontmatterRules.dhall +6/−0
• dhall/defaults/Profile.dhall +16/−0
• dhall/defaults/TypeRule.dhall +12/−0
• dhall/package.dhall +19/−0
• okf-core.cabal +20/−11
• src/Okf/Bundle.hs +19/−0
• src/Okf/Discovery.hs +135/−0
• src/Okf/Profile.hs +150/−2
• test/Main.hs +286/−3
• test/fixtures/doc-id-deviations/decisions/first.md +7/−0
• test/fixtures/doc-id-deviations/decisions/fourth.md +6/−0
• test/fixtures/doc-id-deviations/decisions/second.md +7/−0
• test/fixtures/doc-id-deviations/decisions/third.md +7/−0
• test/fixtures/doc-ids/decisions/adopt-okf.md +9/−0
• test/fixtures/doc-ids/decisions/use-markdown.md +9/−0
• test/fixtures/doc-ids/decisions/use-postgres.md +9/−0
• test/fixtures/invalid-dangling-link/orders.md +10/−0
• test/fixtures/invalid-missing-type/missing-type.md +9/−0
• test/fixtures/invalid-unterminated-frontmatter/broken.md +6/−0
• test/fixtures/profile-deviations/schemas/sales/tables/bad.md +11/−0
• test/fixtures/profile-deviations/schemas/sales/tables/customers.md +11/−0
• test/fixtures/profile-deviations/schemas/sales/tables/orders.md +10/−0
• test/fixtures/profiles/decisions.dhall +21/−0
• test/fixtures/profiles/postgresql.dhall +38/−0
• test/fixtures/valid-bundle/datasets/index.md +3/−0
• test/fixtures/valid-bundle/datasets/sales.md +11/−0
• test/fixtures/valid-bundle/index.md +5/−0
• test/fixtures/valid-bundle/log.md +4/−0
• test/fixtures/valid-bundle/references/index.md +3/−0
• test/fixtures/valid-bundle/references/source-system.md +10/−0
• test/fixtures/valid-bundle/tables/customers.md +11/−0
• test/fixtures/valid-bundle/tables/index.md +4/−0
• test/fixtures/valid-bundle/tables/orders.md +17/−0

@@ -7,6 +7,24 @@  ## [Unreleased] +## [0.2.0.0] - 2026-07-26++### Added++- Profile-declared stable document IDs with strict parsing, missing/malformed/+  duplicate validation, allocation helpers, and bundle lookup by handle.+- `Okf.Discovery`, which finds OKF bundle roots in a directory tree: directories+  holding an `index.md` or a concept document with a non-empty `type`, pruned at+  the first match so nested directories of a bundle are not reported separately.++### Changed++- The published profile Dhall schema gained required `idField` and `idPrefix`+  record fields. Existing descriptors, including those in the separate+  `okf-profiles` repository, must add `idField` and `idPrefix` values or adopt+  the new record-completion defaults under `dhall/defaults/`. This is a breaking+  schema change.+ ## [0.1.2.0] - 2026-07-14  ### Added

@@ -0,0 +1,6 @@+--| Canonical schema for a profile's frontmatter expectations.+--+-- Mirrors the `FrontmatterRules` decoder in `okf-core/src/Okf/Profile.hs`.+{ required : List Text+, recommended : List Text+}

@@ -0,0 +1,24 @@+--| Canonical schema for a complete OKF profile.+--+-- This record type is the contract that `okf validate --profile` accepts. It is+-- owned and published by okf; okf-profiles and downstream projects import it.+-- It mirrors the `ProfileSpec` decoder in `okf-core/src/Okf/Profile.hs`, kept in+-- sync by the drift guard in `okf-core/test/Main.hs`.+--+-- Profiles are NOT part of the OKF standard. A bundle that deviates from a profile+-- remains fully OKF-conformant; `okf validate --profile` reports deviations as+-- advisory by default.+--+-- `idField = Some "docId"` names the frontmatter key that holds stable document+-- handles.  `None Text` disables every document-ID check.+let TypeRule = ./TypeRule.dhall++let FrontmatterRules = ./FrontmatterRules.dhall++in  { name : Text+    , okfVersion : Text+    , frontmatter : FrontmatterRules+    , allowUnknownTypes : Bool+    , idField : Optional Text+    , types : List TypeRule+    }

@@ -0,0 +1,17 @@+--| Canonical schema for one per-`type` rule in an OKF profile.+--+-- This is the single source of truth for the rule shape. It mirrors the+-- `TypeRule` decoder in `okf-core/src/Okf/Profile.hs`; the two are kept in sync by+-- the drift guard in `okf-core/test/Main.hs` (the schema-annotated profile fixture+-- must decode). Other repositories (e.g. okf-profiles) import this type; okf+-- imports nothing remote in return.+--+-- `idPrefix = Some "ADR"` means concepts governed by this rule are expected to+-- carry a handle of the form `ADR-<number>` in the profile's ID field.+{ type : Text+, pathPattern : Optional Text+, resourceScheme : Optional Text+, requireSchemaSection : Bool+, schemaColumns : List Text+, idPrefix : Optional Text+}

@@ -0,0 +1,6 @@+--| Record-completion defaults for profile frontmatter expectations.+let FrontmatterRulesType = ../FrontmatterRules.dhall++in  { Type = FrontmatterRulesType+    , default = { required = [] : List Text, recommended = [] : List Text }+    }

@@ -0,0 +1,16 @@+--| Record-completion defaults for a complete OKF profile.+let ProfileType = ../Profile.dhall++let FrontmatterRules = ./FrontmatterRules.dhall++let TypeRule = ../TypeRule.dhall++in  { Type = ProfileType+    , default =+      { okfVersion = "0.1"+      , frontmatter = FrontmatterRules.default+      , allowUnknownTypes = True+      , idField = None Text+      , types = [] : List TypeRule+      }+    }

@@ -0,0 +1,12 @@+--| Record-completion defaults for one per-`type` profile rule.+let TypeRuleType = ../TypeRule.dhall++in  { Type = TypeRuleType+    , default =+      { pathPattern = None Text+      , resourceScheme = None Text+      , requireSchemaSection = False+      , schemaColumns = [] : List Text+      , idPrefix = None Text+      }+    }

@@ -0,0 +1,19 @@+--| Entry point for okf's published profile schema.+--+-- Import this (by relative path within okf, or by pinned URL from another repo) to+-- get the profile schema types and record-completion defaults:+--+--     let okf = https://raw.githubusercontent.com/shinzui/okf/<tag>/okf-core/dhall/package.dhall sha256:<hash>+--     in  ({ name = "acme", okfVersion = "0.1", … } : okf.Profile)+--+-- okf itself imports nothing remote; the relationship with okf-profiles is one-way+-- (okf-profiles imports this).+{ Profile = ./Profile.dhall+, TypeRule = ./TypeRule.dhall+, FrontmatterRules = ./FrontmatterRules.dhall+, defaults =+  { Profile = ./defaults/Profile.dhall+  , TypeRule = ./defaults/TypeRule.dhall+  , FrontmatterRules = ./defaults/FrontmatterRules.dhall+  }+}

@@ -1,6 +1,6 @@-cabal-version:   3.4-name:            okf-core-version:         0.1.2.0+cabal-version:      3.4+name:               okf-core+version:            0.2.0.0 synopsis:   Read, validate, index, and traverse Open Knowledge Format bundles @@ -10,15 +10,23 @@   index and link graph, validates referential integrity, and writes bundles back   out with a round-trip guarantee. -category:        Data, Text-license:         BSD-3-Clause-license-file:    LICENSE-author:          Nadeem Bitar-maintainer:      nadeem@gmail.com-copyright:       (c) 2026 Nadeem Bitar-build-type:      Simple-extra-doc-files: CHANGELOG.md+category:           Data, Text+license:            BSD-3-Clause+license-file:       LICENSE+author:             Nadeem Bitar+maintainer:         nadeem@gmail.com+copyright:          (c) 2026 Nadeem Bitar+build-type:         Simple+extra-doc-files:    CHANGELOG.md +-- The canonical profile schema, plus the fixtures okf-core-test reads. The+-- fixture descriptors import the schema through ../../../dhall, so both trees+-- must ship for `cabal test` to work from the sdist.+extra-source-files:+  dhall/**/*.dhall+  test/fixtures/**/*.dhall+  test/fixtures/**/*.md+ common common-options   ghc-options:     -Wall -Wcompat -Widentities -Wincomplete-uni-patterns@@ -39,6 +47,7 @@   exposed-modules:     Okf.Bundle     Okf.ConceptId+    Okf.Discovery     Okf.Document     Okf.Graph     Okf.Index

@@ -13,6 +13,7 @@     conceptTitle,     conceptType,     findConcept,+    findConceptsByDocumentId,     isReservedMarkdownFile,     serializeConcept,     walkBundle,@@ -22,6 +23,7 @@ where  import Control.Exception (IOException, try)+import Data.Aeson.KeyMap qualified as KeyMap import Data.List qualified as List import Data.Text qualified as Text import Data.Text.IO qualified as Text.IO@@ -89,6 +91,23 @@ findConcept :: ConceptId -> [Concept] -> Maybe Concept findConcept conceptId =   List.find (\concept -> conceptIdOf concept == conceptId)++-- | Find concepts whose frontmatter carries the given document handle. When a+-- field is supplied, only that key is examined; otherwise every frontmatter+-- value is searched. All matches are returned so callers can report ambiguity.+findConceptsByDocumentId :: Maybe Text -> Text -> [Concept] -> [Concept]+findConceptsByDocumentId fieldFilter handle =+  filter conceptMatches+  where+    conceptMatches concept =+      case fieldFilter of+        Just fieldName ->+          valueMatches (frontmatterLookup fieldName (documentFrontmatter concept))+        Nothing ->+          any (valueMatches . Just) (KeyMap.elems (fields (documentFrontmatter concept)))+    valueMatches (Just (String value)) = Text.strip value == handle+    valueMatches _ = False+    documentFrontmatter concept = frontmatter (conceptDocument concept)  -- | Extract a concept identifier without colliding with Prelude's `id`. conceptIdOf :: Concept -> ConceptId

@@ -0,0 +1,135 @@+-- | Discovery of OKF bundle roots in a directory tree.+--+-- A bundle root is a directory that looks like the top of an OKF bundle: it+-- either holds a reserved @index.md@, or it holds at least one concept+-- document (a non-reserved @.md@ file whose YAML frontmatter carries a+-- non-empty @type@ field). Discovery stops descending as soon as a directory+-- qualifies, so nested subdirectories of a bundle are never reported as+-- bundles of their own.+--+-- Discovery is a convenience for interactive callers, not a validation step:+-- directories that cannot be listed or files that cannot be read are skipped+-- rather than reported as errors.+module Okf.Discovery+  ( DiscoveryOptions (..),+    defaultDiscoveryOptions,+    discoverBundleRoots,+    directoryQualifiesAsBundleRoot,+  )+where++import Control.Exception (IOException, try)+import Control.Monad (filterM)+import Data.List qualified as List+import Data.Text qualified as Text+import Data.Text.IO qualified as Text.IO+import Okf.Bundle (isReservedMarkdownFile)+import Okf.Document (Frontmatter, OKFDocument (..), frontmatterLookup, parseDocument)+import Okf.Prelude+import System.Directory (doesDirectoryExist, listDirectory, pathIsSymbolicLink)+import System.FilePath ((</>))+import System.FilePath qualified as FilePath++-- | How far and where 'discoverBundleRoots' may look.+data DiscoveryOptions = DiscoveryOptions+  { -- | How many directory levels below the search root to inspect. The search+    -- root itself is depth 0, so @maxDepth = 4@ inspects four levels beneath it.+    maxDepth :: !Int,+    -- | Directory names never entered, regardless of depth. Directories whose+    -- name begins with @.@ are always skipped and need no entry here.+    skipDirectories :: ![FilePath]+  }+  deriving stock (Generic, Eq, Show)++-- | Depth four with the usual build-output directories skipped. Depth four is+-- enough to reach a bundle nested a few levels inside a source repository+-- without walking an entire home directory.+defaultDiscoveryOptions :: DiscoveryOptions+defaultDiscoveryOptions =+  DiscoveryOptions+    { maxDepth = 4,+      skipDirectories =+        [ "dist-newstyle",+          "dist",+          "node_modules",+          "target",+          "vendor",+          "_build"+        ]+    }++-- | Bundle roots under a search root, sorted and normalised. The search root+-- itself is a candidate: pointing this at a bundle returns that bundle.+discoverBundleRoots :: DiscoveryOptions -> FilePath -> IO [FilePath]+discoverBundleRoots DiscoveryOptions {maxDepth, skipDirectories} searchRoot =+  List.sort <$> walk 0 searchRoot+  where+    walk depth directory = do+      qualifies <- directoryQualifiesAsBundleRoot directory+      if qualifies+        then pure [FilePath.normalise directory]+        else+          if depth >= maxDepth+            then pure []+            else do+              entries <- listDirectorySafe directory+              subdirectories <-+                filterM+                  (isSearchableDirectory skipDirectories)+                  [directory </> entry | entry <- List.sort entries, not (isHidden entry)]+              concat <$> traverse (walk (depth + 1)) subdirectories++    isHidden entry = case entry of+      ('.' : _) -> True+      _ -> False++-- | Does this directory look like the top of an OKF bundle?+directoryQualifiesAsBundleRoot :: FilePath -> IO Bool+directoryQualifiesAsBundleRoot directory = do+  entries <- listDirectorySafe directory+  if "index.md" `List.elem` entries+    then pure True+    else anyM isConceptDocument [directory </> entry | entry <- entries, isConceptCandidate entry]+  where+    isConceptCandidate entry =+      FilePath.takeExtension entry == ".md" && not (isReservedMarkdownFile entry)++-- | A file is a concept document when it parses and declares a non-empty @type@.+isConceptDocument :: FilePath -> IO Bool+isConceptDocument path = do+  loaded <- try @IOException (Text.IO.readFile path)+  pure $ case loaded of+    Left _ -> False+    Right content -> case parseDocument content of+      Left _ -> False+      Right OKFDocument {frontmatter} -> hasNonEmptyType frontmatter++hasNonEmptyType :: Frontmatter -> Bool+hasNonEmptyType frontmatter =+  case frontmatterLookup "type" frontmatter of+    Just (String value) -> not (Text.null (Text.strip value))+    _ -> False++-- | A directory we may descend into: a real directory, not a symbolic link+-- (which could form a cycle), and not on the skip list.+isSearchableDirectory :: [FilePath] -> FilePath -> IO Bool+isSearchableDirectory skipDirectories path+  | FilePath.takeFileName path `List.elem` skipDirectories = pure False+  | otherwise = do+      isDirectory <- orFalse (doesDirectoryExist path)+      isSymlink <- orFalse (pathIsSymbolicLink path)+      pure (isDirectory && not isSymlink)++listDirectorySafe :: FilePath -> IO [FilePath]+listDirectorySafe directory = do+  listed <- try @IOException (listDirectory directory)+  pure (either (const []) Prelude.id listed)++orFalse :: IO Bool -> IO Bool+orFalse action = either (const False) Prelude.id <$> try @IOException action++anyM :: (a -> IO Bool) -> [a] -> IO Bool+anyM _ [] = pure False+anyM predicate (x : xs) = do+  matched <- predicate x+  if matched then pure True else anyM predicate xs

@@ -17,6 +17,11 @@     loadProfileFile,      -- * Validation+    DocumentId (..),+    parseDocumentId,+    renderDocumentId,+    documentIdsInBundle,+    nextDocumentId,     ProfileViolation (..),     validateProfile, @@ -27,10 +32,13 @@  import CMarkGFM qualified import Control.Exception (SomeException, catch)+import Data.Char (isAsciiLower, isAsciiUpper) import Data.List qualified as List import Data.Text qualified as Text+import Data.Text.Read qualified as Text.Read import Dhall (FromDhall (..), auto, genericAutoWith) import Dhall qualified+import Numeric.Natural (Natural) import Okf.Bundle   ( Concept,     conceptDocument,@@ -49,6 +57,7 @@     okfVersion :: !Text,     frontmatter :: !FrontmatterRules,     allowUnknownTypes :: !Bool,+    idField :: !(Maybe Text),     types :: ![TypeRule]   }   deriving stock (Generic, Eq, Show)@@ -68,7 +77,8 @@     pathPattern :: !(Maybe Text),     resourceScheme :: !(Maybe Text),     requireSchemaSection :: !Bool,-    schemaColumns :: ![Text]+    schemaColumns :: ![Text],+    idPrefix :: !(Maybe Text)   }   deriving stock (Generic, Eq, Show) @@ -90,6 +100,94 @@   (Right <$> Dhall.inputFile auto path)     `catch` \(e :: SomeException) -> pure (Left (Text.pack (show e))) +-- | A parsed document handle: an ASCII-letter-led alphanumeric prefix and a+-- positive number, rendered as @PREFIX-N@.+data DocumentId = DocumentId+  { prefix :: !Text,+    number :: !Natural+  }+  deriving stock (Generic, Eq, Ord, Show)++-- | Parse a strict document handle. The prefix contains one or more ASCII+-- letters or digits and begins with a letter. It is followed by exactly one+-- hyphen and a positive decimal number with no leading zero. Thus @ADR-7@+-- parses, while @ADR-007@, @ADR-0@, @ADR-@, @-7@, @adr 7@, and+-- @ADR-7-extra@ do not.+parseDocumentId :: Text -> Maybe DocumentId+parseDocumentId raw =+  case Text.splitOn "-" raw of+    [prefixText, numberText]+      | validPrefix prefixText,+        validNumberText numberText ->+          case Text.Read.decimal numberText of+            Right (parsedNumber, remainder)+              | Text.null remainder,+                parsedNumber > 0 ->+                  Just (DocumentId prefixText parsedNumber)+            _ -> Nothing+    _ -> Nothing+  where+    validPrefix value =+      case Text.uncons value of+        Just (firstCharacter, rest) ->+          isAsciiLetter firstCharacter && Text.all isAsciiAlphaNumeric rest+        Nothing -> False+    validNumberText value =+      case Text.uncons value of+        Just (firstCharacter, rest) ->+          firstCharacter >= '1'+            && firstCharacter <= '9'+            && Text.all isAsciiDigit rest+        Nothing -> False+    isAsciiLetter character =+      isAsciiLower character || isAsciiUpper character+    isAsciiDigit character =+      character >= '0' && character <= '9'+    isAsciiAlphaNumeric character =+      isAsciiLetter character || isAsciiDigit character++-- | Render a document handle as @PREFIX-N@.+renderDocumentId :: DocumentId -> Text+renderDocumentId DocumentId {prefix, number} =+  prefix <> "-" <> Text.pack (show number)++-- | Every well-formed handle under the profile's ID field, paired with the+-- concept carrying it and sorted by prefix, number, then concept ID. Concepts+-- without a well-formed handle are omitted. A profile with no ID field yields+-- an empty list.+documentIdsInBundle :: ProfileSpec -> [Concept] -> [(DocumentId, ConceptId)]+documentIdsInBundle spec concepts =+  case spec ^. #idField of+    Nothing -> []+    Just fieldName ->+      List.sortOn+        (\(documentId, cid) -> (documentId, renderConceptId cid))+        [ (documentId, conceptIdOf concept)+        | concept <- concepts,+          Just (String rawDocumentId) <- [frontmatterLookup fieldName (conceptFrontmatter concept)],+          Just documentId <- [parseDocumentId rawDocumentId]+        ]++-- | Allocate one more than the highest document-ID number already used for the+-- given prefix, or number 1 when the prefix is unused. Gaps are deliberately+-- not filled: reusing a retired number could make an old reference silently+-- point at a different document.+nextDocumentId :: ProfileSpec -> [Concept] -> Text -> DocumentId+nextDocumentId spec concepts requestedPrefix =+  DocumentId+    { prefix = requestedPrefix,+      number = highestNumber + 1+    }+  where+    highestNumber =+      List.foldl'+        max+        0+        [ documentId ^. #number+        | (documentId, _) <- documentIdsInBundle spec concepts,+          documentId ^. #prefix == requestedPrefix+        ]+ -- | A single deviation from a profile. Advisory by default at the CLI layer. data ProfileViolation   = -- | concept's @type@ is not listed in the profile and unknown types are disallowed@@ -106,6 +204,12 @@     MissingSchemaSection ConceptId Text   | -- | @# Schema@ table columns do not match (concept, type, expected, actual)     SchemaColumnsMismatch ConceptId Text [Text] [Text]+  | -- | type rule declares an @idPrefix@ but the concept has no handle (concept, type, prefix)+    MissingDocumentId ConceptId Text Text+  | -- | handle present but malformed for the declared prefix (concept, prefix, actual value)+    MalformedDocumentId ConceptId Text Text+  | -- | the same handle appears on more than one concept (handle, concept, other concept)+    DuplicateDocumentId Text ConceptId ConceptId   deriving stock (Generic, Eq, Show)  -- | Check every concept against the profile, returning all deviations. Concepts@@ -113,7 +217,8 @@ -- is no rule to check against) and only produce a 'TypeNotInProfile' violation -- when @allowUnknownTypes@ is @False@. validateProfile :: ProfileSpec -> [Concept] -> [ProfileViolation]-validateProfile spec = concatMap checkConcept+validateProfile spec concepts =+  concatMap checkConcept concepts <> checkDuplicateDocumentIds spec concepts   where     rulesByType = [(rule ^. #type_, rule) | rule <- spec ^. #types] @@ -128,12 +233,55 @@                 <> checkPath cid ctype rule                 <> checkResource cid ctype rule concept                 <> checkSchema cid ctype rule concept+                <> checkDocumentId spec cid ctype rule concept      checkRequiredFields cid concept =       [ MissingProfileField cid key       | key <- spec ^. #frontmatter . #required,         not (hasNonEmptyField key (conceptFrontmatter concept))       ]++-- | Check a profile-declared document ID for one concept.+checkDocumentId :: ProfileSpec -> ConceptId -> Text -> TypeRule -> Concept -> [ProfileViolation]+checkDocumentId spec cid ctype rule concept =+  case (spec ^. #idField, rule ^. #idPrefix) of+    (Just fieldName, Just expectedPrefix) ->+      case frontmatterLookup fieldName (conceptFrontmatter concept) of+        Just (String value)+          | not (Text.null (Text.strip value)) ->+              case parseDocumentId value of+                Just documentId+                  | documentId ^. #prefix == expectedPrefix -> []+                _ -> [MalformedDocumentId cid expectedPrefix value]+        _ -> [MissingDocumentId cid ctype expectedPrefix]+    _ -> []++-- | Check every non-empty value under the profile's ID field for bundle-wide+-- uniqueness. Concept IDs are sorted before grouping so output is deterministic.+checkDuplicateDocumentIds :: ProfileSpec -> [Concept] -> [ProfileViolation]+checkDuplicateDocumentIds spec concepts =+  case spec ^. #idField of+    Nothing -> []+    Just fieldName ->+      concatMap duplicateViolations (groupedHandles fieldName)+  where+    groupedHandles fieldName =+      List.groupBy+        (\(leftHandle, _) (rightHandle, _) -> leftHandle == rightHandle)+        (handles fieldName)+    handles fieldName =+      List.sortOn+        (\(handle, cid) -> (handle, renderConceptId cid))+        [ (handle, conceptIdOf concept)+        | concept <- concepts,+          Just (String handle) <- [frontmatterLookup fieldName (conceptFrontmatter concept)],+          not (Text.null (Text.strip handle))+        ]+    duplicateViolations ((handle, firstConcept) : duplicates) =+      [ DuplicateDocumentId handle firstConcept duplicateConcept+      | (_, duplicateConcept) <- duplicates+      ]+    duplicateViolations [] = []  -- | Project a concept's frontmatter (the document's @frontmatter@ field). conceptFrontmatter :: Concept -> Frontmatter

@@ -3,11 +3,13 @@ module Main (main) where  import Data.Aeson (object, toJSON, (.=))+import Data.Foldable (for_) import Data.List qualified as List import Data.Text qualified as Text import Data.Text.IO qualified as Text.IO import Okf.Bundle import Okf.ConceptId+import Okf.Discovery import Okf.Document import Okf.Graph import Okf.Index@@ -23,7 +25,7 @@     removeDirectoryRecursive,   ) import System.Exit (exitFailure)-import System.FilePath ((</>))+import System.FilePath (normalise, takeDirectory, (</>)) import System.IO.Temp (createTempDirectory) import "generic-lens" Data.Generics.Labels () @@ -44,6 +46,13 @@         testIO "walkBundle reports a structured IO error for a missing root" testWalkBundleMissingRoot,         testIO "walkBundle skips index.md and log.md" testWalkBundleSkipsReserved,         testIO "walkBundle discovers nested concept IDs" testWalkBundleDiscoversNestedConceptIds,+        testIO "discoverBundleRoots finds a directory holding index.md" testDiscoverIndexMd,+        testIO "discoverBundleRoots finds a directory holding a typed concept" testDiscoverTypedConcept,+        testIO "discoverBundleRoots ignores markdown without a type field" testDiscoverIgnoresPlainMarkdown,+        testIO "discoverBundleRoots does not descend into a bundle it found" testDiscoverPrunesNestedBundles,+        testIO "discoverBundleRoots skips hidden and build directories" testDiscoverSkipsNoise,+        testIO "discoverBundleRoots honours maxDepth" testDiscoverHonoursMaxDepth,+        testIO "discoverBundleRoots reports a fixture bundle as its own root" testDiscoverFixtureBundle,         test "parseLog/serializeLog round-trips a canonical log" testLogRoundTrip,         test "validateLog flags a non-ISO date heading" testValidateLogNonIsoDate,         test "validateLog flags an empty date group" testValidateLogEmptyDay,@@ -72,6 +81,11 @@         testIO "writeBundle then walkBundle round-trips" testWriteBundleRoundTrip,         testIO "fixture dangling link reports a bundle validation error" testFixtureDanglingLink,         testIO "loadProfileFile decodes the postgresql fixture" testLoadProfileFixture,+        testIO "loadProfileFile decodes record-completed document ID rules" testLoadDocumentIdProfileFixture,+        test "parseDocumentId accepts only canonical handles" testParseDocumentId,+        testIO "documentIdsInBundle sorts handles by prefix and number" testDocumentIdsInBundle,+        test "nextDocumentId skips gaps and starts unused prefixes at one" testNextDocumentId,+        testIO "findConceptsByDocumentId resolves and reports duplicate handles" testFindConceptsByDocumentId,         test "validateProfile accepts a conforming table concept" testProfileConformingTable,         test "validateProfile flags a type not in the vocabulary" testProfileUnknownType,         test "validateProfile flags a missing required field" testProfileMissingField,@@ -79,8 +93,14 @@         test "validateProfile flags a path pattern mismatch" testProfilePathMismatch,         test "validateProfile flags a missing # Schema section" testProfileMissingSchema,         test "validateProfile flags mismatched # Schema columns" testProfileSchemaColumnsMismatch,+        test "validateProfile accepts a conforming document ID" testProfileConformingDocumentId,+        test "validateProfile flags a missing document ID" testProfileMissingDocumentId,+        test "validateProfile flags malformed document IDs" testProfileMalformedDocumentIds,+        test "validateProfile flags duplicate document IDs" testProfileDuplicateDocumentIds,+        test "validateProfile document ID checks are off by default" testProfileDocumentIdsOffByDefault,         test "schemaSectionColumns reads the header row of the Schema table" testSchemaSectionColumns,-        testIO "validateProfile reports the expected deviations for the fixture bundle" testProfileDeviationsFixture+        testIO "validateProfile reports the expected deviations for the fixture bundle" testProfileDeviationsFixture,+        testIO "validateProfile reports document ID fixture deviations" testDocumentIdDeviationsFixture       ]   unless (and results) exitFailure @@ -189,6 +209,83 @@           )     ) +-- | Build a throwaway directory tree, run an action on it, and clean up.+withDiscoveryTree :: String -> [(FilePath, Text)] -> (FilePath -> IO a) -> IO a+withDiscoveryTree label files action = do+  temporaryDirectory <- getTemporaryDirectory+  root <- createTempDirectory temporaryDirectory label+  for_ files $ \(relativePath, content) -> do+    createDirectoryIfMissing True (root </> takeDirectory relativePath)+    Text.IO.writeFile (root </> relativePath) content+  result <- action root+  removeDirectoryRecursive root+  pure result++typedConcept :: Text -> Text+typedConcept titleText =+  Text.unlines ["---", "type: Table", "title: " <> titleText, "---", "", "# " <> titleText]++plainMarkdown :: Text+plainMarkdown = "# Just prose\n\nNo frontmatter here.\n"++testDiscoverIndexMd :: IO (Either Text ())+testDiscoverIndexMd =+  withDiscoveryTree "okf-discovery-index" [("kb/index.md", "# Index\n")] $ \root -> do+    found <- discoverBundleRoots defaultDiscoveryOptions root+    pure (assertEqual [normalise (root </> "kb")] found)++testDiscoverTypedConcept :: IO (Either Text ())+testDiscoverTypedConcept =+  withDiscoveryTree "okf-discovery-typed" [("kb/tables/orders.md", typedConcept "Orders")] $ \root -> do+    found <- discoverBundleRoots defaultDiscoveryOptions root+    pure (assertEqual [normalise (root </> "kb" </> "tables")] found)++testDiscoverIgnoresPlainMarkdown :: IO (Either Text ())+testDiscoverIgnoresPlainMarkdown =+  withDiscoveryTree+    "okf-discovery-plain"+    [("notes/README.md", plainMarkdown), ("notes/CHANGELOG.md", plainMarkdown)]+    $ \root -> do+      found <- discoverBundleRoots defaultDiscoveryOptions root+      pure (assertEqual [] found)++testDiscoverPrunesNestedBundles :: IO (Either Text ())+testDiscoverPrunesNestedBundles =+  withDiscoveryTree+    "okf-discovery-prune"+    [ ("kb/index.md", "# Index\n"),+      ("kb/tables/index.md", "# Tables\n"),+      ("kb/tables/orders.md", typedConcept "Orders")+    ]+    $ \root -> do+      found <- discoverBundleRoots defaultDiscoveryOptions root+      pure (assertEqual [normalise (root </> "kb")] found)++testDiscoverSkipsNoise :: IO (Either Text ())+testDiscoverSkipsNoise =+  withDiscoveryTree+    "okf-discovery-noise"+    [ (".hidden/index.md", "# Hidden\n"),+      ("dist-newstyle/index.md", "# Build output\n"),+      ("kb/index.md", "# Index\n")+    ]+    $ \root -> do+      found <- discoverBundleRoots defaultDiscoveryOptions root+      pure (assertEqual [normalise (root </> "kb")] found)++testDiscoverHonoursMaxDepth :: IO (Either Text ())+testDiscoverHonoursMaxDepth =+  withDiscoveryTree "okf-discovery-depth" [("a/b/c/index.md", "# Deep\n")] $ \root -> do+    shallow <- discoverBundleRoots defaultDiscoveryOptions {maxDepth = 2} root+    deep <- discoverBundleRoots defaultDiscoveryOptions {maxDepth = 3} root+    pure (assertEqual [] shallow >> assertEqual [normalise (root </> "a" </> "b" </> "c")] deep)++testDiscoverFixtureBundle :: IO (Either Text ())+testDiscoverFixtureBundle = do+  bundle <- fixturePath "valid-bundle"+  found <- discoverBundleRoots defaultDiscoveryOptions bundle+  pure (assertEqual [normalise bundle] found)+ testLogRoundTrip :: Either Text () testLogRoundTrip = do   let canonicalLog =@@ -662,6 +759,78 @@         ["PostgreSQL Schema", "PostgreSQL Table", "PostgreSQL View"]         (map (^. #type_) (spec ^. #types)) +testLoadDocumentIdProfileFixture :: IO (Either Text ())+testLoadDocumentIdProfileFixture = do+  path <- fixtureFilePath "profiles/decisions.dhall"+  result <- loadProfileFile path+  pure $ case result of+    Left err -> Left ("failed to load document ID profile: " <> err)+    Right spec -> do+      assertEqual (Just "docId") (spec ^. #idField)+      assertEqual [Just "ADR"] (map (^. #idPrefix) (spec ^. #types))++testParseDocumentId :: Either Text ()+testParseDocumentId = do+  assertEqual+    (Just (DocumentId {prefix = "ADR", number = 7}))+    (parseDocumentId "ADR-7")+  mapM_+    (\invalid -> assertEqual Nothing (parseDocumentId invalid))+    ["ADR-007", "ADR-0", "ADR-", "-7", "ADR 7", "ADR-7-extra"]+  assertEqual (Just "ADR-7") (renderDocumentId <$> parseDocumentId "ADR-7")++testDocumentIdsInBundle :: IO (Either Text ())+testDocumentIdsInBundle = do+  descriptorPath <- fixtureFilePath "profiles/decisions.dhall"+  loaded <- loadProfileFile descriptorPath+  root <- fixturePath "doc-ids"+  concepts <- readBundle root+  pure $ case loaded of+    Left err -> Left ("failed to load document ID profile: " <> err)+    Right spec -> do+      useMarkdown <- parseTestConceptId "decisions/use-markdown"+      usePostgres <- parseTestConceptId "decisions/use-postgres"+      adoptOkf <- parseTestConceptId "decisions/adopt-okf"+      assertEqual+        [ (DocumentId "ADR" 1, useMarkdown),+          (DocumentId "ADR" 2, usePostgres),+          (DocumentId "ADR" 3, adoptOkf)+        ]+        (documentIdsInBundle spec concepts)++testNextDocumentId :: Either Text ()+testNextDocumentId = do+  firstConcept <-+    profileConcept+      "decisions/first"+      [("type", String "Decision Record"), ("title", String "First"), ("docId", String "ADR-1")]+      "# First\n"+  thirdConcept <-+    profileConcept+      "decisions/third"+      [("type", String "Decision Record"), ("title", String "Third"), ("docId", String "ADR-3")]+      "# Third\n"+  let concepts = [firstConcept, thirdConcept]+  assertEqual (DocumentId "ADR" 4) (nextDocumentId testDocumentIdProfileSpec concepts "ADR")+  assertEqual (DocumentId "RFC" 1) (nextDocumentId testDocumentIdProfileSpec concepts "RFC")++testFindConceptsByDocumentId :: IO (Either Text ())+testFindConceptsByDocumentId = do+  validRoot <- fixturePath "doc-ids"+  validConcepts <- readBundle validRoot+  deviationRoot <- fixturePath "doc-id-deviations"+  deviationConcepts <- readBundle deviationRoot+  pure $ do+    usePostgres <- parseTestConceptId "decisions/use-postgres"+    firstId <- parseTestConceptId "decisions/first"+    secondId <- parseTestConceptId "decisions/second"+    assertEqual+      [usePostgres]+      (conceptIdOf <$> findConceptsByDocumentId Nothing "ADR-2" validConcepts)+    assertEqual+      [firstId, secondId]+      (conceptIdOf <$> findConceptsByDocumentId (Just "docId") "ADR-1" deviationConcepts)+ -- | A standalone profile literal so the validation tests do not depend on the -- Dhall fixture. One rule: PostgreSQL Table, fully constrained. testProfileSpec :: ProfileSpec@@ -671,17 +840,39 @@       okfVersion = "0.1",       frontmatter = FrontmatterRules {required = ["type", "title"], recommended = []},       allowUnknownTypes = False,+      idField = Nothing,       types =         [ TypeRule             { type_ = "PostgreSQL Table",               pathPattern = Just "schemas/*/tables/*",               resourceScheme = Just "postgresql",               requireSchemaSection = True,-              schemaColumns = ["Column", "Type", "Nullable", "Description"]+              schemaColumns = ["Column", "Type", "Nullable", "Description"],+              idPrefix = Nothing             }         ]     } +testDocumentIdProfileSpec :: ProfileSpec+testDocumentIdProfileSpec =+  ProfileSpec+    { name = "test-decisions",+      okfVersion = "0.1",+      frontmatter = FrontmatterRules {required = ["type", "title"], recommended = []},+      allowUnknownTypes = False,+      idField = Just "docId",+      types =+        [ TypeRule+            { type_ = "Decision Record",+              pathPattern = Just "decisions/*",+              resourceScheme = Nothing,+              requireSchemaSection = False,+              schemaColumns = [],+              idPrefix = Just "ADR"+            }+        ]+    }+ -- | Build an in-memory concept from a raw ID, frontmatter pairs, and a body. profileConcept :: Text -> [(Text, Value)] -> Text -> Either Text Concept profileConcept rawId fieldPairs bodyText = do@@ -782,6 +973,78 @@     [SchemaColumnsMismatch cid "PostgreSQL Table" ["Column", "Type", "Nullable", "Description"] ["Col", "Type"]]     (validateProfile testProfileSpec [concept]) +testProfileConformingDocumentId :: Either Text ()+testProfileConformingDocumentId = do+  concept <-+    profileConcept+      "decisions/one"+      [("type", String "Decision Record"), ("title", String "One"), ("docId", String "ADR-1")]+      "# One\n"+  assertEqual [] (validateProfile testDocumentIdProfileSpec [concept])++testProfileMissingDocumentId :: Either Text ()+testProfileMissingDocumentId = do+  concept <-+    profileConcept+      "decisions/one"+      [("type", String "Decision Record"), ("title", String "One")]+      "# One\n"+  cid <- parseTestConceptId "decisions/one"+  assertEqual+    [MissingDocumentId cid "Decision Record" "ADR"]+    (validateProfile testDocumentIdProfileSpec [concept])++testProfileMalformedDocumentIds :: Either Text ()+testProfileMalformedDocumentIds = do+  leadingZero <-+    profileConcept+      "decisions/leading-zero"+      [("type", String "Decision Record"), ("title", String "Leading zero"), ("docId", String "ADR-007")]+      "# Leading zero\n"+  wrongPrefix <-+    profileConcept+      "decisions/wrong-prefix"+      [("type", String "Decision Record"), ("title", String "Wrong prefix"), ("docId", String "RFC-1")]+      "# Wrong prefix\n"+  leadingZeroId <- parseTestConceptId "decisions/leading-zero"+  wrongPrefixId <- parseTestConceptId "decisions/wrong-prefix"+  assertEqual+    [ MalformedDocumentId leadingZeroId "ADR" "ADR-007",+      MalformedDocumentId wrongPrefixId "ADR" "RFC-1"+    ]+    (validateProfile testDocumentIdProfileSpec [leadingZero, wrongPrefix])++testProfileDuplicateDocumentIds :: Either Text ()+testProfileDuplicateDocumentIds = do+  second <-+    profileConcept+      "decisions/second"+      [("type", String "Decision Record"), ("title", String "Second"), ("docId", String "ADR-1")]+      "# Second\n"+  firstConcept <-+    profileConcept+      "decisions/first"+      [("type", String "Decision Record"), ("title", String "First"), ("docId", String "ADR-1")]+      "# First\n"+  firstId <- parseTestConceptId "decisions/first"+  secondId <- parseTestConceptId "decisions/second"+  assertEqual+    [DuplicateDocumentId "ADR-1" firstId secondId]+    (validateProfile testDocumentIdProfileSpec [second, firstConcept])++testProfileDocumentIdsOffByDefault :: Either Text ()+testProfileDocumentIdsOffByDefault = do+  concept <-+    profileConcept+      "schemas/sales/tables/orders"+      [ ("type", String "PostgreSQL Table"),+        ("title", String "Orders"),+        ("resource", String "postgresql://warehouse/sales/orders"),+        ("docId", String "not-a-handle")+      ]+      schemaSectionBody+  assertEqual [] (validateProfile testProfileSpec [concept])+ testSchemaSectionColumns :: Either Text () testSchemaSectionColumns =   assertEqual@@ -803,6 +1066,26 @@       ordersId <- parseTestConceptId "schemas/sales/tables/orders"       assertEqual         [TypeNotInProfile badId "pg table", MissingProfileField ordersId "title"]+        (validateProfile spec concepts)++testDocumentIdDeviationsFixture :: IO (Either Text ())+testDocumentIdDeviationsFixture = do+  descriptorPath <- fixtureFilePath "profiles/decisions.dhall"+  loaded <- loadProfileFile descriptorPath+  root <- fixturePath "doc-id-deviations"+  concepts <- readBundle root+  pure $ case loaded of+    Left err -> Left ("failed to load document ID profile: " <> err)+    Right spec -> do+      firstId <- parseTestConceptId "decisions/first"+      secondId <- parseTestConceptId "decisions/second"+      thirdId <- parseTestConceptId "decisions/third"+      fourthId <- parseTestConceptId "decisions/fourth"+      assertEqual+        [ MissingDocumentId fourthId "Decision Record" "ADR",+          MalformedDocumentId thirdId "ADR" "ADR-007",+          DuplicateDocumentId "ADR-1" firstId secondId+        ]         (validateProfile spec concepts)  substringIndex :: Text -> Text -> Maybe Int

@@ -0,0 +1,7 @@+---+type: Decision Record+title: First decision+docId: ADR-1+---++# First decision

@@ -0,0 +1,6 @@+---+type: Decision Record+title: Fourth decision+---++# Fourth decision

@@ -0,0 +1,7 @@+---+type: Decision Record+title: Second decision+docId: ADR-1+---++# Second decision

@@ -0,0 +1,7 @@+---+type: Decision Record+title: Third decision+docId: ADR-007+---++# Third decision

@@ -0,0 +1,9 @@+---+type: Decision Record+title: Adopt Open Knowledge Format+docId: ADR-3+---++# Adopt Open Knowledge Format++Use OKF bundles for durable project knowledge.

@@ -0,0 +1,9 @@+---+type: Decision Record+title: Use Markdown for knowledge records+docId: ADR-1+---++# Use Markdown for knowledge records++Store knowledge records as Markdown documents.

@@ -0,0 +1,9 @@+---+type: Decision Record+title: Use PostgreSQL for the warehouse+docId: ADR-2+---++# Use PostgreSQL for the warehouse++Use PostgreSQL as the warehouse database.

@@ -0,0 +1,10 @@+---+type: BigQuery Table+title: Orders+description: Order fact table.+timestamp: 2026-06-16T00:00:00Z+---++# Orders++Orders join to [Customers](/customers.md).

@@ -0,0 +1,9 @@+---+title: Missing Type+description: This document has frontmatter but no required type.+timestamp: 2026-06-16T00:00:00Z+---++# Missing Type++Permissive validation should reject this as an OKF concept document.

@@ -0,0 +1,6 @@+---+type: BigQuery Table+title: Broken+description: This document never closes frontmatter.++# Broken

@@ -0,0 +1,11 @@+---+type: pg table+title: Bad+resource: postgresql://warehouse/sales/public/bad+---++# Schema++| Column   | Type   | Nullable | Description |+|----------|--------|----------|-------------|+| `bad_id` | bigint | no       | Primary key.|

@@ -0,0 +1,11 @@+---+type: PostgreSQL Table+title: Customers+resource: postgresql://warehouse/sales/public/customers+---++# Schema++| Column        | Type   | Nullable | Description  |+|---------------|--------|----------|--------------|+| `customer_id` | bigint | no       | Primary key. |

@@ -0,0 +1,10 @@+---+type: PostgreSQL Table+resource: postgresql://warehouse/sales/public/orders+---++# Schema++| Column     | Type   | Nullable | Description |+|------------|--------|----------|-------------|+| `order_id` | bigint | no       | Primary key.|

@@ -0,0 +1,21 @@+-- The annotation and TypeRule record completion jointly guard the canonical+-- schema, its defaults, and the Haskell decoder against drift.+let Profile = ../../../dhall/Profile.dhall++let TypeRule = ../../../dhall/defaults/TypeRule.dhall++in    { name = "decisions"+      , okfVersion = "0.1"+      , frontmatter =+        { required = [ "type", "title" ], recommended = [] : List Text }+      , allowUnknownTypes = False+      , idField = Some "docId"+      , types =+        [ TypeRule::{+          , type = "Decision Record"+          , pathPattern = Some "decisions/*"+          , idPrefix = Some "ADR"+          }+        ]+      }+    : Profile

@@ -0,0 +1,38 @@+-- The `: Profile` annotation here is load-bearing: it ties this fixture to the+-- canonical schema, so the `testLoadProfileFixture` round-trip in test/Main.hs+-- fails if okf's published Dhall schema and the Haskell decoder ever drift apart.+let Profile = ../../../dhall/Profile.dhall++in    { name = "shinzui-postgresql"+      , okfVersion = "0.1"+      , frontmatter =+        { required = [ "type", "title" ]+        , recommended = [ "description", "timestamp", "resource" ]+        }+      , allowUnknownTypes = False+      , idField = None Text+      , types =+        [ { type = "PostgreSQL Schema"+          , pathPattern = Some "schemas/*"+          , resourceScheme = Some "postgresql"+          , requireSchemaSection = False+          , schemaColumns = [] : List Text+          , idPrefix = None Text+          }+        , { type = "PostgreSQL Table"+          , pathPattern = Some "schemas/*/tables/*"+          , resourceScheme = Some "postgresql"+          , requireSchemaSection = True+          , schemaColumns = [ "Column", "Type", "Nullable", "Description" ]+          , idPrefix = None Text+          }+        , { type = "PostgreSQL View"+          , pathPattern = Some "schemas/*/views/*"+          , resourceScheme = Some "postgresql"+          , requireSchemaSection = True+          , schemaColumns = [ "Column", "Type", "Description" ]+          , idPrefix = None Text+          }+        ]+      }+    : Profile

@@ -0,0 +1,3 @@+# Dataset++- [Sales Dataset](sales.md) - Daily sales export used by warehouse tables.

@@ -0,0 +1,11 @@+---+type: Dataset+title: Sales Dataset+description: Daily sales export used by warehouse tables.+timestamp: 2026-06-16T00:00:00Z+tags: [sales, source]+---++# Sales Dataset++Source data for the warehouse order tables.

@@ -0,0 +1,5 @@+# OKF fixture bundle++- [datasets/](datasets/index.md)+- [references/](references/index.md)+- [tables/](tables/index.md)

@@ -0,0 +1,4 @@+# Bundle Update Log++## 2026-06-16+* **Update**: Refreshed the valid bundle fixture.

@@ -0,0 +1,3 @@+# Reference++- [Source System](source-system.md) - External system reference.

@@ -0,0 +1,10 @@+---+type: Reference+title: Source System+description: External system reference.+timestamp: 2026-06-16T00:00:00Z+---++# Source System++Reference notes for upstream sales data.

@@ -0,0 +1,11 @@+---+type: BigQuery Table+title: Customers+description: Customer dimension table.+timestamp: 2026-06-16T00:00:00Z+tags: [customers]+---++# Customers++Customer records used for order attribution.

@@ -0,0 +1,4 @@+# BigQuery Table++- [Customers](customers.md) - Customer dimension table.+- [Orders](orders.md) - Order fact table.

@@ -0,0 +1,17 @@+---+type: BigQuery Table+title: Orders+description: Order fact table.+timestamp: 2026-06-16T00:00:00Z+resource: bigquery://analytics.tables.orders+tags: [orders, sales]+---++# Orders++Orders join to [Customers](/tables/customers.md), load from the+[Sales Dataset](../datasets/sales.md), and are documented in the+[source reference](../references/source-system.md).++External citations such as [vendor docs](https://example.com/vendor/orders.md)+are useful prose but should not become OKF graph edges.