cabal-buck2 0.4.0.0 → 0.5.0.0

raw patch · 16 files changed

+158/−74 lines, 16 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• README.md +13/−32
• cabal-buck2.cabal +1/−1
• changelog.md +27/−0
• src/Distribution/Client/Buck2/Generate.hs +13/−6
• src/Distribution/Client/Buck2/LocalPackages.hs +5/−3
• src/Distribution/Client/Buck2/Prebuilt.hs +45/−31
• src/Distribution/Client/Buck2/Spec.hs +5/−0
• src/Distribution/Client/Buck2/Write.hs +2/−0
• test/Main.hs +19/−1
• test/fixtures/normalise-paths/.buckconfig +4/−0
• test/fixtures/normalise-paths/PACKAGE +1/−0
• test/fixtures/normalise-paths/buck2/.gitkeep +0/−0
• test/fixtures/normalise-paths/cabal.project +5/−0
• test/fixtures/normalise-paths/pkg/data/x.txt +1/−0
• test/fixtures/normalise-paths/pkg/pkg.cabal +13/−0
• test/fixtures/normalise-paths/pkg/src/Pkg.hs +4/−0

@@ -229,19 +229,26 @@  # Build modes -The Buck2 build system has two build modes:+The Buck2 build system has the following build mode options, which can+be selected using `-m` on the Buck2 command line (e.g. `-m opt`) -  * `dev`: the default, builds everything with `-O0` and dynamic linking. This is intended to give you the quickest edit-compile-test turnaround.-  * `opt`: enable `-O` and link statically. This takes longer but the code runs faster.+  * Optimisation:+    * `dev`: (default) no optimisation, `-g` for C/C++.+    * `opt`: adds `-O` for Haskell, `-O3 -DNDEBUG` for C/C++, and also enables+      `static` unless overridden by `shared`.+  * Linking:+    * `shared`: (default) shared libraries, dynamic linking+    * `static`: static linking, but also builds shared libraries using+    `-dynamic-too` because those are required for Template Haskell.+  * Profiling:+    * `prof`: enables Haskell profiling; also forces `static`. -To build with `opt`, use `-m opt`, e.g.+To use any of these modes, add them using `-m` on the command line:  ``` buck2 build my-package:my-program -m opt ``` -There are other build options that can be selected in a similar way, such as `-m prof` to enable profiling. See `constraints/BUCK` for details.- # Sharing build results (a build cache)  By default buck2 keeps nothing between runs of its daemon, nor between@@ -428,32 +435,6 @@ including the package's `setup-depends`. It has to be built with the same Cabal version as `cabal-buck2`, so `cabal-buck2` injects a solver constraint to ensure that.--## **Template Haskell and `prof`**--A module that defines a splice must live in a *different*-`haskell_library()` from any module that uses it, when profiling (`-m-prof`). If not, the build will likely complain about a link error or a-missing object file at compile-time.--The situation with Template Haskell and profiling is complex, as is-the reason for this limitation.--* Without `-fexternal-interpreter`: GHC loads object code at-  compile-time into its own process. Since GHC is itself a-  dynamically-linked non-profiled executable, the objects it loads-  must be shared, non-profiled, objects. So we have to build all the-  dependencies of the current packages as shared libraries. This is-  fine, except for the current package: GHC expects to find the-  `.dyn_o` objects for the current package in the current `-odir`. But-  Buck2 doesn't work this way: it builds the two instances of the-  package separately. It's not clear if this is easily fixable.--* With `-fexternal-interpreter`, we could load the profiled non-shared-  objects. However, this method uses the RTS runtime linker, which is-  known to have some limitations and can't load some objects,-  particularly on certain architectures. This is the main reason that-  GHC switched to dynamic linking. So we don't go this route.  ## No support for Cabal's `foreign-library` 

@@ -1,6 +1,6 @@ cabal-version:      3.8 name:               cabal-buck2-version:            0.4.0.0+version:            0.5.0.0  -- To ship a new version: --   - bump the version above

@@ -1,3 +1,30 @@+# 0.5.0.0++  * For `cabal-install` version 3.18+  * Now works on MacOS X+  * A component's C, C++, assembler and Cmm sources are part of its+    library, as in Cabal: their objects are in the library's archive+    and shared library, rather than in libraries of their own linked+    alongside it. A library with no Haskell modules (only C sources,+    say) no longer gets a placeholder module.+  * A library that isn't exposed (`exposed: False`) is registered as+    such, and one whose `ghc-options` give it a unit id+    (`-this-unit-id`) has that unit id.+  * `-m static` and `-m shared` say how executables are linked,+    independently of the build mode (whose default is shared for dev,+    static for opt and prof).+  * `haskell_library` has a `soname` attribute, for a shared library+    whose soname isn't its file's name.+  * Configurations can be named by a project's own constraints too:+    `named_constraint_settings` in the `[cfg]` section of its+    `.buckconfig`.+  * Fix for using a combination of a Cabal `main-is` declaration+    together with `ghc-options: -main-is ...`.+  * Modules in `other-modules` are now correctly hidden outside the+    component.+  * A Template Haskell splice can use a module of the same library+    when profiling (`-m prof`).+ # 0.4.0.0    * For `cabal-install` version 3.18

@@ -71,6 +71,7 @@   , hcOptions   , hsSourceDirs   , includeDirs+  , libExposed   , otherModules   , buildType   , lookupComponent@@ -343,16 +344,18 @@         Left problems -> skipBecause problems ("library " ++ targetName ++ " (couldn't resolve all its modules)")         Right (srcs, srcAutogen) ->           let reexports = [SpecReexport (prettyShow (moduleReexportName r)) (prettyShow original) (specDep <$> from) | (r, Just (from, original)) <- reexportOrigins localIndex pkgDesc lbi clbi lib]-              -- A library with no modules (a compatibility package, say) has-              -- nothing to build, which a rule can't have.+              -- A library with no code at all (a compatibility package, say)+              -- has nothing to build, which a rule can't have.               extra :: [(ModuleName.ModuleName, AutogenFile)]               extra-                | null srcs = [(ModuleName.fromString "CabalBuck2Empty", emptyModule targetName)]+                | null srcs && null (cSources bi) && null (cxxSources bi) && null (asmSources bi) && null (cmmSources bi) = [(ModuleName.fromString "CabalBuck2Empty", emptyModule targetName)]                 | otherwise = []            in componentTargets                 (specComponent localIndex externalBuildTools Library targetName bi)                   { scSrcs = srcs ++ [(prettyShow m, SrcAutogen (autogenName f)) | (m, f) <- extra]                   , scReexports = reexports+                  , scHiddenModules = map prettyShow (otherModules bi)+                  , scExposed = libExposed lib                   , scGeneratedIncludeDirs = generatedIncludeDirs projectRoot pkgDesc lbi bi                   , scHscOptions = platformDefines lbi                   }@@ -491,9 +494,11 @@     { specPackageName = unPackageName (packageName pkgDesc)     , specPackageVersion = prettyShow (packageVersion pkgDesc)     , specGhcOptions = projectGhcOptions-    , specDataDir = getSymbolicPath (dataDir pkgDesc)-    , specDataFiles = map getSymbolicPath (dataFiles pkgDesc)-    , specExtraSourceFiles = map getSymbolicPath (extraSrcFiles pkgDesc)+    , -- These are patterns buck2 globs for, as the .cabal file wrote them:+      -- `./foo.jpg` is a fine Cabal pattern but not a valid buck2 path.+      specDataDir = normalise (getSymbolicPath (dataDir pkgDesc))+    , specDataFiles = map (normalise . getSymbolicPath) (dataFiles pkgDesc)+    , specExtraSourceFiles = map (normalise . getSymbolicPath) (extraSrcFiles pkgDesc)     , specComponents = components     } @@ -519,6 +524,7 @@     , scExtraLibDirs = map getSymbolicPath (extraLibDirs bi)     , scDeps = map specDep (libraryDeps bi)     , scReexports = []+    , scHiddenModules = []     , scBuildTools = mapMaybe buildToolSpec (ordNub [(pn, exe) | ExeDependency pn exe _ <- buildToolDepends bi])     , scCSources = map getSymbolicPath (cSources bi)     , scCxxSources = map getSymbolicPath (cxxSources bi)@@ -528,6 +534,7 @@     , scCcOptions = ccOptions bi     , scCxxOptions = cxxOptions bi     , scIncludeDirs = map getSymbolicPath (includeDirs bi)+    , scExposed = True     , scHscOptions = []     , scGeneratedIncludeDirs = []     , scPkgconfig = ordNub [unPkgconfigName n | PkgconfigDependency n _ <- pkgconfigDepends bi]

@@ -28,7 +28,7 @@ import qualified Data.Map as Map import qualified Data.Set as Set import System.Directory (doesDirectoryExist, doesFileExist, listDirectory, removeDirectoryRecursive)-import System.FilePath (makeRelative, (</>))+import System.FilePath (makeRelative, normalise, (</>))  import Distribution.Client.DistDirLayout (DistDirLayout (distUnpackedSrcDirectory, distUnpackedSrcRootDirectory)) import qualified Distribution.Client.InstallPlan as InstallPlan@@ -300,9 +300,11 @@   where     name = unUnqualComponentName component --- | A directory relative to the project root, @.@ for the root itself.+-- | A directory relative to the project root, @.@ for the root itself. The+-- directory is as @cabal.project@ names it, so it may be relative already,+-- with a @.\/@ in front (@packages: .\/foo@) that a buck2 label can't have. rootRelativeDir :: FilePath -> FilePath -> FilePath-rootRelativeDir projectRoot dir = case makeRelative projectRoot dir of+rootRelativeDir projectRoot dir = case makeRelative projectRoot (normalise dir) of   "" -> "."   rel -> rel 

@@ -32,7 +32,6 @@  import qualified Data.ByteString as BS import qualified Data.ByteString.Char8 as BS8-import Data.Char (isHexDigit) import Data.List (stripPrefix) import qualified Data.Map as Map import qualified Data.Set as Set@@ -76,7 +75,7 @@  import Distribution.InstalledPackageInfo (parseInstalledPackageInfo) import Distribution.Package (HasUnitId (installedUnitId), packageName, packageVersion)-import Distribution.Simple.BuildPaths (exeExtension)+import Distribution.Simple.BuildPaths (dllExtension, exeExtension) import Distribution.Simple.Compiler   ( Compiler (compilerProperties)   , compilerVersion@@ -87,6 +86,7 @@ import Distribution.Simple.Program.Db (lookupProgram) import Distribution.Simple.Program.Types (programPath) import Distribution.Simple.Utils (die', notice, ordNub, warn)+import Distribution.System (OS (..), Platform (..)) import Distribution.Utils.MD5 (md5, showMD5) import Distribution.Types.InstalledPackageInfo   ( InstalledPackageInfo@@ -181,6 +181,10 @@           , rpGlobalRootAbs = globalRootAbs           , rpStoreRootAbs = storeRootAbs           , rpStoreDbRel = makeRelative storeRootAbs storeDB+          , rpDllExtension = dllExtension (pkgConfigPlatform shared)+          , rpPicByDefault = case pkgConfigPlatform shared of+              Platform _ OSX -> True+              _ -> False           , rpExternal = []           } @@ -261,32 +265,25 @@   , rpStoreRootAbs :: FilePath   , rpStoreDbRel :: FilePath   -- ^ The store's package db, relative to 'rpStoreRootAbs'.+  , rpDllExtension :: String+  -- ^ The extension of a shared library, without the dot: @so@, or @dylib@ on macos.+  , rpPicByDefault :: Bool+  -- ^ Whether GHC's code is always position-independent (as on macOS).   , rpExternal :: [(FilePath, FilePath)]   -- ^ Directories that are neither under GHC's libdir nor the cabal store,   -- and the symlinks (relative to @third-party\/haskell@) to them, which   -- they are reached through. See 'externalDirs'.   } --- | Which of the two package dbs a unit id's @.conf@ lives in - decided--- entirely by the unit id's own shape, no searching required (unlike the--- Python predecessor, which had to glob for the store root, not knowing--- its ABI-tag suffix in advance): GHC's global\/boot packages have a plain--- @name-version@ id; a package installed to the store gets a--- hash-suffixed one. (An @-inplace@ id - a package buck2 builds itself ---- never gets here: see 'generatePrebuilt's plan parameter.)+-- | Which of the two package dbs a unit id's @.conf@ lives in - found+-- by looking in the store's db first, then GHC's global one (see+-- 'readPackage'). The unit id's shape can't be relied on to tell them+-- apart: on macOS and Windows cabal shortens store unit ids (e.g.+-- LstLk-4.7.8.4-0e779db1). (An @-inplace@ id - a package buck2 builds+-- itself - never gets here: see 'generatePrebuilt's plan parameter.) data PkgDbKind = GlobalDb | StoreDb   deriving (Eq) -classifyUnitId :: UnitId -> PkgDbKind-classifyUnitId uid-  | hasStoreHashSuffix s = StoreDb-  | otherwise = GlobalDb-  where-    s = unUnitId uid-    hasStoreHashSuffix rs = case break (== '-') (reverse rs) of-      (revSuffix, '-' : _) -> length revSuffix >= 20 && all isHexDigit revSuffix-      _ -> False- -- | A resolved package: its metadata plus which package db it came from -- (needed to pick the right @db =@ value in the generated rule, and to -- know where to find its @.conf@ file).@@ -307,26 +304,32 @@     ++ [("profiled", f) | f <- rpProfiledLibs p]     ++ [("shared", f) | (_, f) <- rpSharedLibs p] -confPath :: FilePath -> FilePath -> UnitId -> FilePath-confPath globalRootAbs storeDB uid = case classifyUnitId uid of+confPath :: FilePath -> FilePath -> PkgDbKind -> UnitId -> FilePath+confPath globalRootAbs storeDB kind uid = case kind of   GlobalDb -> globalRootAbs </> "package.conf.d" </> unUnitId uid <.> "conf"   StoreDb -> storeDB </> unUnitId uid <.> "conf"  readPackage :: Verbosity -> RepoPaths -> FilePath -> UnitId -> IO (Maybe ResolvedPackage) readPackage verbosity paths storeDB uid = do-  let dbKind = classifyUnitId uid-      path = confPath (rpGlobalRootAbs paths) storeDB uid-  exists <- doesFileExist path-  if not exists-    then do+  let storePath = confPath (rpGlobalRootAbs paths) storeDB StoreDb uid+      globalPath = confPath (rpGlobalRootAbs paths) storeDB GlobalDb uid+  storeHas <- doesFileExist storePath+  globalHas <- doesFileExist globalPath+  case (storeHas, globalHas) of+    (True, _) -> parseFrom StoreDb storePath+    (_, True) -> parseFrom GlobalDb globalPath+    _ -> do       warn verbosity $         "cabal buck2: no .conf file for "           ++ unUnitId uid-          ++ " (expected at "-          ++ path+          ++ " (looked in "+          ++ storePath+          ++ " and "+          ++ globalPath           ++ "); packages are only found in GHC's global package db and the cabal store, not in other package dbs"       return Nothing-    else do+  where+    parseFrom dbKind path = do       contents <- BS.readFile path       case parseInstalledPackageInfo contents of         Left errs -> do@@ -347,7 +350,17 @@             if isRts               then return []               else findLibs paths (libraryDirs ipi) ["lib" ++ stem ++ "_p" <.> "a" | stem <- hsLibraries ipi]-          sharedLibs <- findSharedLibs paths ipi+          -- A Haskell shared library mustn't be linked against the+          -- RTS: which RTS (threaded or not) is only decided when the+          -- final executable is, and on macOS, where each symbol is+          -- bound to the library it was linked against, one loaded+          -- into a process already running another RTS (GHC itself,+          -- running a Template Haskell splice) uses its own,+          -- uninitialized, RTS. Cabal doesn't link against it either.+          sharedLibs <-+            if isRts+              then return []+              else findSharedLibs paths ipi           return $ Just (ResolvedPackage ipi uid dbKind staticLibs profiledLibs sharedLibs)  -- | Resolve each candidate filename against @dirs@ in turn, keeping only@@ -377,7 +390,7 @@   where     dirs = libraryDynDirs ipi ++ libraryDirs ipi     oneStem stem = do-      let soname = "lib" ++ stem ++ "-ghc" ++ rpGhcVersion paths ++ ".so"+      let soname = "lib" ++ stem ++ "-ghc" ++ rpGhcVersion paths <.> rpDllExtension paths       mpath <- findOne paths dirs soname       return ((,) soname <$> mpath) @@ -493,6 +506,7 @@       , ("db", str (if rpDbKind p == GlobalDb then globalDbRel else "cabal-store" </> rpStoreDbRel paths))       , ("static_libs", strList (rel (rpStaticLibs p)))       ]+        ++ [("pic_static_libs", strList (rel (rpStaticLibs p))) | rpPicByDefault paths]         ++ [("profiled_static_libs", strList profiled) | not (null profiled)]         ++ [("pic_profiled_static_libs", strList profiled) | not (null profiled)]         ++ [("shared_libs", VDict [(soname, str sopath) | (soname, abs') <- rpSharedLibs p, Just sopath <- [toRepoRelative paths abs']])]

@@ -78,6 +78,9 @@   , scReexports :: [SpecReexport]   -- ^ Modules a library re-exports from the library of another package (or   -- another library of its own).+  , scHiddenModules :: [String]+  -- ^ Of a library: the modules that its package doesn't expose (Cabal's+  -- @other-modules@).   , scBuildTools :: [SpecBuildTool]   , scCSources :: [FilePath]   , scCcOptions :: [String]@@ -87,6 +90,8 @@   , scAsmSources :: [FilePath]   , scAsmOptions :: [String]   , scIncludeDirs :: [FilePath]+  , scExposed :: Bool+  -- ^ Of a library: whether its package is exposed (Cabal's @exposed@).   , scHscOptions :: [String]   -- ^ Defines for the C compiler of @hsc2hs@, which say what the code is   -- compiled for (GHC does this itself for Haskell code).

@@ -235,6 +235,7 @@       ++ listField "ld_options" (scLdOptions c)       ++ valuesField "deps" (map depValue (scDeps c))       ++ valuesField "reexports" (map reexportValue (scReexports c))+      ++ listField "hidden_modules" (scHiddenModules c)       ++ valuesField "build_tools" (map buildToolValue (scBuildTools c))       ++ listField "c_sources" (scCSources c)       ++ listField "cc_options" (scCcOptions c)@@ -244,6 +245,7 @@       ++ listField "asm_sources" (scAsmSources c)       ++ listField "asm_options" (scAsmOptions c)       ++ listField "include_dirs" (scIncludeDirs c)+      ++ [("exposed", VBool False) | not (scExposed c)]       ++ listField "hsc_options" (scHscOptions c)       ++ listField "generated_include_dirs" (scGeneratedIncludeDirs c)       ++ listField "pkgconfig" (scPkgconfig c)

@@ -78,6 +78,7 @@   , ("pkgconfig", pkgconfig)   , ("project-overrides", projectOverrides)   , ("same-name", sameName)+  , ("normalise-paths", normalisePaths)   ]  -- | A library and an executable with the same name in one package, and@@ -99,6 +100,23 @@   assertContains "user/BUCK.cabal.bzl" "'exe': 'tool-exe'" userBzl   assertContains "user/BUCK.cabal.bzl" "'package': 'tool'" userBzl +-- | Paths written with a @.\/@ prefix, in cabal.project and in the file+-- patterns of a package: buck2 accepts neither in a label nor in a glob.+normalisePaths :: Project -> IO ()+normalisePaths project = do+  _ <- buck2 project []++  -- `./pkg/pkg.cabal` in cabal.project is the package in `pkg`.+  packages <- readIn project ("cabal-buck2" </> "packages.bzl")+  assertContains "cabal-buck2/packages.bzl" "'pkg': 'pkg'" packages+  assertNotContains "cabal-buck2/packages.bzl" "./" packages++  -- The patterns buck2 globs for come out without the prefix too.+  pkgBzl <- readIn project ("pkg" </> "BUCK.cabal.bzl")+  assertContains "pkg/BUCK.cabal.bzl" "'data/x.txt'" pkgBzl+  assertContains "pkg/BUCK.cabal.bzl" "'README'" pkgBzl+  assertNotContains "pkg/BUCK.cabal.bzl" "'./" pkgBzl+ -- | A copy of a fixture project, and how to run @cabal-buck2@ in it. data Project = Project   { projectDir :: FilePath@@ -652,7 +670,7 @@ -- @ghcWithPackages@) are reached through a symlink for each directory. externalDirsTest :: IO () externalDirsTest = do-  let paths = RepoPaths{rpGhcVersion = "9.6.7", rpGlobalRootAbs = "/ghc/lib", rpStoreRootAbs = "/home/u/.cabal/store/ghc-9.6.7", rpStoreDbRel = "package.db", rpExternal = []}+  let paths = RepoPaths{rpGhcVersion = "9.6.7", rpGlobalRootAbs = "/ghc/lib", rpStoreRootAbs = "/home/u/.cabal/store/ghc-9.6.7", rpStoreDbRel = "package.db", rpExternal = [], rpDllExtension = "so", rpPicByDefault = False}       aeson = "/nix/store/aaa-aeson-lib-aeson-2.2.5.1/lib/x86_64-linux-ghc-9.6.7"       text = "/nix/store/bbb-text-lib-text-2.1.4/lib/x86_64-linux-ghc-9.6.7"       dirs =

@@ -0,0 +1,4 @@+# Placeholder: this test only checks `cabal buck2`'s generated files, it+# never actually invokes a real buck2 binary, so this doesn't need to be+# a functional buck2 config - it only needs to exist so that+# `ensureBuckconfigAndPackage` sees it's already there and leaves it alone.

@@ -0,0 +1,1 @@+# Placeholder - see .buckconfig.

@@ -0,0 +1,5 @@+-- A package path with a `./` prefix, naming the .cabal file: how `cabal init`+-- and many hand-written projects write it.+packages: ./pkg/pkg.cabal++active-repositories: :none

@@ -0,0 +1,1 @@+x

@@ -0,0 +1,13 @@+cabal-version:      2.4+name:               pkg+version:            0.1.0.0+build-type:         Simple+-- File patterns with a `./` prefix, as Cabal allows.+data-files:         ./data/x.txt+extra-source-files: ./README++library+  exposed-modules:   Pkg+  hs-source-dirs:    src+  build-depends:     base+  default-language:  Haskell2010

@@ -0,0 +1,4 @@+module Pkg (x) where++x :: Int+x = 1