bytestring-mmap-compat (empty) → 0.2.3

raw patch · 11 files changed

+492/−0 lines, 11 filesdep +basedep +bytestringdep +unixsetup-changed

Dependencies added: base, bytestring, unix

Files

• CHANGELOG.md +18/−0
• LICENSE +27/−0
• README.md +39/−0
• Setup.lhs +3/−0
• System/IO/Posix/MMap.hs +114/−0
• System/IO/Posix/MMap/Internal.hs +54/−0
• System/IO/Posix/MMap/Lazy.hs +130/−0
• System/Posix/IO/Compat.hs +12/−0
• bytestring-mmap-compat.cabal +58/−0
• cbits/hs_bytestring_mmap.c +22/−0
• include/hs_bytestring_mmap.h +15/−0

@@ -0,0 +1,18 @@+# Changelog++## 0.2.3  2026-06-08++Initial release of `bytestring-mmap-compat`, forked from+`bytestring-mmap-0.2.2` on Hackage.++  - Adds `System.Posix.IO.Compat` shim so the package compiles+    against `unix >= 2.8` (whose `openFd` lost the `Maybe FileMode`+    argument).+  - Widens the `unix` bound to `>= 2.7 && < 2.9`.+  - Modernises the cabal file: `cabal-version: 2.0`, explicit+    `default-language: Haskell2010`, explicit base / bytestring+    bounds.+  - Public API unchanged from upstream 0.2.2.+  - Dropped `-O2` from `ghc-options` (a per-package `-O2` is rarely+    a real win on a thin FFI wrapper and burns compile time on+    every downstream user; the hot path is in `cbits/`).

@@ -0,0 +1,27 @@+Copyright (c) Don Stewart++All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions+are met:+1. Redistributions of source code must retain the above copyright+   notice, this list of conditions and the following disclaimer.+2. Redistributions in binary form must reproduce the above copyright+   notice, this list of conditions and the following disclaimer in the+   documentation and/or other materials provided with the distribution.+3. Neither the name of the author nor the names of his contributors+   may be used to endorse or promote products derived from this software+   without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE+ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS+OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)+HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF+SUCH DAMAGE.

@@ -0,0 +1,39 @@+# bytestring-mmap-compat++A fork of [`bytestring-mmap`](https://hackage.haskell.org/package/bytestring-mmap)+that compiles against `unix >= 2.8`.++## Why this exists++The original `bytestring-mmap` (Don Stewart, last Hackage upload in+2011) calls `System.Posix.openFd` with the pre-2.8 four-argument+signature. Upstream is unmaintained, and the latest Hackage revision+caps the `unix` bound at `< 2.8`, which leaves the package unusable+in any package set that pulls in a newer `unix` transitively.++This fork:++  - Adds a tiny `System.Posix.IO.Compat` shim that selects the right+    `openFd` signature via CPP.+  - Widens the `unix` bound to cover both old and new signatures.+  - Modernises the cabal file (cabal-version 2.0, sensible base /+    bytestring bounds).++The public API (`System.IO.Posix.MMap`,+`System.IO.Posix.MMap.Lazy`) is byte-for-byte compatible with the+upstream; the only difference is the package name.++## When to switch back to upstream++When (or if) upstream adopts a compatible `openFd` call site, switch+your `build-depends` from `bytestring-mmap-compat` to `bytestring-mmap`+and remove this dependency. No source-level changes required.++## Credit++All non-trivial code is by Don Stewart. This fork only adds the+`Compat` shim and updates packaging metadata.++## License++BSD3; see `LICENSE`.

@@ -0,0 +1,3 @@+#!/usr/bin/env runhaskell+> import Distribution.Simple+> main = defaultMain

@@ -0,0 +1,114 @@+{-# LANGUAGE ForeignFunctionInterface #-}+--------------------------------------------------------------------+-- |+-- Module    :  System.IO.Posix.MMap+-- Copyright :  (c) Galois, Inc. 2007+-- License   :  BSD3+--+-- Maintainer:  Don Stewart <dons@galois.com>+-- Stability :  provisional+-- Portability: non-portable -- posix only+--+-- mmap a file or device into memory as a strict ByteString.+--+module System.IO.Posix.MMap (++      -- $mmap_intro+      -- $mmap_unmap++      -- * Memory mapped files+      unsafeMMapFile -- :: FilePath -> IO ByteString++-- $mmap_intro+--+-- 'unsafeMMapFile' mmaps a file or device into memory as a strict+-- 'ByteString'. The file is not actually copied strictly into memory,+-- but instead pages from the file will be loaded into the address+-- space on demand.+--+-- We can consider mmap as lazy IO pushed into the virtual memory+-- subsystem.+--+-- The file is mapped using MAP_SHARED: modifications to the file+-- will be immediately shared with any other process accessing the+-- file. This has no effect from the Haskell point of view, since+-- ByteStrings are treated as immutable values.+--+-- However, if the file is written to by any other process on the+-- system while it is in use in Haskell, those changes will be+-- immediately reflected on the Haskell side, destroying referential+-- transparency.+--+-- It is only safe to mmap a file if you know you are the sole user.+--+-- For more details about mmap, and its consequences, see:+-- +-- * <http://opengroup.org/onlinepubs/009695399/functions/mmap.html>+--+-- * <http://www.gnu.org/software/libc/manual/html_node/Memory_002dmapped-I_002fO.html>+--++-- $mmap_unmap+--+-- When the entire file is out of scope, the Haskell storage manager+-- will call munmap to free the file, using a finaliser. Until then, as+-- much of the file as you access will be allocated.+--+-- Note that the Haskell storage manager doesn't know how large a+-- resource is associated with an mmapped file. If you allocate many+-- such files, the garbage collector will only see the 'ForeignPtr's+-- that have been allocated, not the corresponding ByteArrays. The+-- result will be that the GC runs less often that you hoped, as it +-- looks like only a few bytes have been allocated on the Haskell heap.+-- +-- Use of 'performGC' or 'finalizeForeignPtr' when you know that+-- the object is going out of scope can ensure that resources are+-- released appropriately.+--++    ) where++import System.IO.Posix.MMap.Internal++-- import System.IO+-- import qualified System.IO as IO+import Foreign.Ptr++import Control.Exception+import Data.ByteString++import System.Posix hiding (openFd)+import System.Posix.IO.Compat (openFd)++-- | The 'unsafeMMapFile' function maps a file or device into memory,+-- returning a strict 'ByteString' that accesses the mapped file.+-- If the mmap fails for some reason, an error is thrown.+--+-- Memory mapped files will behave as if they were read lazily -- +-- pages from the file will be loaded into memory on demand.+--+-- The storage manager is used to free the mapped memory. When+-- the garbage collector notices there are no further references to the +-- mapped memory, a call to munmap is made. It is not necessary to do+-- this yourself. In tight memory situations, it may be profitable to+-- use 'performGC' or 'finalizeForeignPtr' to force an unmap.+--+-- Note: this operation may break referential transparency! If +-- any other process on the system changes the file when it is mapped+-- into Haskell, the contents of your 'ByteString' will change.+--+unsafeMMapFile :: FilePath -> IO ByteString+unsafeMMapFile f = do+    fd   <- openFd f ReadOnly defaultFileFlags+    always (closeFd fd) $ do+        stat <- getFdStatus fd+        let size = fromIntegral (fileSize stat)+        if size <= 0+            then return empty -- BSD mmap won't accept a length of zero+            else do+        ptr <- c_mmap size (fromIntegral fd)+        if ptr == nullPtr+            then error "System.IO.Posix.MMap.mmapFile: unable to mmap file"+            else unsafePackMMapPtr ptr size++  where always = flip finally

@@ -0,0 +1,54 @@+{-# LANGUAGE ForeignFunctionInterface #-}+--------------------------------------------------------------------+-- |+-- Module    :  System.IO.Posix.MMap.Internal+-- Copyright :  (c) Galois, Inc. 2007+-- License   :  BSD3+--+-- Maintainer:  Don Stewart <dons@galois.com>+-- Stability :  provisional+-- Portability: non-portable -- posix only+--+-- Low level mmap access.+--+module System.IO.Posix.MMap.Internal (++    -- * Converting an mmapped pointer to a 'ByteString'+    unsafePackMMapPtr,      -- :: Ptr Word8 -> CSize -> IO ByteString++    -- * Low level bindings+    c_mmap,                 -- :: CSize -> CInt -> IO (Ptr Word8)+    c_munmap                -- :: Ptr Word8 -> CSize -> IO CInt++  ) where++import System.IO+import qualified System.IO as IO+import Foreign.C.Types+import Foreign.Ptr+import qualified Foreign.Concurrent as FC++import Control.Monad+import Data.Word+import Data.ByteString.Internal+-- import Data.ByteString++-- | Create a bytestring from a memory mapped Ptr.+-- A finalizer will be associated with the resource, that will call+-- munmap when the storage manager detects that the resource is no longer+-- in use.+unsafePackMMapPtr :: Ptr Word8 -> CSize -> IO ByteString+unsafePackMMapPtr p s = do+    fp <- FC.newForeignPtr p $ do+                 v <- c_munmap p s+                 when (v == -1) $ IO.hPutStrLn stderr $+                         "System.IO.Posix.MMap: warning, failed to unmap "+                          ++ show s ++" bytes at "++show p+    return (fromForeignPtr fp 0 (fromIntegral s))+{-# INLINE unsafePackMMapPtr #-}++foreign import ccall unsafe "hs_bytestring_mmap.h hs_bytestring_mmap"+    c_mmap   :: CSize -> CInt -> IO (Ptr Word8)++foreign import ccall unsafe "hs_bytestring_mmap.h munmap"+    c_munmap :: Ptr Word8 -> CSize -> IO CInt

@@ -0,0 +1,130 @@+{-# LANGUAGE CPP, BangPatterns, ForeignFunctionInterface #-}+--------------------------------------------------------------------+-- |+-- Module    :  System.IO.Posix.MMap+-- Copyright :  (c) Galois, Inc. 2007+-- License   :  BSD3+--+-- Maintainer:  Don Stewart <dons@galois.com>+-- Stability :  provisional+-- Portability: non-portable -- posix only+--+-- Lazy, chunk-wise memory mapping.+--+-- Memory map a file as a lazy ByteString. Finalisers are associated+-- cached-sized portions of the file, which will be deallocated as+-- those chunks go out of scope.+--+-- Unlike strict Bytestrings, mmapFile for Lazy ByteStrings will+-- deallocate chunks of the file.+--+-- The storage manager is used to free chunks of the mapped memory. When+-- the garbage collector notices there are no further references to +-- a chunk, a call to munmap is made.+--+-- In effect, the file is mmapped once, lazily, then covered with finalizers+-- for each chunk. When any chunk goes out of scope, that part is+-- deallocated. We must allocate the spine of the structure strictly+-- though, to ensure finalizers are registered for the entire file.+--+-- The Haskell garbage collector decides when to run based on heap+-- pressure, however the mmap stores memory outside the Haskell heap, +-- so those resources are not counted when deciding to run the garbage+-- collect. The result is that finalizers run less often than you might+-- expect, and it is possible to write a lazy bytestring mmap program +-- that never deallocates (and thus doesn't run in constant space).+-- 'performGC' or 'finalizerForeignPtr' can be used to trigger collection+-- at sensible points.+--+-- Note: this operation may break referential transparency! If +-- any other process on the system changes the file when it is mapped+-- into Haskell, the contents of your 'ByteString' will change.+--+module System.IO.Posix.MMap.Lazy (++      unsafeMMapFile -- :: FilePath -> IO ByteString++    ) where++import System.IO.Posix.MMap.Internal++-- import System.IO+import Foreign.C.Types+import Foreign.Ptr+-- import Control.Monad++import Control.Exception+import Data.Word+import Data.ByteString.Lazy.Internal++import System.Posix hiding (openFd)+import System.Posix.IO.Compat (openFd)++--+-- | The 'unsafeMMapFile' function maps a file or device into memory as+-- a lazy ByteString, made of 64*pagesize unmappable chunks of bytes.+--+-- Memory mapped files will behave as if they were read lazily -- +-- pages from the file will be loaded into memory on demand.+-- +-- The storage manager is used to free chunks that go out of scope,+-- and unlike strict bytestrings, memory mapped lazy ByteStrings will+-- be deallocated in chunks (so you can write traversals that run in+-- constant space).+--+-- However, the size of the mmapped resource is not known by the Haskell+-- GC, it appears only as a small ForeignPtr. This means that the+-- Haskell GC may not not run as often as you'd like, leading to delays+-- in unmapping chunks.+-- +-- Appropriate use of performGC or finalizerForeignPtr may be required+-- to ensure deallocation, as resources allocated by mmap are not+-- tracked by the Haskell garbage collector.+--+-- For example, when writing out a lazy bytestring allocated with mmap,+-- you may wish to finalizeForeignPtr when each chunk is written, as the +-- chunk goes out of scope, rather than relying on the garbage collector+-- to notice the chunk has gone.+--+-- This operation is unsafe: if the file is written to by any other+-- process on the system, the 'ByteString' contents will change in+-- Haskell.+--+unsafeMMapFile :: FilePath -> IO ByteString+unsafeMMapFile path = do+    fd   <- openFd path ReadOnly defaultFileFlags+    always (closeFd fd) $ do+        stat <- getFdStatus fd+        let size = fromIntegral (fileSize stat)+        ptr  <- c_mmap size (fromIntegral fd)+        if ptr == nullPtr+          then error "System.IO.Posix.MMap.Lazy: unable to mmap file!"+          else chunks chunk_size ptr (fromIntegral size)+  where+    always = flip finally++    -- must be page aligned.+    chunk_size = 64 * fromIntegral pagesize -- empircally derived++--+-- Break the file up into chunks.+    -- Have separate munmap finalizers for each chunk.+--+chunks :: CSize -> Ptr Word8 -> CSize -> IO ByteString+chunks chunk_size p bytes = loop p bytes+#ifndef __HADDOCK__+  where+    loop !ptr !rest+          | rest <= 0 = return Empty+          | otherwise = let s     = min chunk_size rest+                            ptr'  = ptr `plusPtr` fromIntegral s+                            rest' = rest - s+                        in do c  <- unsafePackMMapPtr ptr s+                              cs <- loop ptr' rest' -- need to be strict+                              return (chunk c cs)   -- to ensure we cover the whole file+                                                    -- with finalizers+#endif++foreign import ccall unsafe "unistd.h getpagesize"+    pagesize :: CInt+

@@ -0,0 +1,12 @@+{-# LANGUAGE CPP #-}+module System.Posix.IO.Compat where++import qualified System.Posix as Unix+++openFd :: FilePath -> Unix.OpenMode -> Unix.OpenFileFlags -> IO Unix.Fd+#if MIN_VERSION_unix(2,8,0)+openFd = Unix.openFd+#else+openFd file openMode = Unix.openFd file openMode Nothing+#endif

@@ -0,0 +1,58 @@+cabal-version:      2.0+name:               bytestring-mmap-compat+version:            0.2.3+synopsis:           mmap support for strict ByteStrings (unix >= 2.8 fork)+description:+    A fork of @bytestring-mmap@ that compiles against @unix >= 2.8@.+    .+    The original package (Don Stewart, last uploaded 2011) calls+    @System.Posix.openFd@ with the pre-2.8 four-argument signature.+    Upstream is unmaintained and the latest Hackage revision restricts+    the @unix@ bound to @< 2.8@, which makes the package unusable in+    package sets that pull in a newer @unix@ transitively.+    .+    This fork adds a tiny @System.Posix.IO.Compat@ shim that selects+    the right @openFd@ signature via CPP and widens the @unix@ bound,+    leaving the public API (@System.IO.Posix.MMap@,+    @System.IO.Posix.MMap.Lazy@) byte-for-byte compatible with the+    upstream.+    .+    The package name is suffixed with @-compat@ so it can coexist on+    Hackage with the original; switch back to upstream+    @bytestring-mmap@ once it adopts a compatible @openFd@ call site.+category:           System+homepage:           https://github.com/NCrashed/bytestring-mmap-compat+bug-reports:        https://github.com/NCrashed/bytestring-mmap-compat/issues+license:            BSD3+license-file:       LICENSE+author:             Don Stewart+maintainer:         Anton Gushcha <ncrashed@gmail.com>+copyright:          (c) Don Stewart 2008-2011, (c) Anton Gushcha 2024-2026+build-type:         Simple+tested-with:        GHC == 9.6.6+extra-doc-files:    README.md+                    CHANGELOG.md+extra-source-files: include/hs_bytestring_mmap.h+                    cbits/hs_bytestring_mmap.c++source-repository head+  type:     git+  location: https://github.com/NCrashed/bytestring-mmap-compat.git++library+  default-language:  Haskell2010+  hs-source-dirs:    .+  exposed-modules:   System.IO.Posix.MMap+                     System.IO.Posix.MMap.Lazy+                     System.IO.Posix.MMap.Internal+  other-modules:     System.Posix.IO.Compat+  build-depends:     base       >= 4.14 && < 5+                   , bytestring >= 0.10 && < 0.13+                   , unix       >= 2.7  && < 2.9+  default-extensions: CPP, ForeignFunctionInterface, BangPatterns,+                      NondecreasingIndentation+  ghc-options:       -Wall+  c-sources:         cbits/hs_bytestring_mmap.c+  include-dirs:      include+  includes:          hs_bytestring_mmap.h+  install-includes:  hs_bytestring_mmap.h

@@ -0,0 +1,22 @@+/*+ * hs_bytestring_mmap.c+ *+ * License   : BSD3+ *+ * Copyright (C) 2003   David Roundy+ *               2005-7 Don Stewart+ *+ * Maintainer: Don Stewart <dons@galois.com>+ */+#include "hs_bytestring_mmap.h"++/* + * mmap len bytes from fd into memory, read only.+ */+unsigned char *hs_bytestring_mmap(size_t len, int fd) {+      void *result = mmap(0, len, PROT_READ, MAP_SHARED, fd, 0);+      if (result == MAP_FAILED)+            return (unsigned char *)0;+      else+            return (unsigned char *)result;+}

@@ -0,0 +1,15 @@+/*+ * hs_bytestring_mmap.h+ *+ * License   : BSD3+ *+ * Copyright (C) 2003   David Roundy+ *               2005-7 Don Stewart+ *+ * Maintainer: Don Stewart <dons@galois.com>+ */++#include <sys/types.h>+#include <sys/mman.h>++unsigned char *hs_bytestring_mmap(size_t len, int fd);