packages feed

streamly-fsnotify 1.1.1.3 → 2.0

raw patch · 4 files changed

+96/−287 lines, 4 filesdep +exceptionsdep +streamly-coredep ~filepathdep ~semiringsdep ~streamly

Dependencies added: exceptions, streamly-core

Dependency ranges changed: filepath, semirings, streamly, time

Files

CHANGELOG.md view
@@ -1,22 +1,33 @@ # Revision history for streamly-fsnotify +## 2.0 -- 2023-12-29++With Streamly also making major breaking changes, this seemed like the time for a radical overhaul (we needed to move to the new stream type _at some point_, and that would break everything anyway). Essentially, having taken over this package nearly four years ago, and used it extensively, I've decided that it was unnecessarily complex. The line count is now _much_ shorter, but the interesting stuff is still here. In particular:++- The predicate algebra, while somewhat neat, didn't really seem to belong in a file-watching library.+- Trying to fully abstract over `fsnotify` was annoying in practice, both as a maintainer and a user. We now use `fsnotify`'s `Event` type directly. This makes interoperability a lot easier. And it means we won't need to make any major changes here for future changes in `fsnotify`, like converting to `OsPath`.++We do some new clever stuff as well, like stopping the file watcher automatically at the end of the stream.++It is possible that we've gone too far. If there's anything you really miss from version 1, please mention it on the issue tracker!+ ## 1.1.1.0 -- 2020-05-27 -* Export additional `EventPredicate`s.+- Export additional `EventPredicate`s.  ## 1.1.0.0 -- 2020-05-27 -* Use abstract newtype for `StopWatching`.-* Remove typed filepaths. Power-to-weight ratio was too low.+- Use abstract newtype for `StopWatching`.+- Remove typed filepaths. Power-to-weight ratio was too low.  ## 1.0.1.0 -- 2020-05-27 -* George Thomas takes over as maintainer. Metadata changes.+- George Thomas takes over as maintainer. Metadata changes.  ## 1.0.0.1 -- 2019-12-09 -* Widen bounds on ``time`` to support Windows build.+- Widen bounds on `time` to support Windows build.  ## 1.0.0.0 -- 2019-12-06 -* First version. Released on an unsuspecting world.+- First version. Released on an unsuspecting world.
README.md view
@@ -1,61 +1,1 @@-## What's the deal with this library?--[``streamly``][1] is an undoubtedly awesome library - [fast][2], flexible, and-well-documented. File system watching is a natural fit for a streaming library,-and this is exactly what ``streamly-fsnotify`` provides you.--As an example, here is a program which watches ``/home/koz/c-project/`` and any-of its subdirectories for added or modified C source files (which we take to be-anything with a ``.c`` extension). This program then writes that the event-occurred, to what file, and when, forever.--```haskell--{-# LANGUAGE LambdaCase #-}--import System.FilePath ((</>))-import Streamly.FSNotify (EventPredicate, hasExtension, isDirectory, invert, isDeletion, conj, watchTree)-import qualified Streamly.Prelude as SP---- conj -> both must be true--- invert -> true when the argument would be false and vice versa-isCSourceFile :: EventPredicate-isCSourceFile = hasExtension "c" `conj` invert isDirectory--notDeletion :: EventPredicate-notDeletion = invert isDeletion--srcPath :: FilePath-srcPath = "home" </> "koz" </> "c-project"---- first value given by watchTree stops the watcher--- we don't use it here, but if you want to, just call it-main :: IO ()-main = do-    (_, stream) <- watchTree srcPath $ isCSourceFile `conj` notDeletion-    SP.drain . SP.mapM go $ stream-  where-    go = \case-        Added p t _ -> putStrLn $ "Created: " ++ show p ++ " at " ++ show t-        Modified p t _ -> putStrLn $ "Modified: " ++ show p ++ " at " ++ show t-        _ -> pure ()-```--## That seems pretty cool! What kind of features can I expect?--* Cross-platform - should work anywhere both ``streamly`` and ``fsnotify`` do.-* Efficient (event-driven, so won't shred your CPU or load your RAM).-* Able to do one-level and recursive watching.-* Compositional and principled treatment of event filtering predicates.-* Extensive set of filtering predicates, so you don't have to see events you-  don't care about!--## Sounds good? Can I use it?--We've test-built this library for GHCs 8.6.5 through 8.10.1 on GNU/Linux. In-theory, ``streamly-fsnotify`` should work everywhere both ``streamly`` and-``fsnotify`` will, which includes other OSes (such as Windows). However, we-haven't tried it ourselves - let us know if you do!--[1]: http://hackage.haskell.org/package/streamly-[2]: https://github.com/composewell/streaming-benchmarks+This library provides a higher-level, stream-based API for [`fsnotify`](https://hackage.haskell.org/package/fsnotify). It should work anywhere both `streamly` and `fsnotify` do. Note that there's no need for your project to register a dependency on `fsnotify` directly, but you will need to depend on `streamly` in order to get anything done.
src/Streamly/FSNotify.hs view
@@ -1,38 +1,33 @@ {- |-__Introduction__--This provides file watching as a Streamly stream. You can either watch recursively (namely, a directory's contents and-all its subdirectories as well), or not. You can also filter out file system events you are not interested in. Lastly,-we provide a compositional scheme for constructing filters for file system events.- __Example__ -This example program watches @\/home\/koz\/c-project@ (and any of its subdirectories) for added or modified files with a-@.c@ extension, and emits the change to the terminal, along with a timestamp of when it happened, forever:+Here is a program which watches @\/home\/georgefst\/c-project@ and any of its subdirectories for added or modified+C source files (which we take to be anything with a @.c@ extension). This program then writes that the event occurred,+to what file, and when, forever. -> {-# LANGUAGE LambdaCase #-}+> {-# LANGUAGE GHC2021, BlockArguments, LambdaCase #-} >-> import System.FilePath ((</>))-> import Streamly.FSNotify (EventPredicate, hasExtension, isDirectory, invert, isDeletion, conj, watchTree)-> import qualified Streamly.Prelude as SP+> import Data.Functor.Contravariant (Predicate (Predicate))+> import Streamly.Data.Fold qualified as SF+> import Streamly.Data.Stream.Prelude qualified as SP+> import System.FilePath (isExtensionOf, (</>)) >-> -- conj -> both must be true-> -- invert -> true when the argument would be false and vice versa-> isCSourceFile :: EventPredicate-> isCSourceFile = hasExtension "c" `conj` invert isDirectory+> import Streamly.FSNotify >-> notDeletion :: EventPredicate-> notDeletion = invert isDeletion+> isCSourceFile :: Predicate Event+> isCSourceFile = Predicate \e ->+>     "c" `isExtensionOf` eventPath e && eventIsDirectory e == IsFile >+> notDeletion :: Predicate Event+> notDeletion = Predicate \case+>     Removed{} -> False+>     _ -> True+> > srcPath :: FilePath-> srcPath = "home" </> "koz" </> "c-project"+> srcPath = "/" </> "home" </> "gthomas" </> "c-project" >-> -- first value given by watchTree stops the watcher-> -- we don't use it here, but if you want to, just call it > main :: IO ()-> main = do->     (_, stream) <- watchTree srcPath $ isCSourceFile `conj` notDeletion->     SP.drain . SP.mapM go $ stream+> main = SP.fold (SF.drainMapM go) $ watchTree srcPath $ isCSourceFile <> notDeletion >   where >     go = \case >         Added p t _ -> putStrLn $ "Created: " ++ show p ++ " at " ++ show t@@ -40,200 +35,61 @@ >         _ -> pure () -} module Streamly.FSNotify (-    -- * Basic types-    FSEntryType(..), Event(..), StopWatching(stopWatching),-    eventPath, eventTime, eventFSEntry,-    -- * Events and predicates-    EventPredicate(..),-    everything, nothing, isDirectory, hasExtension, isCreation, isModification, isDeletion, isBasic, invert, conj, disj,-    -- * Watchers-    watchDirectory, watchDirectoryWith, watchTree, watchTreeWith,+    Event (..),+    EventIsDirectory (..),+    watchDir,+    watchTree, ) where -import Control.Arrow ((>>>)) import Control.Concurrent.Chan (newChan, readChan)-import Control.Monad.IO.Class (MonadIO(..))-import Data.Semiring (Semiring(..))-import Data.Text (Text, pack)-import Data.Time.Clock (UTCTime)-import Streamly.Prelude (IsStream, MonadAsync)-import System.FilePath (isExtensionOf)--import qualified Streamly.Prelude as SP-import qualified System.FSNotify as FSN----- | Allows us to designate 'Event's as being fired by a directory or a non-directory entry.-data FSEntryType = Dir | NotDir-    deriving (Eq, Show, Read, Bounded, Enum)---- | A file system notification.-data Event-    = Added FilePath UTCTime FSEntryType -- ^ Creation event-    | Modified FilePath UTCTime FSEntryType -- ^ Modification event-    | Removed FilePath UTCTime FSEntryType -- ^ Deletion event-    | Other FilePath UTCTime Text -- ^ Some other kind of event-    deriving (Eq, Show)---- | An object, which, when executed with 'stopWatching', stops a file system watch.-newtype StopWatching m = StopWatching { stopWatching :: m () }---- | Helper to retrieve the file path associated with an event.-eventPath :: Event -> FilePath-eventPath = \case-    Added p _ _ -> p-    Modified p _ _ -> p-    Removed p _ _ -> p-    Other p _ _ -> p---- | Helper to retrieve an event's timestamp.-eventTime :: Event -> UTCTime-eventTime = \case-    Added _ t _ -> t-    Modified _ t _ -> t-    Removed _ t _ -> t-    Other _ t _ -> t---- | Helper to retrieve whether the event stems from a directory or not.--- Returns 'Nothing' if the event is not \'basic\' (that is, not a creation, modification or deletion).-eventFSEntry :: Event -> Maybe FSEntryType-eventFSEntry = \case-    Added _ _ e -> Just e-    Modified _ _ e -> Just e-    Removed _ _ e -> Just e-    Other{} -> Nothing---{- Predicates -}---- | A \'test\' for whether we want to \'notice\' an event. Should return 'True' for events we care about.-newtype EventPredicate = EventPredicate { runPredicate :: Event -> Bool }--{- | 'EventPredicate' can be a 'Semigroup' in two ways:--- Under logical conjunction (both of the conditions must be met); and-- Under logical disjunction (either of the conditions must be met).--Both of these can be made into a 'Monoid' by using the trivial predicate (always true) for the first case, and the null-predicate (always false) for the second. This makes it a valid candidate to be a semiring, which allows our users to-compose 'EventPredicate's using both of these methods, as they see fit.--If you want an instance of 'Semigroup' and 'Monoid' with one of these behaviours, you can use 'Data.Semiring.Add' (for-the logical disjunction behaviour) or 'Data.Semiring.Mul' (for the logical conjunction behaviour).--}-instance Semiring EventPredicate where-    (EventPredicate f) `plus` (EventPredicate g) = EventPredicate $ (||) <$> f <*> g-    zero = nothing-    (EventPredicate f) `times` (EventPredicate g) = EventPredicate $ (&&) <$> f <*> g-    one = everything---- | Predicate conjunction (meaning that /both/ have to be true for the result to be true).--- Synonym for 'Data.Semigroup.times'.-conj :: EventPredicate -> EventPredicate -> EventPredicate-conj = times---- | Predicate disjunction (meaning that /either/ has to be true for the result to be true).--- Synonym for 'Data.Semigroup.plus'.-disj :: EventPredicate -> EventPredicate -> EventPredicate-disj = plus---- | The trivial predicate (allows any event through).-everything :: EventPredicate-everything = EventPredicate $ const True---- | The null predicate (allows no events through).-nothing :: EventPredicate-nothing = EventPredicate $ const False---- | Allows through events that are caused by directories.--- Note that this will assume that non-\'basic\' events (that is, not creations, modifications or deletions) do not stem--- from directories; use with care.-isDirectory :: EventPredicate-isDirectory = EventPredicate $ eventFSEntry >>> \case-    Nothing -> False-    Just Dir -> True-    Just NotDir -> False---- | Allows through events triggered by file system entries with a specific--- extension.-hasExtension :: FilePath -> EventPredicate-hasExtension fe = EventPredicate $ (fe `isExtensionOf`) . \case-    Added p _ _ -> p-    Modified p _ _ -> p-    Removed p _ _ -> p-    Other p _ _ -> p---- | Allows through only creation events.-isCreation :: EventPredicate-isCreation = EventPredicate $ \case-    Added{} -> True-    _ -> False---- | Allows through only modification events.-isModification :: EventPredicate-isModification = EventPredicate $ \case-    Modified{} -> True-    _ -> False---- | Allows through only deletion events.-isDeletion :: EventPredicate-isDeletion = EventPredicate $ \case-    Removed{} -> True-    _ -> False---- | Allows through only \'basic\' events (namely creation, modification and deletion).-isBasic :: EventPredicate-isBasic = EventPredicate $ \case-    Other{} -> False-    _ -> True---- | \'Flips\' the predicate - what it used to allow through is now blocked, and vice versa.-invert :: EventPredicate -> EventPredicate-invert (EventPredicate f) = EventPredicate $ not . f---{- Watchers -}+import Control.Monad.Catch (MonadCatch)+import Control.Monad.IO.Class (liftIO)+import Data.Functor.Contravariant (Predicate (getPredicate))+import Streamly.Data.Stream.Prelude (MonadAsync, Stream)+import Streamly.Data.Stream.Prelude qualified as S+import Streamly.Data.StreamK qualified as SK+import Streamly.Internal.Data.StreamK qualified as SK+import System.FSNotify (+    ActionPredicate,+    Event (..),+    EventChannel,+    EventIsDirectory (..),+    StopListening,+    WatchManager,+    defaultConfig,+    startManagerConf,+    stopManager,+    watchDirChan,+    watchTreeChan,+ )  -- | Watch a given directory, but only at one level (thus, subdirectories will __not__ be watched recursively).-watchDirectory :: (IsStream t, MonadAsync m) => FilePath -> EventPredicate -> m (StopWatching m, t m Event)-watchDirectory = watch FSN.watchDirChan FSN.defaultConfig---- | As 'watchDirectory', but with a specified set of watch options.-watchDirectoryWith :: (IsStream t, MonadAsync m) =>-    FSN.WatchConfig -> FilePath -> EventPredicate -> m (StopWatching m, t m Event)-watchDirectoryWith = watch FSN.watchDirChan+watchDir :: (MonadAsync m, MonadCatch m) => FilePath -> Predicate Event -> Stream m Event+watchDir = watch watchDirChan  -- | Watch a given directory recursively (thus, subdirectories will also have their contents watched).-watchTree :: (IsStream t, MonadAsync m) => FilePath -> EventPredicate -> m (StopWatching m, t m Event)-watchTree = watch FSN.watchTreeChan FSN.defaultConfig---- | As 'watchTree', but with a specified set of watch options.-watchTreeWith :: (IsStream t, MonadAsync m) => FSN.WatchConfig -> FilePath -> EventPredicate -> m (StopWatching m, t m Event)-watchTreeWith = watch FSN.watchTreeChan---{- Util -}--watch :: (IsStream t, MonadAsync m) =>-    (FSN.WatchManager -> FilePath -> FSN.ActionPredicate -> FSN.EventChannel -> IO FSN.StopListening) ->-    FSN.WatchConfig -> FilePath -> EventPredicate -> m (StopWatching m, t m Event)-watch f conf p predicate = do-    manager <- liftIO $ FSN.startManagerConf conf-    let pred' = runPredicate predicate . mungeEvent-    chan <- liftIO newChan-    stop <- liftIO $ f manager p pred' chan-    let reallyStop = StopWatching $ liftIO stop >> liftIO (FSN.stopManager manager)-    pure (reallyStop, SP.repeatM $ liftIO $ mungeEvent <$> readChan chan)--mungeEvent :: FSN.Event -> Event-mungeEvent = \case-    FSN.Added p t b -> Added p t (isDir b)-    FSN.Modified p t b -> Modified p t (isDir b)-    FSN.Removed p t b -> Modified p t (isDir b)-    FSN.Unknown p t b s -> Other p t (pack s)-    e -> Other (FSN.eventPath e) (FSN.eventTime e) (pack $ show e)+watchTree :: (MonadAsync m, MonadCatch m) => FilePath -> Predicate Event -> Stream m Event+watchTree = watch watchTreeChan -isDir :: FSN.EventIsDirectory -> FSEntryType-isDir = \case-    FSN.IsFile -> NotDir-    FSN.IsDirectory -> Dir+watch ::+    (MonadAsync m, MonadCatch m) =>+    (WatchManager -> FilePath -> ActionPredicate -> EventChannel -> IO StopListening) ->+    FilePath ->+    Predicate Event ->+    Stream m Event+watch f p predicate = withInit+    do+        manager <- liftIO $ startManagerConf defaultConfig+        chan <- liftIO newChan+        stop <- liftIO $ f manager p (getPredicate predicate) chan+        pure (chan, liftIO $ stop >> liftIO (stopManager manager))+    \(chan, stop) -> S.finally stop $ S.repeatM $ liftIO $ readChan chan+  where+    -- TODO a few problems with this:+    -- it's vendored from `georgefst-utils`+    -- it's inelegant and inefficient (though inhibiting fusion isn't a major issue since we already have `finally`)+    -- it incurs a direct dependency on `streamly-core`+    withInit init_ stream =+        SK.toStream . SK.unCross $+            SK.mkCross . SK.fromStream . stream+                =<< SK.mkCross (SK.fromStream $ S.fromEffect init_)
streamly-fsnotify.cabal view
@@ -1,6 +1,6 @@ cabal-version:       3.0 name:                streamly-fsnotify-version:             1.1.1.3+version:             2.0 synopsis:            Folder watching as a Streamly stream. description:     Provides Streamly streams for both single-level and recursive folder watching.@@ -20,15 +20,17 @@         Streamly.FSNotify     build-depends:         base >= 4.9 && < 5,-        filepath ^>= {1.4.2.1, 1.5},+        exceptions ^>= 0.10,+        filepath ^>= 1.4.2.1,         fsnotify ^>= 0.4,-        semirings ^>= {0.5.2, 0.6, 0.7},-        streamly ^>= {0.7, 0.8, 0.9, 0.10},+        semirings ^>= {0.5.2, 0.6},+        streamly ^>= {0.9, 0.10},+        streamly-core ^>= 0.2,         text ^>= {1.2.3.0, 2.0, 2.1},-        time ^>= {1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 1.13, 1.14},+        time ^>= {1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 1.12},     hs-source-dirs: src-    default-language: Haskell2010+    default-language: GHC2021     ghc-options: -Wall     default-extensions:-        FlexibleContexts+        BlockArguments         LambdaCase