frecently (empty) → 1.0

raw patch · 5 files changed

+438/−0 lines, 5 filesdep +atomic-writedep +basedep +bytestring

Dependencies added: atomic-write, base, bytestring, cereal, containers, directory, filepath, optparse-applicative, process

Files

• CHANGELOG.md +5/−0
• LICENSE +29/−0
• README.md +120/−0
• app/Main.hs +238/−0
• frecently.cabal +46/−0

@@ -0,0 +1,5 @@+# Changelog++## 1.0++Initial release

@@ -0,0 +1,29 @@+BSD 3-Clause License++Copyright (c) 2022, Jonas Carpay+All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice, this+   list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+   this list of conditions and the following disclaimer in the documentation+   and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its+   contributors may be used to endorse or promote products derived from+   this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

@@ -0,0 +1,120 @@+# frecently+[![frecently on hackage](https://img.shields.io/hackage/v/frecently)](http://hackage.haskell.org/package/frecently)+[![frecently on Stackage Nightly](https://stackage.org/package/frecently/badge/nightly)](https://stackage.org/nightly/package/frecently)++Extremely simple CLI tool for maintaining a [frecency](https://en.wikipedia.org/wiki/Frecency) history.++The intended use case is to add a frecency-based search history to CLI tools like [dmenu](https://tools.suckless.org/dmenu/), [rofi](https://github.com/davatorium/rofi), or [fzf](https://github.com/junegunn/fzf).+It also allows you to easily emulate popular directory-jumping tools like [autojump](https://github.com/wting/autojump) or [z](https://github.com/rupa/z).+++### Examples++We'll look at some examples of how to integrate `frecently` with other tools.+For more detailed information, run `frecently --help`.++#### Basic CLI:++```console+$ frecently view .history      # .history doesn't exist yet. By default, all commands treat a missing file as empty+$ frecently bump .history foo  # creates .history, and bumps foo+$ frecently bump .history bar+$ frecently view .history      # `view` shows all entries, ordered by frecency+bar+foo+$ echo -e "bar\nbaz" | frecently view .history --augment   # with --augment, frecently accepts extra entries on stdin (newline-separated) that should always appear in the output+bar+foo+baz+$ echo -e "foo\nbar\nbaz" | frecently scores .history --augment   # `scores` has the same interface as `view`, but shows the entire scores table+weighted score  hourly          daily           monthly+  178.663539      0.985473        0.999390        0.999980      bar+  178.267277      0.983010        0.999286        0.999976      foo+    0.000000      0.000000        0.000000        0.000000      baz+$ echo -e "bar\nbaz" | frecently view .history --augment --restrict  # with --restrict, we exclusively output entries that appear on stdin+bar+baz+```++#### `dmenu` web searches with history++This bash script shows how easily `frecently` integrates with tools like `dmenu`:++```bash+set -eo pipefail+HISTORY=~/.search-history+QUERY=$(frecently view $HISTORY | dmenu -p "Web search:")++if [[ -n "$QUERY" ]]; then+  frecently bump "$HISTORY" "$QUERY"+  xdg-open "https://www.duckduckgo.com/?q=$QUERY"+fi+```++#### Directory picker++In this more complicated bash script, we use `frecently` and `dmenu` to first ask the user for a directory, and then open a terminal in that directory.+It shows a good use-case for `--augment` (`-a`), augmenting the list with any non-hidden directory at most 2 deep from `$HOME`.++```bash+set -eo pipefail+HISTORY=~/.directory-history+# First, purge non-existent directories from the history+for dir in $(frecently view $HISTORY); do+  if [ ! -d "$dir" ]; then+    echo "Removing $dir"+    frecently delete $HISTORY "$dir"+  fi+done+# Load the history, augmenting it with every non-hidden directory at most 2 deep from $HOME.+DIR=$(find $HOME -maxdepth 2 -type d -not -path '*/.*' | frecently view $HISTORY -a | dmenu -i)+if [ -d $DIR ]; then+  frecently bump $HISTORY "$DIR"+  $TERMCMD -d "$DIR"+fi+```++This kind of script is especially useful when combined with a shell hook that bumps on every directory change, to get `autojump`-like behavior.+For `fish`, that looks like this:++```fish+function __frecently-directory-hook --on-variable PWD --description 'bump current directory in history'+  frecently bump /path/to/history/file "$PWD"+end+```++### Installation++#### Binaries++The [GitHub Action CI](https://github.com/jonascarpay/frecently/actions) builds static binaries for x86 and aarch64, so look for the artifacts there.+If this project gains traction (stars) I'll add proper releases so the artifacts don't get deleted after 90 days.++#### Compiling from source++`frecently` can be built using a Haskell build tool, or using Nix.++Using Cabal, run `cabal build`.++Using Nix, the `flake.nix` file exposes the executable both directly and as an overlay.++### Implementation details++`frecently` works by maintaining three energy levels per entry.+The energy levels decay exponentially, with half-lives of an hour, a day, and a month, respectively.+We `bump` an entry by adding 1 to each of these.++An entry's frecency score is calculated by multiplying each of these three energies by a weight.+The weights default to 720, 30, and 1, for the hourly, daily, and monthly energies, respectively, but can be overridden on the CLI.++Energies are updated _only_ when the history file is used in a `bump` or `touch` command, and when we do, we update every entry's energy simultaneously.+This is invisible to the user, but it ensures that we only need to calculate decay factors once when opening a file, making score calculations very efficient.++When, during an update, an entry's monthly energy drops below the threshold value (defaults to 0.1), it is deleted from the history.+If you don't want items to be deleted, use a threshold of 0.++### Comparison with other tools++This tool was inspired by [frece](https://github.com/YodaEmbedding/frece).+I really like the idea behind `frece`, but I think the execution is more complicated than it needs to be.+`frecently` is both simpler and easier to integrate into CLI applications.

@@ -0,0 +1,238 @@+{-# LANGUAGE DeriveAnyClass #-}+{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE DerivingStrategies #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE TupleSections #-}+{-# OPTIONS_GHC -Wno-unrecognised-pragmas #-}++module Main (main) where++import Control.Monad+import qualified Data.ByteString as BS+import Data.Char (isSpace)+import Data.Foldable (toList)+import Data.List (dropWhileEnd, sortOn)+import Data.Map (Map)+import qualified Data.Map as Map+import Data.Maybe (mapMaybe)+import Data.Serialize (Serialize, decode, encode)+import Data.Word (Word64)+import GHC.Generics (Generic)+import Options.Applicative hiding (str)+import System.AtomicWrite.Writer.ByteString (atomicWriteFile)+import System.Directory (createDirectoryIfMissing, doesFileExist)+import System.Exit (die)+import System.FilePath (takeDirectory)+import System.Process (readProcess)+import Text.Printf (printf)++main :: IO ()+main = do+  now <- getUTC+  cmd <- customExecParser (prefs $ showHelpOnEmpty <> showHelpOnError) (info (pCommand <**> helper) mempty)+  case cmd of+    Bump str thresh fa -> withFrecencies fa $ expire thresh . bump str . decay now+    Delete str fa -> withFrecencies fa $ delete str+    View augmentArgs weights fa -> do+      frecs <- loadFrecencies fa+      fAugment <- augment augmentArgs+      putStr . unlines . view weights . fAugment . decay now $ frecs+    Scores augmentArgs weights fa -> do+      frecs <- loadFrecencies fa+      fAugment <- augment augmentArgs+      printScores weights . fAugment . decay now $ frecs+    Touch expireArgs fileArgs -> do+      withFrecencies fileArgs $ expire expireArgs . decay now++data Command+  = Bump NEString ExpireArgs FileArgs+  | View AugmentArgs Weights FileArgs+  | Delete NEString FileArgs+  | Scores AugmentArgs Weights FileArgs+  | Touch ExpireArgs FileArgs++pCommand :: Parser Command+pCommand =+  subparser $+    command+      "bump"+      ( info+          (withFile $ Bump <$> pStringArg (help "The entry to bump") <*> pExpireArgs)+          ( progDesc "Bump a single entry and update the database"+              <> footer "Bumping adds 1 to the entry's hourly/weekly/monthly energy, updates all entries' energy, and removes entries whose monthly energy has dropped below the threshold."+          )+      )+      <> command+        "view"+        ( info+            (withFile $ View <$> pAugmentArgs <*> pWeights)+            ( progDesc "View the history"+                <> footer "When used with --augment and/or --restrict, this blocks on stdin."+            )+        )+      <> command+        "delete"+        ( info+            (withFile $ Delete <$> pStringArg (help "The entry to delete"))+            (progDesc "Delete an entry from the history")+        )+      <> command+        "scores"+        ( info+            (withFile $ Scores <$> pAugmentArgs <*> pWeights)+            ( progDesc "View score table"+                <> footer "The hourly/daily/weekly energies are presented unweighted. Supports the same stdin functionality as the view command."+            )+        )+      <> command+        "touch"+        ( info+            (withFile $ Touch <$> pExpireArgs)+            (progDesc "Create and/or update a history file")+        )++withFile :: Parser (FileArgs -> a) -> Parser a+withFile inner =+  (\fp a err -> a (FileArgs fp err))+    <$> strArgument (help "History file to use" <> metavar "FILE")+    <*> inner+    <*> flag False True (long "missing-file-error" <> short 'e' <> help "Throw an error if the file is missing, instead of treating it as an empty history")++data FileArgs = FileArgs+  { faPath :: FilePath,+    _faErrorIfMissing :: Bool+    -- TODO different output file+  }++newtype Weights = Weights Energy++pStringArg :: Mod ArgumentFields NEString -> Parser NEString+pStringArg extraInfo = argument (maybeReader $ \str -> guard ('\n' `notElem` str) >> stripWhitespace str) (metavar "KEY" <> extraInfo)++pWeights :: Parser Weights+pWeights =+  fmap Weights $+    Energy+      <$> option auto (short 'h' <> long "hourly" <> metavar "FLOAT" <> help "Hourly energy weight" <> showDefault <> value 720)+      <*> option auto (short 'd' <> long "daily" <> metavar "FLOAT" <> help "Daily energy weight" <> showDefault <> value 30)+      <*> option auto (short 'm' <> long "monthly" <> metavar "FLOAT" <> help "Monthly energy weight" <> showDefault <> value 1)++-- TODO no-decay+newtype ExpireArgs = ExpireArgs {_uaThreshold :: Double}++pExpireArgs :: Parser ExpireArgs+pExpireArgs =+  ExpireArgs+    <$> option+      auto+      ( long "threshold"+          <> short 't'+          <> help "Expiration threshold. Entries with a monthly energy below this will be removed."+          <> metavar "FLOAT"+          <> value 0.1+          <> showDefault+      )++readInput :: IO [NEString]+readInput = mapMaybe stripWhitespace . lines <$> getContents++augment :: AugmentArgs -> IO (Frecencies -> Frecencies)+augment (AugmentArgs False False) = pure id+augment (AugmentArgs aug res) = do+  strs <- readInput+  pure $ \(Frecencies t fs) -> Frecencies t $+    case (aug, res) of+      (False, True) -> Map.fromList [(str, nrg) | str <- strs, nrg <- toList (Map.lookup str fs)]+      (True, False) -> foldr (\str -> Map.insertWith (<>) str (Energy 0 0 0)) fs strs+      (True, True) -> Map.fromList $ (\key -> maybe (key, Energy 0 0 0) (key,) (Map.lookup key fs)) <$> strs+      _ -> fs++data AugmentArgs = AugmentArgs+  { _aaAugment :: Bool,+    _aaRestrict :: Bool+  }++pAugmentArgs :: Parser AugmentArgs+pAugmentArgs =+  AugmentArgs+    <$> flag False True (long "augment" <> short 'a' <> help "Augment the output with entries read from stdin, treating them as if they had a score of 0 if they are not present in the history")+    <*> flag False True (long "restrict" <> short 'r' <> help "Only output entries present in the keys read from stdin")++type Time = Word64++newtype NEString = NEString {unNEString :: String}+  deriving newtype (Eq, Ord, Show, Serialize)++getUTC :: IO Time+getUTC = read <$> readProcess "date" ["+%s"] ""++data Energy = Energy+  { hourly :: Double,+    daily :: Double,+    monthly :: Double+  }+  deriving stock (Generic)+  deriving anyclass (Serialize)++instance Semigroup Energy where Energy h d m <> Energy h' d' m' = Energy (h + h') (d + d') (m + m')++data Frecencies = Frecencies+  { lastUpdate :: Time,+    energies :: Map NEString Energy+  }+  deriving stock (Generic)+  deriving anyclass (Serialize)++decay :: Time -> Frecencies -> Frecencies+decay tNow (Frecencies tUpdate frecs) = Frecencies tNow (fmap f frecs)+  where+    f (Energy h d m) = Energy (alphaHour * h) (alphaDay * d) (alphaMonth * m)+    deltaSec = fromIntegral $ tNow - tUpdate+    alphaHour = 0.5 ** (deltaSec / 3600)+    alphaDay = 0.5 ** (deltaSec / 86400)+    alphaMonth = 0.5 ** (deltaSec / 2592000)++score :: Weights -> Energy -> Double+score (Weights (Energy wh wd wm)) (Energy h d m) = wh * h + wd * d + wm * m++delete :: NEString -> Frecencies -> Frecencies+delete str (Frecencies t fs) = Frecencies t (Map.delete str fs)++expire :: ExpireArgs -> Frecencies -> Frecencies+expire (ExpireArgs threshold) (Frecencies t fs) = Frecencies t (Map.filter ((> threshold) . monthly) fs)++-- TODO bump by 1 1 1, multiply scores before presenting+bump :: NEString -> Frecencies -> Frecencies+bump str (Frecencies t fs) = Frecencies t (Map.insertWith (<>) str (Energy 1 1 1) fs)++view :: Weights -> Frecencies -> [String]+view weights = fmap (unNEString . fst) . sortOn (negate . score weights . snd) . Map.toList . energies++stripWhitespace :: String -> Maybe NEString+stripWhitespace str = if null str' then Nothing else Just (NEString str)+  where+    str' = (dropWhileEnd isSpace . dropWhile isSpace) str++loadFrecencies :: FileArgs -> IO Frecencies+loadFrecencies (FileArgs fp errorOnMissing) = do+  exists <- doesFileExist fp+  if exists+    then BS.readFile fp >>= either die pure . decode+    else+      if errorOnMissing+        then die $ "Error: missing history file " <> fp+        else pure $ Frecencies 0 mempty++withFrecencies :: FileArgs -> (Frecencies -> Frecencies) -> IO ()+withFrecencies fa f = loadFrecencies fa >>= writeFrecencies (faPath fa) . f++writeFrecencies :: FilePath -> Frecencies -> IO ()+writeFrecencies path fs = do+  createDirectoryIfMissing True (takeDirectory path)+  atomicWriteFile path (encode fs)++printScores :: Weights -> Frecencies -> IO ()+printScores weights (Frecencies _ fs) = do+  printf "weighted score\thourly\t\tdaily\t\tmonthly\n"+  forM_ (sortOn (negate . score weights . snd) $ Map.toList fs) $ \(str, nrg@(Energy h d m)) ->+    printf "%12.6f\t%10.6f\t%10.6f\t%10.6f\t%s\n" (score weights nrg) h d m (unNEString str)

@@ -0,0 +1,46 @@+cabal-version:   2.4+name:            frecently+version:         1.0+license:         BSD-3-Clause+build-type:      Simple+license-file:    LICENSE+category:        Command Line Tools+author:          Jonas Carpay+maintainer:      Jonas Carpay <jonascarpay@gmail.com>+copyright:       2022 Jonas Carpay+tested-with:     GHC ==8.6.3 || ==8.8.3 || ==8.10.5+synopsis:        CLI frecency history+extra-doc-files:+  CHANGELOG.md+  README.md++description:+  Extremely simple CLI frecency histories.+  Intended for adding frecency to tools like dmenu / rofi / fzf.+  See the README for more information.++homepage:        https://github.com/jonascarpay/frecently#readme++source-repository head+  type:     git+  location: git://github.com/jonascarpay/frecently.git++executable frecently+  hs-source-dirs:   app+  main-is:          Main.hs+  build-depends:+    , atomic-write+    , base                  >=4.9 && <5+    , bytestring+    , cereal+    , containers+    , directory+    , filepath+    , optparse-applicative+    , process++  default-language: Haskell2010+  ghc-options:+    -Wall -Wcompat -Widentities -Wincomplete-uni-patterns+    -Wincomplete-record-updates -Wredundant-constraints+    -fhide-source-paths -Wpartial-fields -O2