hledger-lib 1.19.1 → 1.20

raw patch · 45 files changed

+2757/−1905 lines, 45 filesdep +pretty-simpledep −fgldep −pretty-showPVP ok

version bump matches the API change (PVP)

Dependencies added: pretty-simple

Dependencies removed: fgl, pretty-show

API changes (from Hackage documentation)

Files

• CHANGES.md +124/−0
• Hledger/Data/Account.hs +1/−3
• Hledger/Data/Amount.hs +172/−86
• Hledger/Data/Dates.hs +25/−10
• Hledger/Data/Journal.hs +46/−22
• Hledger/Data/Posting.hs +14/−1
• Hledger/Data/RawOptions.hs +6/−1
• Hledger/Data/StringFormat.hs +48/−27
• Hledger/Data/Transaction.hs +12/−3
• Hledger/Data/Types.hs +12/−2
• Hledger/Data/Valuation.hs +133/−106
• Hledger/Query.hs +24/−34
• Hledger/Read.hs +0/−1
• Hledger/Read/Common.hs +203/−75
• Hledger/Read/CsvReader.hs +92/−65
• Hledger/Read/JournalReader.hs +0/−33
• Hledger/Reports.hs +0/−1
• Hledger/Reports/AccountTransactionsReport.hs +5/−6
• Hledger/Reports/BalanceReport.hs +19/−18
• Hledger/Reports/BudgetReport.hs +108/−50
• Hledger/Reports/EntriesReport.hs +8/−9
• Hledger/Reports/MultiBalanceReport.hs +185/−176
• Hledger/Reports/PostingsReport.hs +29/−30
• Hledger/Reports/ReportOptions.hs +153/−196
• Hledger/Reports/ReportTypes.hs +12/−14
• Hledger/Utils.hs +0/−3
• Hledger/Utils/Debug.hs +19/−8
• Hledger/Utils/Regex.hs +12/−3
• Hledger/Utils/String.hs +33/−88
• Hledger/Utils/Text.hs +1/−58
• Text/Tabular/AsciiWide.hs +130/−52
• Text/WideString.hs +71/−0
• hledger-lib.cabal +6/−8
• hledger_csv.5 +96/−21
• hledger_csv.info +144/−100
• hledger_csv.txt +173/−136
• hledger_journal.5 +97/−38
• hledger_journal.info +252/−179
• hledger_journal.txt +281/−231
• hledger_timeclock.5 +1/−1
• hledger_timeclock.info +2/−2
• hledger_timeclock.txt +1/−1
• hledger_timedot.5 +2/−2
• hledger_timedot.info +3/−3
• hledger_timedot.txt +2/−2

@@ -1,6 +1,130 @@ Internal/api/developer-ish changes in the hledger-lib (and hledger) packages. For user-visible changes, see the hledger package changelog. +# 1.20 2020-12-05++++- added: journalApplyAliases, transactionApplyAliases, postingApplyAliases++- a new more robust price lookup implementation, fgl library dropped (#1402)++- Reverted a stripAnsi change in 1.19.1 that caused a 3x slowdown of amount rendering +  in terminal reports. (#1350)++- Amount and table rendering has been improved, so that stripAnsi is no longer needed.+  This speeds up amount rendering in the terminal, speeding up some reports by 10% or more since 1.19.+  (Stephen Morgan)++- global commodity display styles can now be set in InputOpts or Journal,+  overriding all others (declared or inferred). This is used by the import+  command and probably command-line options in future.++- Journal keeps a new piece of parsing state, a decimal mark character,+  which can optionally be set to force the number format expected by all+  amount parsers.++- Remove Empty Query constructor, which does nothing and has done so for a very long time. (Stephen Morgan)++- In ReportOpts, store query terms term-by-term in a list in querystring_. (Stephen Morgan)+  This helps deal with tricky quoting issues, as we no longer have to make+  sure everything is quoted properly before merging it into a string.++- Implement concat(Top|Bottom)Padded in terms of renderRow, allowing them to be width aware. (Stephen Morgan)++- Expand Tabular.AsciiWide to allow multiline, custom-width,+  vertically/horizontally-aligned cells, and optional table borders.+  (Stephen Morgan)++- Introduce showMixed*Unnormalised, eliminate most direct calls of strWidth. (Stephen Morgan)++- showMixedAmountElided now makes better use of space, showing as many+  Amounts possible as long as they and the elision string fit within+  32 characters. (Stephen Morgan)++- Add Functor instance for CompoundPeriodicReport. (Stephen Morgan)++- Generalise CBCSubreportSpec to allow more subreport control. (Stephen Morgan)++- Export some MultiBalanceReport helper functions. (Stephen Morgan)++- Make Default instances clearer, remove Default instance for Bool. (Stephen Morgan)++- Many ReportOpts-related changes, such as the addition of ReportSpec, aimed+  at preventing runtime errors (from parsing: regexps, dates, format strings;+  from not having today's date set; etc.)+  ReportSpec holds a ReportOpts, the day of the report, and the Query generated from these.++- StringFormat now takes an optional overline width, which is+  currently only used by defaultBalanceLineFormat. (Stephen Morgan)++- quoteIfNeeded should not escape the backslashes in unicode code points. (Stephen Morgan)++- Export OrdPlus and constructors. (Stephen Morgan)++- Debug output now uses pretty-simple instead pretty-show.+  This hopefully gives overall nicer debug output (eg in colour), +  including for values which don't have Read-able Show output.+  This means that we can start removing custom Show instances +  that were a workaround for pretty-show. Eg account names+  in debug output no longer show their colons as underscores.++  Here's some old pretty-show output:++   CsvRules+     { rdirectives = [ ( "skip" , "1" ) ]+     , rcsvfieldindexes = [ ( "date" , 1 ) , ( "amount" , 2 ) ]+     , rassignments = [ ( "amount" , "%2" ) , ( "date" , "%1" ) ]+     , rconditionalblocks = []+     }++  And the new pretty-simple output:++   CsvRules+     { rdirectives=+       [ ( "skip", "1" ) ]+     , rcsvfieldindexes=+       [ ( "date", 1 ), ( "amount", 2 ) ]+     , rassignments=+       [ ( "amount", "%2" ), ( "date", "%1" ) ]+     , rconditionalblocks= []+     }++  We require pretty-simple 4.0.0.0 to get this compact output.+  It's a little less compact than pretty-show, but not too bad.+  Non-compact pretty-simple output would be:++   CsvRules+       { rdirectives=+           [+               ( "skip"+               , "1B"+               )+           ]+       , rcsvfieldindexes=+           [+               ( "date"+               , 1+               )+           ,+               ( "amount"+               , 2+               )+           ]+       , rassignments=+           [+               ( "amount"+               , "%2"+               )+           ,+               ( "date"+               , "%1"+               )+           ]+       , rconditionalblocks=[]+       }++ # 1.19.1 2020-09-07  - Allow megaparsec 9

@@ -14,7 +14,6 @@ import Data.Maybe (fromMaybe) import Data.Ord (Down(..)) import qualified Data.Map as M-import qualified Data.Text as T import Safe (headMay, lookupJustDef) import Text.Printf @@ -28,12 +27,11 @@ -- deriving instance Show Account instance Show Account where     show Account{..} = printf "Account %s (boring:%s, postings:%d, ebalance:%s, ibalance:%s)"-                       (T.map colonToUnderscore aname)  -- hide : so pretty-show doesn't break line+                       aname                        (if aboring then "y" else "n" :: String)                        anumpostings                        (showMixedAmount aebalance)                        (showMixedAmount aibalance)-      where colonToUnderscore x = if x == ':' then '_' else x  instance Eq Account where   (==) a b = aname a == aname b -- quick equality test for speed

@@ -69,6 +69,7 @@   amountstyle,   styleAmount,   styleAmountExceptPrecision,+  amountUnstyled,   showAmount,   cshowAmount,   showAmountWithZeroCommodity,@@ -108,6 +109,7 @@   mixedAmountTotalPriceToUnitPrice,   -- ** rendering   styleMixedAmount,+  mixedAmountUnstyled,   showMixedAmount,   showMixedAmountOneLine,   showMixedAmountDebug,@@ -116,6 +118,10 @@   showMixedAmountElided,   showMixedAmountWithZeroCommodity,   showMixedAmountWithPrecision,+  showMixed,+  showMixedUnnormalised,+  showMixedOneLine,+  showMixedOneLineUnnormalised,   setMixedAmountPrecision,   canonicaliseMixedAmount,   -- * misc.@@ -127,14 +133,15 @@ import Data.Char (isDigit) import Data.Decimal (decimalPlaces, normalizeDecimal, roundTo) import Data.Function (on)-import Data.List+import Data.List (genericSplitAt, groupBy, intercalate, mapAccumL,+                  partition, sortBy) import qualified Data.Map as M import Data.Map (findWithDefault)-import Data.Maybe+import Data.Maybe (fromMaybe) import qualified Data.Text as T import Data.Word (Word8)-import Safe (lastDef, maximumMay)-import Text.Printf+import Safe (lastDef, lastMay)+import Text.Printf (printf)  import Hledger.Data.Types import Hledger.Data.Commodity@@ -277,11 +284,6 @@ amountIsZero :: Amount -> Bool amountIsZero Amount{aquantity=q} = q == 0 --- | Get the string representation of an amount, based on its commodity's--- display settings except using the specified precision.-showAmountWithPrecision :: AmountPrecision -> Amount -> String-showAmountWithPrecision p = showAmount . setAmountPrecision p- -- | Set an amount's display precision, flipped. withPrecision :: Amount -> AmountPrecision -> Amount withPrecision = flip setAmountPrecision@@ -300,19 +302,6 @@     displayprecision = asprecision $ astyle a     naturalprecision = Precision . decimalPlaces . normalizeDecimal $ aquantity a --- | Get a string representation of an amount for debugging,--- appropriate to the current debug level. 9 shows maximum detail.-showAmountDebug :: Amount -> String-showAmountDebug Amount{acommodity="AUTO"} = "(missing)"-showAmountDebug Amount{..} = printf "Amount {acommodity=%s, aquantity=%s, aprice=%s, astyle=%s}" (show acommodity) (show aquantity) (showAmountPriceDebug aprice) (show astyle)---- | Get the string representation of an amount, without any \@ price.--- With a True argument, adds ANSI codes to show negative amounts in red.-showAmountWithoutPrice :: Bool -> Amount -> String-showAmountWithoutPrice c a = showamt a{aprice=Nothing}-  where-    showamt = if c then cshowAmount else showAmount- -- | Set an amount's internal precision, ie rounds the Decimal representing -- the amount's quantity to some number of decimal places. -- Rounding is done with Data.Decimal's default roundTo function:@@ -364,6 +353,10 @@     Just s  -> a{astyle=s{asprecision=origp}}     Nothing -> a +-- | Reset this amount's display style to the default.+amountUnstyled :: Amount -> Amount+amountUnstyled a = a{astyle=amountstyle}+ -- | Get the string representation of an amount, based on its -- commodity's display settings. String representations equivalent to -- zero are converted to just \"0\". The special "missing" amount is@@ -374,10 +367,18 @@ -- | Colour version. For a negative amount, adds ANSI codes to change the colour, -- currently to hard-coded red. cshowAmount :: Amount -> String-cshowAmount a =-  (if isNegativeAmount a then color Dull Red else id) $-  showAmountHelper False a+cshowAmount a = (if isNegativeAmount a then color Dull Red else id) $+                showAmountHelper False a +-- | Get the string representation of an amount, without any \@ price.+showAmountWithoutPrice :: Amount -> String+showAmountWithoutPrice a = showAmount a{aprice=Nothing}++-- | Get the string representation of an amount, based on its commodity's+-- display settings except using the specified precision.+showAmountWithPrecision :: AmountPrecision -> Amount -> String+showAmountWithPrecision p = showAmount . setAmountPrecision p+ showAmountHelper :: Bool -> Amount -> String showAmountHelper _ Amount{acommodity="AUTO"} = "" showAmountHelper showzerocommodity a@Amount{acommodity=c, aprice=mp, astyle=AmountStyle{..}} =@@ -395,6 +396,12 @@ showAmountWithZeroCommodity :: Amount -> String showAmountWithZeroCommodity = showAmountHelper True +-- | Get a string representation of an amount for debugging,+-- appropriate to the current debug level. 9 shows maximum detail.+showAmountDebug :: Amount -> String+showAmountDebug Amount{acommodity="AUTO"} = "(missing)"+showAmountDebug Amount{..} = printf "Amount {acommodity=%s, aquantity=%s, aprice=%s, astyle=%s}" (show acommodity) (show aquantity) (showAmountPriceDebug aprice) (show astyle)+ -- | Get the string representation of the number part of of an amount, -- using the display settings from its commodity. showamountquantity :: Amount -> String@@ -608,95 +615,174 @@ styleMixedAmount :: M.Map CommoditySymbol AmountStyle -> MixedAmount -> MixedAmount styleMixedAmount styles (Mixed as) = Mixed $ map (styleAmount styles) as +-- | Reset each individual amount's display style to the default.+mixedAmountUnstyled :: MixedAmount -> MixedAmount+mixedAmountUnstyled = mapMixedAmount amountUnstyled+ -- | Get the string representation of a mixed amount, after -- normalising it to one amount per commodity. Assumes amounts have -- no or similar prices, otherwise this can show misleading prices. showMixedAmount :: MixedAmount -> String-showMixedAmount = showMixedAmountHelper False False---- | Like showMixedAmount, but zero amounts are shown with their--- commodity if they have one.-showMixedAmountWithZeroCommodity :: MixedAmount -> String-showMixedAmountWithZeroCommodity = showMixedAmountHelper True False+showMixedAmount = fst . showMixed showAmount Nothing Nothing False  -- | Get the one-line string representation of a mixed amount. showMixedAmountOneLine :: MixedAmount -> String-showMixedAmountOneLine = showMixedAmountHelper False True--showMixedAmountHelper :: Bool -> Bool -> MixedAmount -> String-showMixedAmountHelper showzerocommodity useoneline m =-  join $ map showamt $ amounts $ normaliseMixedAmountSquashPricesForDisplay m-  where-    join | useoneline = intercalate ", "-         | otherwise  = vConcatRightAligned-    showamt | showzerocommodity = showAmountWithZeroCommodity-            | otherwise         = showAmount---- | Compact labelled trace of a mixed amount, for debugging.-ltraceamount :: String -> MixedAmount -> MixedAmount-ltraceamount s = traceWith (((s ++ ": ") ++).showMixedAmount)+showMixedAmountOneLine = fst . showMixedOneLine showAmountWithoutPrice Nothing Nothing False --- | Set the display precision in the amount's commodities.-setMixedAmountPrecision :: AmountPrecision -> MixedAmount -> MixedAmount-setMixedAmountPrecision p (Mixed as) = Mixed $ map (setAmountPrecision p) as+-- | Like showMixedAmount, but zero amounts are shown with their+-- commodity if they have one.+showMixedAmountWithZeroCommodity :: MixedAmount -> String+showMixedAmountWithZeroCommodity = fst . showMixed showAmountWithZeroCommodity Nothing Nothing False  -- | Get the string representation of a mixed amount, showing each of its -- component amounts with the specified precision, ignoring their -- commoditys' display precision settings. showMixedAmountWithPrecision :: AmountPrecision -> MixedAmount -> String-showMixedAmountWithPrecision p m =-    vConcatRightAligned $ map (showAmountWithPrecision p) $ amounts $ normaliseMixedAmountSquashPricesForDisplay m+showMixedAmountWithPrecision p = fst . showMixed (showAmountWithPrecision p) Nothing Nothing False +-- | Get the string representation of a mixed amount, without showing any transaction prices.+-- With a True argument, adds ANSI codes to show negative amounts in red.+showMixedAmountWithoutPrice :: Bool -> MixedAmount -> String+showMixedAmountWithoutPrice c = fst . showMixed showAmountWithoutPrice Nothing Nothing c++-- | Get the one-line string representation of a mixed amount, but without+-- any \@ prices.+-- With a True argument, adds ANSI codes to show negative amounts in red.+showMixedAmountOneLineWithoutPrice :: Bool -> MixedAmount -> String+showMixedAmountOneLineWithoutPrice c = fst . showMixedOneLine showAmountWithoutPrice Nothing Nothing c++-- | Like showMixedAmountOneLineWithoutPrice, but show at most the given width,+-- with an elision indicator if there are more.+-- With a True argument, adds ANSI codes to show negative amounts in red.+showMixedAmountElided :: Int -> Bool -> MixedAmount -> String+showMixedAmountElided w c = fst . showMixedOneLine showAmountWithoutPrice Nothing (Just w) c+ -- | Get an unambiguous string representation of a mixed amount for debugging. showMixedAmountDebug :: MixedAmount -> String showMixedAmountDebug m | m == missingmixedamt = "(missing)"                        | otherwise       = printf "Mixed [%s]" as     where as = intercalate "\n       " $ map showAmountDebug $ amounts m --- TODO these and related fns are comically complicated:+-- | General function to display a MixedAmount, one Amount on each line.+-- It takes a function to display each Amount, an optional minimum width+-- to pad to, an optional maximum width to display, and a Bool to determine+-- whether to colourise negative numbers. Amounts longer than the maximum+-- width (if given) will be elided. The function also returns the actual+-- width of the output string.+showMixed :: (Amount -> String) -> Maybe Int -> Maybe Int -> Bool -> MixedAmount -> (String, Int)+showMixed showamt mmin mmax c =+    showMixedUnnormalised showamt mmin mmax c . normaliseMixedAmountSquashPricesForDisplay --- | Get the string representation of a mixed amount, without showing any transaction prices.--- With a True argument, adds ANSI codes to show negative amounts in red.-showMixedAmountWithoutPrice :: Bool -> MixedAmount -> String-showMixedAmountWithoutPrice c m = intercalate "\n" $ map showamt as+-- | Like showMixed, but does not normalise the MixedAmount before displaying.+showMixedUnnormalised :: (Amount -> String) -> Maybe Int -> Maybe Int -> Bool -> MixedAmount -> (String, Int)+showMixedUnnormalised showamt mmin mmax c (Mixed as) =+    (intercalate "\n" $ map finalise elided, width)   where-    Mixed as = normaliseMixedAmountSquashPricesForDisplay $ mixedAmountStripPrices m-    showamt a =-      (if c && isNegativeAmount a then color Dull Red else id) $-      printf (printf "%%%ds" width) $ showAmountWithoutPrice c a+    width    = maximum $ fromMaybe 0 mmin : map adLength elided+    astrs    = amtDisplayList sepwidth showamt as+    sepwidth = 0  -- "\n" has width 0++    finalise = adString . pad . if c then colourise else id+    pad amt = amt{ adString = applyN (width - adLength amt) (' ':) $ adString amt+                 , adLength = width+                 }++    elided = maybe id elideTo mmax astrs+    elideTo m xs = maybeAppend elisionStr short       where-        width = fromMaybe 0 . maximumMay $ map (length . showAmount) as+        elisionStr = elisionDisplay (Just m) sepwidth (length long) $ lastDef nullAmountDisplay short+        (short, long) = partition ((m>=) . adLength) xs -mixedAmountStripPrices :: MixedAmount -> MixedAmount-mixedAmountStripPrices (Mixed as) = Mixed $ map (\a -> a{aprice=Nothing}) as+-- | General function to display a MixedAmount on a single line. It+-- takes a function to display each Amount, an optional minimum width to+-- pad to, an optional maximum width to display, and a Bool to determine+-- whether to colourise negative numbers. It will display as many Amounts+-- as it can in the maximum width (if given), and further Amounts will be+-- elided. The function also returns the actual width of the output string.+showMixedOneLine :: (Amount -> String) -> Maybe Int -> Maybe Int -> Bool -> MixedAmount -> (String, Int)+showMixedOneLine showamt mmin mmax c =+    showMixedOneLineUnnormalised showamt mmin mmax c . normaliseMixedAmountSquashPricesForDisplay --- | Get the one-line string representation of a mixed amount, but without--- any \@ prices.--- With a True argument, adds ANSI codes to show negative amounts in red.-showMixedAmountOneLineWithoutPrice :: Bool -> MixedAmount -> String-showMixedAmountOneLineWithoutPrice c m =-  intercalate ", " $ map (showAmountWithoutPrice c) as-    where-      (Mixed as) = normaliseMixedAmountSquashPricesForDisplay $ stripPrices m-      stripPrices (Mixed as) = Mixed $ map stripprice as where stripprice a = a{aprice=Nothing}+-- | Like showMixedOneLine, but does not normalise the MixedAmount before+-- displaying.+showMixedOneLineUnnormalised :: (Amount -> String) -> Maybe Int -> Maybe Int -> Bool -> MixedAmount -> (String, Int)+showMixedOneLineUnnormalised showamt mmin mmax c (Mixed as) =+    (pad . intercalate ", " $ map finalise elided, max width $ fromMaybe 0 mmin)+  where+    width    = maybe 0 adTotal $ lastMay elided+    astrs    = amtDisplayList sepwidth showamt as+    sepwidth = 2  -- ", " has width 2+    n        = length as --- | Like showMixedAmountOneLineWithoutPrice, but show at most two commodities,--- with a elision indicator if there are more.--- With a True argument, adds ANSI codes to show negative amounts in red.-showMixedAmountElided :: Bool -> MixedAmount -> String-showMixedAmountElided c m = intercalate ", " $ take 2 astrs ++ elisionstr+    finalise = adString . if c then colourise else id+    pad = applyN (fromMaybe 0 mmin - width) (' ':)++    elided = maybe id elideTo mmax astrs+    elideTo m = addElide . takeFitting m . withElided+    -- Add the last elision string to the end of the display list+    addElide [] = []+    addElide xs = maybeAppend (snd $ last xs) $ map fst xs+    -- Return the elements of the display list which fit within the maximum width+    -- (including their elision strings)+    takeFitting m = dropWhileRev (\(a,e) -> m < adTotal (fromMaybe a e))+    dropWhileRev p = foldr (\x xs -> if null xs && p x then [] else x:xs) []++    -- Add the elision strings (if any) to each amount+    withElided = zipWith (\num amt -> (amt, elisionDisplay Nothing sepwidth num amt)) [n-1,n-2..0]++data AmountDisplay = AmountDisplay+  { adAmount :: !Amount  -- ^ Amount displayed+  , adString :: !String  -- ^ String representation of the Amount+  , adLength :: !Int     -- ^ Length of the string representation+  , adTotal  :: !Int     -- ^ Cumulative length of MixedAmount this Amount is part of,+                         --   including separators+  } deriving (Show)++nullAmountDisplay :: AmountDisplay+nullAmountDisplay = AmountDisplay nullamt "" 0 0++amtDisplayList :: Int -> (Amount -> String) -> [Amount] -> [AmountDisplay]+amtDisplayList sep showamt = snd . mapAccumL display (-sep)   where-    astrs = map (showAmountWithoutPrice c) as-      where-        (Mixed as) = normaliseMixedAmountSquashPricesForDisplay $ stripPrices m-          where-            stripPrices (Mixed as) = Mixed $ map stripprice as-              where-                stripprice a = a{aprice=Nothing}-    elisionstr | n > 2     = [show (n - 2) ++ " more.."]-               | otherwise = []+    display tot amt = (tot', AmountDisplay amt str width tot')       where-        n = length astrs+        str  = showamt amt+        width  = strWidth str+        tot' = tot + width + sep++-- The string "m more", added to the previous running total+elisionDisplay :: Maybe Int -> Int -> Int -> AmountDisplay -> Maybe AmountDisplay+elisionDisplay mmax sep n lastAmt+  | n > 0     = Just $ AmountDisplay 0 str len (adTotal lastAmt + len)+  | otherwise = Nothing+  where+    fullString = show n ++ " more.."+    -- sep from the separator, 7 from " more..", 1 + floor (logBase 10 n) from number+    fullLength = sep + 8 + floor (logBase 10 $ fromIntegral n)++    str | Just m <- mmax, fullLength > m = take (m - 2) fullString ++ ".."+        | otherwise                      = fullString+    len = case mmax of Nothing -> fullLength+                       Just m  -> max 2 $ min m fullLength++maybeAppend :: Maybe a -> [a] -> [a]+maybeAppend Nothing  = id+maybeAppend (Just a) = (++[a])++colourise :: AmountDisplay -> AmountDisplay+colourise amt = amt{adString=markColour $ adString amt}+  where markColour = if isNegativeAmount (adAmount amt) then color Dull Red else id++-- | Compact labelled trace of a mixed amount, for debugging.+ltraceamount :: String -> MixedAmount -> MixedAmount+ltraceamount s = traceWith (((s ++ ": ") ++).showMixedAmount)++-- | Set the display precision in the amount's commodities.+setMixedAmountPrecision :: AmountPrecision -> MixedAmount -> MixedAmount+setMixedAmountPrecision p (Mixed as) = Mixed $ map (setAmountPrecision p) as++mixedAmountStripPrices :: MixedAmount -> MixedAmount+mixedAmountStripPrices (Mixed as) = Mixed $ map (\a -> a{aprice=Nothing}) as  -- | Canonicalise a mixed amount's display styles using the provided commodity style map. canonicaliseMixedAmount :: M.Map CommoditySymbol AmountStyle -> MixedAmount -> MixedAmount

@@ -1,9 +1,10 @@-{-# LANGUAGE NoMonoLocalBinds #-}-{-# LANGUAGE FlexibleContexts #-}-{-# LANGUAGE TypeFamilies #-}-{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE FlexibleContexts    #-}+{-# LANGUAGE NoMonoLocalBinds    #-}+{-# LANGUAGE OverloadedStrings   #-}+{-# LANGUAGE PackageImports      #-} {-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE PackageImports #-}+{-# LANGUAGE TupleSections       #-}+{-# LANGUAGE TypeFamilies        #-} {-|  Date parsing and utilities for hledger.@@ -110,7 +111,6 @@ -- Help ppShow parse and line-wrap DateSpans better in debug output. instance Show DateSpan where     show s = "DateSpan " ++ showDateSpan s-    -- show s = "DateSpan \"" ++ showDateSpan s ++ "\"" -- quotes to help pretty-show  showDate :: Day -> String showDate = show@@ -777,10 +777,17 @@   failIfInvalidDate date  ymd :: TextParser m SmartDate-ymd = liftA2 SmartAssumeStart yearp (optional $ try monthday) >>= failIfInvalidDate-  where monthday = do-            sep <- datesepchar-            liftA2 (,) decimal . optional $ char sep *> decimal+ymd = do+    y <- yearp+    fmap (SmartAssumeStart y) . optional . try $ do+        sep <- datesepchar+        m <- decimal+        unless (validMonth m) $ Fail.fail ("Bad month " <> show m)+        fmap (m,) . optional . try $ do+            _ <- char sep+            d <- decimal+            failIfInvalidDate $ SmartAssumeStart y (Just (m, Just d))+            return d  md :: TextParser m (Month, MonthDay) md = do@@ -943,6 +950,14 @@ -- | -- >>> parsewith (doubledatespanp (fromGregorian 2018 01 01) <* eof) "20180101-201804" -- Right DateSpan 2018Q1+-- >>> parsewith (doubledatespanp (fromGregorian 2018 01 01) <* eof) "2017..2018"+-- Right DateSpan 2017+-- >>> parsewith (doubledatespanp (fromGregorian 2018 01 01) <* eof) "2017-2018"+-- Right DateSpan 2017+-- >>> parsewith (doubledatespanp (fromGregorian 2018 01 01) <* eof) "2017-01-2018"+-- Right DateSpan 2017+-- >>> parsewith (doubledatespanp (fromGregorian 2018 01 01) <* eof) "2017-01-01-2018"+-- Right DateSpan 2017 doubledatespanp :: Day -> TextParser m DateSpan doubledatespanp rdate = liftA2 fromToSpan     (optional (string' "from" *> skipNonNewlineSpaces) *> smartdate)

@@ -52,6 +52,7 @@   -- overJournalAmounts,   -- traverseJournalAmounts,   -- journalCanonicalCommodities,+  journalCommoditiesDeclared,   journalDateSpan,   journalStartDate,   journalEndDate,@@ -79,6 +80,7 @@   journalNumberAndTieTransactions,   journalUntieTransactions,   journalModifyTransactions,+  journalApplyAliases,   -- * Tests   samplejournal,   tests_Journal,@@ -175,6 +177,7 @@   j1 <> j2 = Journal {      jparsedefaultyear          = jparsedefaultyear          j2     ,jparsedefaultcommodity     = jparsedefaultcommodity     j2+    ,jparsedecimalmark          = jparsedecimalmark          j2     ,jparseparentaccounts       = jparseparentaccounts       j2     ,jparsealiases              = jparsealiases              j2     -- ,jparsetransactioncount     = jparsetransactioncount     j1 +  jparsetransactioncount     j2@@ -182,6 +185,7 @@     ,jincludefilestack          = jincludefilestack j2     ,jdeclaredaccounts          = jdeclaredaccounts          j1 <> jdeclaredaccounts          j2     ,jdeclaredaccounttypes      = jdeclaredaccounttypes      j1 <> jdeclaredaccounttypes      j2+    ,jglobalcommoditystyles     = jglobalcommoditystyles     j1 <> jglobalcommoditystyles     j2     ,jcommodities               = jcommodities               j1 <> jcommodities               j2     ,jinferredcommodities       = jinferredcommodities       j1 <> jinferredcommodities       j2     ,jpricedirectives           = jpricedirectives           j1 <> jpricedirectives           j2@@ -201,6 +205,7 @@ nulljournal = Journal {    jparsedefaultyear          = Nothing   ,jparsedefaultcommodity     = Nothing+  ,jparsedecimalmark          = Nothing   ,jparseparentaccounts       = []   ,jparsealiases              = []   -- ,jparsetransactioncount     = 0@@ -208,6 +213,7 @@   ,jincludefilestack          = []   ,jdeclaredaccounts          = []   ,jdeclaredaccounttypes      = M.empty+  ,jglobalcommoditystyles     = M.empty   ,jcommodities               = M.empty   ,jinferredcommodities       = M.empty   ,jpricedirectives           = []@@ -263,6 +269,10 @@ journalPostings :: Journal -> [Posting] journalPostings = concatMap tpostings . jtxns +-- | Sorted unique commodity symbols declared by commodity directives in this journal.+journalCommoditiesDeclared :: Journal -> [AccountName]+journalCommoditiesDeclared = nubSort . M.keys . jcommodities+ -- | Sorted unique account names posted to by this journal's transactions. journalAccountNamesUsed :: Journal -> [AccountName] journalAccountNamesUsed = accountNamesFromPostings . journalPostings@@ -562,7 +572,7 @@     ,jtxns             = reverse $ jtxns j     ,jtxnmodifiers     = reverse $ jtxnmodifiers j     ,jperiodictxns     = reverse $ jperiodictxns j-    ,jpricedirectives     = reverse $ jpricedirectives j+    ,jpricedirectives  = reverse $ jpricedirectives j     }  -- | Set this journal's last read time, ie when its files were last read.@@ -608,7 +618,7 @@ -- WARN tricky algorithm and code ahead.  -- -- Code overview as of 20190219, this could/should be simplified/documented more:---  parseAndFinaliseJournal['] (Cli/Utils.hs), journalAddForecast (Common.hs), budgetJournal (BudgetReport.hs), tests (BalanceReport.hs)+--  parseAndFinaliseJournal['] (Cli/Utils.hs), journalAddForecast (Common.hs), journalAddBudgetGoalTransactions (BudgetReport.hs), tests (BalanceReport.hs) --   journalBalanceTransactions --    runST --     runExceptT@@ -938,10 +948,8 @@ --  -- | Choose and apply a consistent display style to the posting--- amounts in each commodity. Each commodity's style is specified by a--- commodity (or D) directive, or otherwise inferred from posting--- amounts. Can return an error message eg if inconsistent number--- formats are found.+-- amounts in each commodity (see journalCommodityStyles).+-- Can return an error message eg if inconsistent number formats are found. journalApplyCommodityStyles :: Journal -> Either String Journal journalApplyCommodityStyles j@Journal{jtxns=ts, jpricedirectives=pds} =   case journalInferCommodityStyles j of@@ -958,18 +966,20 @@         fixbalanceassertion ba = ba{baamount=styleAmount styles $ baamount ba}         fixpricedirective pd@PriceDirective{pdamount=a} = pd{pdamount=styleAmountExceptPrecision styles a} --- | Get the canonical amount styles for this journal, whether--- declared by commodity directives, by the last default commodity (D)--- directive, or inferred from posting amounts, as a map from symbol--- to style. Styles declared by directives take precedence (and--- commodity takes precedence over D). Styles from directives are--- guaranteed to specify the decimal mark character.+-- | Get the canonical amount styles for this journal, whether (in order of precedence):+-- set globally in InputOpts,+-- declared by commodity directives, +-- declared by a default commodity (D) directive, +-- or inferred from posting amounts, +-- as a map from symbol to style. +-- Styles from directives are assumed to specify the decimal mark. journalCommodityStyles :: Journal -> M.Map CommoditySymbol AmountStyle journalCommodityStyles j =   -- XXX could be some redundancy here, cf journalStyleInfluencingAmounts-  commoditystyles <> defaultcommoditystyle <> inferredstyles+  globalstyles <> declaredstyles <> defaultcommoditystyle <> inferredstyles   where-    commoditystyles       = M.mapMaybe cformat $ jcommodities j+    globalstyles          = jglobalcommoditystyles j+    declaredstyles        = M.mapMaybe cformat $ jcommodities j     defaultcommoditystyle = M.fromList $ catMaybes [jparsedefaultcommodity j]     inferredstyles        = jinferredcommodities j @@ -978,14 +988,12 @@ -- "the format of the first amount, adjusted to the highest precision of all amounts". -- Can return an error message eg if inconsistent number formats are found. journalInferCommodityStyles :: Journal -> Either String Journal-journalInferCommodityStyles j =+journalInferCommodityStyles j =    case-    commodityStylesFromAmounts $-    dbg7 "journalInferCommodityStyles using amounts" $-    journalStyleInfluencingAmounts j+    commodityStylesFromAmounts $ journalStyleInfluencingAmounts j   of     Left e   -> Left e-    Right cs -> Right j{jinferredcommodities = cs}+    Right cs -> Right j{jinferredcommodities = dbg7 "journalInferCommodityStyles" cs}  -- | Given a list of amounts, in parse order (roughly speaking; see journalStyleInfluencingAmounts), -- build a map from their commodity names to standard commodity@@ -1004,10 +1012,14 @@     commamts = groupSort [(acommodity as, as) | as <- amts]     commstyles = [(c, canonicalStyleFrom $ map astyle as) | (c,as) <- commamts] --- TODO: should probably detect and report inconsistencies here+-- TODO: should probably detect and report inconsistencies here.+-- Though, we don't have the info for a good error message, so maybe elsewhere. -- | Given a list of amount styles (assumed to be from parsed amounts -- in a single commodity), in parse order, choose a canonical style.--- Traditionally it's "the style of the first, with the maximum precision of all".+-- This is:+-- the general style of the first amount, +-- with the first digit group style seen,+-- with the maximum precision of all. -- canonicalStyleFrom :: [AmountStyle] -> AmountStyle canonicalStyleFrom [] = amountstyle@@ -1114,7 +1126,9 @@ -- Transaction price amounts (posting amounts' aprice field) are not included. -- journalStyleInfluencingAmounts :: Journal -> [Amount]-journalStyleInfluencingAmounts j = catMaybes $ concat [+journalStyleInfluencingAmounts j = +  dbg7 "journalStyleInfluencingAmounts" $+  catMaybes $ concat [    [mdefaultcommodityamt]   ,map (Just . pdamount) $ jpricedirectives j   ,map Just $ concatMap amounts $ map pamount $ journalPostings j@@ -1218,6 +1232,16 @@ postingFindTag :: TagName -> Posting -> Maybe (TagName, TagValue) postingFindTag tagname p = find ((tagname==) . fst) $ postingAllTags p +-- | Apply some account aliases to all posting account names in the journal, as described by accountNameApplyAliases.+-- This can fail due to a bad replacement pattern in a regular expression alias.+journalApplyAliases :: [AccountAlias] -> Journal -> Either RegexError Journal+-- short circuit the common case, just in case there's a performance impact from txnTieKnot etc.+journalApplyAliases [] j = Right j+journalApplyAliases aliases j = +  case mapM (transactionApplyAliases aliases) $ jtxns j of+    Right ts -> Right j{jtxns = ts}+    Left err -> Left err+   -- -- | Build a database of market prices in effect on the given date, -- -- from the journal's price directives. -- journalPrices :: Day -> Journal -> Prices

@@ -1,3 +1,4 @@+{-# LANGUAGE NamedFieldPuns #-} {-|  A 'Posting' represents a change (by some 'MixedAmount') of the balance in@@ -37,6 +38,7 @@   transactionAllTags,   relatedPostings,   removePrices,+  postingApplyAliases,   -- * date operations   postingDate,   postingDate2,@@ -168,7 +170,7 @@                           BalancedVirtualPosting -> (\s -> "["++s++"]", acctnamewidth-2)                           VirtualPosting -> (\s -> "("++s++")", acctnamewidth-2)                           _ -> (id,acctnamewidth)-      showamount = padLeftWide 12 . showMixedAmount+      showamount = fst . showMixed showAmount (Just 12) Nothing False   showComment :: Text -> String@@ -287,6 +289,17 @@ concatAccountNames :: [AccountName] -> AccountName concatAccountNames as = accountNameWithPostingType t $ T.intercalate ":" $ map accountNameWithoutPostingType as     where t = headDef RegularPosting $ filter (/= RegularPosting) $ map accountNamePostingType as++-- | Apply some account aliases to the posting's account name, as described by accountNameApplyAliases.+-- This can fail due to a bad replacement pattern in a regular expression alias.+postingApplyAliases :: [AccountAlias] -> Posting -> Either RegexError Posting+postingApplyAliases aliases p@Posting{paccount} =+  case accountNameApplyAliases aliases paccount of+    Right a -> Right p{paccount=a}+    Left e  -> Left err+      where+        err = "problem while applying account aliases:\n" ++ pshow aliases +          ++ "\n to account name: "++T.unpack paccount++"\n "++e  -- | Rewrite an account name using all matching aliases from the given list, in sequence. -- Each alias sees the result of applying the previous aliases.

@@ -11,6 +11,7 @@   RawOpts,   setopt,   setboolopt,+  appendopts,   inRawOpts,   boolopt,   choiceopt,@@ -22,7 +23,8 @@   posintopt,   maybeintopt,   maybeposintopt,-  maybecharopt+  maybecharopt,+  overRawOpts ) where @@ -46,6 +48,9 @@  setboolopt :: String -> RawOpts -> RawOpts setboolopt name = overRawOpts (++ [(name,"")])++appendopts :: [(String,String)] -> RawOpts -> RawOpts+appendopts new = overRawOpts $ \old -> concat [old,new]  -- | Is the named option present ? inRawOpts :: String -> RawOpts -> Bool

@@ -10,27 +10,33 @@         , StringFormat(..)         , StringFormatComponent(..)         , ReportItemField(..)+        , overlineWidth+        , defaultBalanceLineFormat         , tests_StringFormat         ) where  import Prelude () import "base-compat-batteries" Prelude.Compat-import Numeric+import Numeric (readDec) import Data.Char (isPrint)-import Data.Maybe+import Data.Default (Default(..))+import Data.Maybe (isJust) -- import qualified Data.Text as T import Text.Megaparsec-import Text.Megaparsec.Char+import Text.Megaparsec.Char (char, digitChar, string) -import Hledger.Utils.Parse+import Hledger.Utils.Parse (SimpleStringParser) import Hledger.Utils.String (formatString) import Hledger.Utils.Test  -- | A format specification/template to use when rendering a report line item as text. ----- A format is a sequence of components; each is either a literal--- string, or a hledger report item field with specified width and--- justification whose value will be interpolated at render time.+-- A format is an optional width, along with a sequence of components;+-- each is either a literal string, or a hledger report item field with+-- specified width and justification whose value will be interpolated+-- at render time. The optional width determines the length of the+-- overline to draw above the totals row; if it is Nothing, then the+-- maximum width of all lines is used. -- -- A component's value may be a multi-line string (or a -- multi-commodity amount), in which case the final string will be@@ -41,9 +47,9 @@ -- mode, which provides a limited StringFormat renderer. -- data StringFormat =-    OneLine [StringFormatComponent]       -- ^ multi-line values will be rendered on one line, comma-separated-  | TopAligned [StringFormatComponent]    -- ^ values will be top-aligned (and bottom-padded to the same height)-  | BottomAligned [StringFormatComponent] -- ^ values will be bottom-aligned (and top-padded)+    OneLine       (Maybe Int) [StringFormatComponent] -- ^ multi-line values will be rendered on one line, comma-separated+  | TopAligned    (Maybe Int) [StringFormatComponent] -- ^ values will be top-aligned (and bottom-padded to the same height)+  | BottomAligned (Maybe Int) [StringFormatComponent] -- ^ values will be bottom-aligned (and top-padded)   deriving (Show, Eq)  data StringFormatComponent =@@ -73,6 +79,21 @@   | FieldNo Int       -- ^ A report item's nth field. May be unimplemented.     deriving (Show, Eq) +instance Default StringFormat where def = defaultBalanceLineFormat++overlineWidth :: StringFormat -> Maybe Int+overlineWidth (OneLine       w _) = w+overlineWidth (TopAligned    w _) = w+overlineWidth (BottomAligned w _) = w++-- | Default line format for balance report: "%20(total)  %2(depth_spacer)%-(account)"+defaultBalanceLineFormat :: StringFormat+defaultBalanceLineFormat = BottomAligned (Just 20) [+      FormatField False (Just 20) Nothing TotalField+    , FormatLiteral "  "+    , FormatField True (Just 2) Nothing DepthSpacerField+    , FormatField True Nothing Nothing AccountField+    ] ----------------------------------------------------------------------  -- renderStringFormat :: StringFormat -> Map String String -> String@@ -93,10 +114,10 @@   alignspec <- optional (try $ char '%' >> oneOf ("^_,"::String))   let constructor =         case alignspec of-          Just '^' -> TopAligned-          Just '_' -> BottomAligned-          Just ',' -> OneLine-          _        -> defaultStringFormatStyle+          Just '^' -> TopAligned Nothing+          Just '_' -> BottomAligned Nothing+          Just ',' -> OneLine Nothing+          _        -> defaultStringFormatStyle Nothing   constructor <$> many componentp  componentp :: SimpleStringParser StringFormatComponent@@ -157,23 +178,23 @@    ,let s `gives` expected = test s $ parseStringFormat s @?= Right expected    in tests "parseStringFormat" [-      ""                           `gives` (defaultStringFormatStyle [])-    , "D"                          `gives` (defaultStringFormatStyle [FormatLiteral "D"])-    , "%(date)"                    `gives` (defaultStringFormatStyle [FormatField False Nothing Nothing DescriptionField])-    , "%(total)"                   `gives` (defaultStringFormatStyle [FormatField False Nothing Nothing TotalField])+      ""                           `gives` (defaultStringFormatStyle Nothing [])+    , "D"                          `gives` (defaultStringFormatStyle Nothing [FormatLiteral "D"])+    , "%(date)"                    `gives` (defaultStringFormatStyle Nothing [FormatField False Nothing Nothing DescriptionField])+    , "%(total)"                   `gives` (defaultStringFormatStyle Nothing [FormatField False Nothing Nothing TotalField])     -- TODO     -- , "^%(total)"                  `gives` (TopAligned [FormatField False Nothing Nothing TotalField])     -- , "_%(total)"                  `gives` (BottomAligned [FormatField False Nothing Nothing TotalField])     -- , ",%(total)"                  `gives` (OneLine [FormatField False Nothing Nothing TotalField])-    , "Hello %(date)!"             `gives` (defaultStringFormatStyle [FormatLiteral "Hello ", FormatField False Nothing Nothing DescriptionField, FormatLiteral "!"])-    , "%-(date)"                   `gives` (defaultStringFormatStyle [FormatField True Nothing Nothing DescriptionField])-    , "%20(date)"                  `gives` (defaultStringFormatStyle [FormatField False (Just 20) Nothing DescriptionField])-    , "%.10(date)"                 `gives` (defaultStringFormatStyle [FormatField False Nothing (Just 10) DescriptionField])-    , "%20.10(date)"               `gives` (defaultStringFormatStyle [FormatField False (Just 20) (Just 10) DescriptionField])-    , "%20(account) %.10(total)"   `gives` (defaultStringFormatStyle [FormatField False (Just 20) Nothing AccountField-                                                                     ,FormatLiteral " "-                                                                     ,FormatField False Nothing (Just 10) TotalField-                                                                     ])+    , "Hello %(date)!"             `gives` (defaultStringFormatStyle Nothing [FormatLiteral "Hello ", FormatField False Nothing Nothing DescriptionField, FormatLiteral "!"])+    , "%-(date)"                   `gives` (defaultStringFormatStyle Nothing [FormatField True Nothing Nothing DescriptionField])+    , "%20(date)"                  `gives` (defaultStringFormatStyle Nothing [FormatField False (Just 20) Nothing DescriptionField])+    , "%.10(date)"                 `gives` (defaultStringFormatStyle Nothing [FormatField False Nothing (Just 10) DescriptionField])+    , "%20.10(date)"               `gives` (defaultStringFormatStyle Nothing [FormatField False (Just 20) (Just 10) DescriptionField])+    , "%20(account) %.10(total)"   `gives` (defaultStringFormatStyle Nothing [FormatField False (Just 20) Nothing AccountField+                                                                             ,FormatLiteral " "+                                                                             ,FormatField False Nothing (Just 10) TotalField+                                                                             ])     , test "newline not parsed" $ assertLeft $ parseStringFormat "\n"     ]  ]

@@ -33,6 +33,7 @@   transactionTransformPostings,   transactionApplyValuation,   transactionToCost,+  transactionApplyAliases,   -- nonzerobalanceerror,   -- * date operations   transactionDate2,@@ -256,11 +257,11 @@     -- currently prices are considered part of the amount string when right-aligning amounts     shownAmounts       | elideamount    = [""]-      | onelineamounts = [fitString (Just amtwidth) Nothing False False $ showMixedAmountOneLine $ pamount p]+      | onelineamounts = [fst . showMixedOneLineUnnormalised showAmount (Just amtwidth) Nothing False $ pamount p]       | null (amounts $ pamount p) = [""]-      | otherwise      = map (fitStringMulti (Just amtwidth) Nothing False False . showAmount ) . amounts $ pamount p+      | otherwise      = lines . fst . showMixedUnnormalised showAmount (Just amtwidth) Nothing False $ pamount p       where-        amtwidth = maximum $ 12 : map (strWidth . showMixedAmount . pamount) pstoalignwith  -- min. 12 for backwards compatibility+        amtwidth = maximum $ 12 : map (snd . showMixedUnnormalised showAmount Nothing Nothing False . pamount) pstoalignwith  -- min. 12 for backwards compatibility      (samelinecomment, newlinecomments) =       case renderCommentLines (pcomment p) of []   -> ("",[])@@ -590,6 +591,14 @@ -- | Convert this transaction's amounts to cost, and apply the appropriate amount styles. transactionToCost :: M.Map CommoditySymbol AmountStyle -> Transaction -> Transaction transactionToCost styles t@Transaction{tpostings=ps} = t{tpostings=map (postingToCost styles) ps}++-- | Apply some account aliases to all posting account names in the transaction, as described by accountNameApplyAliases.+-- This can fail due to a bad replacement pattern in a regular expression alias.+transactionApplyAliases :: [AccountAlias] -> Transaction -> Either RegexError Transaction+transactionApplyAliases aliases t =+  case mapM (postingApplyAliases aliases) $ tpostings t of+    Right ps -> Right $ txnTieKnot $ t{tpostings=ps}+    Left err -> Left err  -- tests 

@@ -89,7 +89,7 @@ type WeekDay = Int   -- 1-7  -- Typical report periods (spans of time), both finite and open-ended.--- A richer abstraction than DateSpan.+-- A higher-level abstraction than DateSpan. data Period =     DayPeriod Day   | WeekPeriod Day@@ -159,6 +159,12 @@  data Side = L | R deriving (Eq,Show,Read,Ord,Generic) +-- | One of the decimal marks we support: either period or comma.+type DecimalMark = Char++isDecimalMark :: Char -> Bool+isDecimalMark c = c == '.' || c == ','+ -- | The basic numeric type used in amounts. type Quantity = Decimal -- The following is for hledger-web, and requires blaze-markup.@@ -440,6 +446,7 @@   -- parsing-related data    jparsedefaultyear      :: Maybe Year                            -- ^ the current default year, specified by the most recent Y directive (or current date)   ,jparsedefaultcommodity :: Maybe (CommoditySymbol,AmountStyle)   -- ^ the current default commodity and its format, specified by the most recent D directive+  ,jparsedecimalmark      :: Maybe DecimalMark                     -- ^ the character to always parse as decimal point, if set by CsvReader's decimal-mark (or a future journal directive)   ,jparseparentaccounts   :: [AccountName]                         -- ^ the current stack of parent account names, specified by apply account directives   ,jparsealiases          :: [AccountAlias]                        -- ^ the current account name aliases in effect, specified by alias directives (& options ?)   -- ,jparsetransactioncount :: Integer                               -- ^ the current count of transactions parsed so far (only journal format txns, currently)@@ -448,8 +455,9 @@   -- principal data   ,jdeclaredaccounts      :: [(AccountName,AccountDeclarationInfo)] -- ^ Accounts declared by account directives, in parse order (after journal finalisation)   ,jdeclaredaccounttypes  :: M.Map AccountType [AccountName]        -- ^ Accounts whose type has been declared in account directives (usually 5 top-level accounts)+  ,jglobalcommoditystyles :: M.Map CommoditySymbol AmountStyle      -- ^ per-commodity display styles declared globally, eg by command line option or import command   ,jcommodities           :: M.Map CommoditySymbol Commodity        -- ^ commodities and formats declared by commodity directives-  ,jinferredcommodities   :: M.Map CommoditySymbol AmountStyle      -- ^ commodities and formats inferred from journal amounts  TODO misnamed, should be eg jusedstyles+  ,jinferredcommodities   :: M.Map CommoditySymbol AmountStyle      -- ^ commodities and formats inferred from journal amounts   ,jpricedirectives       :: [PriceDirective]                       -- ^ Declarations of market prices by P directives, in parse order (after journal finalisation)   ,jinferredmarketprices  :: [MarketPrice]                          -- ^ Market prices implied by transactions, in parse order (after journal finalisation)   ,jtxnmodifiers          :: [TransactionModifier]@@ -459,6 +467,8 @@   ,jfiles                 :: [(FilePath, Text)]                     -- ^ the file path and raw text of the main and                                                                     --   any included journal files. The main file is first,                                                                     --   followed by any included files in the order encountered.+                                                                    --   TODO: FilePath is a sloppy type here, don't assume it's a+                                                                    --   real file; values like "", "-", "(string)" can be seen   ,jlastreadtime          :: ClockTime                              -- ^ when this journal was last read from its file(s)   } deriving (Eq, Generic) 

@@ -28,22 +28,23 @@ where  import Control.Applicative ((<|>))-import Data.Decimal (roundTo)+import Data.Foldable (asum) import Data.Function ((&), on)-import Data.Graph.Inductive  (Gr, Node, NodeMap, mkMapGraph, mkNode, lab, out, sp)-import Data.List+import Data.List ( (\\), sortBy ) import Data.List.Extra (nubSortBy) import qualified Data.Map as M-import Data.Maybe+import qualified Data.Set as S+import Data.Maybe ( fromMaybe ) import qualified Data.Text as T import Data.Time.Calendar (Day, fromGregorian) import Data.MemoUgly (memo) import GHC.Generics (Generic)-import Safe (headMay)+import Safe (lastMay)  import Hledger.Utils import Hledger.Data.Types import Hledger.Data.Amount+import Hledger.Data.Dates (nulldate)   ------------------------------------------------------------------------------@@ -60,31 +61,6 @@   | AtDefault  (Maybe CommoditySymbol)  -- ^ works like AtNow in single period reports, like AtEnd in multiperiod reports   deriving (Show,Eq) --- | A snapshot of the known exchange rates between commodity pairs at a given date,--- as a graph allowing fast lookup and path finding, along with some helper data.-data PriceGraph = PriceGraph {-   prGraph   :: Gr CommoditySymbol Quantity-    -- ^ A directed graph of exchange rates between commodity pairs.-    -- Node labels are commodities and edge labels are exchange rates,-    -- which were either:-    -- declared by P directives,-    -- inferred from transaction prices,-    -- inferred by reversing a declared rate,-    -- or inferred by reversing a transaction-inferred rate.-    -- There will be at most one edge between each directed pair of commodities,-    -- eg there can be one USD->EUR and one EUR->USD.-  ,prNodemap :: NodeMap CommoditySymbol-    -- ^ Mapping of graph node ids to commodity symbols.-  ,prDefaultValuationCommodities :: M.Map CommoditySymbol CommoditySymbol-    -- ^ The default valuation commodity for each source commodity.-    --   These are used when a valuation commodity is not specified-    --   (-V). They are the destination commodity of the latest-    --   (declared or inferred, but not reverse) each-    --   source commodity's latest market price (on the date of this-    --   graph).-  }-  deriving (Show,Generic)- -- | A price oracle is a magic memoising function that efficiently -- looks up market prices (exchange rates) from one commodity to -- another (or if unspecified, to a default valuation commodity) on a@@ -172,7 +148,7 @@  -- | Standard error message for a report not supporting --value=then. unsupportedValueThenError :: String-unsupportedValueThenError = "Sorry, --value=then is not yet implemented for this kind of report."+unsupportedValueThenError = "Sorry, --value=then is not yet supported for this kind of report."  -- | Find the market value of each component amount in the given -- commodity, or its default valuation commodity, at the given@@ -227,11 +203,11 @@ priceLookup makepricegraph d from mto =   -- trace ("priceLookup ("++show d++", "++show from++", "++show mto++")") $   let-    -- build a graph of the commodity exchange rates in effect on this day-    -- XXX should hide these fgl details better-    PriceGraph{prGraph=g, prNodemap=m, prDefaultValuationCommodities=defaultdests} =+    PriceGraph{pgEdges=forwardprices+              ,pgEdgesRev=allprices+              ,pgDefaultValuationCommodities=defaultdests+              } =       traceAt 1 ("valuation date: "++show d) $ makepricegraph d-    fromnode = node m from     mto' = mto <|> mdefaultto       where         mdefaultto = dbg1 ("default valuation commodity for "++T.unpack from) $@@ -241,25 +217,15 @@       Nothing            -> Nothing       Just to | to==from -> Nothing       Just to            ->-        -- We have a commodity to convert to. Find the most direct price available.-        case mindirectprice of+        -- We have a commodity to convert to. Find the most direct price available,+        -- according to the rules described in makePriceGraph.+        case +          pricesShortestPath forwardprices from to <|> +          pricesShortestPath allprices     from to +        of           Nothing -> Nothing-          Just q  -> Just (to, q)-        where-          tonode = node m to-          mindirectprice :: Maybe Quantity =-            -- Find the shortest path, if any, between from and to.-            case sp fromnode tonode g :: Maybe [Node] of-              Nothing    -> Nothing-              Just nodes ->-                dbg ("market price for "++intercalate " -> " (map T.unpack comms)) $-                  -- TODO: it would be nice to include price date as part of the label-                  -- in PriceGraph, so we could show the dates of market prices here-                Just $ product $ pathEdgeLabels g nodes  -- convert to a single exchange rate-                where comms = catMaybes $ map (lab g) nodes--          -- log a message and a Maybe Quantity, hiding Just/Nothing and limiting decimal places-          dbg msg = dbg1With (((msg++": ")++) . maybe "" (show . roundTo 8))+          Just [] -> Nothing+          Just ps -> Just (mpto $ last ps, product $ map mprate ps)  tests_priceLookup =   let@@ -278,13 +244,85 @@     priceLookup makepricegraph (fromGregorian 2000 01 01) "B" (Just "A") @?= Just ("A",0.1)     priceLookup makepricegraph (fromGregorian 2000 01 01) "A" (Just "E") @?= Just ("E",500) +------------------------------------------------------------------------------+-- Market price graph++type Edge = MarketPrice+type Path = [Edge]++data PriceGraph = PriceGraph {+   pgDate :: Day+    -- ^ The date on which these prices are in effect.+  ,pgEdges :: [Edge]+    -- ^ "Forward" exchange rates between commodity pairs, either+    --   declared by P directives or inferred from transaction prices,+    --   forming the edges of a directed graph.  +  ,pgEdgesRev :: [Edge]+    -- ^ The same edges, plus any additional edges that can be+    --   inferred by reversing them and inverting the rates.+    --+    --   In both of these there will be at most one edge between each+    --   directed pair of commodities, eg there can be one USD->EUR and one EUR->USD.+  ,pgDefaultValuationCommodities :: M.Map CommoditySymbol CommoditySymbol+    -- ^ The default valuation commodity for each source commodity.+    --   These are used when a valuation commodity is not specified+    --   (-V). They are the destination commodity of each source commodity's+    --   latest (declared or inferred, but not reverse) market price+    --   (on the date of this graph).+  }+  deriving (Show,Generic)++-- | Find the shortest path and corresponding conversion rate, if any, +-- from one commodity to another using the provided market prices which+-- form the edges of a directed graph. There should be at most one edge+-- between each directed pair of commodities, eg there can be one+-- USD->EUR price and one EUR->USD price.+pricesShortestPath :: [Edge] -> CommoditySymbol -> CommoditySymbol -> Maybe Path+pricesShortestPath edges start end =+  dbg1 ("shortest price path for "++T.unpack start++" -> "++T.unpack end) $ +  asum $ map (findPath end edgesremaining) initialpaths+  where+    initialpaths = dbg9 "initial price paths" $ [[p] | p <- edges, mpfrom p == start]+    edgesremaining = dbg9 "initial edges remaining" $ edges \\ concat initialpaths++-- Helper: breadth-first search for a continuation of the given path+-- using zero or more of the given edges, to the specified end commodity.+-- Returns the first & shortest complete path found, or Nothing.+findPath :: CommoditySymbol -> [Edge] -> Path -> Maybe Path+findPath end _ path | mpathend == Just end = Just path  -- path is complete+  where +    mpathend = mpto <$> lastMay path+findPath _ [] _ = Nothing   -- no more edges are available+findPath end edgesremaining path =   -- try continuing with all the remaining edges+  asum [ +      findPath end edgesremaining' path'+    | e <- nextedges+    , let path' = path++[e]+    , let edgesremaining' = filter (/=e) edgesremaining+    ]+  where+    nextedges = [ e | e <- edgesremaining, Just (mpfrom e) == mpathend ]+      where+        mpathend = mpto <$> lastMay path++-- | A snapshot of the known exchange rates between commodity pairs at a given date.+-- This is a home-made version, more tailored to our needs. -- | Build the graph of commodity conversion prices for a given day. -- Converts a list of declared market prices in parse order, and a--- list of transaction-inferred market prices in parse order, to a--- graph of all known exchange rates between commodity pairs in effect--- on that day. Cf hledger.m4.md -> Valuation:+-- list of transaction-inferred market prices in parse order, to: ----- hledger looks for a market price (exchange rate) from commodity A+-- 1. a graph of all known exchange rates declared or inferred from +-- one commodity to another in effect on that day+--+-- 2. a second graph which includes any additional exchange rates+-- that can be inferred by reversing known rates+--+-- 3. a map of each commodity's default valuation commodity, if any.+--+-- These allow price lookup and valuation to be performed as+-- described in hledger.m4.md -> Valuation:+--+-- "hledger looks for a market price (exchange rate) from commodity A -- to commodity B in one or more of these ways, in this order of -- preference: --@@ -296,15 +334,15 @@ -- 2. A *reverse market price*: --    the inverse of a declared or inferred market price from B to A. -- --- 3. A *chained market price*:---    a synthetic price formed by combining the shortest chain of market---    prices (any of the above types) leading from A to B.+-- 3. A *a forward chain of market prices*:+--    a synthetic price formed by combining the shortest chain of+--    "forward" (only 1 above) market prices, leading from A to B. ----- 1 and 2 form the edges of the price graph, and we can query it for--- 3 (which is the reason we use a graph).+-- 4. A *any chain of market prices*:+--    a chain of any market prices, including both forward and+--    reverse prices (1 and 2 above), leading from A to B." ----- We also identify each commodity's default valuation commodity, if--- any. For each commodity A, hledger picks a default valuation+-- and: "For each commodity A, hledger picks a default valuation -- commodity as follows, in this order of preference: -- -- 1. The price commodity from the latest declared market price for A@@ -316,32 +354,31 @@ -- -- 3. If there are no P directives at all (any commodity or date), and --    the `--infer-value` flag is used, then the price commodity from---    the latest transaction price for A on or before valuation date.+--    the latest transaction price for A on or before valuation date." -- makePriceGraph :: [MarketPrice] -> [MarketPrice] -> Day -> PriceGraph makePriceGraph alldeclaredprices allinferredprices d =   dbg9 ("makePriceGraph "++show d) $-  PriceGraph{prGraph=g, prNodemap=m, prDefaultValuationCommodities=defaultdests}+  PriceGraph{+     pgDate = d+    ,pgEdges=forwardprices+    ,pgEdgesRev=allprices+    ,pgDefaultValuationCommodities=defaultdests+    }   where     -- prices in effect on date d, either declared or inferred-    visibledeclaredprices = filter ((<=d).mpdate) alldeclaredprices-    visibleinferredprices = filter ((<=d).mpdate) allinferredprices-    declaredandinferredprices = dbg2 "declaredandinferredprices" $-      effectiveMarketPrices visibledeclaredprices visibleinferredprices+    visibledeclaredprices = dbg2 "visibledeclaredprices" $ filter ((<=d).mpdate) alldeclaredprices+    visibleinferredprices = dbg2 "visibleinferredprices" $ filter ((<=d).mpdate) allinferredprices+    forwardprices = effectiveMarketPrices visibledeclaredprices visibleinferredprices      -- infer any additional reverse prices not already declared or inferred-    reverseprices = dbg2 "reverseprices" $-      map marketPriceReverse declaredandinferredprices \\ declaredandinferredprices--    -- build the graph and associated node map-    (g, m) =-      mkMapGraph-      (dbg9 "price graph labels" $ sort allcomms) -- this must include all nodes mentioned in edges-      (dbg9 "price graph edges" $ [(mpfrom, mpto, mprate) | MarketPrice{..} <- prices])-      :: (Gr CommoditySymbol Quantity, NodeMap CommoditySymbol)+    reverseprices = dbg2 "additional reverse prices" $+      [p | p@MarketPrice{..} <- map marketPriceReverse forwardprices+         , not $ (mpfrom,mpto) `S.member` forwardpairs+      ]       where-        prices   = declaredandinferredprices ++ reverseprices-        allcomms = map mpfrom prices+        forwardpairs = S.fromList [(mpfrom,mpto) | MarketPrice{..} <- forwardprices]+    allprices = forwardprices ++ reverseprices      -- determine a default valuation commodity for each source commodity     -- somewhat but not quite like effectiveMarketPrices@@ -370,6 +407,7 @@     declaredprices' = [(1, i, p) | (i,p) <- zip [1..] declaredprices]     inferredprices' = [(0, i, p) | (i,p) <- zip [1..] inferredprices]   in+    dbg2 "effective forward prices" $     -- combine     declaredprices' ++ inferredprices'     -- sort by decreasing date then decreasing precedence then decreasing parse order@@ -380,35 +418,24 @@     & nubSortBy (compare `on` (\(MarketPrice{..})->(mpfrom,mpto)))  marketPriceReverse :: MarketPrice -> MarketPrice-marketPriceReverse mp@MarketPrice{..} = mp{mpfrom=mpto, mpto=mpfrom, mprate=1/mprate}----------------------------------------------------------------------------------- fgl helpers---- | Look up an existing graph node by its label.--- (If the node does not exist, a new one will be generated, but not--- persisted in the nodemap.)-node :: Ord a => NodeMap a -> a -> Node-node m = fst . fst . mkNode m---- | Convert a valid path within the given graph to the corresponding--- edge labels. When there are multiple edges between two nodes, the--- lowest-sorting label is used.-pathEdgeLabels :: (Show b, Ord b) => Gr a b -> [Node] -> [b]-pathEdgeLabels g = map frommaybe . map (nodesEdgeLabel g) . pathEdges-  where frommaybe = fromMaybe (error' "pathEdgeLabels: expected no Nothings here")  -- PARTIAL:---- | Convert a path to node pairs representing the path's edges.-pathEdges :: [Node] -> [(Node,Node)]-pathEdges p = [(f,t) | f:t:_ <- tails p]+marketPriceReverse mp@MarketPrice{..} = +  mp{mpfrom=mpto, mpto=mpfrom, mprate=if mprate==0 then 0 else 1/mprate}  -- PARTIAL: / --- | Get the label of a graph edge from one node to another.--- When there are multiple such edges, the lowest-sorting label is used.-nodesEdgeLabel :: Ord b => Gr a b -> (Node, Node) -> Maybe b-nodesEdgeLabel g (from,to) = headMay $ sort [l | (_,t,l) <- out g from, t==to]+nullmarketprice :: MarketPrice+nullmarketprice = MarketPrice {+   mpdate=nulldate+  ,mpfrom=""+  ,mpto=""+  ,mprate=0+  }  ------------------------------------------------------------------------------  tests_Valuation = tests "Valuation" [    tests_priceLookup+  ,test "marketPriceReverse" $ do+    marketPriceReverse nullmarketprice{mprate=2} @?= nullmarketprice{mprate=0.5}+    marketPriceReverse nullmarketprice @?= nullmarketprice  -- the reverse of a 0 price is a 0 price++   ]

@@ -5,10 +5,6 @@  -} --- Silence safe 0.3.18's deprecation warnings for (max|min)imum(By)?Def for now--- (may hide other deprecation warnings too). https://github.com/ndmitchell/safe/issues/26-{-# OPTIONS_GHC -Wno-warnings-deprecations #-}- {-# LANGUAGE CPP                #-} {-# LANGUAGE FlexibleContexts   #-} {-# LANGUAGE OverloadedStrings  #-}@@ -18,11 +14,13 @@   -- * Query and QueryOpt   Query(..),   QueryOpt(..),+  OrdPlus(..),   payeeTag,   noteTag,   generatedTransactionTag,   -- * parsing   parseQuery,+  parseQueryList,   simplifyQuery,   filterQuery,   -- * accessors@@ -37,7 +35,6 @@   queryIsSym,   queryIsReal,   queryIsStatus,-  queryIsEmpty,   queryStartDate,   queryEndDate,   queryDateSpan,@@ -62,6 +59,7 @@ where  import Control.Applicative ((<|>), many, optional)+import Data.Default (Default(..)) import Data.Either (partitionEithers) import Data.List (partition) import Data.Maybe (fromMaybe, isJust, mapMaybe)@@ -99,8 +97,6 @@            | Real Bool        -- ^ match if "realness" (involves a real non-virtual account ?) has this value            | Amt OrdPlus Quantity  -- ^ match if the amount's numeric quantity is less than/greater than/equal to/unsignedly equal to some value            | Sym Regexp       -- ^ match if the entire commodity symbol is matched by this regexp-           | Empty Bool       -- ^ if true, show zero-amount postings/accounts which are usually not shown-                              --   more of a query option than a query criteria ?            | Depth Int        -- ^ match if account depth is less than or equal to this value.                               --   Depth is sometimes used like a query (for filtering report data)                               --   and sometimes like a query option (for controlling display)@@ -108,6 +104,8 @@                                         -- matching the regexp if provided, exists     deriving (Eq,Show) +instance Default Query where def = Any+ -- | Construct a payee tag payeeTag :: Maybe String -> Either RegexError Query payeeTag = fmap (Tag (toRegexCI' "payee")) . maybe (pure Nothing) (fmap Just . toRegexCI)@@ -141,10 +139,24 @@ -- showAccountMatcher _ = Nothing  --- | Convert a query expression containing zero or more--- space-separated terms to a query and zero or more query options; or--- return an error message if query parsing fails.+-- | A version of parseQueryList which acts on a single Text of+-- space-separated terms. --+-- The usual shell quoting rules are assumed. When a pattern contains+-- whitespace, it (or the whole term including prefix) should be enclosed+-- in single or double quotes.+--+-- >>> parseQuery nulldate "expenses:dining out"+-- Right (Or [Acct (RegexpCI "expenses:dining"),Acct (RegexpCI "out")],[])+--+-- >>> parseQuery nulldate "\"expenses:dining out\""+-- Right (Acct (RegexpCI "expenses:dining out"),[])+parseQuery :: Day -> T.Text -> Either String (Query,[QueryOpt])+parseQuery d = parseQueryList d . words'' prefixes++-- | Convert a list of query expression containing to a query and zero+-- or more query options; or return an error message if query parsing fails.+-- -- A query term is either: -- -- 1. a search pattern, which matches on one or more fields, eg:@@ -161,10 +173,6 @@ -- --      inacct:FULLACCTNAME ----- The usual shell quoting rules are assumed. When a pattern contains--- whitespace, it (or the whole term including prefix) should be enclosed--- in single or double quotes.--- -- Period expressions may contain relative dates, so a reference date is -- required to fully parse these. --@@ -173,15 +181,8 @@ -- 2. multiple description patterns are OR'd together -- 3. multiple status patterns are OR'd together -- 4. then all terms are AND'd together------ >>> parseQuery nulldate "expenses:dining out"--- Right (Or [Acct (RegexpCI "expenses:dining"),Acct (RegexpCI "out")],[])------ >>> parseQuery nulldate "\"expenses:dining out\""--- Right (Acct (RegexpCI "expenses:dining out"),[])-parseQuery :: Day -> T.Text -> Either String (Query,[QueryOpt])-parseQuery d s = do-  let termstrs = words'' prefixes s+parseQueryList :: Day -> [T.Text] -> Either String (Query, [QueryOpt])+parseQueryList d termstrs = do   eterms <- sequence $ map (parseQueryTerm d) termstrs   let (pats, opts) = partitionEithers eterms       (descpats, pats') = partition queryIsDesc pats@@ -277,7 +278,6 @@                               Right st -> Right $ Left $ StatusQ st parseQueryTerm _ (T.stripPrefix "real:" -> Just s) = Right $ Left $ Real $ parseBool s || T.null s parseQueryTerm _ (T.stripPrefix "amt:" -> Just s) = Right $ Left $ Amt ord q where (ord, q) = either error id $ parseAmountQueryTerm s  -- PARTIAL:-parseQueryTerm _ (T.stripPrefix "empty:" -> Just s) = Right $ Left $ Empty $ parseBool s parseQueryTerm _ (T.stripPrefix "depth:" -> Just s)   | n >= 0    = Right $ Left $ Depth n   | otherwise = Left "depth: should have a positive number"@@ -440,10 +440,6 @@ queryIsStatus (StatusQ _) = True queryIsStatus _ = False -queryIsEmpty :: Query -> Bool-queryIsEmpty (Empty _) = True-queryIsEmpty _ = False- -- | Does this query specify a start date and nothing else (that would -- filter postings prior to the date) ? -- When the flag is true, look for a starting secondary date instead.@@ -617,11 +613,6 @@ matchesPosting (Real v) p = v == isReal p matchesPosting q@(Depth _) Posting{paccount=a} = q `matchesAccount` a matchesPosting q@(Amt _ _) Posting{pamount=amt} = q `matchesMixedAmount` amt--- matchesPosting q@(Amt _ _) Posting{pamount=amt} = q `matchesMixedAmount` amt--- matchesPosting (Empty v) Posting{pamount=a} = v == mixedAmountLooksZero a--- matchesPosting (Empty False) Posting{pamount=a} = True--- matchesPosting (Empty True) Posting{pamount=a} = mixedAmountLooksZero a-matchesPosting (Empty _) _ = True matchesPosting (Sym r) Posting{pamount=Mixed as} = any (matchesCommodity (Sym r)) $ map acommodity as matchesPosting (Tag n v) p = case (reString n, v) of   ("payee", Just v) -> maybe False (regexMatch v . T.unpack . transactionPayee) $ ptransaction p@@ -643,7 +634,6 @@ matchesTransaction (StatusQ s) t = tstatus t == s matchesTransaction (Real v) t = v == hasRealPostings t matchesTransaction q@(Amt _ _) t = any (q `matchesPosting`) $ tpostings t-matchesTransaction (Empty _) _ = True matchesTransaction (Depth d) t = any (Depth d `matchesPosting`) $ tpostings t matchesTransaction q@(Sym _) t = any (q `matchesPosting`) $ tpostings t matchesTransaction (Tag n v) t = case (reString n, v) of

@@ -32,7 +32,6 @@   readJournal',    -- * Re-exported-  JournalReader.accountaliasp,   JournalReader.postingp,   findReader,   splitReaderPrefix,

@@ -75,6 +75,9 @@   modifiedaccountnamep,   accountnamep, +  -- ** account aliases+  accountaliasp,+   -- ** amounts   spaceandamountormissingp,   amountp,@@ -103,9 +106,9 @@   singlespacedtextp,   singlespacedtextsatisfyingp,   singlespacep,-   skipNonNewlineSpaces,   skipNonNewlineSpaces1,+  aliasesFromOpts,    -- * tests   tests_Common,@@ -145,6 +148,7 @@  import Hledger.Data import Hledger.Utils+import Safe (headMay)  --- ** doctest setup -- $setup@@ -194,12 +198,26 @@     ,new_save_          :: Bool                 -- ^ save latest new transactions state for next time     ,pivot_             :: String               -- ^ use the given field's value as the account name     ,auto_              :: Bool                 -- ^ generate automatic postings when journal is parsed+    ,commoditystyles_   :: Maybe (M.Map CommoditySymbol AmountStyle) -- ^ optional commodity display styles affecting all files+    ,strict_            :: Bool                 -- ^ do extra error checking (eg, all posted accounts are declared)  } deriving (Show)  instance Default InputOpts where def = definputopts  definputopts :: InputOpts-definputopts = InputOpts def def def def def def True def def+definputopts = InputOpts+    { mformat_           = Nothing+    , mrules_file_       = Nothing+    , aliases_           = []+    , anon_              = False+    , ignore_assertions_ = False+    , new_               = False+    , new_save_          = True+    , pivot_             = ""+    , auto_              = False+    , commoditystyles_   = Nothing+    , strict_            = False+    }  rawOptsToInputOpts :: RawOpts -> InputOpts rawOptsToInputOpts rawopts = InputOpts{@@ -213,6 +231,8 @@   ,new_save_          = True   ,pivot_             = stringopt "pivot" rawopts   ,auto_              = boolopt "auto" rawopts+  ,commoditystyles_   = Nothing+  ,strict_            = boolopt "strict" rawopts   }  --- ** parsing utilities@@ -266,6 +286,7 @@                   Right pj -> journalFinalise iopts f txt pj  -- | Like parseAndFinaliseJournal but takes a (non-Erroring) JournalParser.+-- Also, applies command-line account aliases before finalising. -- Used for timeclock/timedot. -- TODO: get rid of this, use parseAndFinaliseJournal instead parseAndFinaliseJournal' :: JournalParser IO ParsedJournal -> InputOpts@@ -279,7 +300,11 @@   -- see notes above   case ep of     Left e   -> throwError $ customErrorBundlePretty e-    Right pj -> journalFinalise iopts f txt pj+    Right pj -> +      -- apply any command line account aliases. Can fail with a bad replacement pattern.+      case journalApplyAliases (aliasesFromOpts iopts) pj of+        Left e    -> throwError e+        Right pj' -> journalFinalise iopts f txt pj'  -- | Post-process a Journal that has just been parsed or generated, in this order: --@@ -295,50 +320,105 @@ -- -- - infer transaction-implied market prices from transaction prices ---journalFinalise :: InputOpts -> FilePath -> Text -> Journal -> ExceptT String IO Journal-journalFinalise iopts f txt pj = do+journalFinalise :: InputOpts -> FilePath -> Text -> ParsedJournal -> ExceptT String IO Journal+journalFinalise InputOpts{auto_,ignore_assertions_,commoditystyles_,strict_} f txt pj = do   t <- liftIO getClockTime   d <- liftIO getCurrentDay-  -- Infer and apply canonical styles for each commodity (or fail).-  -- This affects transaction balancing/assertions/assignments, so needs to be done early.-  -- (TODO: since #903's refactoring for hledger 1.12,-  -- journalApplyCommodityStyles here is seeing the-  -- transactions before they get reversesd to normal order.)-  case journalApplyCommodityStyles pj of-    Left e    -> throwError e-    Right pj' -> either throwError return $-      pj'-      & journalAddFile (f, txt)  -- save the file path and content-      & journalSetLastReadTime t -- save the last read time-      & journalReverse           -- convert all lists to parse order-      & (if not (auto_ iopts) || null (jtxnmodifiers pj)-         then-           -- Auto postings are not active.-           -- Balance all transactions and maybe check balance assertions.-           journalBalanceTransactions (not $ ignore_assertions_ iopts)-         else \j -> do  -- Either monad-           -- Auto postings are active.-           -- Balance all transactions without checking balance assertions,-           j' <- journalBalanceTransactions False j-           -- then add the auto postings-           -- (Note adding auto postings after balancing means #893b fails;-           -- adding them before balancing probably means #893a, #928, #938 fail.)-           case journalModifyTransactions d j' of-             Left e -> throwError e-             Right j'' -> do-               -- then apply commodity styles once more, to style the auto posting amounts. (XXX inefficient ?)-               j''' <- journalApplyCommodityStyles j''-               -- then check balance assertions.-               journalBalanceTransactions (not $ ignore_assertions_ iopts) j'''-        )-     & fmap journalInferMarketPricesFromTransactions  -- infer market prices from commodity-exchanging transactions+  let pj' =+        pj{jglobalcommoditystyles=fromMaybe M.empty commoditystyles_}  -- save any global commodity styles+        & journalAddFile (f, txt)  -- save the main file's info+        & journalSetLastReadTime t -- save the last read time+        & journalReverse -- convert all lists to the order they were parsed +  -- If in strict mode, check all postings are to declared accounts+  case if strict_ then journalCheckAccountsDeclared pj' else Right () of+    Left e   -> throwError e+    Right () ->++      -- and using declared commodities+      case if strict_ then journalCheckCommoditiesDeclared pj' else Right () of+        Left e   -> throwError e+        Right () ->++          -- Infer and apply canonical styles for each commodity (or throw an error).+          -- This affects transaction balancing/assertions/assignments, so needs to be done early.+          case journalApplyCommodityStyles pj' of+            Left e     -> throwError e+            Right pj'' -> either throwError return $+              pj''+              & (if not auto_ || null (jtxnmodifiers pj'')+                then+                  -- Auto postings are not active.+                  -- Balance all transactions and maybe check balance assertions.+                  journalBalanceTransactions (not ignore_assertions_)+                else \j -> do  -- Either monad+                  -- Auto postings are active.+                  -- Balance all transactions without checking balance assertions,+                  j' <- journalBalanceTransactions False j+                  -- then add the auto postings+                  -- (Note adding auto postings after balancing means #893b fails;+                  -- adding them before balancing probably means #893a, #928, #938 fail.)+                  case journalModifyTransactions d j' of+                    Left e -> throwError e+                    Right j'' -> do+                      -- then apply commodity styles once more, to style the auto posting amounts. (XXX inefficient ?)+                      j''' <- journalApplyCommodityStyles j''+                      -- then check balance assertions.+                      journalBalanceTransactions (not ignore_assertions_) j'''+                )+            & fmap journalInferMarketPricesFromTransactions  -- infer market prices from commodity-exchanging transactions++-- | Check that all the journal's postings are to accounts declared with+-- account directives, returning an error message otherwise.+journalCheckAccountsDeclared :: Journal -> Either String ()+journalCheckAccountsDeclared j = sequence_ $ map checkacct $ journalPostings j+  where+    checkacct Posting{paccount,ptransaction}+      | paccount `elem` as = Right ()+      | otherwise          = +          Left $ "\nstrict mode: undeclared account \""++T.unpack paccount++"\""+            ++ case ptransaction of+                Just Transaction{tsourcepos} -> "\nin transaction at: "++showGenericSourcePos tsourcepos+                Nothing -> ""+      where+        as = journalAccountNamesDeclared j++-- | Check that all the commodities used in this journal's postings have been declared+-- by commodity directives, returning an error message otherwise.+journalCheckCommoditiesDeclared :: Journal -> Either String ()+journalCheckCommoditiesDeclared j = +  sequence_ $ map checkcommodities $ journalPostings j+  where+    checkcommodities Posting{..} =+      case mfirstundeclaredcomm of+        Nothing -> Right ()+        Just c  -> Left $ +          "\nstrict mode: undeclared commodity \""++T.unpack c++"\""+          ++ case ptransaction of+                Just Transaction{tsourcepos} -> "\nin transaction at: "++showGenericSourcePos tsourcepos+                Nothing -> ""      +      where+        mfirstundeclaredcomm = +          headMay $ filter (not . (`elem` cs)) $ catMaybes $+          (acommodity . baamount <$> pbalanceassertion) :+          (map (Just . acommodity) $ amounts pamount)+        cs = journalCommoditiesDeclared j+ setYear :: Year -> JournalParser m () setYear y = modify' (\j -> j{jparsedefaultyear=Just y})  getYear :: JournalParser m (Maybe Year) getYear = fmap jparsedefaultyear get +-- | Get the decimal mark that has been specified for parsing, if any+-- (eg by the CSV decimal-mark rule, or possibly a future journal directive).+-- Return it as an AmountStyle that amount parsers can use.+getDecimalMarkStyle :: JournalParser m (Maybe AmountStyle)+getDecimalMarkStyle = do+  Journal{jparsedecimalmark} <- get+  let mdecmarkStyle = maybe Nothing (\c -> Just $ amountstyle{asdecimalpoint=Just c}) jparsedecimalmark+  return mdecmarkStyle+ setDefaultCommodityAndStyle :: (CommoditySymbol,AmountStyle) -> JournalParser m () setDefaultCommodityAndStyle cs = modify' (\j -> j{jparsedefaultcommodity=Just cs}) @@ -352,16 +432,14 @@ getDefaultAmountStyle :: JournalParser m (Maybe AmountStyle) getDefaultAmountStyle = fmap snd <$> getDefaultCommodityAndStyle --- | Lookup currency-specific amount style.------ Returns 'AmountStyle' used in commodity directive within current journal--- prior to current position or in its parents files.+-- | Get the 'AmountStyle' declared by the most recently parsed (in the current or parent files,+-- prior to the current position) commodity directive for the given commodity, if any. getAmountStyle :: CommoditySymbol -> JournalParser m (Maybe AmountStyle) getAmountStyle commodity = do-    specificStyle <-  maybe Nothing cformat . M.lookup commodity . jcommodities <$> get-    defaultStyle <- fmap snd <$> getDefaultCommodityAndStyle-    let effectiveStyle = listToMaybe $ catMaybes [specificStyle, defaultStyle]-    return effectiveStyle+  Journal{jcommodities} <- get+  let mspecificStyle = M.lookup commodity jcommodities >>= cformat+  mdefaultStyle <- fmap snd <$> getDefaultCommodityAndStyle+  return $ listToMaybe $ catMaybes [mspecificStyle, mdefaultStyle]  addDeclaredAccountType :: AccountName -> AccountType -> JournalParser m () addDeclaredAccountType acct atype =@@ -466,7 +544,7 @@       let dateStr = show year ++ [sep1] ++ show month ++ [sep2] ++ show day        when (sep1 /= sep2) $ customFailure $ parseErrorAtRegion startOffset endOffset $-        "invalid date (mixing date separators is not allowed): " ++ dateStr+        "invalid date: separators are different, should be the same"        case fromGregorianValid year month day of         Nothing -> customFailure $ parseErrorAtRegion startOffset endOffset $@@ -632,9 +710,26 @@ -- or right, followed by, in any order: an optional transaction price, -- an optional ledger-style lot price, and/or an optional ledger-style -- lot date. A lot price and lot date will be ignored.+--+-- To parse the amount's quantity (number) we need to know which character +-- represents a decimal mark. We find it in one of three ways:+--+-- 1. If a decimal mark has been set explicitly in the journal parse state, +--    we use that+--+-- 2. Or if the journal has a commodity declaration for the amount's commodity,+--    we get the decimal mark from  that+--+-- 3. Otherwise we will parse any valid decimal mark appearing in the+--    number, as long as the number appears well formed.+--+-- Note 3 is the default zero-config case; it means we automatically handle+-- files with any supported decimal mark, but it also allows different decimal marks+-- in  different amounts, which is a bit too loose. There's an open issue. amountp :: JournalParser m Amount amountp = label "amount" $ do-  let spaces = lift $ skipNonNewlineSpaces+  let +    spaces = lift $ skipNonNewlineSpaces   amount <- amountwithoutpricep <* spaces   (mprice, _elotprice, _elotdate) <- runPermutation $     (,,) <$> toPermutationWithDefault Nothing (Just <$> priceamountp <* spaces)@@ -642,9 +737,8 @@          <*> toPermutationWithDefault Nothing (Just <$> lotdatep <* spaces)   pure $ amount { aprice = mprice } --- XXX Just like amountp but don't allow lot prices. Needed for balanceassertionp.-amountpnolotprices :: JournalParser m Amount-amountpnolotprices = label "amount" $ do+amountpnolotpricesp :: JournalParser m Amount+amountpnolotpricesp = label "amount" $ do   let spaces = lift $ skipNonNewlineSpaces   amount <- amountwithoutpricep   spaces@@ -661,7 +755,9 @@   leftsymbolamountp :: Bool -> (Decimal -> Decimal) -> JournalParser m Amount   leftsymbolamountp mult sign = label "amount" $ do     c <- lift commoditysymbolp-    suggestedStyle <- getAmountStyle c+    mdecmarkStyle <- getDecimalMarkStyle+    mcommodityStyle <- getAmountStyle c+    let suggestedStyle = mdecmarkStyle <|> mcommodityStyle     commodityspaced <- lift skipNonNewlineSpaces'     sign2 <- lift $ signp     offBeforeNum <- getOffset@@ -684,14 +780,18 @@     case mSpaceAndCommodity of       -- right symbol amount       Just (commodityspaced, c) -> do-        suggestedStyle <- getAmountStyle c-        (q,prec,mdec,mgrps) <- lift $ interpretNumber numRegion suggestedStyle ambiguousRawNum mExponent+        mdecmarkStyle <- getDecimalMarkStyle+        mcommodityStyle <- getAmountStyle c+        let msuggestedStyle = mdecmarkStyle <|> mcommodityStyle+        (q,prec,mdec,mgrps) <- lift $ interpretNumber numRegion msuggestedStyle ambiguousRawNum mExponent         let s = amountstyle{ascommodityside=R, ascommodityspaced=commodityspaced, asprecision=prec, asdecimalpoint=mdec, asdigitgroups=mgrps}         return $ nullamt{acommodity=c, aquantity=sign q, aismultiplier=mult, astyle=s, aprice=Nothing}       -- no symbol amount       Nothing -> do-        suggestedStyle <- getDefaultAmountStyle-        (q,prec,mdec,mgrps) <- lift $ interpretNumber numRegion suggestedStyle ambiguousRawNum mExponent+        mdecmarkStyle <- getDecimalMarkStyle+        mcommodityStyle <- getDefaultAmountStyle+        let msuggestedStyle = mdecmarkStyle <|> mcommodityStyle+        (q,prec,mdec,mgrps) <- lift $ interpretNumber numRegion msuggestedStyle ambiguousRawNum mExponent         -- if a default commodity has been set, apply it and its style to this amount         -- (unless it's a multiplier in an automated posting)         defcs <- getDefaultCommodityAndStyle@@ -708,8 +808,8 @@     -> Either AmbiguousNumber RawNumber     -> Maybe Integer     -> TextParser m (Quantity, AmountPrecision, Maybe Char, Maybe DigitGroupStyle)-  interpretNumber posRegion suggestedStyle ambiguousNum mExp =-    let rawNum = either (disambiguateNumber suggestedStyle) id ambiguousNum+  interpretNumber posRegion msuggestedStyle ambiguousNum mExp =+    let rawNum = either (disambiguateNumber msuggestedStyle) id ambiguousNum     in  case fromRawNumber rawNum mExp of           Left errMsg -> customFailure $                            uncurry parseErrorAtRegion posRegion errMsg@@ -768,7 +868,7 @@   lift skipNonNewlineSpaces   -- this amount can have a price; balance assertions ignore it,   -- but balance assignments will use it-  a <- amountpnolotprices <?> "amount (for a balance assertion or assignment)"+  a <- amountpnolotpricesp <?> "amount (for a balance assertion or assignment)"   return BalanceAssertion     { baamount    = a     , batotal     = istotal@@ -846,7 +946,7 @@   -> Either String             (Quantity, Word8, Maybe Char, Maybe DigitGroupStyle) fromRawNumber (WithSeparators _ _ _) (Just _) =-    Left "invalid number: mixing digit separators with exponents is not allowed"+    Left "invalid number: digit separators and exponents may not be used together" fromRawNumber raw mExp = do     (quantity, precision) <- toQuantity (fromMaybe 0 mExp) (digitGroup raw) (decimalGroup raw)     return (quantity, precision, mDecPt raw, digitGroupStyle raw)@@ -855,7 +955,7 @@     toQuantity e preDecimalGrp postDecimalGrp       | precision < 0   = Right (Decimal 0 (digitGrpNum * 10^(-precision)), 0)       | precision < 256 = Right (Decimal precision8 digitGrpNum, precision8)-      | otherwise = Left "invalid number: numbers with more than 255 decimal digits are not allowed at this time"+      | otherwise = Left "invalid number: numbers with more than 255 decimal places are currently not supported"       where         digitGrpNum = digitGroupNumber $ preDecimalGrp <> postDecimalGrp         precision   = toInteger (digitGroupLength postDecimalGrp) - e@@ -876,13 +976,12 @@       (a:b:cs) | a < b -> b:cs       gs               -> gs - disambiguateNumber :: Maybe AmountStyle -> AmbiguousNumber -> RawNumber-disambiguateNumber suggestedStyle (AmbiguousNumber grp1 sep grp2) =+disambiguateNumber msuggestedStyle (AmbiguousNumber grp1 sep grp2) =   -- If present, use the suggested style to disambiguate;   -- otherwise, assume that the separator is a decimal point where possible.-  if isDecimalPointChar sep &&-     maybe True (sep `isValidDecimalBy`) suggestedStyle+  if isDecimalMark sep &&+     maybe True (sep `isValidDecimalBy`) msuggestedStyle   then NoSeparators grp1 (Just (sep, grp2))   else WithSeparators sep [grp1, grp2] Nothing   where@@ -917,7 +1016,7 @@   rawNumber <- fmap Right leadingDecimalPt <|> leadingDigits    -- Guard against mistyped numbers-  mExtraDecimalSep <- optional $ lookAhead $ satisfy isDecimalPointChar+  mExtraDecimalSep <- optional $ lookAhead $ satisfy isDecimalMark   when (isJust mExtraDecimalSep) $     Fail.fail "invalid number (invalid use of separator)" @@ -933,7 +1032,7 @@    leadingDecimalPt :: TextParser m RawNumber   leadingDecimalPt = do-    decPt <- satisfy isDecimalPointChar+    decPt <- satisfy isDecimalMark     decGrp <- digitgroupp     pure $ NoSeparators mempty (Just (decPt, decGrp)) @@ -954,7 +1053,7 @@    withDecimalPt :: Char -> [DigitGrp] -> TextParser m RawNumber   withDecimalPt digitSep digitGroups = do-    decPt <- satisfy $ \c -> isDecimalPointChar c && c /= digitSep+    decPt <- satisfy $ \c -> isDecimalMark c && c /= digitSep     decDigitGrp <- option mempty digitgroupp      pure $ WithSeparators digitSep digitGroups (Just (decPt, decDigitGrp))@@ -966,21 +1065,17 @@     -> [DigitGrp]     -> Either AmbiguousNumber RawNumber   withoutDecimalPt grp1 sep grp2 grps-    | null grps && isDecimalPointChar sep =+    | null grps && isDecimalMark sep =         Left $ AmbiguousNumber grp1 sep grp2     | otherwise = Right $ WithSeparators sep (grp1:grp2:grps) Nothing    trailingDecimalPt :: DigitGrp -> TextParser m RawNumber   trailingDecimalPt grp1 = do-    decPt <- satisfy isDecimalPointChar+    decPt <- satisfy isDecimalMark     pure $ NoSeparators grp1 (Just (decPt, mempty)) --isDecimalPointChar :: Char -> Bool-isDecimalPointChar c = c == '.' || c == ','- isDigitSeparatorChar :: Char -> Bool-isDigitSeparatorChar c = isDecimalPointChar c || c == ' '+isDigitSeparatorChar c = isDecimalMark c || c == ' '  -- | Some kinds of number literal we might parse. data RawNumber@@ -1333,6 +1428,39 @@     isBracketedDateChar c = isDigit c || isDateSepChar c || c == '='  {-# INLINABLE bracketeddatetagsp #-}++-- | Get the account name aliases from options, if any.+aliasesFromOpts :: InputOpts -> [AccountAlias]+aliasesFromOpts = map (\a -> fromparse $ runParser accountaliasp ("--alias "++quoteIfNeeded a) $ T.pack a)+                  . aliases_++accountaliasp :: TextParser m AccountAlias+accountaliasp = regexaliasp <|> basicaliasp++basicaliasp :: TextParser m AccountAlias+basicaliasp = do+  -- dbgparse 0 "basicaliasp"+  old <- rstrip <$> (some $ noneOf ("=" :: [Char]))+  char '='+  skipNonNewlineSpaces+  new <- rstrip <$> anySingle `manyTill` eolof  -- eol in journal, eof in command lines, normally+  return $ BasicAlias (T.pack old) (T.pack new)++regexaliasp :: TextParser m AccountAlias+regexaliasp = do+  -- dbgparse 0 "regexaliasp"+  char '/'+  off1 <- getOffset+  re <- some $ noneOf ("/\n\r" :: [Char]) -- paranoid: don't try to read past line end+  off2 <- getOffset+  char '/'+  skipNonNewlineSpaces+  char '='+  skipNonNewlineSpaces+  repl <- anySingle `manyTill` eolof+  case toRegexCI re of+    Right r -> return $! RegexAlias r repl+    Left e  -> customFailure $! parseErrorAtRegion off1 off2 e  --- ** tests 

@@ -78,7 +78,7 @@  import Hledger.Data import Hledger.Utils-import Hledger.Read.Common (Reader(..),InputOpts(..),amountp, statusp, genericSourcePos, journalFinalise)+import Hledger.Read.Common (aliasesFromOpts,  Reader(..),InputOpts(..), amountp, statusp, genericSourcePos, journalFinalise )  --- ** doctest setup -- $setup@@ -108,13 +108,16 @@   let rulesfile = mrules_file_ iopts   r <- liftIO $ readJournalFromCsv rulesfile f t   case r of Left e   -> throwError e-            Right pj -> journalFinalise iopts{ignore_assertions_=True} f t pj'-              where-                -- journalFinalise assumes the journal's items are-                -- reversed, as produced by JournalReader's parser.-                -- But here they are already properly ordered. So we'd-                -- better preemptively reverse them once more. XXX inefficient-                pj' = journalReverse pj+            Right pj ->+              -- journalFinalise assumes the journal's items are+              -- reversed, as produced by JournalReader's parser.+              -- But here they are already properly ordered. So we'd+              -- better preemptively reverse them once more. XXX inefficient+              let pj' = journalReverse pj+              -- apply any command line account aliases. Can fail with a bad replacement pattern.+              in case journalApplyAliases (aliasesFromOpts iopts) pj' of+                  Left e -> throwError e+                  Right pj'' -> journalFinalise iopts{ignore_assertions_=True} f t pj''  --- ** reading rules files --- *** rules utilities@@ -263,12 +266,12 @@   r1 == r2 = (rdirectives r1, rcsvfieldindexes r1, rassignments r1) ==              (rdirectives r2, rcsvfieldindexes r2, rassignments r2)  --- It is used for debug output only+-- Custom Show instance used for debug output: omit the rblocksassigning field, which isn't showable. instance Show CsvRules where-  show r = "CsvRules { rdirectives=" ++ show (rdirectives r) ++-           ", rcsvfieldindexes=" ++ show (rcsvfieldindexes r) ++-           ", rassignments=" ++ show (rassignments r) ++-           ", rconditionalblocks="++ show (rconditionalblocks r) +++  show r = "CsvRules { rdirectives = " ++ show (rdirectives r) +++           ", rcsvfieldindexes = "     ++ show (rcsvfieldindexes r) +++           ", rassignments = "         ++ show (rassignments r) +++           ", rconditionalblocks = "   ++ show (rconditionalblocks r) ++            " }"  type CsvRulesParser a = StateT CsvRulesParsed SimpleTextParser a@@ -364,7 +367,7 @@  RULES: RULE* -RULE: ( FIELD-LIST | FIELD-ASSIGNMENT | CONDITIONAL-BLOCK | SKIP | NEWEST-FIRST | DATE-FORMAT | COMMENT | BLANK ) NEWLINE+RULE: ( FIELD-LIST | FIELD-ASSIGNMENT | CONDITIONAL-BLOCK | SKIP | NEWEST-FIRST | DATE-FORMAT | DECIMAL-MARK | COMMENT | BLANK ) NEWLINE  FIELD-LIST: fields SPACE FIELD-NAME ( SPACE? , SPACE? FIELD-NAME )* @@ -462,6 +465,7 @@ directives :: [String] directives =   ["date-format"+  ,"decimal-mark"   ,"separator"   -- ,"default-account"   -- ,"default-currency"@@ -711,7 +715,7 @@     else       return $ defaultRulesText rulesfile   rules <- either throwerr return $ parseAndValidateCsvRules rulesfile rulestext-  dbg6IO "rules" rules+  dbg6IO "csv rules" rules    -- parse the skip directive's value, if any   let skiplines = case getDirective "skip" rules of@@ -744,7 +748,7 @@    let     -- convert CSV records to transactions-    txns = snd $ mapAccumL+    txns = dbg7 "csv txns" $ snd $ mapAccumL                    (\pos r ->                       let                         SourcePos name line col = pos@@ -761,7 +765,9 @@     -- than one date and the first date is more recent than the last):     -- reverse them to get same-date transactions ordered chronologically.     txns' =-      (if newestfirst || mdataseemsnewestfirst == Just True then reverse else id) txns+      (if newestfirst || mdataseemsnewestfirst == Just True +        then dbg7 "reversed csv txns" . reverse else id) +        txns       where         newestfirst = dbg6 "newestfirst" $ isJust $ getDirective "newest-first" rules         mdataseemsnewestfirst = dbg6 "mdataseemsnewestfirst" $@@ -769,7 +775,7 @@             ds | length ds > 1 -> Just $ head ds > last ds             _                  -> Nothing     -- Second, sort by date.-    txns'' = sortBy (comparing tdate) txns'+    txns'' = dbg7 "date-sorted csv txns" $ sortBy (comparing tdate) txns'    when (not rulesfileexists) $ do     dbg1IO "creating conversion rules file" rulesfile@@ -977,7 +983,7 @@ getAmount rules record currency p1IsVirtual n =   -- Warning, many tricky corner cases here.   -- docs: hledger_csv.m4.md #### amount-  -- tests: tests/csv.test ~ 13, 31-34+  -- tests: hledger/test/csv.test ~ 13, 31-34   let     unnumberedfieldnames = ["amount","amount-in","amount-out"] @@ -1024,56 +1030,77 @@              "\t=> value: " ++ showMixedAmount a -- XXX not sure this is showing all the right info            | (f,a) <- fs] -  where-    -- | Given a non-empty amount string to parse, along with a possibly-    -- non-empty currency symbol to prepend, parse as a hledger amount (as-    -- in journal format), or raise an error.-    -- The CSV rules and record are provided for the error message.-    parseAmount :: CsvRules -> CsvRecord -> String -> String -> MixedAmount-    parseAmount rules record currency amountstr =-      either mkerror (Mixed . (:[])) $  -- PARTIAL:-      runParser (evalStateT (amountp <* eof) nulljournal) "" $-      T.pack $ (currency++) $ simplifySign amountstr-      where-        mkerror e = error' $ unlines-          ["error: could not parse \""++amountstr++"\" as an amount"-          ,showRecord record-          ,showRules rules record-          -- ,"the default-currency is: "++fromMaybe "unspecified" (getDirective "default-currency" rules)-          ,"the parse error is:      "++customErrorBundlePretty e-          ,"you may need to "-           ++"change your amount*, balance*, or currency* rules, "-           ++"or add or change your skip rule"-          ]- -- | Figure out the expected balance (assertion or assignment) specified for posting N, -- if any (and its parse position). getBalance :: CsvRules -> CsvRecord -> String -> Int -> Maybe (Amount, GenericSourcePos)-getBalance rules record currency n =-  (fieldval ("balance"++show n)-    -- for posting 1, also recognise the old field name-    <|> if n==1 then fieldval "balance" else Nothing)-  >>= parsebalance currency n . strip+getBalance rules record currency n = do+  v <- (fieldval ("balance"++show n)+        -- for posting 1, also recognise the old field name+        <|> if n==1 then fieldval "balance" else Nothing)+  case v of+    "" -> Nothing+    s  -> Just (+            parseBalanceAmount rules record currency n s+           ,nullsourcepos  -- parse position to show when assertion fails,+           )               -- XXX the csv record's line number would be good+     where-    parsebalance currency n s-      | null s    = Nothing-      | otherwise = Just-          (either (mkerror n s) id $-            runParser (evalStateT (amountp <* eof) nulljournal) "" $-            T.pack $ (currency++) $ simplifySign s-          ,nullsourcepos)  -- XXX parse position to show when assertion fails,-                           -- the csv record's line number would be good-      where-        mkerror n s e = error' $ unlines-          ["error: could not parse \""++s++"\" as balance"++show n++" amount"-          ,showRecord record-          ,showRules rules record-          -- ,"the default-currency is: "++fromMaybe "unspecified" mdefaultcurrency-          ,"the parse error is:      "++customErrorBundlePretty e-          ]-    -- mdefaultcurrency = rule "default-currency"-    fieldval = hledgerFieldValue rules record :: HledgerFieldName -> Maybe String+    fieldval = fmap strip . hledgerFieldValue rules record :: HledgerFieldName -> Maybe String +-- | Given a non-empty amount string (from CSV) to parse, along with a+-- possibly non-empty currency symbol to prepend,+-- parse as a hledger MixedAmount (as in journal format), or raise an error.+-- The whole CSV record is provided for the error message.+parseAmount :: CsvRules -> CsvRecord -> String -> String -> MixedAmount+parseAmount rules record currency s =+  either mkerror (Mixed . (:[])) $  -- PARTIAL:+  runParser (evalStateT (amountp <* eof) journalparsestate) "" $+  T.pack $ (currency++) $ simplifySign s+  where+    journalparsestate = nulljournal{jparsedecimalmark=parseDecimalMark rules}+    mkerror e = error' $ unlines+      ["error: could not parse \""++s++"\" as an amount"+      ,showRecord record+      ,showRules rules record+      -- ,"the default-currency is: "++fromMaybe "unspecified" (getDirective "default-currency" rules)+      ,"the parse error is:      "++customErrorBundlePretty e+      ,"you may need to "+        ++"change your amount*, balance*, or currency* rules, "+        ++"or add or change your skip rule"+      ]++-- XXX unify these ^v++-- | Almost but not quite the same as parseAmount.+-- Given a non-empty amount string (from CSV) to parse, along with a+-- possibly non-empty currency symbol to prepend,+-- parse as a hledger Amount (as in journal format), or raise an error.+-- The CSV record and the field's numeric suffix are provided for the error message.+parseBalanceAmount :: CsvRules -> CsvRecord -> String -> Int -> String -> Amount+parseBalanceAmount rules record currency n s =+  either (mkerror n s) id $+    runParser (evalStateT (amountp <* eof) journalparsestate) "" $+    T.pack $ (currency++) $ simplifySign s+                  -- the csv record's line number would be good+  where+    journalparsestate = nulljournal{jparsedecimalmark=parseDecimalMark rules}+    mkerror n s e = error' $ unlines+      ["error: could not parse \""++s++"\" as balance"++show n++" amount"+      ,showRecord record+      ,showRules rules record+      -- ,"the default-currency is: "++fromMaybe "unspecified" mdefaultcurrency+      ,"the parse error is:      "++customErrorBundlePretty e+      ]++-- Read a valid decimal mark from the decimal-mark rule, if any.+-- If the rule is present with an invalid argument, raise an error.+parseDecimalMark :: CsvRules -> Maybe DecimalMark+parseDecimalMark rules =+  case rules `csvRule` "decimal-mark" of+    Nothing -> Nothing+    Just [c] | isDecimalMark c -> Just c+    Just s -> error' $ "decimal-mark's argument should be \".\" or \",\" (not \""++s++"\")"+ -- | Make a balance assertion for the given amount, with the given parse -- position (to be shown in assertion failures), with the assertion type -- possibly set by a balance-type rule.@@ -1168,7 +1195,7 @@ getEffectiveAssignment rules record f = lastMay $ map snd $ assignments   where     -- all active assignments to field f, in order-    assignments = dbg7 "assignments" $ filter ((==f).fst) $ toplevelassignments ++ conditionalassignments+    assignments = dbg9 "csv assignments" $ filter ((==f).fst) $ toplevelassignments ++ conditionalassignments       where         -- all top level field assignments         toplevelassignments    = rassignments rules

@@ -179,11 +179,6 @@       mapM_ addAccountAlias (reverse $ aliasesFromOpts iopts)       journalp --- | Get the account name aliases from options, if any.-aliasesFromOpts :: InputOpts -> [AccountAlias]-aliasesFromOpts = map (\a -> fromparse $ runParser accountaliasp ("--alias "++quoteIfNeeded a) $ T.pack a)-                  . aliases_- --- ** parsers --- *** journal @@ -504,34 +499,6 @@   lift skipNonNewlineSpaces1   alias <- lift accountaliasp   addAccountAlias alias--accountaliasp :: TextParser m AccountAlias-accountaliasp = regexaliasp <|> basicaliasp--basicaliasp :: TextParser m AccountAlias-basicaliasp = do-  -- dbgparse 0 "basicaliasp"-  old <- rstrip <$> (some $ noneOf ("=" :: [Char]))-  char '='-  skipNonNewlineSpaces-  new <- rstrip <$> anySingle `manyTill` eolof  -- eol in journal, eof in command lines, normally-  return $ BasicAlias (T.pack old) (T.pack new)--regexaliasp :: TextParser m AccountAlias-regexaliasp = do-  -- dbgparse 0 "regexaliasp"-  char '/'-  off1 <- getOffset-  re <- some $ noneOf ("/\n\r" :: [Char]) -- paranoid: don't try to read past line end-  off2 <- getOffset-  char '/'-  skipNonNewlineSpaces-  char '='-  skipNonNewlineSpaces-  repl <- anySingle `manyTill` eolof-  case toRegexCI re of-    Right r -> return $! RegexAlias r repl-    Left e  -> customFailure $! parseErrorAtRegion off1 off2 e  endaliasesdirectivep :: JournalParser m () endaliasesdirectivep = do

@@ -41,5 +41,4 @@   ,tests_EntriesReport   ,tests_MultiBalanceReport   ,tests_PostingsReport-  ,tests_ReportOptions   ]

@@ -82,8 +82,8 @@ totallabel   = "Period Total" balancelabel = "Historical Total" -accountTransactionsReport :: ReportOpts -> Journal -> Query -> Query -> AccountTransactionsReport-accountTransactionsReport ropts j reportq thisacctq = (label, items)+accountTransactionsReport :: ReportSpec -> Journal -> Query -> Query -> AccountTransactionsReport+accountTransactionsReport rspec@ReportSpec{rsOpts=ropts} j reportq thisacctq = (label, items)   where     -- a depth limit should not affect the account transactions report     -- seems unnecessary for some reason XXX@@ -115,12 +115,11 @@     styles = journalCommodityStyles j     periodlast =       fromMaybe (error' "journalApplyValuation: expected a non-empty journal") $ -- XXX shouldn't happen-      reportPeriodOrJournalLastDay ropts j-    mreportlast = reportPeriodLastDay ropts-    today = fromMaybe (error' "journalApplyValuation: could not pick a valuation date, ReportOpts today_ is unset") $ today_ ropts -- XXX shouldn't happen+      reportPeriodOrJournalLastDay rspec j+    mreportlast = reportPeriodLastDay rspec     multiperiod = interval_ ropts /= NoInterval     tval = case value_ ropts of-             Just v  -> \t -> transactionApplyValuation prices styles periodlast mreportlast today multiperiod t v+             Just v  -> \t -> transactionApplyValuation prices styles periodlast mreportlast (rsToday rspec) multiperiod t v              Nothing -> id     ts4 =       ptraceAtWith 5 (("ts4:\n"++).pshowTransactions) $

@@ -23,7 +23,7 @@ import Hledger.Read (mamountp') import Hledger.Query import Hledger.Utils-import Hledger.Reports.MultiBalanceReport (multiBalanceReportWith)+import Hledger.Reports.MultiBalanceReport (multiBalanceReport) import Hledger.Reports.ReportOptions import Hledger.Reports.ReportTypes @@ -61,10 +61,10 @@ -- their balances (change of balance) during the specified period. -- If the normalbalance_ option is set, it adjusts the sorting and sign of -- amounts (see ReportOpts and CompoundBalanceCommand).-balanceReport :: ReportOpts -> Query -> Journal -> BalanceReport-balanceReport ropts q j = (rows, total)+balanceReport :: ReportSpec -> Journal -> BalanceReport+balanceReport rspec j = (rows, total)   where-    report = multiBalanceReportWith ropts q j (journalPriceOracle (infer_value_ ropts) j)+    report = multiBalanceReport rspec j     rows = [( prrFullName row             , prrDisplayName row             , prrDepth row - 1  -- BalanceReport uses 0-based account depths@@ -101,9 +101,10 @@ tests_BalanceReport = tests "BalanceReport" [    let-    (opts,journal) `gives` r = do-      let (eitems, etotal) = r-          (aitems, atotal) = balanceReport opts (queryFromOpts nulldate opts) journal+    (rspec,journal) `gives` r = do+      let opts' = rspec{rsQuery=And [queryFromFlags $ rsOpts rspec, rsQuery rspec]}+          (eitems, etotal) = r+          (aitems, atotal) = balanceReport opts' journal           showw (acct,acct',indent,amt) = (acct, acct', indent, showMixedAmountDebug amt)       (map showw aitems) @?= (map showw eitems)       (showMixedAmountDebug atotal) @?= (showMixedAmountDebug etotal)@@ -111,10 +112,10 @@     tests "balanceReport" [       test "no args, null journal" $-     (defreportopts, nulljournal) `gives` ([], 0)+     (defreportspec, nulljournal) `gives` ([], 0)      ,test "no args, sample journal" $-     (defreportopts, samplejournal) `gives`+     (defreportspec, samplejournal) `gives`       ([         ("assets:bank:checking","assets:bank:checking",0, mamountp' "$1.00")        ,("assets:bank:saving","assets:bank:saving",0, mamountp' "$1.00")@@ -127,7 +128,7 @@        Mixed [usd 0])      ,test "with --tree" $-     (defreportopts{accountlistmode_=ALTree}, samplejournal) `gives`+     (defreportspec{rsOpts=defreportopts{accountlistmode_=ALTree}}, samplejournal) `gives`       ([         ("assets","assets",0, mamountp' "$0.00")        ,("assets:bank","bank",1, mamountp' "$2.00")@@ -144,7 +145,7 @@        Mixed [usd 0])      ,test "with --depth=N" $-     (defreportopts{depth_=Just 1}, samplejournal) `gives`+     (defreportspec{rsOpts=defreportopts{depth_=Just 1}}, samplejournal) `gives`       ([        ("expenses",    "expenses",    0, mamountp'  "$2.00")        ,("income",      "income",      0, mamountp' "$-2.00")@@ -152,7 +153,7 @@        Mixed [usd 0])      ,test "with depth:N" $-     (defreportopts{query_="depth:1"}, samplejournal) `gives`+     (defreportspec{rsQuery=Depth 1}, samplejournal) `gives`       ([        ("expenses",    "expenses",    0, mamountp'  "$2.00")        ,("income",      "income",      0, mamountp' "$-2.00")@@ -160,11 +161,11 @@        Mixed [usd 0])      ,test "with date:" $-     (defreportopts{query_="date:'in 2009'"}, samplejournal2) `gives`+     (defreportspec{rsQuery=Date $ DateSpan (Just $ fromGregorian 2009 01 01) (Just $ fromGregorian 2010 01 01)}, samplejournal2) `gives`       ([], 0)      ,test "with date2:" $-     (defreportopts{query_="date2:'in 2009'"}, samplejournal2) `gives`+     (defreportspec{rsQuery=Date2 $ DateSpan (Just $ fromGregorian 2009 01 01) (Just $ fromGregorian 2010 01 01)}, samplejournal2) `gives`       ([         ("assets:bank:checking","assets:bank:checking",0,mamountp' "$1.00")        ,("income:salary","income:salary",0,mamountp' "$-1.00")@@ -172,7 +173,7 @@        Mixed [usd 0])      ,test "with desc:" $-     (defreportopts{query_="desc:income"}, samplejournal) `gives`+     (defreportspec{rsQuery=Desc $ toRegexCI' "income"}, samplejournal) `gives`       ([         ("assets:bank:checking","assets:bank:checking",0,mamountp' "$1.00")        ,("income:salary","income:salary",0, mamountp' "$-1.00")@@ -180,7 +181,7 @@        Mixed [usd 0])      ,test "with not:desc:" $-     (defreportopts{query_="not:desc:income"}, samplejournal) `gives`+     (defreportspec{rsQuery=Not . Desc $ toRegexCI' "income"}, samplejournal) `gives`       ([         ("assets:bank:saving","assets:bank:saving",0, mamountp' "$1.00")        ,("assets:cash","assets:cash",0, mamountp' "$-2.00")@@ -191,7 +192,7 @@        Mixed [usd 0])      ,test "with period on a populated period" $-      (defreportopts{period_= PeriodBetween (fromGregorian 2008 1 1) (fromGregorian 2008 1 2)}, samplejournal) `gives`+      (defreportspec{rsOpts=defreportopts{period_= PeriodBetween (fromGregorian 2008 1 1) (fromGregorian 2008 1 2)}}, samplejournal) `gives`        (         [          ("assets:bank:checking","assets:bank:checking",0, mamountp' "$1.00")@@ -200,7 +201,7 @@         Mixed [usd 0])       ,test "with period on an unpopulated period" $-      (defreportopts{period_= PeriodBetween (fromGregorian 2008 1 2) (fromGregorian 2008 1 3)}, samplejournal) `gives`+      (defreportspec{rsOpts=defreportopts{period_= PeriodBetween (fromGregorian 2008 1 2) (fromGregorian 2008 1 3)}}, samplejournal) `gives`        ([], 0)  

@@ -1,11 +1,12 @@ {- | -} -{-# LANGUAGE CPP #-}-{-# LANGUAGE NamedFieldPuns #-}-{-# LANGUAGE OverloadedStrings #-}-{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE CPP                 #-}+{-# LANGUAGE NamedFieldPuns      #-}+{-# LANGUAGE OverloadedStrings   #-}+{-# LANGUAGE RecordWildCards     #-} {-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TupleSections       #-}  module Hledger.Reports.BudgetReport (   BudgetGoal,@@ -17,6 +18,7 @@   budgetReport,   budgetReportAsTable,   budgetReportAsText,+  budgetReportAsCsv,   -- * Helpers   reportPeriodName,   -- * Tests@@ -25,6 +27,7 @@ where  import Data.Decimal+import Data.Default (def) import Data.HashMap.Strict (HashMap) import qualified Data.HashMap.Strict as HM import Data.List@@ -33,7 +36,6 @@ #if !(MIN_VERSION_base(4,11,0)) import Data.Monoid ((<>)) #endif-import Data.Time.Calendar import Safe --import Data.List --import Data.Maybe@@ -43,11 +45,14 @@ --import qualified Data.Text.Lazy as TL --import System.Console.CmdArgs.Explicit as C --import Lucid as L+ import Text.Printf (printf) import Text.Tabular as T+import Text.Tabular.AsciiWide as T  import Hledger.Data import Hledger.Utils+import Hledger.Read.CsvReader (CSV) import Hledger.Reports.ReportOptions import Hledger.Reports.ReportTypes import Hledger.Reports.MultiBalanceReport@@ -62,31 +67,33 @@ type BudgetReportRow = PeriodicReportRow DisplayName BudgetCell type BudgetReport    = PeriodicReport    DisplayName BudgetCell --- | Calculate budget goals from all periodic transactions,--- actual balance changes from the regular transactions,--- and compare these to get a 'BudgetReport'.--- Unbudgeted accounts may be hidden or renamed (see budgetRollup).-budgetReport :: ReportOpts -> Bool -> DateSpan -> Day -> Journal -> BudgetReport-budgetReport ropts' assrt reportspan d j = dbg1 "sortedbudgetreport" budgetreport+type BudgetDisplayCell = ((String, Int), Maybe ((String, Int), Maybe (String, Int)))++-- | Calculate per-account, per-period budget (balance change) goals+-- from all periodic transactions, calculate actual balance changes +-- from the regular transactions, and compare these to get a 'BudgetReport'.+-- Unbudgeted accounts may be hidden or renamed (see journalWithBudgetAccountNames).+budgetReport :: ReportSpec -> Bool -> DateSpan -> Journal -> BudgetReport+budgetReport rspec assrt reportspan j = dbg4 "sortedbudgetreport" budgetreport   where     -- Budget report demands ALTree mode to ensure subaccounts and subaccount budgets are properly handled     -- and that reports with and without --empty make sense when compared side by side-    ropts = ropts' { accountlistmode_ = ALTree }+    ropts = (rsOpts rspec){ accountlistmode_ = ALTree }     showunbudgeted = empty_ ropts     budgetedaccts =-      dbg2 "budgetedacctsinperiod" $+      dbg3 "budgetedacctsinperiod" $       nub $       concatMap expandAccountName $       accountNamesFromPostings $       concatMap tpostings $       concatMap (`runPeriodicTransaction` reportspan) $       jperiodictxns j-    actualj = dbg1With (("actualj"++).show.jtxns)  $ budgetRollUp budgetedaccts showunbudgeted j-    budgetj = dbg1With (("budgetj"++).show.jtxns)  $ budgetJournal assrt ropts reportspan j+    actualj = journalWithBudgetAccountNames budgetedaccts showunbudgeted j+    budgetj = journalAddBudgetGoalTransactions assrt ropts reportspan j     actualreport@(PeriodicReport actualspans _ _) =-        dbg1 "actualreport" $ multiBalanceReport d ropts{empty_=True} actualj+        dbg5 "actualreport" $ multiBalanceReport rspec{rsOpts=ropts{empty_=True}} actualj     budgetgoalreport@(PeriodicReport _ budgetgoalitems budgetgoaltotals) =-        dbg1 "budgetgoalreport" $ multiBalanceReport d ropts{empty_=True} budgetj+        dbg5 "budgetgoalreport" $ multiBalanceReport rspec{rsOpts=ropts{empty_=True}} budgetj     budgetgoalreport'       -- If no interval is specified:       -- budgetgoalreport's span might be shorter actualreport's due to periodic txns;@@ -96,16 +103,17 @@     budgetreport = combineBudgetAndActual ropts j budgetgoalreport' actualreport  -- | Use all periodic transactions in the journal to generate--- budget transactions in the specified report period.--- Budget transactions are similar to forecast transactions except--- their purpose is to set goal amounts (of change) per account and period.-budgetJournal :: Bool -> ReportOpts -> DateSpan -> Journal -> Journal-budgetJournal assrt _ropts reportspan j =+-- budget goal transactions in the specified date span.+-- Budget goal transactions are similar to forecast transactions except+-- their purpose and effect is to define balance change goals, per account and period,+-- for BudgetReport.+journalAddBudgetGoalTransactions :: Bool -> ReportOpts -> DateSpan -> Journal -> Journal+journalAddBudgetGoalTransactions assrt _ropts reportspan j =   either error' id $ journalBalanceTransactions assrt j{ jtxns = budgetts }  -- PARTIAL:   where-    budgetspan = dbg2 "budgetspan" $ reportspan+    budgetspan = dbg3 "budget span" $ reportspan     budgetts =-      dbg1 "budgetts" $+      dbg5 "budget goal txns" $       [makeBudgetTxn t       | pt <- jperiodictxns j       , t <- runPeriodicTransaction pt budgetspan@@ -119,10 +127,12 @@ -- -- 2. subaccounts with no budget goal are merged with their closest parent account --    with a budget goal, so that only budgeted accounts are shown.---    This can be disabled by --empty.+--    This can be disabled by -E/--empty. ---budgetRollUp :: [AccountName] -> Bool -> Journal -> Journal-budgetRollUp budgetedaccts showunbudgeted j = j { jtxns = remapTxn <$> jtxns j }+journalWithBudgetAccountNames :: [AccountName] -> Bool -> Journal -> Journal+journalWithBudgetAccountNames budgetedaccts showunbudgeted j = +  dbg5With (("budget account names: "++).pshow.journalAccountNamesUsed) $ +  j { jtxns = remapTxn <$> jtxns j }   where     remapTxn = mapPostings (map remapPosting)       where@@ -210,8 +220,9 @@ -- | Render a budget report as plain text suitable for console output. budgetReportAsText :: ReportOpts -> BudgetReport -> String budgetReportAsText ropts@ReportOpts{..} budgetr =-  title ++ "\n\n" ++-  tableAsText ropts showcell (maybetranspose $ budgetReportAsTable ropts budgetr)+    title ++ "\n\n" +++    renderTable def{tableBorders=False,prettyTable=pretty_tables_}+        (alignCell TopLeft) (alignCell TopRight) (uncurry showcell) displayTableWithWidths   where     multiperiod = interval_ /= NoInterval     title = printf "Budget performance in %s%s:"@@ -226,31 +237,39 @@         Just (AtDefault _mc)  -> ", current value"         Just (AtDate d _mc) -> ", valued at "++showDate d         Nothing             -> "")-    actualwidth = maximum' $ map fst amountsAndGoals-    budgetwidth = maximum' $ map snd amountsAndGoals-    amountsAndGoals =-      map (\(a,g) -> (amountWidth a, amountWidth g)) . concatMap prrAmounts $ prRows budgetr++    displayTableWithWidths :: Table String String ((Int, Int, Int), BudgetDisplayCell)+    displayTableWithWidths = Table rh ch $ map (zipWith (,) widths) displaycells+    Table rh ch displaycells = case budgetReportAsTable ropts budgetr of+        Table rh' ch' vals -> maybetranspose . Table rh' ch' $ map (map displayCell) vals++    displayCell (actual, budget) = (showamt actual', budgetAndPerc <$> budget)       where-        amountWidth = maybe 0 (length . showMixedAmountElided False)+        actual' = fromMaybe 0 actual+        budgetAndPerc b = (showamt b, showper <$> percentage actual' b)+        showamt = showMixedOneLine showAmountWithoutPrice Nothing (Just 32) color_+        showper p = let str = show (roundTo 0 p) in (str, length str)+    cellWidth ((_,wa), Nothing)                    = (wa,  0,  0)+    cellWidth ((_,wa), Just ((_,wb), Nothing))     = (wa, wb,  0)+    cellWidth ((_,wa), Just ((_,wb), Just (_,wp))) = (wa, wb, wp)++    widths = zip3 actualwidths budgetwidths percentwidths+    actualwidths  = map (maximum' . map (first3  . cellWidth)) cols+    budgetwidths  = map (maximum' . map (second3 . cellWidth)) cols+    percentwidths = map (maximum' . map (third3  . cellWidth)) cols+    cols = transpose displaycells+     -- XXX lay out actual, percentage and/or goal in the single table cell for now, should probably use separate cells-    showcell :: BudgetCell -> String-    showcell (mactual, mbudget) = actualstr ++ " " ++ budgetstr+    showcell :: (Int, Int, Int) -> BudgetDisplayCell -> Cell+    showcell (actualwidth, budgetwidth, percentwidth) ((actual,wa), mbudget) =+        Cell TopRight [(replicate (actualwidth - wa) ' ' ++ actual ++ budgetstr, actualwidth + totalbudgetwidth)]       where-        percentwidth = 4-        actual = fromMaybe 0 mactual-        actualstr = printf ("%"++show actualwidth++"s") (showamt actual)+        totalpercentwidth = if percentwidth == 0 then 0 else percentwidth + 5+        totalbudgetwidth  = if budgetwidth == 0 then 0 else budgetwidth + totalpercentwidth + 3         budgetstr = case mbudget of-          Nothing     -> replicate (percentwidth + 7 + budgetwidth) ' '-          Just budget ->-            case percentage actual budget of-              Just pct ->-                printf ("[%"++show percentwidth++"s%% of %"++show budgetwidth++"s]")-                       (show $ roundTo 0 pct) (showamt' budget)-              Nothing ->-                printf ("["++replicate (percentwidth+5) ' '++"%"++show budgetwidth++"s]")-                       (showamt' budget)-        showamt = showMixedAmountElided color_-        showamt' = showMixedAmountElided False  -- XXX colored budget amounts disrupts layout+          Nothing                             -> replicate totalbudgetwidth ' '+          Just ((budget, wb), Nothing)        -> " [" ++ replicate totalpercentwidth ' ' ++ replicate (budgetwidth - wb) ' ' ++ budget ++ "]"+          Just ((budget, wb), Just (pct, wp)) -> " [" ++ replicate (percentwidth - wp) ' ' ++ pct ++ "% of " ++ replicate (budgetwidth - wb) ' ' ++ budget ++ "]"      -- | Calculate the percentage of actual change to budget goal to show, if any.     -- If valuing at cost, both amounts are converted to cost before comparing.@@ -321,6 +340,45 @@       where         multiyear = (>1) $ length $ nubSort $ map spanStartYear spans     _ -> maybe "" (showDate . prevday) . spanEnd++-- XXX generalise this with multiBalanceReportAsCsv ?+-- | Render a budget report as CSV. Like multiBalanceReportAsCsv,+-- but includes alternating actual and budget amount columns.+budgetReportAsCsv :: ReportOpts -> BudgetReport -> CSV+budgetReportAsCsv +  ReportOpts{average_, row_total_, no_total_, transpose_}+  (PeriodicReport colspans items (PeriodicReportRow _ abtotals (magrandtot,mbgrandtot) (magrandavg,mbgrandavg)))+  = (if transpose_ then transpose else id) $++  -- heading row+  ("Account" : +   concatMap (\span -> [showDateSpan span, "budget"]) colspans+   ++ concat [["Total"  ,"budget"] | row_total_]+   ++ concat [["Average","budget"] | average_]+  ) :++  -- account rows+  [T.unpack (displayFull a) :+   map showmamt (flattentuples abamts)+   ++ concat [[showmamt mactualrowtot, showmamt mbudgetrowtot] | row_total_]+   ++ concat [[showmamt mactualrowavg, showmamt mbudgetrowavg] | average_]+  | PeriodicReportRow a abamts (mactualrowtot,mbudgetrowtot) (mactualrowavg,mbudgetrowavg) <- items+  ]++  -- totals row+  ++ concat [+    [+    "Total:" :+    map showmamt (flattentuples abtotals)+    ++ concat [[showmamt magrandtot,showmamt mbgrandtot] | row_total_] +    ++ concat [[showmamt magrandavg,showmamt mbgrandavg] | average_]+    ]+  | not no_total_+  ]++  where+    flattentuples abs = concat [[a,b] | (a,b) <- abs]+    showmamt = maybe "" (showMixedAmountOneLineWithoutPrice False)  -- tests 

@@ -32,26 +32,25 @@ type EntriesReportItem = Transaction  -- | Select transactions for an entries report.-entriesReport :: ReportOpts -> Query -> Journal -> EntriesReport-entriesReport ropts@ReportOpts{..} q j@Journal{..} =-  sortBy (comparing getdate) $ filter (q `matchesTransaction`) $ map tvalue jtxns+entriesReport :: ReportSpec -> Journal -> EntriesReport+entriesReport rspec@ReportSpec{rsOpts=ropts@ReportOpts{..}} j@Journal{..} =+  sortBy (comparing getdate) $ filter (rsQuery rspec `matchesTransaction`) $ map tvalue jtxns   where     getdate = transactionDateFn ropts     -- We may be converting posting amounts to value, per hledger_options.m4.md "Effect of --value on reports".     tvalue t@Transaction{..} = t{tpostings=map pvalue tpostings}       where         pvalue p = maybe p-          (postingApplyValuation (journalPriceOracle infer_value_ j) (journalCommodityStyles j) periodlast mreportlast today False p)+          (postingApplyValuation (journalPriceOracle infer_value_ j) (journalCommodityStyles j) periodlast mreportlast (rsToday rspec) False p)           value_           where-            periodlast  = fromMaybe today $ reportPeriodOrJournalLastDay ropts j-            mreportlast = reportPeriodLastDay ropts-            today       = fromMaybe (error' "erValue: could not pick a valuation date, ReportOpts today_ is unset") today_  -- PARTIAL: should not happen+            periodlast  = fromMaybe (rsToday rspec) $ reportPeriodOrJournalLastDay rspec j+            mreportlast = reportPeriodLastDay rspec  tests_EntriesReport = tests "EntriesReport" [   tests "entriesReport" [-     test "not acct" $ (length $ entriesReport defreportopts (Not . Acct $ toRegex' "bank") samplejournal) @?= 1-    ,test "date" $ (length $ entriesReport defreportopts (Date $ DateSpan (Just $ fromGregorian 2008 06 01) (Just $ fromGregorian 2008 07 01)) samplejournal) @?= 3+     test "not acct" $ (length $ entriesReport defreportspec{rsQuery=Not . Acct $ toRegex' "bank"} samplejournal) @?= 1+    ,test "date" $ (length $ entriesReport defreportspec{rsQuery=Date $ DateSpan (Just $ fromGregorian 2008 06 01) (Just $ fromGregorian 2008 07 01)} samplejournal) @?= 3   ]  ] 

@@ -16,15 +16,21 @@   multiBalanceReport,   multiBalanceReportWith, -  CompoundBalanceReport,   compoundBalanceReport,   compoundBalanceReportWith, -  tableAsText,-   sortRows,   sortRowsLike, +  -- * Helper functions+  calculateReportSpan,+  makeReportQuery,+  getPostingsByColumn,+  getPostings,+  calculateColSpans,+  startingBalances,+  generateMultiBalanceReport,+   -- -- * Tests   tests_MultiBalanceReport )@@ -45,18 +51,23 @@ #endif import Data.Semigroup (sconcat) import Data.Time.Calendar (Day, addDays, fromGregorian)-import Safe (headMay, lastDef, lastMay)-import Text.Tabular as T-import Text.Tabular.AsciiWide (render)+import Safe (headMay, lastDef, lastMay, minimumMay)  import Hledger.Data import Hledger.Query-import Hledger.Utils+import Hledger.Utils hiding (dbg3,dbg4,dbg5)+import qualified Hledger.Utils import Hledger.Read (mamountp') import Hledger.Reports.ReportOptions import Hledger.Reports.ReportTypes  +-- add a prefix to this function's debug output+dbg3 s = let p = "multiBalanceReport" in Hledger.Utils.dbg3 (p++" "++s)+dbg4 s = let p = "multiBalanceReport" in Hledger.Utils.dbg4 (p++" "++s)+dbg5 s = let p = "multiBalanceReport" in Hledger.Utils.dbg5 (p++" "++s)++ -- | A multi balance report is a kind of periodic report, where the amounts -- correspond to balance changes or ending balances in a given period. It has: --@@ -77,13 +88,11 @@  type MultiBalanceReport    = PeriodicReport    DisplayName MixedAmount type MultiBalanceReportRow = PeriodicReportRow DisplayName MixedAmount-type CompoundBalanceReport = CompoundPeriodicReport DisplayName MixedAmount  -- type alias just to remind us which AccountNames might be depth-clipped, below. type ClippedAccountName = AccountName  - -- | Generate a multicolumn balance report for the matched accounts, -- showing the change of balance, accumulated balance, or historical balance -- in each of the specified periods. If the normalbalance_ option is set, it@@ -91,79 +100,78 @@ -- CompoundBalanceCommand). hledger's most powerful and useful report, used -- by the balance command (in multiperiod mode) and (via compoundBalanceReport) -- by the bs/cf/is commands.-multiBalanceReport :: Day -> ReportOpts -> Journal -> MultiBalanceReport-multiBalanceReport today ropts j =-    multiBalanceReportWith ropts q j (journalPriceOracle infer j)-  where-    q = queryFromOpts today ropts-    infer = infer_value_ ropts+multiBalanceReport :: ReportSpec -> Journal -> MultiBalanceReport+multiBalanceReport rspec j = multiBalanceReportWith rspec j (journalPriceOracle infer j)+  where infer = infer_value_ $ rsOpts rspec --- | A helper for multiBalanceReport. This one takes an explicit Query--- instead of deriving one from ReportOpts, and an extra argument, a--- PriceOracle to be used for looking up market prices. Commands which+-- | A helper for multiBalanceReport. This one takes an extra argument,+-- a PriceOracle to be used for looking up market prices. Commands which -- run multiple reports (bs etc.) can generate the price oracle just -- once for efficiency, passing it to each report by calling this -- function directly.-multiBalanceReportWith :: ReportOpts -> Query -> Journal -> PriceOracle -> MultiBalanceReport-multiBalanceReportWith ropts q j priceoracle = report+multiBalanceReportWith :: ReportSpec -> Journal -> PriceOracle -> MultiBalanceReport+multiBalanceReportWith rspec' j priceoracle = report   where     -- Queries, report/column dates.-    reportspan = dbg "reportspan" $ calculateReportSpan ropts q j-    reportq    = dbg "reportq"    $ makeReportQuery ropts reportspan q+    reportspan = dbg3 "reportspan" $ calculateReportSpan rspec' j+    rspec      = dbg3 "reportopts" $ makeReportQuery rspec' reportspan+    valuation  = makeValuation rspec' j priceoracle  -- Must use rspec' instead of rspec,+                                                     -- so the reportspan isn't used for valuation      -- Group postings into their columns.-    colps    = dbg'' "colps"  $ getPostingsByColumn ropts reportq j reportspan-    colspans = dbg "colspans" $ M.keys colps+    colps    = dbg5 "colps"  $ getPostingsByColumn rspec j reportspan+    colspans = dbg3 "colspans" $ M.keys colps      -- The matched accounts with a starting balance. All of these should appear     -- in the report, even if they have no postings during the report period.-    startbals = dbg' "startbals" $ startingBalances ropts reportq j reportspan+    startbals = dbg5 "startbals" $ startingBalances rspec j reportspan      -- Generate and postprocess the report, negating balances and taking percentages if needed-    report = dbg' "report" $-      generateMultiBalanceReport ropts reportq j priceoracle colspans colps startbals+    report = dbg4 "multiBalanceReportWith" $+      generateMultiBalanceReport rspec j valuation colspans colps startbals  -- | Generate a compound balance report from a list of CBCSubreportSpec. This -- shares postings between the subreports.-compoundBalanceReport :: Day -> ReportOpts -> Journal -> [CBCSubreportSpec]-                      -> CompoundBalanceReport-compoundBalanceReport today ropts j =-    compoundBalanceReportWith ropts q j (journalPriceOracle infer j)-  where-    q = queryFromOpts today ropts-    infer = infer_value_ ropts+compoundBalanceReport :: ReportSpec -> Journal -> [CBCSubreportSpec a]+                      -> CompoundPeriodicReport a MixedAmount+compoundBalanceReport rspec j = compoundBalanceReportWith rspec j (journalPriceOracle infer j)+  where infer = infer_value_ $ rsOpts rspec  -- | A helper for compoundBalanceReport, similar to multiBalanceReportWith.-compoundBalanceReportWith :: ReportOpts -> Query -> Journal -> PriceOracle-                          -> [CBCSubreportSpec] -> CompoundBalanceReport-compoundBalanceReportWith ropts q j priceoracle subreportspecs = cbr+compoundBalanceReportWith :: ReportSpec -> Journal -> PriceOracle+                          -> [CBCSubreportSpec a]+                          -> CompoundPeriodicReport a MixedAmount+compoundBalanceReportWith rspec' j priceoracle subreportspecs = cbr   where     -- Queries, report/column dates.-    reportspan = dbg "reportspan" $ calculateReportSpan ropts q j-    reportq    = dbg "reportq"    $ makeReportQuery ropts reportspan q+    reportspan = dbg3 "reportspan" $ calculateReportSpan rspec' j+    rspec      = dbg3 "reportopts" $ makeReportQuery rspec' reportspan+    valuation  = makeValuation rspec' j priceoracle  -- Must use ropts' instead of ropts,+                                                     -- so the reportspan isn't used for valuation      -- Group postings into their columns.-    colps    = dbg'' "colps"  $ getPostingsByColumn ropts{empty_=True} reportq j reportspan-    colspans = dbg "colspans" $ M.keys colps+    colps    = dbg5 "colps"  $ getPostingsByColumn rspec{rsOpts=(rsOpts rspec){empty_=True}} j reportspan+    colspans = dbg3 "colspans" $ M.keys colps      -- The matched accounts with a starting balance. All of these should appear     -- in the report, even if they have no postings during the report period.-    startbals = dbg' "startbals" $ startingBalances ropts reportq j reportspan+    startbals = dbg5 "startbals" $ startingBalances rspec j reportspan      subreports = map generateSubreport subreportspecs       where         generateSubreport CBCSubreportSpec{..} =             ( cbcsubreporttitle             -- Postprocess the report, negating balances and taking percentages if needed-            , prNormaliseSign cbcsubreportnormalsign $-                generateMultiBalanceReport ropts' reportq j priceoracle colspans colps' startbals'+            , cbcsubreporttransform $+                generateMultiBalanceReport rspec{rsOpts=ropts} j valuation colspans colps' startbals'             , cbcsubreportincreasestotal             )           where-            ropts'     = ropts{normalbalance_=Just cbcsubreportnormalsign}             -- Filter the column postings according to each subreport-            colps'     = filter (matchesPosting $ cbcsubreportquery j) <$> colps-            startbals' = HM.filterWithKey (\k _ -> matchesAccount (cbcsubreportquery j) k) startbals+            colps'     = filter (matchesPosting q) <$> colps+            startbals' = HM.filterWithKey (\k _ -> matchesAccount q k) startbals+            ropts      = cbcsubreportoptions $ rsOpts rspec+            q          = cbcsubreportquery j      -- Sum the subreport totals by column. Handle these cases:     -- - no subreports@@ -186,21 +194,20 @@ -- TODO: Do we want to check whether to bother calculating these? isHistorical -- and startDate is not nothing, otherwise mempty? This currently gives a -- failure with some totals which are supposed to be 0 being blank.-startingBalances :: ReportOpts -> Query -> Journal -> DateSpan -> HashMap AccountName Account-startingBalances ropts q j reportspan = acctchanges+startingBalances :: ReportSpec -> Journal -> DateSpan -> HashMap AccountName Account+startingBalances rspec@ReportSpec{rsQuery=query,rsOpts=ropts} j reportspan =+    acctChangesFromPostings rspec' . map fst $ getPostings rspec' j   where-    acctchanges = acctChangesFromPostings ropts' startbalq . map fst $-        getPostings ropts' startbalq j+    rspec' = rspec{rsQuery=startbalq,rsOpts=ropts'}+    ropts' = case accountlistmode_ ropts of+        ALTree -> ropts{period_=precedingperiod, no_elide_=True}+        ALFlat -> ropts{period_=precedingperiod}      -- q projected back before the report start date.     -- When there's no report start date, in case there are future txns (the hledger-ui case above),     -- we use emptydatespan to make sure they aren't counted as starting balance.-    startbalq = dbg'' "startbalq" $ And [datelessq, precedingspanq]-    datelessq = dbg "datelessq" $ filterQuery (not . queryIsDateOrDate2) q--    ropts' = case accountlistmode_ ropts of-        ALTree -> ropts{no_elide_=True, period_=precedingperiod}-        ALFlat -> ropts{period_=precedingperiod}+    startbalq = dbg3 "startbalq" $ And [datelessq, precedingspanq]+    datelessq = dbg3 "datelessq" $ filterQuery (not . queryIsDateOrDate2) query      precedingperiod = dateSpanAsPeriod . spanIntersect precedingspan .                          periodAsDateSpan $ period_ ropts@@ -210,20 +217,20 @@         a -> a  -- | Calculate the span of the report to be generated.-calculateReportSpan :: ReportOpts -> Query -> Journal -> DateSpan-calculateReportSpan ropts q j = reportspan+calculateReportSpan :: ReportSpec -> Journal -> DateSpan+calculateReportSpan ReportSpec{rsQuery=query,rsOpts=ropts} j = reportspan   where     -- The date span specified by -b/-e/-p options and query args if any.-    requestedspan  = dbg "requestedspan" $ queryDateSpan (date2_ ropts) q+    requestedspan  = dbg3 "requestedspan" $ queryDateSpan (date2_ ropts) query     -- If the requested span is open-ended, close it using the journal's end dates.     -- This can still be the null (open) span if the journal is empty.-    requestedspan' = dbg "requestedspan'" $+    requestedspan' = dbg3 "requestedspan'" $         requestedspan `spanDefaultsFrom` journalDateSpan (date2_ ropts) j     -- The list of interval spans enclosing the requested span.     -- This list can be empty if the journal was empty,     -- or if hledger-ui has added its special date:-tomorrow to the query     -- and all txns are in the future.-    intervalspans  = dbg "intervalspans" $ splitSpan (interval_ ropts) requestedspan'+    intervalspans  = dbg3 "intervalspans" $ splitSpan (interval_ ropts) requestedspan'     -- The requested span enlarged to enclose a whole number of intervals.     -- This can be the null span if there were no intervals.     reportspan = DateSpan (spanStart =<< headMay intervalspans)@@ -233,25 +240,37 @@ -- The user's query expanded to the report span -- if there is one (otherwise any date queries are left as-is, which -- handles the hledger-ui+future txns case above).-makeReportQuery :: ReportOpts -> DateSpan -> Query -> Query-makeReportQuery ropts reportspan q-    | reportspan == nulldatespan = q-    | otherwise = And [dateless q, reportspandatesq]+makeReportQuery :: ReportSpec -> DateSpan -> ReportSpec+makeReportQuery rspec reportspan+    | reportspan == nulldatespan = rspec+    | otherwise = rspec{rsQuery=query}   where-    reportspandatesq = dbg "reportspandatesq" $ dateqcons reportspan-    dateless   = dbg "dateless" . filterQuery (not . queryIsDateOrDate2)-    dateqcons  = if date2_ ropts then Date2 else Date+    query            = simplifyQuery $ And [dateless $ rsQuery rspec, reportspandatesq]+    reportspandatesq = dbg3 "reportspandatesq" $ dateqcons reportspan+    dateless         = dbg3 "dateless" . filterQuery (not . queryIsDateOrDate2)+    dateqcons        = if date2_ (rsOpts rspec) then Date2 else Date +-- | Make a valuation function for valuating MixedAmounts and a given Day+makeValuation :: ReportSpec -> Journal -> PriceOracle -> (Day -> MixedAmount -> MixedAmount)+makeValuation rspec j priceoracle day = case value_ (rsOpts rspec) of+    Nothing -> id+    Just v  -> mixedAmountApplyValuation priceoracle styles day mreportlast (rsToday rspec) multiperiod v+  where+    -- Some things needed if doing valuation.+    styles = journalCommodityStyles j+    mreportlast = reportPeriodLastDay rspec+    multiperiod = interval_ (rsOpts rspec) /= NoInterval+ -- | Group postings, grouped by their column-getPostingsByColumn :: ReportOpts -> Query -> Journal -> DateSpan -> Map DateSpan [Posting]-getPostingsByColumn ropts q j reportspan = columns+getPostingsByColumn :: ReportSpec -> Journal -> DateSpan -> Map DateSpan [Posting]+getPostingsByColumn rspec j reportspan = columns   where     -- Postings matching the query within the report period.-    ps :: [(Posting, Day)] = dbg'' "ps" $ getPostings ropts q j+    ps :: [(Posting, Day)] = dbg5 "getPostingsByColumn" $ getPostings rspec j     days = map snd ps      -- The date spans to be included as report columns.-    colspans = calculateColSpans ropts reportspan days+    colspans = calculateColSpans (rsOpts rspec) reportspan days     addPosting (p, d) = maybe id (M.adjust (p:)) $ latestSpanContaining colspans d     emptyMap = M.fromList . zip colspans $ repeat [] @@ -259,19 +278,19 @@     columns = foldr addPosting emptyMap ps  -- | Gather postings matching the query within the report period.-getPostings :: ReportOpts -> Query -> Journal -> [(Posting, Day)]-getPostings ropts q =+getPostings :: ReportSpec -> Journal -> [(Posting, Day)]+getPostings ReportSpec{rsQuery=query,rsOpts=ropts} =     map (\p -> (p, date p)) .     journalPostings .     filterJournalAmounts symq .    -- remove amount parts excluded by cur:     filterJournalPostings reportq  -- remove postings not matched by (adjusted) query   where-    symq = dbg "symq" . filterQuery queryIsSym $ dbg "requested q" q+    symq = dbg3 "symq" . filterQuery queryIsSym $ dbg3 "requested q" query     -- The user's query with no depth limit, and expanded to the report span     -- if there is one (otherwise any date queries are left as-is, which     -- handles the hledger-ui+future txns case above).-    reportq = dbg "reportq" $ depthless q-    depthless = dbg "depthless" . filterQuery (not . queryIsDepth)+    reportq = dbg3 "reportq" $ depthless query+    depthless = dbg3 "depthless" . filterQuery (not . queryIsDepth)      date = case whichDateFromOpts ropts of         PrimaryDate   -> postingDate@@ -283,25 +302,24 @@     splitSpan (interval_ ropts) displayspan   where     displayspan-      | empty_ ropts = dbg "displayspan (-E)" reportspan                        -- all the requested intervals-      | otherwise = dbg "displayspan" $ reportspan `spanIntersect` matchedspan  -- exclude leading/trailing empty intervals-    matchedspan = dbg "matchedspan" $ daysSpan days+      | empty_ ropts = dbg3 "displayspan (-E)" reportspan                        -- all the requested intervals+      | otherwise = dbg3 "displayspan" $ reportspan `spanIntersect` matchedspan  -- exclude leading/trailing empty intervals+    matchedspan = dbg3 "matchedspan" $ daysSpan days   -- | Gather the account balance changes into a regular matrix -- including the accounts from all columns.-calculateAccountChanges :: ReportOpts -> Query -> [DateSpan]-                        -> Map DateSpan [Posting]+calculateAccountChanges :: ReportSpec -> [DateSpan] -> Map DateSpan [Posting]                         -> HashMap ClippedAccountName (Map DateSpan Account)-calculateAccountChanges ropts q colspans colps-    | queryDepth q == Just 0 = acctchanges <> elided-    | otherwise              = acctchanges+calculateAccountChanges rspec colspans colps+    | queryDepth (rsQuery rspec) == Just 0 = acctchanges <> elided+    | otherwise = acctchanges   where     -- Transpose to get each account's balance changes across all columns.     acctchanges = transposeMap colacctchanges      colacctchanges :: Map DateSpan (HashMap ClippedAccountName Account) =-      dbg'' "colacctchanges" $ fmap (acctChangesFromPostings ropts q) colps+      dbg5 "colacctchanges" $ fmap (acctChangesFromPostings rspec) colps      elided = HM.singleton "..." $ M.fromList [(span, nullacct) | span <- colspans] @@ -309,94 +327,111 @@ -- the accounts that have postings and calculate the change amount for -- each. Accounts and amounts will be depth-clipped appropriately if -- a depth limit is in effect.-acctChangesFromPostings :: ReportOpts -> Query -> [Posting] -> HashMap ClippedAccountName Account-acctChangesFromPostings ropts q ps = HM.fromList [(aname a, a) | a <- as]+acctChangesFromPostings :: ReportSpec -> [Posting] -> HashMap ClippedAccountName Account+acctChangesFromPostings ReportSpec{rsQuery=query,rsOpts=ropts} ps =+    HM.fromList [(aname a, a) | a <- as]   where     as = filterAccounts . drop 1 $ accountsFromPostings ps     filterAccounts = case accountlistmode_ ropts of         ALTree -> filter ((depthq `matchesAccount`) . aname)      -- exclude deeper balances         ALFlat -> clipAccountsAndAggregate (queryDepth depthq) .  -- aggregate deeper balances at the depth limit.                       filter ((0<) . anumpostings)-    depthq = dbg "depthq" $ filterQuery queryIsDepth q+    depthq = dbg3 "depthq" $ filterQuery queryIsDepth query  -- | Accumulate and value amounts, as specified by the report options. -- -- Makes sure all report columns have an entry.-accumValueAmounts :: ReportOpts -> Journal -> PriceOracle -> [DateSpan]+accumValueAmounts :: ReportOpts -> (Day -> MixedAmount -> MixedAmount) -> [DateSpan]                   -> HashMap ClippedAccountName Account                   -> HashMap ClippedAccountName (Map DateSpan Account)                   -> HashMap ClippedAccountName (Map DateSpan Account)-accumValueAmounts ropts j priceoracle colspans startbals acctchanges =  -- PARTIAL:-    HM.mapWithKey processRow $ acctchanges <> (mempty <$ startbals)+accumValueAmounts ropts valuation colspans startbals acctchanges =  -- PARTIAL:+    -- Ensure all columns have entries, including those with starting balances+    HM.mapWithKey rowbals $ ((<>zeros) <$> acctchanges) <> (zeros <$ startbals)   where-    -- Must accumulate before valuing, since valuation can change without any-    -- postings. Make sure every column has an entry.-    processRow name changes = M.mapWithKey valueAcct . rowbals name $ changes <> zeros--    -- The row amounts to be displayed: per-period changes,+    -- The valued row amounts to be displayed: per-period changes,     -- zero-based cumulative totals, or     -- starting-balance-based historical balances.-    rowbals name changes = dbg'' "rowbals" $ case balancetype_ ropts of-        PeriodChange      -> changes-        CumulativeChange  -> snd $ M.mapAccum f nullacct                  changes-        HistoricalBalance -> snd $ M.mapAccum f (startingBalanceFor name) changes-      where f a b = let s = sumAcct a b in (s, s)+    rowbals name changes = dbg5 "rowbals" $ case balancetype_ ropts of+        PeriodChange      -> changeamts+        CumulativeChange  -> cumulative+        HistoricalBalance -> historical+      where+        historical = cumulativeSum startingBalance+        cumulative | fixedValuationDate = cumulativeSum nullacct+                   | otherwise          = fmap (`subtractAcct` valuedStart) historical+        changeamts | fixedValuationDate = M.mapWithKey valueAcct changes+                   | otherwise          = M.fromDistinctAscList . zip dates $+                                            zipWith subtractAcct histamts (valuedStart:histamts)+          where (dates, histamts) = unzip $ M.toAscList historical +        cumulativeSum start = snd $ M.mapAccumWithKey accumValued start changes+          where accumValued startAmt date newAmt = (s, valueAcct date s)+                  where s = sumAcct startAmt newAmt++        -- Whether the market price is measured at the same date for all report+        -- periods, and we can therefore use the simpler calculations for+        -- cumulative and change reports.+        fixedValuationDate = case value_ ropts of+            Just (AtCost (Just _)) -> singleperiod+            Just (AtEnd  _)        -> singleperiod+            Just (AtDefault _)     -> singleperiod+            _                      -> True+          where singleperiod = interval_ ropts == NoInterval++        startingBalance = HM.lookupDefault nullacct name startbals+        valuedStart = valueAcct (DateSpan Nothing historicalDate) startingBalance+     -- Add the values of two accounts. Should be right-biased, since it's used     -- in scanl, so other properties (such as anumpostings) stay in the right place     sumAcct Account{aibalance=i1,aebalance=e1} a@Account{aibalance=i2,aebalance=e2} =         a{aibalance = i1 + i2, aebalance = e1 + e2} +    -- Subtract the values in one account from another. Should be left-biased.+    subtractAcct a@Account{aibalance=i1,aebalance=e1} Account{aibalance=i2,aebalance=e2} =+        a{aibalance = i1 - i2, aebalance = e1 - e2}+     -- We may be converting amounts to value, per hledger_options.m4.md "Effect of --value on reports".     valueAcct (DateSpan _ (Just end)) acct =         acct{aibalance = value (aibalance acct), aebalance = value (aebalance acct)}-      where value = avalue (addDays (-1) end)-    valueAcct _ _ = error' "multiBalanceReport: expected all spans to have an end date"  -- XXX should not happen--    avalue periodlast = maybe id-        (mixedAmountApplyValuation priceoracle styles periodlast mreportlast today multiperiod) $-        value_ ropts-      where-        -- Some things needed if doing valuation.-        styles = journalCommodityStyles j-        mreportlast = reportPeriodLastDay ropts-        today = fromMaybe (error' "multiBalanceReport: could not pick a valuation date, ReportOpts today_ is unset") $ today_ ropts  -- XXX shouldn't happen-        multiperiod = interval_ ropts /= NoInterval+      where value = valuation (addDays (-1) end)+    valueAcct _ _ = error "multiBalanceReport: expected all spans to have an end date"  -- XXX should not happen -    startingBalanceFor a = HM.lookupDefault nullacct a startbals     zeros = M.fromList [(span, nullacct) | span <- colspans]+    historicalDate = minimumMay $ mapMaybe spanStart colspans   -- | Lay out a set of postings grouped by date span into a regular matrix with rows -- given by AccountName and columns by DateSpan, then generate a MultiBalanceReport -- from the columns.-generateMultiBalanceReport :: ReportOpts -> Query -> Journal -> PriceOracle-                           -> [DateSpan]-                           -> Map DateSpan [Posting]-                           -> HashMap AccountName Account+generateMultiBalanceReport :: ReportSpec -> Journal -> (Day -> MixedAmount -> MixedAmount) -> [DateSpan]+                           -> Map DateSpan [Posting] -> HashMap AccountName Account                            -> MultiBalanceReport-generateMultiBalanceReport ropts q j priceoracle colspans colps startbals = report+generateMultiBalanceReport rspec@ReportSpec{rsOpts=ropts} j valuation colspans colps startbals =+    report   where     -- Each account's balance changes across all columns.-    acctchanges = dbg'' "acctchanges" $ calculateAccountChanges ropts q colspans colps+    acctchanges = dbg5 "acctchanges" $ calculateAccountChanges rspec colspans colps      -- Process changes into normal, cumulative, or historical amounts, plus value them-    accumvalued = accumValueAmounts ropts j priceoracle colspans startbals acctchanges+    accumvalued = accumValueAmounts ropts valuation colspans startbals acctchanges      -- All account names that will be displayed, possibly depth-clipped.-    displaynames = dbg'' "displaynames" $ displayedAccounts ropts q accumvalued+    displaynames = dbg5 "displaynames" $ displayedAccounts rspec accumvalued      -- All the rows of the report.-    rows = dbg'' "rows" $ buildReportRows ropts displaynames accumvalued+    rows = dbg5 "rows"+             . (if invert_ ropts then map (fmap negate) else id)  -- Negate amounts if applicable+             $ buildReportRows ropts displaynames accumvalued      -- Calculate column totals-    totalsrow = dbg' "totalsrow" $ calculateTotalsRow ropts rows+    totalsrow = dbg5 "totalsrow" $ calculateTotalsRow ropts rows      -- Sorted report rows.-    sortedrows = dbg' "sortedrows" $ sortRows ropts j rows+    sortedrows = dbg5 "sortedrows" $ sortRows ropts j rows -    -- Postprocess the report, negating balances and taking percentages if needed-    report = postprocessReport ropts $ PeriodicReport colspans sortedrows totalsrow+    -- Take percentages if needed+    report = reportPercent ropts $ PeriodicReport colspans sortedrows totalsrow  -- | Build the report rows. --@@ -423,10 +458,9 @@  -- | Calculate accounts which are to be displayed in the report, as well as -- their name and depth-displayedAccounts :: ReportOpts -> Query-                  -> HashMap AccountName (Map DateSpan Account)+displayedAccounts :: ReportSpec -> HashMap AccountName (Map DateSpan Account)                   -> HashMap AccountName DisplayName-displayedAccounts ropts q valuedaccts+displayedAccounts ReportSpec{rsQuery=query,rsOpts=ropts} valuedaccts     | depth == 0 = HM.singleton "..." $ DisplayName "..." "..." 1     | otherwise  = HM.mapWithKey (\a _ -> displayedName a) displayedAccts   where@@ -459,7 +493,7 @@                 | otherwise = aebalance      -- Accounts interesting because they are a fork for interesting subaccounts-    interestingParents = dbg'' "interestingParents" $ case accountlistmode_ ropts of+    interestingParents = dbg5 "interestingParents" $ case accountlistmode_ ropts of         ALTree -> HM.filterWithKey hasEnoughSubs numSubs         ALFlat -> mempty       where@@ -467,7 +501,7 @@         minSubs = if no_elide_ ropts then 1 else 2      isZeroRow balance = all (mixedAmountLooksZero . balance)-    depth = fromMaybe maxBound $ queryDepth q+    depth = fromMaybe maxBound $  queryDepth query     numSubs = subaccountTallies . HM.keys $ HM.filterWithKey isInteresting valuedaccts  -- | Sort the rows by amount or by account declaration order.@@ -518,7 +552,7 @@      colamts = transpose . map prrAmounts $ filter isTopRow rows -    coltotals :: [MixedAmount] = dbg'' "coltotals" $ map sum colamts+    coltotals :: [MixedAmount] = dbg5 "coltotals" $ map sum colamts      -- Calculate the grand total and average. These are always the sum/average     -- of the column totals.@@ -528,21 +562,17 @@         _            -> lastDef 0 coltotals     grandaverage = averageMixedAmounts coltotals --- | Map the report rows to percentages and negate if needed-postprocessReport :: ReportOpts -> MultiBalanceReport -> MultiBalanceReport-postprocessReport ropts = maybePercent . maybeInvert+-- | Map the report rows to percentages if needed+reportPercent :: ReportOpts -> MultiBalanceReport -> MultiBalanceReport+reportPercent ropts report@(PeriodicReport spans rows totalrow)+  | percent_ ropts = PeriodicReport spans (map percentRow rows) (percentRow totalrow)+  | otherwise      = report   where-    maybeInvert  = if invert_  ropts then fmap negate else id-    maybePercent = if percent_ ropts then prPercent   else id--    prPercent (PeriodicReport spans rows totalrow) =-        PeriodicReport spans (map percentRow rows) (percentRow totalrow)-      where-        percentRow (PeriodicReportRow name rowvals rowtotal rowavg) =-            PeriodicReportRow name-                (zipWith perdivide rowvals $ prrAmounts totalrow)-                (perdivide rowtotal $ prrTotal totalrow)-                (perdivide rowavg $ prrAverage totalrow)+    percentRow (PeriodicReportRow name rowvals rowtotal rowavg) =+      PeriodicReportRow name+        (zipWith perdivide rowvals $ prrAmounts totalrow)+        (perdivide rowtotal $ prrTotal totalrow)+        (perdivide rowavg $ prrAverage totalrow)   -- | Transpose a Map of HashMaps to a HashMap of Maps.@@ -583,37 +613,16 @@     return $ mixed [per $ if aquantity b' == 0 then 0 else aquantity a' / abs (aquantity b') * 100]   where errmsg = "Cannot calculate percentages if accounts have different commodities (Hint: Try --cost, -V or similar flags.)" --- Local debug helper--- add a prefix to this function's debug output-dbg   s = let p = "multiBalanceReport" in Hledger.Utils.dbg3 (p++" "++s)-dbg'  s = let p = "multiBalanceReport" in Hledger.Utils.dbg4 (p++" "++s)-dbg'' s = let p = "multiBalanceReport" in Hledger.Utils.dbg5 (p++" "++s)--- dbg = const id  -- exclude this function from debug output---- common rendering helper, XXX here for now-tableAsText :: ReportOpts -> (a -> String) -> Table String String a -> String-tableAsText (ReportOpts{pretty_tables_ = pretty}) showcell =-  unlines-  . trimborder-  . lines-  . render pretty id id showcell-  . align-  where-    trimborder = drop 1 . init . map (drop 1 . init)-    align (Table l t d) = Table l' t d-      where-        acctswidth = maximum' $ map strWidth (headerContents l)-        l'         = padRightWide acctswidth <$> l- -- tests  tests_MultiBalanceReport = tests "MultiBalanceReport" [    let     amt0 = Amount {acommodity="$", aquantity=0, aprice=Nothing, astyle=AmountStyle {ascommodityside = L, ascommodityspaced = False, asprecision = Precision 2, asdecimalpoint = Just '.', asdigitgroups = Nothing}, aismultiplier=False}-    (opts,journal) `gives` r = do-      let (eitems, etotal) = r-          (PeriodicReport _ aitems atotal) = multiBalanceReport nulldate opts journal+    (rspec,journal) `gives` r = do+      let rspec' = rspec{rsQuery=And [queryFromFlags $ rsOpts rspec, rsQuery rspec]}+          (eitems, etotal) = r+          (PeriodicReport _ aitems atotal) = multiBalanceReport rspec' journal           showw (PeriodicReportRow a lAmt amt amt')               = (displayFull a, displayName a, displayDepth a, map showMixedAmountDebug lAmt, showMixedAmountDebug amt, showMixedAmountDebug amt')       (map showw aitems) @?= (map showw eitems)@@ -621,10 +630,10 @@   in    tests "multiBalanceReport" [       test "null journal"  $-      (defreportopts, nulljournal) `gives` ([], Mixed [nullamt])+      (defreportspec, nulljournal) `gives` ([], Mixed [nullamt])       ,test "with -H on a populated period"  $-      (defreportopts{period_= PeriodBetween (fromGregorian 2008 1 1) (fromGregorian 2008 1 2), balancetype_=HistoricalBalance}, samplejournal) `gives`+      (defreportspec{rsOpts=defreportopts{period_= PeriodBetween (fromGregorian 2008 1 1) (fromGregorian 2008 1 2), balancetype_=HistoricalBalance}}, samplejournal) `gives`        (         [ PeriodicReportRow (flatDisplayName "assets:bank:checking") [mamountp' "$1.00"]  (mamountp' "$1.00")  (Mixed [amt0 {aquantity=1}])         , PeriodicReportRow (flatDisplayName "income:salary")        [mamountp' "$-1.00"] (mamountp' "$-1.00") (Mixed [amt0 {aquantity=(-1)}])

@@ -65,20 +65,19 @@  -- | Select postings from the journal and add running balance and other -- information to make a postings report. Used by eg hledger's register command.-postingsReport :: ReportOpts -> Query -> Journal -> PostingsReport-postingsReport ropts@ReportOpts{..} q j =+postingsReport :: ReportSpec -> Journal -> PostingsReport+postingsReport rspec@ReportSpec{rsOpts=ropts@ReportOpts{..}} j =   (totallabel, items)     where-      reportspan  = adjustReportDates ropts q j+      reportspan  = adjustReportDates rspec j       whichdate   = whichDateFromOpts ropts-      mdepth      = queryDepth q+      mdepth      = queryDepth $ rsQuery rspec       styles      = journalCommodityStyles j       priceoracle = journalPriceOracle infer_value_ j       multiperiod = interval_ /= NoInterval-      today       = fromMaybe (error' "postingsReport: could not pick a valuation date, ReportOpts today_ is unset") today_  -- PARTIAL:        -- postings to be included in the report, and similarly-matched postings before the report start date-      (precedingps, reportps) = matchedPostingsBeforeAndDuring ropts q j reportspan+      (precedingps, reportps) = matchedPostingsBeforeAndDuring rspec j reportspan        -- Postings, or summary postings with their subperiod's end date, to be displayed.       displayps :: [(Posting, Maybe Day)]@@ -90,12 +89,12 @@         where           showempty = empty_ || average_           -- We may be converting posting amounts to value, per hledger_options.m4.md "Effect of --value on reports".-          pvalue p periodlast = maybe p (postingApplyValuation priceoracle styles periodlast mreportlast today multiperiod p) value_+          pvalue p periodlast = maybe p (postingApplyValuation priceoracle styles periodlast mreportlast (rsToday rspec) multiperiod p) value_             where-              mreportlast = reportPeriodLastDay ropts+              mreportlast = reportPeriodLastDay rspec           reportorjournallast =             fromMaybe (error' "postingsReport: expected a non-empty journal") $  -- PARTIAL: shouldn't happen-            reportPeriodOrJournalLastDay ropts j+            reportPeriodOrJournalLastDay rspec j        -- Posting report items ready for display.       items =@@ -113,13 +112,13 @@               precedingsum = sumPostings precedingps               precedingavg | null precedingps = 0                            | otherwise        = divideMixedAmount (fromIntegral $ length precedingps) precedingsum-              bvalue = maybe id (mixedAmountApplyValuation priceoracle styles daybeforereportstart Nothing today multiperiod) value_+              bvalue = maybe id (mixedAmountApplyValuation priceoracle styles daybeforereportstart Nothing (rsToday rspec) multiperiod) value_                   -- XXX constrain valuation type to AtDate daybeforereportstart here ?                 where                   daybeforereportstart =                     maybe (error' "postingsReport: expected a non-empty journal")  -- PARTIAL: shouldn't happen                     (addDays (-1))-                    $ reportPeriodOrJournalStart ropts j+                    $ reportPeriodOrJournalStart rspec j            runningcalc = registerRunningCalculationFn ropts           startnum = if historical then length precedingps + 1 else 1@@ -140,17 +139,17 @@ -- 1. If the start date is unspecified, use the earliest date in the journal (if any) -- 2. If the end date is unspecified, use the latest date in the journal (if any) -- 3. If a report interval is specified, enlarge the dates to enclose whole intervals-adjustReportDates :: ReportOpts -> Query -> Journal -> DateSpan-adjustReportDates opts q j = reportspan+adjustReportDates :: ReportSpec -> Journal -> DateSpan+adjustReportDates rspec@ReportSpec{rsOpts=ropts} j = reportspan   where     -- see also multiBalanceReport-    requestedspan       = dbg3 "requestedspan"       $ queryDateSpan' q                                       -- span specified by -b/-e/-p options and query args+    requestedspan       = dbg3 "requestedspan"       $ queryDateSpan' $ rsQuery rspec                         -- span specified by -b/-e/-p options and query args     journalspan         = dbg3 "journalspan"         $ dates `spanUnion` date2s                               -- earliest and latest dates (or date2s) in the journal       where         dates  = journalDateSpan False j         date2s = journalDateSpan True  j     requestedspanclosed = dbg3 "requestedspanclosed" $ requestedspan `spanDefaultsFrom` journalspan           -- if open-ended, close it using the journal's dates (if any)-    intervalspans       = dbg3 "intervalspans"       $ splitSpan (interval_ opts) requestedspanclosed  -- get the whole intervals enclosing that+    intervalspans       = dbg3 "intervalspans"       $ splitSpan (interval_ ropts) requestedspanclosed  -- get the whole intervals enclosing that     mreportstart        = dbg3 "reportstart"         $ maybe Nothing spanStart $ headMay intervalspans        -- start of the first interval, or open ended     mreportend          = dbg3 "reportend"           $ maybe Nothing spanEnd   $ lastMay intervalspans        -- end of the last interval, or open ended     reportspan          = dbg3 "reportspan"          $ DateSpan mreportstart mreportend                       -- the requested span enlarged to whole intervals if possible@@ -159,29 +158,29 @@ -- and also any similarly-matched postings before that date span. -- Date restrictions and depth restrictions in the query are ignored. -- A helper for the postings report.-matchedPostingsBeforeAndDuring :: ReportOpts -> Query -> Journal -> DateSpan -> ([Posting],[Posting])-matchedPostingsBeforeAndDuring opts q j (DateSpan mstart mend) =+matchedPostingsBeforeAndDuring :: ReportSpec -> Journal -> DateSpan -> ([Posting],[Posting])+matchedPostingsBeforeAndDuring ReportSpec{rsOpts=ropts,rsQuery=q} j (DateSpan mstart mend) =   dbg5 "beforeps, duringps" $ span (beforestartq `matchesPosting`) beforeandduringps   where     beforestartq = dbg3 "beforestartq" $ dateqtype $ DateSpan Nothing mstart     beforeandduringps =-      dbg5 "ps5" $ sortOn sortdate $                                           -- sort postings by date or date2-      dbg5 "ps4" $ (if invert_ opts then map negatePostingAmount else id) $    -- with --invert, invert amounts-      dbg5 "ps3" $ map (filterPostingAmount symq) $                            -- remove amount parts which the query's cur: terms would exclude-      dbg5 "ps2" $ (if related_ opts then concatMap relatedPostings else id) $ -- with -r, replace each with its sibling postings-      dbg5 "ps1" $ filter (beforeandduringq `matchesPosting`) $                -- filter postings by the query, with no start date or depth limit+      dbg5 "ps5" $ sortOn sortdate $                                             -- sort postings by date or date2+      dbg5 "ps4" $ (if invert_ ropts then map negatePostingAmount else id) $     -- with --invert, invert amounts+      dbg5 "ps3" $ map (filterPostingAmount symq) $                              -- remove amount parts which the query's cur: terms would exclude+      dbg5 "ps2" $ (if related_ ropts then concatMap relatedPostings else id) $  -- with -r, replace each with its sibling postings+      dbg5 "ps1" $ filter (beforeandduringq `matchesPosting`) $                  -- filter postings by the query, with no start date or depth limit                   journalPostings $-                  journalSelectingAmountFromOpts opts j    -- maybe convert to cost early, will be seen by amt:. XXX what about converting to value ?+                  journalSelectingAmountFromOpts ropts j    -- maybe convert to cost early, will be seen by amt:. XXX what about converting to value ?       where         beforeandduringq = dbg4 "beforeandduringq" $ And [depthless $ dateless q, beforeendq]           where             depthless  = filterQuery (not . queryIsDepth)             dateless   = filterQuery (not . queryIsDateOrDate2)             beforeendq = dateqtype $ DateSpan Nothing mend-        sortdate = if date2_ opts then postingDate2 else postingDate+        sortdate = if date2_ ropts then postingDate2 else postingDate         symq = dbg4 "symq" $ filterQuery queryIsSym q     dateqtype-      | queryIsDate2 dateq || (queryIsDate dateq && date2_ opts) = Date2+      | queryIsDate2 dateq || (queryIsDate dateq && date2_ ropts) = Date2       | otherwise = Date       where         dateq = dbg4 "dateq" $ filterQuery queryIsDateOrDate2 $ dbg4 "q" q  -- XXX confused by multiple date:/date2: ?@@ -270,7 +269,7 @@ tests_PostingsReport = tests "PostingsReport" [     test "postingsReport" $ do-    let (query, journal) `gives` n = (length $ snd $ postingsReport defreportopts query journal) @?= n+    let (query, journal) `gives` n = (length $ snd $ postingsReport defreportspec{rsQuery=query} journal) @?= n     -- with the query specified explicitly     (Any, nulljournal) `gives` 0     (Any, samplejournal) `gives` 13@@ -279,10 +278,10 @@     (And [Depth 1, StatusQ Cleared, Acct (toRegex' "expenses")], samplejournal) `gives` 2     (And [And [Depth 1, StatusQ Cleared], Acct (toRegex' "expenses")], samplejournal) `gives` 2     -- with query and/or command-line options-    (length $ snd $ postingsReport defreportopts Any samplejournal) @?= 13-    (length $ snd $ postingsReport defreportopts{interval_=Months 1} Any samplejournal) @?= 11-    (length $ snd $ postingsReport defreportopts{interval_=Months 1, empty_=True} Any samplejournal) @?= 20-    (length $ snd $ postingsReport defreportopts (Acct (toRegex' "assets:bank:checking")) samplejournal) @?= 5+    (length $ snd $ postingsReport defreportspec samplejournal) @?= 13+    (length $ snd $ postingsReport defreportspec{rsOpts=defreportopts{interval_=Months 1}} samplejournal) @?= 11+    (length $ snd $ postingsReport defreportspec{rsOpts=defreportopts{interval_=Months 1, empty_=True}} samplejournal) @?= 20+    (length $ snd $ postingsReport defreportspec{rsQuery=Acct $ toRegex' "assets:bank:checking"} samplejournal) @?= 5       -- (defreportopts, And [Acct "a a", Acct "'b"], samplejournal2) `gives` 0      -- [(Just (fromGregorian 2008 01 01,"income"),assets:bank:checking             $1,$1)

@@ -10,13 +10,16 @@  module Hledger.Reports.ReportOptions (   ReportOpts(..),+  ReportSpec(..),   BalanceType(..),   AccountListMode(..),   ValuationType(..),-  FormatStr,   defreportopts,   rawOptsToReportOpts,-  checkReportOpts,+  defreportspec,+  reportOptsToSpec,+  updateReportSpecFromOpts,+  rawOptsToReportSpec,   flat_,   tree_,   reportOptsToggleStatus,@@ -25,25 +28,18 @@   journalSelectingAmountFromOpts,   intervalFromRawOpts,   forecastPeriodFromRawOpts,-  queryFromOpts,-  queryFromOptsOnly,-  queryOptsFromOpts,+  queryFromFlags,   transactionDateFn,   postingDateFn,   reportSpan,   reportStartDate,   reportEndDate,-  specifiedStartEndDates,-  specifiedStartDate,-  specifiedEndDate,   reportPeriodStart,   reportPeriodOrJournalStart,   reportPeriodLastDay,   reportPeriodOrJournalLastDay,   valuationTypeIsCost,   valuationTypeIsDefaultValue,--  tests_ReportOptions ) where @@ -51,7 +47,7 @@ import Data.List.Extra (nubSort) import Data.Maybe (fromMaybe, isJust) import qualified Data.Text as T-import Data.Time.Calendar (Day, addDays, fromGregorian)+import Data.Time.Calendar (Day, addDays) import Data.Default (Default(..)) import Safe (lastDef, lastMay) @@ -65,8 +61,6 @@ import Hledger.Utils  -type FormatStr = String- -- | Which "balance" is being shown in a balance report. data BalanceType = PeriodChange      -- ^ The change of balance in each period.                  | CumulativeChange  -- ^ The accumulated change across multiple periods.@@ -88,11 +82,7 @@ -- commands, as noted below. data ReportOpts = ReportOpts {      -- for most reports:-     today_          :: Maybe Day  -- ^ The current date. A late addition to ReportOpts.-                                   -- Optional, but when set it may affect some reports:-                                   -- Reports use it when picking a -V valuation date.-                                   -- This is not great, adds indeterminacy.-    ,period_         :: Period+     period_         :: Period     ,interval_       :: Interval     ,statuses_       :: [Status]  -- ^ Zero, one, or two statuses to be matched     ,value_          :: Maybe ValuationType  -- ^ What value should amounts be converted to ?@@ -102,9 +92,8 @@     ,empty_          :: Bool     ,no_elide_       :: Bool     ,real_           :: Bool-    ,format_         :: Maybe FormatStr-    ,query_          :: String -- ^ All query arguments space sepeareted-                               --   and quoted if needed (see 'quoteIfNeeded')+    ,format_         :: StringFormat+    ,querystring_    :: [T.Text]     --     ,average_        :: Bool     -- for posting reports (register)@@ -142,100 +131,134 @@  defreportopts :: ReportOpts defreportopts = ReportOpts-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def-    def+    { period_          = PeriodAll+    , interval_        = NoInterval+    , statuses_        = []+    , value_           = Nothing+    , infer_value_     = False+    , depth_           = Nothing+    , date2_           = False+    , empty_           = False+    , no_elide_        = False+    , real_            = False+    , format_          = def+    , querystring_     = []+    , average_         = False+    , related_         = False+    , txn_dates_       = False+    , balancetype_     = def+    , accountlistmode_ = ALFlat+    , drop_            = 0+    , row_total_       = False+    , no_total_        = False+    , pretty_tables_   = False+    , sort_amount_     = False+    , percent_         = False+    , invert_          = False+    , normalbalance_   = Nothing+    , color_           = False+    , forecast_        = Nothing+    , transpose_       = False+    }  rawOptsToReportOpts :: RawOpts -> IO ReportOpts-rawOptsToReportOpts rawopts = checkReportOpts <$> do-  let rawopts' = checkRawOpts rawopts-  d <- getCurrentDay-  no_color <- isJust <$> lookupEnv "NO_COLOR"-  supports_color <- hSupportsANSIColor stdout-  let colorflag = stringopt "color" rawopts-  return defreportopts{-     today_       = Just d-    ,period_      = periodFromRawOpts d rawopts'-    ,interval_    = intervalFromRawOpts rawopts'-    ,statuses_    = statusesFromRawOpts rawopts'-    ,value_       = valuationTypeFromRawOpts rawopts'-    ,infer_value_ = boolopt "infer-value" rawopts'-    ,depth_       = maybeposintopt "depth" rawopts'-    ,date2_       = boolopt "date2" rawopts'-    ,empty_       = boolopt "empty" rawopts'-    ,no_elide_    = boolopt "no-elide" rawopts'-    ,real_        = boolopt "real" rawopts'-    ,format_      = maybestringopt "format" rawopts' -- XXX move to CliOpts or move validation from Cli.CliOptions to here-    ,query_       = unwords . map quoteIfNeeded $ listofstringopt "args" rawopts' -- doesn't handle an arg like "" right-    ,average_     = boolopt "average" rawopts'-    ,related_     = boolopt "related" rawopts'-    ,txn_dates_   = boolopt "txn-dates" rawopts'-    ,balancetype_ = balancetypeopt rawopts'-    ,accountlistmode_ = accountlistmodeopt rawopts'-    ,drop_        = posintopt "drop" rawopts'-    ,row_total_   = boolopt "row-total" rawopts'-    ,no_total_    = boolopt "no-total" rawopts'-    ,sort_amount_ = boolopt "sort-amount" rawopts'-    ,percent_     = boolopt "percent" rawopts'-    ,invert_      = boolopt "invert" rawopts'-    ,pretty_tables_ = boolopt "pretty-tables" rawopts'-    ,color_       = and [not no_color-                        ,not $ colorflag `elem` ["never","no"]-                        ,colorflag `elem` ["always","yes"] || supports_color-                        ]-    ,forecast_    = forecastPeriodFromRawOpts d rawopts'-    ,transpose_   = boolopt "transpose" rawopts'+rawOptsToReportOpts rawopts = do+    d <- getCurrentDay+    no_color <- isJust <$> lookupEnv "NO_COLOR"+    supports_color <- hSupportsANSIColor stdout++    let colorflag    = stringopt "color" rawopts+        formatstring = maybestringopt "format" rawopts+        querystring  = map T.pack $ listofstringopt "args" rawopts  -- doesn't handle an arg like "" right++    format <- case parseStringFormat <$> formatstring of+        Nothing         -> return defaultBalanceLineFormat+        Just (Right x)  -> return x+        Just (Left err) -> fail $ "could not parse format option: " ++ err++    let reportopts = defreportopts+          {period_      = periodFromRawOpts d rawopts+          ,interval_    = intervalFromRawOpts rawopts+          ,statuses_    = statusesFromRawOpts rawopts+          ,value_       = valuationTypeFromRawOpts rawopts+          ,infer_value_ = boolopt "infer-value" rawopts+          ,depth_       = maybeposintopt "depth" rawopts+          ,date2_       = boolopt "date2" rawopts+          ,empty_       = boolopt "empty" rawopts+          ,no_elide_    = boolopt "no-elide" rawopts+          ,real_        = boolopt "real" rawopts+          ,format_      = format+          ,querystring_ = querystring+          ,average_     = boolopt "average" rawopts+          ,related_     = boolopt "related" rawopts+          ,txn_dates_   = boolopt "txn-dates" rawopts+          ,balancetype_ = balancetypeopt rawopts+          ,accountlistmode_ = accountlistmodeopt rawopts+          ,drop_        = posintopt "drop" rawopts+          ,row_total_   = boolopt "row-total" rawopts+          ,no_total_    = boolopt "no-total" rawopts+          ,sort_amount_ = boolopt "sort-amount" rawopts+          ,percent_     = boolopt "percent" rawopts+          ,invert_      = boolopt "invert" rawopts+          ,pretty_tables_ = boolopt "pretty-tables" rawopts+          ,color_       = and [not no_color+                              ,not $ colorflag `elem` ["never","no"]+                              ,colorflag `elem` ["always","yes"] || supports_color+                              ]+          ,forecast_    = forecastPeriodFromRawOpts d rawopts+          ,transpose_   = boolopt "transpose" rawopts+          }+    return reportopts++-- | The result of successfully parsing a ReportOpts on a particular+-- Day. Any ambiguous dates are completed and Queries are parsed,+-- ensuring that there are no regular expression errors. Values here+-- should be used in preference to re-deriving them from ReportOpts.+-- If you change the query_ in ReportOpts, you should call+-- `reportOptsToSpec` to regenerate the ReportSpec with the new+-- Query.+data ReportSpec = ReportSpec+  { rsOpts      :: ReportOpts  -- ^ The underlying ReportOpts used to generate this ReportSpec+  , rsToday     :: Day         -- ^ The Day this ReportSpec is generated for+  , rsQuery     :: Query       -- ^ The generated Query for the given day+  , rsQueryOpts :: [QueryOpt]  -- ^ A list of QueryOpts for the given day+  } deriving (Show)++instance Default ReportSpec where def = defreportspec++defreportspec :: ReportSpec+defreportspec = ReportSpec+    { rsOpts      = def+    , rsToday     = nulldate+    , rsQuery     = Any+    , rsQueryOpts = []     } --- | Do extra validation of raw option values, raising an error if there's a problem.-checkRawOpts :: RawOpts -> RawOpts-checkRawOpts rawopts--- our standard behaviour is to accept conflicting options actually,--- using the last one - more forgiving for overriding command-line aliases---   | countopts ["change","cumulative","historical"] > 1---     = usageError "please specify at most one of --change, --cumulative, --historical"---   | countopts ["flat","tree"] > 1---     = usageError "please specify at most one of --flat, --tree"---   | countopts ["daily","weekly","monthly","quarterly","yearly"] > 1---     = usageError "please specify at most one of --daily, "-  | otherwise = rawopts---   where---     countopts = length . filter (`boolopt` rawopts)+-- | Generate a ReportSpec from a set of ReportOpts on a given day.+reportOptsToSpec :: Day -> ReportOpts -> Either String ReportSpec+reportOptsToSpec day ropts = do+    (argsquery, queryopts) <- parseQueryList day $ querystring_ ropts+    return ReportSpec+      { rsOpts = ropts+      , rsToday = day+      , rsQuery = simplifyQuery $ And [queryFromFlags ropts, argsquery]+      , rsQueryOpts = queryopts+      } --- | Do extra validation of report options, raising an error if there's a problem.-checkReportOpts :: ReportOpts -> ReportOpts-checkReportOpts ropts@ReportOpts{..} =-  either usageError (const ropts) $ do-    case depth_ of-      Just d | d < 0 -> Left "--depth should have a positive number"-      _              -> Right ()+-- | Regenerate a ReportSpec after updating ReportOpts, or return an error+-- message if there is a problem such as missing or unparseable options data.+-- This helps keep the ReportSpec, its underlying ReportOpts, and the ReportOpts'+-- data fields like querystring_ all in sync.+updateReportSpecFromOpts :: (ReportOpts -> ReportOpts) -> ReportSpec -> Either String ReportSpec+updateReportSpecFromOpts f rspec = reportOptsToSpec (rsToday rspec) . f $ rsOpts rspec +-- | Generate a ReportSpec from RawOpts and the current date.+rawOptsToReportSpec :: RawOpts -> IO ReportSpec+rawOptsToReportSpec rawopts = do+    d <- getCurrentDay+    ropts <- rawOptsToReportOpts rawopts+    either fail return $ reportOptsToSpec d ropts+ accountlistmodeopt :: RawOpts -> AccountListMode accountlistmodeopt =   fromMaybe ALFlat . choiceopt parse where@@ -445,20 +468,11 @@     Just (AtCost _) -> journalToCost     _               -> id --- | Convert report options and arguments to a query.--- If there is a parsing problem, this function calls error.-queryFromOpts :: Day -> ReportOpts -> Query-queryFromOpts d ropts = simplifyQuery . And $ [flagsq, argsq]-  where-    flagsq = queryFromOptsOnly d ropts-    argsq = fst $ either error' id $ parseQuery d (T.pack $ query_ ropts)  -- PARTIAL:- -- | Convert report options to a query, ignoring any non-flag command line arguments.-queryFromOptsOnly :: Day -> ReportOpts -> Query-queryFromOptsOnly _d ReportOpts{..} = simplifyQuery $ And flagsq+queryFromFlags :: ReportOpts -> Query+queryFromFlags ReportOpts{..} = simplifyQuery $ And flagsq   where     flagsq = consIf   Real  real_-           . consIf   Empty empty_            . consJust Depth depth_            $   [ (if date2_ then Date2 else Date) $ periodAsDateSpan period_                , Or $ map StatusQ statuses_@@ -466,108 +480,51 @@     consIf f b = if b then (f True:) else id     consJust f = maybe id ((:) . f) --- | Convert report options and arguments to query options.--- If there is a parsing problem, this function calls error.-queryOptsFromOpts :: Day -> ReportOpts -> [QueryOpt]-queryOptsFromOpts d = snd . either error' id . parseQuery d . T.pack . query_  -- PARTIAL:- -- Report dates.  -- | The effective report span is the start and end dates specified by -- options or queries, or otherwise the earliest and latest transaction or -- posting dates in the journal. If no dates are specified by options/queries -- and the journal is empty, returns the null date span.--- Needs IO to parse smart dates in options/queries.-reportSpan :: Journal -> ReportOpts -> IO DateSpan-reportSpan j ropts = do-  (mspecifiedstartdate, mspecifiedenddate) <--    dbg3 "specifieddates" <$> specifiedStartEndDates ropts-  let+reportSpan :: Journal -> ReportSpec -> DateSpan+reportSpan j ReportSpec{rsQuery=query} = dbg3 "reportspan" $ DateSpan mstartdate menddate+  where     DateSpan mjournalstartdate mjournalenddate =       dbg3 "journalspan" $ journalDateSpan False j  -- ignore secondary dates-    mstartdate = mspecifiedstartdate <|> mjournalstartdate-    menddate   = mspecifiedenddate   <|> mjournalenddate-  return $ dbg3 "reportspan" $ DateSpan mstartdate menddate--reportStartDate :: Journal -> ReportOpts -> IO (Maybe Day)-reportStartDate j ropts = spanStart <$> reportSpan j ropts--reportEndDate :: Journal -> ReportOpts -> IO (Maybe Day)-reportEndDate j ropts = spanEnd <$> reportSpan j ropts---- | The specified report start/end dates are the dates specified by options or queries, if any.--- Needs IO to parse smart dates in options/queries.-specifiedStartEndDates :: ReportOpts -> IO (Maybe Day, Maybe Day)-specifiedStartEndDates ropts = do-  today <- getCurrentDay-  let-    q = queryFromOpts today ropts-    mspecifiedstartdate = queryStartDate False q-    mspecifiedenddate   = queryEndDate   False q-  return (mspecifiedstartdate, mspecifiedenddate)+    mstartdate = queryStartDate False query <|> mjournalstartdate+    menddate   = queryEndDate   False query <|> mjournalenddate -specifiedStartDate :: ReportOpts -> IO (Maybe Day)-specifiedStartDate ropts = fst <$> specifiedStartEndDates ropts+reportStartDate :: Journal -> ReportSpec -> Maybe Day+reportStartDate j = spanStart . reportSpan j -specifiedEndDate :: ReportOpts -> IO (Maybe Day)-specifiedEndDate ropts = snd <$> specifiedStartEndDates ropts+reportEndDate :: Journal -> ReportSpec -> Maybe Day+reportEndDate j = spanEnd . reportSpan j  -- Some pure alternatives to the above. XXX review/clean up  -- Get the report's start date. -- If no report period is specified, will be Nothing.--- Will also be Nothing if ReportOpts does not have today_ set,--- since we need that to get the report period robustly--- (unlike reportStartDate, which looks up the date with IO.)-reportPeriodStart :: ReportOpts -> Maybe Day-reportPeriodStart ropts@ReportOpts{..} = do-  t <- today_-  queryStartDate False $ queryFromOpts t ropts+reportPeriodStart :: ReportSpec -> Maybe Day+reportPeriodStart = queryStartDate False . rsQuery  -- Get the report's start date, or if no report period is specified, -- the journal's start date (the earliest posting date). If there's no -- report period and nothing in the journal, will be Nothing.-reportPeriodOrJournalStart :: ReportOpts -> Journal -> Maybe Day-reportPeriodOrJournalStart ropts j =-  reportPeriodStart ropts <|> journalStartDate False j+reportPeriodOrJournalStart :: ReportSpec -> Journal -> Maybe Day+reportPeriodOrJournalStart rspec j =+  reportPeriodStart rspec <|> journalStartDate False j  -- Get the last day of the overall report period. -- This the inclusive end date (one day before the -- more commonly used, exclusive, report end date). -- If no report period is specified, will be Nothing.--- Will also be Nothing if ReportOpts does not have today_ set,--- since we need that to get the report period robustly--- (unlike reportEndDate, which looks up the date with IO.)-reportPeriodLastDay :: ReportOpts -> Maybe Day-reportPeriodLastDay ropts@ReportOpts{..} = do-  t <- today_-  let q = queryFromOpts t ropts-  qend <- queryEndDate False q-  return $ addDays (-1) qend+reportPeriodLastDay :: ReportSpec -> Maybe Day+reportPeriodLastDay = fmap (addDays (-1)) . queryEndDate False . rsQuery  -- Get the last day of the overall report period, or if no report -- period is specified, the last day of the journal (ie the latest -- posting date). If there's no report period and nothing in the -- journal, will be Nothing.-reportPeriodOrJournalLastDay :: ReportOpts -> Journal -> Maybe Day-reportPeriodOrJournalLastDay ropts j =-  reportPeriodLastDay ropts <|> journalEndDate False j---- tests--tests_ReportOptions = tests "ReportOptions" [-   test "queryFromOpts" $ do-       queryFromOpts nulldate defreportopts @?= Any-       queryFromOpts nulldate defreportopts{query_="a"} @?= Acct (toRegexCI' "a")-       queryFromOpts nulldate defreportopts{query_="desc:'a a'"} @?= Desc (toRegexCI' "a a")-       queryFromOpts nulldate defreportopts{period_=PeriodFrom (fromGregorian 2012 01 01),query_="date:'to 2013'" }-         @?= (Date $ DateSpan (Just $ fromGregorian 2012 01 01) (Just $ fromGregorian 2013 01 01))-       queryFromOpts nulldate defreportopts{query_="date2:'in 2012'"} @?= (Date2 $ DateSpan (Just $ fromGregorian 2012 01 01) (Just $ fromGregorian 2013 01 01))-       queryFromOpts nulldate defreportopts{query_="'a a' 'b"} @?= Or [Acct $ toRegexCI' "a a", Acct $ toRegexCI' "'b"]--  ,test "queryOptsFromOpts" $ do-      queryOptsFromOpts nulldate defreportopts @?= []-      queryOptsFromOpts nulldate defreportopts{query_="a"} @?= []-      queryOptsFromOpts nulldate defreportopts{period_=PeriodFrom (fromGregorian 2012 01 01)-                                              ,query_="date:'to 2013'"} @?= []- ]+reportPeriodOrJournalLastDay :: ReportSpec -> Journal -> Maybe Day+reportPeriodOrJournalLastDay rspec j =+  reportPeriodLastDay rspec <|> journalEndDate False j

@@ -17,7 +17,6 @@ , Average  , periodicReportSpan-, prNormaliseSign , prMapName , prMapMaybeName @@ -40,8 +39,10 @@ import Data.Semigroup (Semigroup(..)) #endif import GHC.Generics (Generic)+ import Hledger.Data import Hledger.Query (Query)+import Hledger.Reports.ReportOptions (ReportOpts)  type Percentage = Decimal @@ -104,17 +105,11 @@       sumPadded as     []     = as       sumPadded []     bs     = bs --- | Figure out the overall date span of a PeridicReport+-- | Figure out the overall date span of a PeriodicReport periodicReportSpan :: PeriodicReport a b -> DateSpan periodicReportSpan (PeriodicReport [] _ _)       = DateSpan Nothing Nothing periodicReportSpan (PeriodicReport colspans _ _) = DateSpan (spanStart $ head colspans) (spanEnd $ last colspans) --- | Given a PeriodicReport and its normal balance sign,--- if it is known to be normally negative, convert it to normally positive.-prNormaliseSign :: Num b => NormalSign -> PeriodicReport a b -> PeriodicReport a b-prNormaliseSign NormallyNegative = fmap negate-prNormaliseSign NormallyPositive = id- -- | Map a function over the row names. prMapName :: (a -> b) -> PeriodicReport a c -> PeriodicReport b c prMapName f report = report{prRows = map (prrMapName f) $ prRows report}@@ -153,15 +148,18 @@   , cbrDates      :: [DateSpan]   , cbrSubreports :: [(String, PeriodicReport a b, Bool)]   , cbrTotals     :: PeriodicReportRow () b-  } deriving (Show, Generic, ToJSON)+  } deriving (Show, Functor, Generic, ToJSON)  -- | Description of one subreport within a compound balance report. -- Part of a "CompoundBalanceCommandSpec", but also used in hledger-lib.-data CBCSubreportSpec = CBCSubreportSpec-  { cbcsubreporttitle          :: String-  , cbcsubreportquery          :: Journal -> Query-  , cbcsubreportnormalsign     :: NormalSign-  , cbcsubreportincreasestotal :: Bool+data CBCSubreportSpec a = CBCSubreportSpec+  { cbcsubreporttitle          :: String                    -- ^ The title to use for the subreport+  , cbcsubreportquery          :: Journal -> Query          -- ^ The Query to use for the subreport+  , cbcsubreportoptions        :: ReportOpts -> ReportOpts  -- ^ A function to transform the ReportOpts used to produce the subreport+  , cbcsubreporttransform      :: PeriodicReport DisplayName MixedAmount -> PeriodicReport a MixedAmount  -- ^ A function to transform the result of the subreport+  , cbcsubreportincreasestotal :: Bool                      -- ^ Whether the subreport and overall report total are of the same sign (e.g. Assets are normally+                                                            --   positive in a balance sheet report, as is the overall total. Liabilities are normally of the+                                                            --   opposite sign.)   }  

@@ -36,7 +36,6 @@  import Control.Monad (liftM, when) -- import Data.Char-import Data.Default import Data.FileEmbed (makeRelativeToProject, embedStringFile) import Data.List -- import Data.Maybe@@ -143,8 +142,6 @@   return $ utcToZonedTime tz t  -- misc--instance Default Bool where def = False  -- | Apply a function the specified number of times, -- which should be > 0 (otherwise does nothing).

@@ -89,6 +89,7 @@ import           Control.Monad.IO.Class import           Data.List hiding (uncons) import qualified Data.Text as T+import qualified Data.Text.Lazy as TL import           Debug.Trace import           Hledger.Utils.Parse import           Safe (readDef)@@ -97,17 +98,27 @@ import           System.IO.Unsafe (unsafePerformIO) import           Text.Megaparsec import           Text.Printf-import           Text.Show.Pretty (ppShow, pPrint)+import           Text.Pretty.Simple  -- (defaultOutputOptionsDarkBg, OutputOptions(..), pShowOpt, pPrintOpt) --- | Pretty print. Easier alias for pretty-show's pPrint.+prettyopts = +    defaultOutputOptionsDarkBg+    -- defaultOutputOptionsLightBg+    -- defaultOutputOptionsNoColor+    { outputOptionsIndentAmount=2+    , outputOptionsCompact=True+    }++-- | Pretty print. Generic alias for pretty-simple's pPrint. pprint :: Show a => a -> IO ()-pprint = pPrint+pprint = pPrintOpt CheckColorTty prettyopts --- | Pretty show. Easier alias for pretty-show's ppShow.+-- | Pretty show. Generic alias for pretty-simple's pShow. pshow :: Show a => a -> String-pshow = ppShow+pshow = TL.unpack . pShowOpt prettyopts --- | Pretty trace. Easier alias for traceShowId + ppShow.+-- XXX some of the below can be improved with pretty-simple, https://github.com/cdepillabout/pretty-simple#readme++-- | Pretty trace. Easier alias for traceShowId + pShow. ptrace :: Show a => a -> a ptrace = traceWith pshow @@ -157,7 +168,7 @@ ptraceAt :: Show a => Int -> String -> a -> a ptraceAt level     | level > 0 && debugLevel < level = flip const-    | otherwise = \s a -> let p = ppShow a+    | otherwise = \s a -> let p = pshow a                               ls = lines p                               nlorspace | length ls > 1 = "\n"                                         | otherwise     = " " ++ take (10 - length s) (repeat ' ')@@ -303,7 +314,7 @@ plogAt lvl     | lvl > 0 && debugLevel < lvl = flip const     | otherwise = \s a ->-        let p = ppShow a+        let p = pshow a             ls = lines p             nlorspace | length ls > 1 = "\n"                       | otherwise     = " " ++ take (10 - length s) (repeat ' ')

@@ -147,7 +147,9 @@ -- | A replacement pattern. May include numeric backreferences (\N). type Replacement = String --- | An regular expression compilation/processing error message.+-- | An error message arising during a regular expression operation.+-- Eg: trying to compile a malformed regular expression, or+-- trying to apply a malformed replacement pattern. type RegexError = String  -- helpers@@ -162,13 +164,20 @@  -- | A memoising version of regexReplace. Caches the result for each -- search pattern, replacement pattern, target string tuple.+-- This won't generate a regular expression parsing error since that+-- is pre-compiled nowadays, but there can still be a runtime error +-- from the replacement pattern, eg with a backreference referring +-- to a nonexistent match group. regexReplace :: Regexp -> Replacement -> String -> Either RegexError String regexReplace re repl = memo $ regexReplaceUnmemo re repl  -- helpers:  -- Replace this regular expression with this replacement pattern in this--- string, or return an error message.+-- string, or return an error message. (There should be no regexp+-- parsing errors these days since Regexp's compiled form is used,+-- but there can still be a runtime error from the replacement+-- pattern, eg a backreference referring to a nonexistent match group.) regexReplaceUnmemo :: Regexp -> Replacement -> String -> Either RegexError String regexReplaceUnmemo re repl s = foldM (replaceMatch repl) s (reverse $ match (reCompiled re) s :: [MatchText String])   where@@ -190,7 +199,7 @@             -- return the referenced group text, or an error message.             lookupMatchGroup :: MatchText String -> String -> Either RegexError String             lookupMatchGroup grps ('\\':s@(_:_)) | all isDigit s =-              case read s of n | n `elem` indices grps -> Right $ fst (grps ! n)+              case read s of n | n `elem` indices grps -> Right $ fst (grps ! n)  -- PARTIAL: should not fail, all digits                              _                         -> Left $ "no match group exists for backreference \"\\"++s++"\""             lookupMatchGroup _ s = Left $ "lookupMatchGroup called on non-numeric-backreference \""++s++"\", shouldn't happen"     backrefRegex = toRegex' "\\\\[0-9]+"  -- PARTIAL: should not fail

@@ -40,6 +40,7 @@  -- * wide-character-aware layout  charWidth,  strWidth,+ strWidthAnsi,  takeWidth,  fitString,  fitStringMulti,@@ -48,14 +49,18 @@  ) where  -import Data.Char (isDigit, isSpace, toLower, toUpper)-import Data.List (intercalate, transpose)-import Text.Megaparsec (Parsec, (<|>), (<?>), anySingle, between, many, noneOf,-                        oneOf, parseMaybe, sepBy, takeWhileP, try)-import Text.Megaparsec.Char (char, string)+import Data.Char (isSpace, toLower, toUpper)+import Data.Default (def)+import Data.List (intercalate)+import Text.Megaparsec ((<|>), between, many, noneOf, sepBy)+import Text.Megaparsec.Char (char) import Text.Printf (printf)  import Hledger.Utils.Parse+import Hledger.Utils.Regex (toRegex', regexReplace)+import Text.Tabular (Header(..), Properties(..))+import Text.Tabular.AsciiWide (Align(..), Cell(..), TableOpts(..), renderRow)+import Text.WideString (strWidth, charWidth)   -- | Take elements from the end of a list.@@ -119,13 +124,17 @@ -- | Double-quote this string if it contains whitespace, single quotes -- or double-quotes, escaping the quotes as needed. quoteIfNeeded :: String -> String-quoteIfNeeded s | any (`elem` s) (quotechars++whitespacechars++redirectchars) = show s+quoteIfNeeded s | any (`elem` s) (quotechars++whitespacechars++redirectchars) = showChar '"' $ escapeQuotes s "\""                 | otherwise = s+  where+    escapeQuotes []       x = x+    escapeQuotes ('"':cs) x = showString "\\\"" $ escapeQuotes cs x+    escapeQuotes (c:cs)   x = showChar c        $ escapeQuotes cs x  -- | Single-quote this string if it contains whitespace or double-quotes. -- No good for strings containing single quotes. singleQuoteIfNeeded :: String -> String-singleQuoteIfNeeded s | any (`elem` s) whitespacechars = "'"++s++"'"+singleQuoteIfNeeded s | any (`elem` s) (quotechars++whitespacechars) = "'"++s++"'"                       | otherwise = s  quotechars, whitespacechars, redirectchars :: [Char]@@ -168,26 +177,16 @@ -- | Join several multi-line strings as side-by-side rectangular strings of the same height, top-padded. -- Treats wide characters as double width. concatTopPadded :: [String] -> String-concatTopPadded strs = intercalate "\n" $ map concat $ transpose padded-    where-      lss = map lines strs-      h = maximum $ map length lss-      ypad ls = replicate (difforzero h (length ls)) "" ++ ls-      xpad ls = map (padLeftWide w) ls where w | null ls = 0-                                               | otherwise = maximum $ map strWidth ls-      padded = map (xpad . ypad) lss+concatTopPadded = renderRow def{tableBorders=False, borderSpaces=False}+                . Group NoLine . map (Header . cell)+  where cell = Cell BottomLeft . map (\x -> (x, strWidth x)) . lines  -- | Join several multi-line strings as side-by-side rectangular strings of the same height, bottom-padded. -- Treats wide characters as double width. concatBottomPadded :: [String] -> String-concatBottomPadded strs = intercalate "\n" $ map concat $ transpose padded-    where-      lss = map lines strs-      h = maximum $ map length lss-      ypad ls = ls ++ replicate (difforzero h (length ls)) ""-      xpad ls = map (padRightWide w) ls where w | null ls = 0-                                                | otherwise = maximum $ map strWidth ls-      padded = map (xpad . ypad) lss+concatBottomPadded = renderRow def{tableBorders=False, borderSpaces=False}+                   . Group NoLine . map (Header . cell)+  where cell = Cell TopLeft . map (\x -> (x, strWidth x)) . lines   -- | Join multi-line strings horizontally, after compressing each of@@ -327,74 +326,20 @@                    | otherwise = ""   where cw = charWidth c --- from Pandoc (copyright John MacFarlane, GPL)--- see also http://unicode.org/reports/tr11/#Description---- | Calculate the render width of a string, considering--- wide characters (counted as double width), ANSI escape codes--- (not counted), and line breaks (in a multi-line string, the longest--- line determines the width).-strWidth :: String -> Int-strWidth = maximum . (0:) . map (foldr (\a b -> charWidth a + b) 0) . lines . stripAnsi+-- | Like strWidth, but also strips ANSI escape sequences before+-- calculating the width.+--+-- This is no longer used in code, as widths are calculated before+-- adding ANSI escape sequences, but is being kept around for now.+strWidthAnsi :: String -> Int+strWidthAnsi = strWidth . stripAnsi  -- | Strip ANSI escape sequences from a string. -- -- >>> stripAnsi "\ESC[31m-1\ESC[m" -- "-1" stripAnsi :: String -> String-stripAnsi s = case parseMaybe (many $ "" <$ try ansi <|> pure <$> anySingle) s of-    Nothing -> error "Bad ansi escape"  -- PARTIAL: should not happen-    Just xs -> concat xs-  where-    -- This parses lots of invalid ANSI escape codes, but that should be fine-    ansi = string "\ESC[" *> digitSemicolons *> suffix <?> "ansi" :: Parsec CustomErr String Char-    digitSemicolons = takeWhileP Nothing (\c -> isDigit c || c == ';')-    suffix = oneOf ['A', 'B', 'C', 'D', 'H', 'J', 'K', 'f', 'm', 's', 'u']---- | Get the designated render width of a character: 0 for a combining--- character, 1 for a regular character, 2 for a wide character.--- (Wide characters are rendered as exactly double width in apps and--- fonts that support it.) (From Pandoc.)-charWidth :: Char -> Int-charWidth c =-  case c of-      _ | c <  '\x0300'                    -> 1-        | c >= '\x0300' && c <= '\x036F'   -> 0  -- combining-        | c >= '\x0370' && c <= '\x10FC'   -> 1-        | c >= '\x1100' && c <= '\x115F'   -> 2-        | c >= '\x1160' && c <= '\x11A2'   -> 1-        | c >= '\x11A3' && c <= '\x11A7'   -> 2-        | c >= '\x11A8' && c <= '\x11F9'   -> 1-        | c >= '\x11FA' && c <= '\x11FF'   -> 2-        | c >= '\x1200' && c <= '\x2328'   -> 1-        | c >= '\x2329' && c <= '\x232A'   -> 2-        | c >= '\x232B' && c <= '\x2E31'   -> 1-        | c >= '\x2E80' && c <= '\x303E'   -> 2-        | c == '\x303F'                    -> 1-        | c >= '\x3041' && c <= '\x3247'   -> 2-        | c >= '\x3248' && c <= '\x324F'   -> 1 -- ambiguous-        | c >= '\x3250' && c <= '\x4DBF'   -> 2-        | c >= '\x4DC0' && c <= '\x4DFF'   -> 1-        | c >= '\x4E00' && c <= '\xA4C6'   -> 2-        | c >= '\xA4D0' && c <= '\xA95F'   -> 1-        | c >= '\xA960' && c <= '\xA97C'   -> 2-        | c >= '\xA980' && c <= '\xABF9'   -> 1-        | c >= '\xAC00' && c <= '\xD7FB'   -> 2-        | c >= '\xD800' && c <= '\xDFFF'   -> 1-        | c >= '\xE000' && c <= '\xF8FF'   -> 1 -- ambiguous-        | c >= '\xF900' && c <= '\xFAFF'   -> 2-        | c >= '\xFB00' && c <= '\xFDFD'   -> 1-        | c >= '\xFE00' && c <= '\xFE0F'   -> 1 -- ambiguous-        | c >= '\xFE10' && c <= '\xFE19'   -> 2-        | c >= '\xFE20' && c <= '\xFE26'   -> 1-        | c >= '\xFE30' && c <= '\xFE6B'   -> 2-        | c >= '\xFE70' && c <= '\xFEFF'   -> 1-        | c >= '\xFF01' && c <= '\xFF60'   -> 2-        | c >= '\xFF61' && c <= '\x16A38'  -> 1-        | c >= '\x1B000' && c <= '\x1B001' -> 2-        | c >= '\x1D000' && c <= '\x1F1FF' -> 1-        | c >= '\x1F200' && c <= '\x1F251' -> 2-        | c >= '\x1F300' && c <= '\x1F773' -> 1-        | c >= '\x20000' && c <= '\x3FFFD' -> 2-        | otherwise                        -> 1-+stripAnsi s = either err id $ regexReplace ansire "" s+ where+   err    = error "stripAnsi: invalid replacement pattern"      -- PARTIAL, shouldn't happen+   ansire = toRegex' "\ESC\\[([0-9]+;)*([0-9]+)?[ABCDHJKfmsu]"  -- PARTIAL, should succeed

@@ -69,8 +69,8 @@  -- import Hledger.Utils.Parse -- import Hledger.Utils.Regex-import Hledger.Utils.String (charWidth) import Hledger.Utils.Test+import Text.WideString (charWidth, textWidth)  -- lowercase, uppercase :: String -> String -- lowercase = map toLower@@ -343,63 +343,6 @@                 cw <= w                 = T.cons c $ textTakeWidth (w-cw) (T.tail t)               | otherwise = ""---- -- from Pandoc (copyright John MacFarlane, GPL)--- -- see also http://unicode.org/reports/tr11/#Description---- | Calculate the designated render width of a string, taking into--- account wide characters and line breaks (the longest line within a--- multi-line string determines the width ).-textWidth :: Text -> Int-textWidth "" = 0-textWidth s = maximum $ map (T.foldr (\a b -> charWidth a + b) 0) $ T.lines s---- -- | Get the designated render width of a character: 0 for a combining--- -- character, 1 for a regular character, 2 for a wide character.--- -- (Wide characters are rendered as exactly double width in apps and--- -- fonts that support it.) (From Pandoc.)--- charWidth :: Char -> Int--- charWidth c =---   case c of---       _ | c <  '\x0300'                    -> 1---         | c >= '\x0300' && c <= '\x036F'   -> 0  -- combining---         | c >= '\x0370' && c <= '\x10FC'   -> 1---         | c >= '\x1100' && c <= '\x115F'   -> 2---         | c >= '\x1160' && c <= '\x11A2'   -> 1---         | c >= '\x11A3' && c <= '\x11A7'   -> 2---         | c >= '\x11A8' && c <= '\x11F9'   -> 1---         | c >= '\x11FA' && c <= '\x11FF'   -> 2---         | c >= '\x1200' && c <= '\x2328'   -> 1---         | c >= '\x2329' && c <= '\x232A'   -> 2---         | c >= '\x232B' && c <= '\x2E31'   -> 1---         | c >= '\x2E80' && c <= '\x303E'   -> 2---         | c == '\x303F'                    -> 1---         | c >= '\x3041' && c <= '\x3247'   -> 2---         | c >= '\x3248' && c <= '\x324F'   -> 1 -- ambiguous---         | c >= '\x3250' && c <= '\x4DBF'   -> 2---         | c >= '\x4DC0' && c <= '\x4DFF'   -> 1---         | c >= '\x4E00' && c <= '\xA4C6'   -> 2---         | c >= '\xA4D0' && c <= '\xA95F'   -> 1---         | c >= '\xA960' && c <= '\xA97C'   -> 2---         | c >= '\xA980' && c <= '\xABF9'   -> 1---         | c >= '\xAC00' && c <= '\xD7FB'   -> 2---         | c >= '\xD800' && c <= '\xDFFF'   -> 1---         | c >= '\xE000' && c <= '\xF8FF'   -> 1 -- ambiguous---         | c >= '\xF900' && c <= '\xFAFF'   -> 2---         | c >= '\xFB00' && c <= '\xFDFD'   -> 1---         | c >= '\xFE00' && c <= '\xFE0F'   -> 1 -- ambiguous---         | c >= '\xFE10' && c <= '\xFE19'   -> 2---         | c >= '\xFE20' && c <= '\xFE26'   -> 1---         | c >= '\xFE30' && c <= '\xFE6B'   -> 2---         | c >= '\xFE70' && c <= '\xFEFF'   -> 1---         | c >= '\xFF01' && c <= '\xFF60'   -> 2---         | c >= '\xFF61' && c <= '\x16A38'  -> 1---         | c >= '\x1B000' && c <= '\x1B001' -> 2---         | c >= '\x1D000' && c <= '\x1F1FF' -> 1---         | c >= '\x1F200' && c <= '\x1F251' -> 2---         | c >= '\x1F300' && c <= '\x1F773' -> 1---         | c >= '\x20000' && c <= '\x3FFFD' -> 2---         | otherwise                        -> 1   -- | Read a decimal number from a Text. Assumes the input consists only of digit

@@ -3,83 +3,161 @@  module Text.Tabular.AsciiWide where +import Data.Maybe (fromMaybe)+import Data.Default (Default(..)) import Data.List (intersperse, transpose)+import Safe (maximumMay) import Text.Tabular-import Hledger.Utils.String+import Text.WideString (strWidth) --- | for simplicity, we assume that each cell is rendered---   on a single line-render :: Bool -- ^ pretty tables-       -> (rh -> String)-       -> (ch -> String)-       -> (a -> String)-       -> Table rh ch a-       -> String-render pretty fr fc f (Table rh ch cells) =-  unlines $ [ bar VT SingleLine   -- +--------------------------------------+-            , renderColumns pretty sizes ch2-            , bar VM DoubleLine   -- +======================================+-            ] ++-            (renderRs $ fmap renderR $ zipHeader [] cells $ fmap fr rh) ++-            [ bar VB SingleLine ] -- +--------------------------------------+++-- | The options to use for rendering a table.+data TableOpts = TableOpts+  { prettyTable  :: Bool  -- ^ Pretty tables+  , tableBorders :: Bool  -- ^ Whether to display the outer borders+  , borderSpaces :: Bool  -- ^ Whether to display spaces around bars+  } deriving (Show)++instance Default TableOpts where+  def = TableOpts { prettyTable  = False+                  , tableBorders = True+                  , borderSpaces = True+                  }++-- | Cell contents along an alignment+data Cell = Cell Align [(String, Int)]+    deriving (Show)++-- | How to align text in a cell+data Align = TopRight | BottomRight | BottomLeft | TopLeft+  deriving (Show)++emptyCell :: Cell+emptyCell = Cell TopRight []++-- | Create a single-line cell from the given contents with its natural width.+alignCell :: Align -> String -> Cell+alignCell a x = Cell a [(x, strWidth x)]++-- | Return the width of a Cell.+cellWidth :: Cell -> Int+cellWidth (Cell _ xs) = fromMaybe 0 . maximumMay $ map snd xs+++-- | Render a table according to common options, for backwards compatibility+render :: Bool -> (rh -> String) -> (ch -> String) -> (a -> String) -> Table rh ch a -> String+render pretty fr fc f = renderTable def{prettyTable=pretty} (cell . fr) (cell . fc) (cell . f)+  where cell = alignCell TopRight++-- | Render a table according to various cell specifications+renderTable :: TableOpts         -- ^ Options controlling Table rendering+            -> (rh -> Cell)  -- ^ Rendering function for row headers+            -> (ch -> Cell)  -- ^ Rendering function for column headers+            -> (a -> Cell)   -- ^ Function determining the string and width of a cell+            -> Table rh ch a+            -> String+renderTable topts@TableOpts{prettyTable=pretty, tableBorders=borders} fr fc f (Table rh ch cells) =+  unlines . addBorders $+    renderColumns topts sizes ch2+    : bar VM DoubleLine   -- +======================================++    : renderRs (fmap renderR $ zipHeader [] cellContents rowHeaders)  where-  bar vpos prop = concat (renderHLine vpos pretty sizes ch2 prop)+  renderR (cs,h) = renderColumns topts sizes $ Group DoubleLine+                     [ Header h+                     , fmap fst $ zipHeader emptyCell cs colHeaders+                     ]++  rowHeaders   = fmap fr rh+  colHeaders   = fmap fc ch+  cellContents = map (map f) cells+   -- ch2 and cell2 include the row and column labels-  ch2 = Group DoubleLine [Header "", fmap fc ch]-  cells2 = headerContents ch2-         : zipWith (\h cs -> h : map f cs) rhStrings cells-  ---  renderR (cs,h) = renderColumns pretty sizes $ Group DoubleLine-                    [ Header h-                    , fmap fst $ zipHeader "" (map f cs) ch]-  rhStrings = map fr $ headerContents rh+  ch2 = Group DoubleLine [Header emptyCell, colHeaders]+  cells2 = headerContents ch2 : zipWith (:) (headerContents rowHeaders) cellContents+   -- maximum width for each column-  sizes   = map (maximum . map strWidth) . transpose $ cells2+  sizes = map (fromMaybe 0 . maximumMay . map cellWidth) $ transpose cells2   renderRs (Header s)   = [s]-  renderRs (Group p hs) = concat . intersperse sep . map renderRs $ hs-    where sep = renderHLine VM pretty sizes ch2 p+  renderRs (Group p hs) = concat . intersperse sep $ map renderRs hs+    where sep = renderHLine VM borders pretty sizes ch2 p +  -- borders and bars+  addBorders xs = if borders then bar VT SingleLine : xs ++ [bar VB SingleLine] else xs+  bar vpos prop = concat $ renderHLine vpos borders pretty sizes ch2 prop++-- | Render a single row according to cell specifications.+renderRow :: TableOpts -> Header Cell -> String+renderRow topts h = renderColumns topts is h+  where is = map (\(Cell _ xs) -> fromMaybe 0 . maximumMay $ map snd xs) $ headerContents h++ verticalBar :: Bool -> Char verticalBar pretty = if pretty then '│' else '|' -leftBar :: Bool -> String-leftBar pretty = verticalBar pretty : " "+leftBar :: Bool -> Bool -> String+leftBar pretty True  = verticalBar pretty : " "+leftBar pretty False = [verticalBar pretty] -rightBar :: Bool -> String-rightBar pretty = " " ++ [verticalBar pretty]+rightBar :: Bool -> Bool -> String+rightBar pretty True  = ' ' : [verticalBar pretty]+rightBar pretty False = [verticalBar pretty] -midBar :: Bool -> String-midBar pretty = " " ++ verticalBar pretty : " "+midBar :: Bool -> Bool -> String+midBar pretty True  = ' ' : verticalBar pretty : " "+midBar pretty False = [verticalBar pretty] -doubleMidBar :: Bool -> String-doubleMidBar pretty = if pretty then " ║ " else " || "+doubleMidBar :: Bool -> Bool -> String+doubleMidBar pretty True  = if pretty then " ║ " else " || "+doubleMidBar pretty False = if pretty then "║" else "||"  -- | We stop rendering on the shortest list!-renderColumns :: Bool -- ^ pretty-              -> [Int] -- ^ max width for each column-              -> Header String+renderColumns :: TableOpts  -- ^ rendering options for the table+              -> [Int]      -- ^ max width for each column+              -> Header Cell               -> String-renderColumns pretty is h = leftBar pretty ++ coreLine ++ rightBar pretty- where-  coreLine = concatMap helper $ flattenHeader $ zipHeader 0 is h-  helper = either hsep (uncurry padLeftWide)-  hsep :: Properties -> String-  hsep NoLine     = "  "-  hsep SingleLine = midBar pretty-  hsep DoubleLine = doubleMidBar pretty+renderColumns TableOpts{prettyTable=pretty, tableBorders=borders, borderSpaces=spaces} is h =+    concat . intersperse "\n"                    -- Put each line on its own line+    . map (addBorders . concat) . transpose      -- Change to a list of lines and add borders+    . map (either hsep padCell) . flattenHeader  -- We now have a matrix of strings+    . zipHeader 0 is $ padRow <$> h  -- Pad cell height and add width marker+  where+    -- Pad each cell to have the appropriate width+    padCell (w, Cell TopLeft     ls) = map (\(x,xw) -> x ++ replicate (w - xw) ' ') ls+    padCell (w, Cell BottomLeft  ls) = map (\(x,xw) -> x ++ replicate (w - xw) ' ') ls+    padCell (w, Cell TopRight    ls) = map (\(x,xw) -> replicate (w - xw) ' ' ++ x) ls+    padCell (w, Cell BottomRight ls) = map (\(x,xw) -> replicate (w - xw) ' ' ++ x) ls +    -- Pad each cell to have the same number of lines+    padRow (Cell TopLeft     ls) = Cell TopLeft     $ ls ++ replicate (nLines - length ls) ("",0)+    padRow (Cell TopRight    ls) = Cell TopRight    $ ls ++ replicate (nLines - length ls) ("",0)+    padRow (Cell BottomLeft  ls) = Cell BottomLeft  $ replicate (nLines - length ls) ("",0) ++ ls+    padRow (Cell BottomRight ls) = Cell BottomRight $ replicate (nLines - length ls) ("",0) ++ ls++    hsep :: Properties -> [String]+    hsep NoLine     = replicate nLines $ if spaces then "  " else ""+    hsep SingleLine = replicate nLines $ midBar pretty spaces+    hsep DoubleLine = replicate nLines $ doubleMidBar pretty spaces++    addBorders xs | borders   = leftBar pretty spaces ++ xs ++ rightBar pretty spaces+                  | spaces    =  ' ' : xs ++ " "+                  | otherwise = xs++    nLines = fromMaybe 0 . maximumMay . map (\(Cell _ ls) -> length ls) $ headerContents h+ renderHLine :: VPos+            -> Bool  -- ^ show outer borders             -> Bool -- ^ pretty             -> [Int] -- ^ width specifications-            -> Header String+            -> Header a             -> Properties             -> [String]-renderHLine _ _ _ _ NoLine = []-renderHLine vpos pretty w h prop = [renderHLine' vpos pretty prop w h]+renderHLine _ _ _ _ _ NoLine = []+renderHLine vpos borders pretty w h prop = [renderHLine' vpos borders pretty prop w h] -renderHLine' :: VPos -> Bool -> Properties -> [Int] -> Header String -> String-renderHLine' vpos pretty prop is h = edge HL ++ sep ++ coreLine ++ sep ++ edge HR+renderHLine' :: VPos -> Bool -> Bool -> Properties -> [Int] -> Header a -> String+renderHLine' vpos borders pretty prop is h = addBorders $ sep ++ coreLine ++ sep  where+  addBorders xs   = if borders then edge HL ++ xs ++ edge HR else xs   edge hpos       = boxchar vpos hpos SingleLine prop pretty   coreLine        = concatMap helper $ flattenHeader $ zipHeader 0 is h   helper          = either vsep dashes

@@ -0,0 +1,71 @@+-- | Calculate the width of String and Text, being aware of wide characters.++module Text.WideString (+  -- * wide-character-aware layout+  strWidth,+  textWidth,+  charWidth+  ) where++import Data.Text (Text)+import qualified Data.Text as T+++-- | Calculate the render width of a string, considering+-- wide characters (counted as double width)+strWidth :: String -> Int+strWidth = foldr (\a b -> charWidth a + b) 0++-- | Calculate the render width of a string, considering+-- wide characters (counted as double width)+textWidth :: Text -> Int+textWidth = T.foldr (\a b -> charWidth a + b) 0++-- from Pandoc (copyright John MacFarlane, GPL)+-- see also http://unicode.org/reports/tr11/#Description++-- | Get the designated render width of a character: 0 for a combining+-- character, 1 for a regular character, 2 for a wide character.+-- (Wide characters are rendered as exactly double width in apps and+-- fonts that support it.) (From Pandoc.)+charWidth :: Char -> Int+charWidth c+    | c <  '\x0300'                    = 1+    | c >= '\x0300' && c <= '\x036F'   = 0  -- combining+    | c >= '\x0370' && c <= '\x10FC'   = 1+    | c >= '\x1100' && c <= '\x115F'   = 2+    | c >= '\x1160' && c <= '\x11A2'   = 1+    | c >= '\x11A3' && c <= '\x11A7'   = 2+    | c >= '\x11A8' && c <= '\x11F9'   = 1+    | c >= '\x11FA' && c <= '\x11FF'   = 2+    | c >= '\x1200' && c <= '\x2328'   = 1+    | c >= '\x2329' && c <= '\x232A'   = 2+    | c >= '\x232B' && c <= '\x2E31'   = 1+    | c >= '\x2E80' && c <= '\x303E'   = 2+    | c == '\x303F'                    = 1+    | c >= '\x3041' && c <= '\x3247'   = 2+    | c >= '\x3248' && c <= '\x324F'   = 1 -- ambiguous+    | c >= '\x3250' && c <= '\x4DBF'   = 2+    | c >= '\x4DC0' && c <= '\x4DFF'   = 1+    | c >= '\x4E00' && c <= '\xA4C6'   = 2+    | c >= '\xA4D0' && c <= '\xA95F'   = 1+    | c >= '\xA960' && c <= '\xA97C'   = 2+    | c >= '\xA980' && c <= '\xABF9'   = 1+    | c >= '\xAC00' && c <= '\xD7FB'   = 2+    | c >= '\xD800' && c <= '\xDFFF'   = 1+    | c >= '\xE000' && c <= '\xF8FF'   = 1 -- ambiguous+    | c >= '\xF900' && c <= '\xFAFF'   = 2+    | c >= '\xFB00' && c <= '\xFDFD'   = 1+    | c >= '\xFE00' && c <= '\xFE0F'   = 1 -- ambiguous+    | c >= '\xFE10' && c <= '\xFE19'   = 2+    | c >= '\xFE20' && c <= '\xFE26'   = 1+    | c >= '\xFE30' && c <= '\xFE6B'   = 2+    | c >= '\xFE70' && c <= '\xFEFF'   = 1+    | c >= '\xFF01' && c <= '\xFF60'   = 2+    | c >= '\xFF61' && c <= '\x16A38'  = 1+    | c >= '\x1B000' && c <= '\x1B001' = 2+    | c >= '\x1D000' && c <= '\x1F1FF' = 1+    | c >= '\x1F200' && c <= '\x1F251' = 2+    | c >= '\x1F300' && c <= '\x1F773' = 1+    | c >= '\x20000' && c <= '\x3FFFD' = 2+    | otherwise                        = 1

@@ -4,10 +4,10 @@ -- -- see: https://github.com/sol/hpack ----- hash: 4cff01322093c9a60a07a65c0ed150394360e71104dd3e7b52fc6782c5cf5a57+-- hash: 496cff1a1bc952e36c246f175d8efcb4c6cab6e9adc79c17b2d1e8b71ddaa2e5  name:           hledger-lib-version:        1.19.1+version:        1.20 synopsis:       A reusable library providing the core functionality of hledger description:    A reusable library containing hledger's core functionality.                 This is used by most hledger* packages so that they support the same@@ -103,6 +103,7 @@       Text.Tabular.AsciiWide   other-modules:       Text.Megaparsec.Custom+      Text.WideString       Paths_hledger_lib   hs-source-dirs:       ./.@@ -126,7 +127,6 @@     , data-default >=0.5     , directory     , extra >=1.6.3-    , fgl >=5.5.4.0     , file-embed >=0.0.10     , filepath     , hashtables >=1.2.3.1@@ -135,7 +135,7 @@     , old-time     , parsec >=3     , parser-combinators >=0.4.0-    , pretty-show >=1.6.4+    , pretty-simple >4 && <5     , regex-tdfa     , safe >=0.2     , split >=0.1@@ -179,7 +179,6 @@     , directory     , doctest >=0.16.3     , extra >=1.6.3-    , fgl >=5.5.4.0     , file-embed >=0.0.10     , filepath     , hashtables >=1.2.3.1@@ -188,7 +187,7 @@     , old-time     , parsec >=3     , parser-combinators >=0.4.0-    , pretty-show >=1.6.4+    , pretty-simple >4 && <5     , regex-tdfa     , safe >=0.2     , split >=0.1@@ -233,7 +232,6 @@     , data-default >=0.5     , directory     , extra >=1.6.3-    , fgl >=5.5.4.0     , file-embed >=0.0.10     , filepath     , hashtables >=1.2.3.1@@ -243,7 +241,7 @@     , old-time     , parsec >=3     , parser-combinators >=0.4.0-    , pretty-show >=1.6.4+    , pretty-simple >4 && <5     , regex-tdfa     , safe >=0.2     , split >=0.1

@@ -1,6 +1,6 @@ .\"t -.TH "hledger_csv" "5" "September 2020" "hledger 1.18.99" "hledger User Manuals"+.TH "hledger_csv" "5" "November 2020" "hledger 1.20" "hledger User Manuals"   @@ -75,9 +75,14 @@ T{ \f[B]\f[CB]date-format\f[B]\f[R] T}@T{-describe the format of CSV dates+how to parse dates in CSV records T} T{+\f[B]\f[CB]decimal-mark\f[B]\f[R]+T}@T{+the decimal mark used in CSV amounts, if ambiguous+T}+T{ \f[B]\f[CB]newest-first\f[B]\f[R] T}@T{ disambiguate record order when there\[aq]s only one date@@ -866,6 +871,27 @@ .P .PD https://hackage.haskell.org/package/time/docs/Data-Time-Format.html#v:formatTime+.SS \f[C]decimal-mark\f[R]+.IP+.nf+\f[C]+decimal-mark .+\f[R]+.fi+.PP+or:+.IP+.nf+\f[C]+decimal-mark ,+\f[R]+.fi+.PP+hledger automatically accepts either period or comma as a decimal mark+when parsing numbers (cf Amounts).+However if any numbers in the CSV contain digit group marks, such as+thousand-separating commas, you should declare the decimal mark+explicitly with this rule, to avoid misparsed numbers. .SS \f[C]newest-first\f[R] .PP hledger always sorts the generated transactions by date.@@ -1087,31 +1113,80 @@ .SS Setting currency/commodity .PP If the currency/commodity symbol is included in the CSV\[aq]s amount-field(s), you don\[aq]t have to do anything special.+field(s):+.IP+.nf+\f[C]+2020-01-01,foo,$123.00+\f[R]+.fi .PP-If the currency is provided as a separate CSV field, you can either:-.IP \[bu] 2-assign that to \f[C]currency\f[R], which adds it to all posting amounts.-The symbol will prepended to the amount quantity (on the left side).-If you write a trailing space after the symbol, there will be a space-between symbol and amount (an exception to the usual whitespace-stripping).-.IP \[bu] 2-or assign it to \f[C]currencyN\f[R] which adds it to posting N\[aq]s-amount only.-.IP \[bu] 2-or for more control, construct the amount from symbol and quantity using-field assignment, eg:-.RS 2+you don\[aq]t have to do anything special for the commodity symbol, it+will be assigned as part of the amount.+Eg: .IP .nf \f[C]-fields date,description,currency,quantity-# add currency symbol on the right:-amount %quantity %currency+fields date,description,amount \f[R] .fi-.RE+.IP+.nf+\f[C]+2020-01-01 foo+    expenses:unknown         $123.00+    income:unknown          $-123.00+\f[R]+.fi+.PP+If the currency is provided as a separate CSV field:+.IP+.nf+\f[C]+2020-01-01,foo,USD,123.00+\f[R]+.fi+.PP+You can assign that to the \f[C]currency\f[R] pseudo-field, which has+the special effect of prepending itself to every amount in the+transaction (on the left, with no separating space):+.IP+.nf+\f[C]+fields date,description,currency,amount+\f[R]+.fi+.IP+.nf+\f[C]+2020-01-01 foo+    expenses:unknown       USD123.00+    income:unknown        USD-123.00+\f[R]+.fi+.PP+Or, you can use a field assignment to construct the amount yourself,+with more control.+Eg to put the symbol on the right, and separated by a space:+.IP+.nf+\f[C]+fields date,description,cur,amt+amount %amt %cur+\f[R]+.fi+.IP+.nf+\f[C]+2020-01-01 foo+    expenses:unknown        123.00 USD+    income:unknown         -123.00 USD+\f[R]+.fi+.PP+Note we used a temporary field name (\f[C]cur\f[R]) that is not+\f[C]currency\f[R] - that would trigger the prepending effect, which we+don\[aq]t want here. .SS Referencing other fields .PP In field assignments, you can interpolate only CSV fields, not hledger

@@ -3,8 +3,8 @@  File: hledger_csv.info,  Node: Top,  Next: EXAMPLES,  Up: (dir) -hledger_csv(5) hledger 1.18.99-******************************+hledger_csv(5) hledger 1.20+***************************  CSV - how hledger reads CSV data, and the CSV rules file format @@ -42,7 +42,9 @@ *'if' table*                    apply some rules to CSV records matched                                 by patterns, alternate syntax *'end'*                         skip the remaining CSV records-*'date-format'*                 describe the format of CSV dates+*'date-format'*                 how to parse dates in CSV records+*'decimal-mark'*                the decimal mark used in CSV amounts,+                                if ambiguous *'newest-first'*                disambiguate record order when there's                                 only one date *'include'*                     inline another CSV rules file@@ -387,6 +389,7 @@ * if table:: * end:: * date-format::+* decimal-mark:: * newest-first:: * include:: * balance-type::@@ -787,7 +790,7 @@  end  -File: hledger_csv.info,  Node: date-format,  Next: newest-first,  Prev: end,  Up: CSV RULES+File: hledger_csv.info,  Node: date-format,  Next: decimal-mark,  Prev: end,  Up: CSV RULES  2.8 'date-format' =================@@ -818,11 +821,29 @@ https://hackage.haskell.org/package/time/docs/Data-Time-Format.html#v:formatTime  -File: hledger_csv.info,  Node: newest-first,  Next: include,  Prev: date-format,  Up: CSV RULES+File: hledger_csv.info,  Node: decimal-mark,  Next: newest-first,  Prev: date-format,  Up: CSV RULES -2.9 'newest-first'+2.9 'decimal-mark' ================== +decimal-mark .++   or:++decimal-mark ,++   hledger automatically accepts either period or comma as a decimal+mark when parsing numbers (cf Amounts).  However if any numbers in the+CSV contain digit group marks, such as thousand-separating commas, you+should declare the decimal mark explicitly with this rule, to avoid+misparsed numbers.+++File: hledger_csv.info,  Node: newest-first,  Next: include,  Prev: decimal-mark,  Up: CSV RULES++2.10 'newest-first'+===================+ hledger always sorts the generated transactions by date.  Transactions on the same date should appear in the same order as their CSV records, as hledger can usually auto-detect whether the CSV's normal order is@@ -842,7 +863,7 @@  File: hledger_csv.info,  Node: include,  Next: balance-type,  Prev: newest-first,  Up: CSV RULES -2.10 'include'+2.11 'include' ==============  include RULESFILE@@ -865,7 +886,7 @@  File: hledger_csv.info,  Node: balance-type,  Prev: include,  Up: CSV RULES -2.11 'balance-type'+2.12 'balance-type' ===================  Balance assertions generated by assigning to balanceN are of the simple@@ -1049,26 +1070,47 @@ ==============================  If the currency/commodity symbol is included in the CSV's amount-field(s), you don't have to do anything special.+field(s): -   If the currency is provided as a separate CSV field, you can either:+2020-01-01,foo,$123.00 -   * assign that to 'currency', which adds it to all posting amounts.-     The symbol will prepended to the amount quantity (on the left-     side).  If you write a trailing space after the symbol, there will-     be a space between symbol and amount (an exception to the usual-     whitespace stripping).+   you don't have to do anything special for the commodity symbol, it+will be assigned as part of the amount.  Eg: -   * or assign it to 'currencyN' which adds it to posting N's amount-     only.+fields date,description,amount -   * or for more control, construct the amount from symbol and quantity-     using field assignment, eg:+2020-01-01 foo+    expenses:unknown         $123.00+    income:unknown          $-123.00 -     fields date,description,currency,quantity-     # add currency symbol on the right:-     amount %quantity %currency+   If the currency is provided as a separate CSV field: +2020-01-01,foo,USD,123.00++   You can assign that to the 'currency' pseudo-field, which has the+special effect of prepending itself to every amount in the transaction+(on the left, with no separating space):++fields date,description,currency,amount++2020-01-01 foo+    expenses:unknown       USD123.00+    income:unknown        USD-123.00++   Or, you can use a field assignment to construct the amount yourself,+with more control.  Eg to put the symbol on the right, and separated by+a space:++fields date,description,cur,amt+amount %amt %cur++2020-01-01 foo+    expenses:unknown        123.00 USD+    income:unknown         -123.00 USD++   Note we used a temporary field name ('cur') that is not 'currency' -+that would trigger the prepending effect, which we don't want here.+  File: hledger_csv.info,  Node: Referencing other fields,  Next: How CSV rules are evaluated,  Prev: Setting currency/commodity,  Up: TIPS @@ -1150,84 +1192,86 @@  Tag Table: Node: Top72-Node: EXAMPLES2677-Ref: #examples2783-Node: Basic2991-Ref: #basic3091-Node: Bank of Ireland3633-Ref: #bank-of-ireland3768-Node: Amazon5230-Ref: #amazon5348-Node: Paypal7067-Ref: #paypal7161-Node: CSV RULES14805-Ref: #csv-rules14914-Node: skip15209-Ref: #skip15302-Node: fields15677-Ref: #fields15799-Node: Transaction field names16964-Ref: #transaction-field-names17124-Node: Posting field names17235-Ref: #posting-field-names17387-Node: account17457-Ref: #account17573-Node: amount18110-Ref: #amount18241-Node: currency19348-Ref: #currency19483-Node: balance19689-Ref: #balance19823-Node: comment20140-Ref: #comment20257-Node: field assignment20420-Ref: #field-assignment20563-Node: separator21381-Ref: #separator21516-Node: if block22056-Ref: #if-block22181-Node: Matching the whole record22582-Ref: #matching-the-whole-record22757-Node: Matching individual fields23561-Ref: #matching-individual-fields23765-Node: Combining matchers23989-Ref: #combining-matchers24185-Node: Rules applied on successful match24498-Ref: #rules-applied-on-successful-match24689-Node: if table25343-Ref: #if-table25462-Node: end27200-Ref: #end27312-Node: date-format27536-Ref: #date-format27668-Node: newest-first28417-Ref: #newest-first28555-Node: include29238-Ref: #include29369-Node: balance-type29813-Ref: #balance-type29933-Node: TIPS30633-Ref: #tips30715-Node: Rapid feedback30971-Ref: #rapid-feedback31088-Node: Valid CSV31548-Ref: #valid-csv31678-Node: File Extension31870-Ref: #file-extension32022-Node: Reading multiple CSV files32451-Ref: #reading-multiple-csv-files32636-Node: Valid transactions32877-Ref: #valid-transactions33055-Node: Deduplicating importing33683-Ref: #deduplicating-importing33862-Node: Setting amounts34895-Ref: #setting-amounts35064-Node: Setting currency/commodity36051-Ref: #setting-currencycommodity36243-Node: Referencing other fields37046-Ref: #referencing-other-fields37246-Node: How CSV rules are evaluated38143-Ref: #how-csv-rules-are-evaluated38316+Node: EXAMPLES2787+Ref: #examples2893+Node: Basic3101+Ref: #basic3201+Node: Bank of Ireland3743+Ref: #bank-of-ireland3878+Node: Amazon5340+Ref: #amazon5458+Node: Paypal7177+Ref: #paypal7271+Node: CSV RULES14915+Ref: #csv-rules15024+Node: skip15336+Ref: #skip15429+Node: fields15804+Ref: #fields15926+Node: Transaction field names17091+Ref: #transaction-field-names17251+Node: Posting field names17362+Ref: #posting-field-names17514+Node: account17584+Ref: #account17700+Node: amount18237+Ref: #amount18368+Node: currency19475+Ref: #currency19610+Node: balance19816+Ref: #balance19950+Node: comment20267+Ref: #comment20384+Node: field assignment20547+Ref: #field-assignment20690+Node: separator21508+Ref: #separator21643+Node: if block22183+Ref: #if-block22308+Node: Matching the whole record22709+Ref: #matching-the-whole-record22884+Node: Matching individual fields23688+Ref: #matching-individual-fields23892+Node: Combining matchers24116+Ref: #combining-matchers24312+Node: Rules applied on successful match24625+Ref: #rules-applied-on-successful-match24816+Node: if table25470+Ref: #if-table25589+Node: end27327+Ref: #end27439+Node: date-format27663+Ref: #date-format27795+Node: decimal-mark28544+Ref: #decimal-mark28687+Node: newest-first29026+Ref: #newest-first29167+Node: include29850+Ref: #include29981+Node: balance-type30425+Ref: #balance-type30545+Node: TIPS31245+Ref: #tips31327+Node: Rapid feedback31583+Ref: #rapid-feedback31700+Node: Valid CSV32160+Ref: #valid-csv32290+Node: File Extension32482+Ref: #file-extension32634+Node: Reading multiple CSV files33063+Ref: #reading-multiple-csv-files33248+Node: Valid transactions33489+Ref: #valid-transactions33667+Node: Deduplicating importing34295+Ref: #deduplicating-importing34474+Node: Setting amounts35507+Ref: #setting-amounts35676+Node: Setting currency/commodity36663+Ref: #setting-currencycommodity36855+Node: Referencing other fields38029+Ref: #referencing-other-fields38229+Node: How CSV rules are evaluated39126+Ref: #how-csv-rules-are-evaluated39299  End Tag Table 

@@ -39,26 +39,28 @@        if table                         apply  some rules to CSV records matched                                         by patterns, alternate syntax        end                              skip the remaining CSV records-       date-format                      describe the format of CSV dates-       newest-first                     disambiguate record order  when  there's+       date-format                      how to parse dates in CSV records+       decimal-mark                     the decimal mark used in CSV amounts, if+                                        ambiguous+       newest-first                     disambiguate  record  order when there's                                         only one date        include                          inline another CSV rules file        balance-type                     choose which type of balance assignments                                         to use -       Note, for best error messages when reading CSV files, use a .csv,  .tsv+       Note,  for best error messages when reading CSV files, use a .csv, .tsv        or .ssv file extension or file prefix - see File Extension below.         There's an introductory Convert CSV files tutorial on hledger.org.  EXAMPLES-       Here  are  some sample hledger CSV rules files.  See also the full col-+       Here are some sample hledger CSV rules files.  See also the  full  col-        lection at:        https://github.com/simonmichael/hledger/tree/master/examples/csv     Basic-       At minimum, the rules file must identify the date  and  amount  fields,-       and  often  it also specifies the date format and how many header lines+       At  minimum,  the  rules file must identify the date and amount fields,+       and often it also specifies the date format and how many  header  lines        there are.  Here's a simple CSV file and a rules file for it:                Date, Description, Id, Amount@@ -77,8 +79,8 @@        Default account names are chosen, since we didn't set them.     Bank of Ireland-       Here's a CSV with two amount fields (Debit and Credit), and  a  balance-       field,  which we can use to add balance assertions, which is not neces-+       Here's  a  CSV with two amount fields (Debit and Credit), and a balance+       field, which we can use to add balance assertions, which is not  neces-        sary but provides extra error checking:                Date,Details,Debit,Credit,Balance@@ -120,13 +122,13 @@                   assets:bank:boi:checking         EUR-5.0 = EUR126.0                   expenses:unknown                  EUR5.0 -       The balance assertions don't raise an error above, because we're  read--       ing  directly  from  CSV, but they will be checked if these entries are+       The  balance assertions don't raise an error above, because we're read-+       ing directly from CSV, but they will be checked if  these  entries  are        imported into a journal file.     Amazon        Here we convert amazon.com order history, and use an if block to gener--       ate  a third posting if there's a fee.  (In practice you'd probably get+       ate a third posting if there's a fee.  (In practice you'd probably  get        this data from your bank instead, but it's an example.)                "Date","Type","To/From","Name","Status","Amount","Fees","Transaction ID"@@ -178,7 +180,7 @@                   expenses:fees           $1.00     Paypal-       Here's a real-world rules file for (customised) Paypal CSV,  with  some+       Here's  a  real-world rules file for (customised) Paypal CSV, with some        Paypal-specific rules, and a second rules file included:                "Date","Time","TimeZone","Name","Type","Status","Currency","Gross","Fee","Net","From Email Address","To Email Address","Transaction ID","Item Title","Item ID","Reference Txn ID","Receipt ID","Balance","Note"@@ -333,9 +335,9 @@    skip               skip N -       The word "skip" followed by a number (or no number,  meaning  1)  tells-       hledger  to  ignore  this  many non-empty lines preceding the CSV data.-       (Empty/blank lines are skipped automatically.) You'll need  this  when-+       The  word  "skip"  followed by a number (or no number, meaning 1) tells+       hledger to ignore this many non-empty lines  preceding  the  CSV  data.+       (Empty/blank  lines  are skipped automatically.) You'll need this when-        ever your CSV data contains header lines.         It also has a second purpose: it can be used inside if blocks to ignore@@ -344,27 +346,27 @@    fields               fields FIELDNAME1, FIELDNAME2, ... -       A fields list (the word  "fields"  followed  by  comma-separated  field-       names)  is  the quick way to assign CSV field values to hledger fields.+       A  fields  list  (the  word  "fields" followed by comma-separated field+       names) is the quick way to assign CSV field values to  hledger  fields.        It does two things: -       1. it names the CSV fields.  This is optional, but  can  be  convenient+       1. it  names  the  CSV fields.  This is optional, but can be convenient           later for interpolating them.         2. when you use a standard hledger field name, it assigns the CSV value           to that part of the hledger transaction. -       Here's an example that says "use the 1st, 2nd and  4th  fields  as  the-       transaction's  date,  description  and amount; name the last two fields+       Here's  an  example  that  says "use the 1st, 2nd and 4th fields as the+       transaction's date, description and amount; name the  last  two  fields        for later reference; and ignore the others":                fields date, description, , amount, , , somefield, anotherfield -       Field names may not contain whitespace.  Fields you  don't  care  about-       can  be  left  unnamed.  Currently there must be least two items (there+       Field  names  may  not contain whitespace.  Fields you don't care about+       can be left unnamed.  Currently there must be least  two  items  (there        must be at least one comma). -       Note, always use comma in the fields list, even if your  CSV  uses  an-+       Note,  always  use  comma in the fields list, even if your CSV uses an-        other separator character.         Here are the standard hledger field/pseudo-field names.  For more about@@ -377,52 +379,52 @@     Posting field names    account-       accountN, where N is 1 to 99, causes a posting to  be  generated,  with+       accountN,  where  N  is 1 to 99, causes a posting to be generated, with        that account name. -       Most  often  there are two postings, so you'll want to set account1 and-       account2.  Typically account1 is associated with the CSV file,  and  is-       set  once  with  a top-level assignment, while account2 is set based on+       Most often there are two postings, so you'll want to set  account1  and+       account2.   Typically  account1 is associated with the CSV file, and is+       set once with a top-level assignment, while account2 is  set  based  on        each transaction's description, and in conditional blocks. -       If a posting's account name is left unset but its amount  is  set  (see-       below),  a default account name will be chosen (like "expenses:unknown"+       If  a  posting's  account name is left unset but its amount is set (see+       below), a default account name will be chosen (like  "expenses:unknown"        or "income:unknown").     amount-       amountN sets posting N's amount.  If the CSV uses separate  fields  for-       inflows  and  outflows, you can use amountN-in and amountN-out instead.-       By assigning to amount1, amount2, ...  etc.  you can generate  anywhere+       amountN  sets  posting N's amount.  If the CSV uses separate fields for+       inflows and outflows, you can use amountN-in and  amountN-out  instead.+       By  assigning to amount1, amount2, ...  etc.  you can generate anywhere        from 0 to 99 postings. -       There  is  also  an older, unnumbered form of these names, suitable for+       There is also an older, unnumbered form of these  names,  suitable  for        2-posting transactions, which sets both posting 1's and (negated) post--       ing  2's  amount:  amount,  or amount-in and amount-out.  This is still-       supported because it keeps pre-hledger-1.17 csv  rules  files  working,-       and  because  it  can be more succinct, and because it converts posting+       ing 2's amount: amount, or amount-in and  amount-out.   This  is  still+       supported  because  it  keeps pre-hledger-1.17 csv rules files working,+       and because it can be more succinct, and because  it  converts  posting        2's amount to cost if there's a transaction price, which can be useful.         If you have an existing rules file using the unnumbered form, you might-       want  to  use  the numbered form in certain conditional blocks, without-       having to update and retest all the old  rules.   To  facilitate  this,-       posting    1    ignores    amount/amount-in/amount-out    if   any   of+       want to use the numbered form in certain  conditional  blocks,  without+       having  to  update  and  retest all the old rules.  To facilitate this,+       posting   1   ignores    amount/amount-in/amount-out    if    any    of        amount1/amount1-in/amount1-out are assigned, and posting 2 ignores them-       if  any  of  amount2/amount2-in/amount2-out are assigned, avoiding con-+       if any of amount2/amount2-in/amount2-out are  assigned,  avoiding  con-        flicts.     currency        If the CSV has the currency symbol in a separate field (ie, not part of-       the  amount  field), you can use currencyN to prepend it to posting N's+       the amount field), you can use currencyN to prepend it to  posting  N's        amount.  Or, currency with no number affects all postings.     balance-       balanceN sets a balance assertion amount (or if the posting  amount  is+       balanceN  sets  a balance assertion amount (or if the posting amount is        left empty, a balance assignment) on posting N. -       Also,  for  compatibility with hledger <1.17: balance with no number is+       Also, for compatibility with hledger <1.17: balance with no  number  is        equivalent to balance1. -       You can adjust the type of assertion/assignment with  the  balance-type+       You  can  adjust the type of assertion/assignment with the balance-type        rule (see below).     comment@@ -434,11 +436,11 @@    field assignment               HLEDGERFIELDNAME FIELDVALUE -       Instead of or in addition to a fields list, you can use  a  "field  as--       signment"  rule  to set the value of a single hledger field, by writing-       its name (any of the standard hledger field names above) followed by  a-       text  value.  The value may contain interpolated CSV fields, referenced-       by their 1-based position in the CSV record (%N), or by the  name  they+       Instead  of  or  in addition to a fields list, you can use a "field as-+       signment" rule to set the value of a single hledger field,  by  writing+       its  name (any of the standard hledger field names above) followed by a+       text value.  The value may contain interpolated CSV fields,  referenced+       by  their  1-based position in the CSV record (%N), or by the name they        were given in the fields list (%CSVFIELDNAME).  Some examples:                # set the amount to the 4th CSV field, with " USD" appended@@ -447,14 +449,14 @@               # combine three fields to make a comment, containing note: and date: tags               comment note: %somefield - %anotherfield, date: %1 -       Interpolation  strips  outer  whitespace (so a CSV value like " 1 " be-+       Interpolation strips outer whitespace (so a CSV value like "  1  "  be-        comes 1 when interpolated) (#1051).  See TIPS below for more about ref-        erencing other fields.     separator-       You  can  use the separator rule to read other kinds of character-sepa--       rated data.  The argument is any single  separator  character,  or  the-       words  tab or space (case insensitive).  Eg, for comma-separated values+       You can use the separator rule to read other kinds  of  character-sepa-+       rated  data.   The  argument  is any single separator character, or the+       words tab or space (case insensitive).  Eg, for comma-separated  values        (CSV):                separator ,@@ -467,7 +469,7 @@                separator TAB -       If the input file has a .csv, .ssv or .tsv file extension (or  a  csv:,+       If  the  input file has a .csv, .ssv or .tsv file extension (or a csv:,        ssv:, tsv: prefix), the appropriate separator will be inferred automat-        ically, and you won't need this rule. @@ -482,8 +484,8 @@                RULE                RULE -       Conditional blocks ("if blocks") are a block of rules that are  applied-       only  to CSV records which match certain patterns.  They are often used+       Conditional  blocks ("if blocks") are a block of rules that are applied+       only to CSV records which match certain patterns.  They are often  used        for customising account names based on transaction descriptions.     Matching the whole record@@ -491,16 +493,16 @@                REGEX -       REGEX is a case-insensitive regular expression  which  tries  to  match-       anywhere  within  the  CSV record.  It is a POSIX ERE (extended regular-       expression) that also supports GNU word boundaries (\b,  \B,  \<,  \>),+       REGEX  is  a  case-insensitive  regular expression which tries to match+       anywhere within the CSV record.  It is a POSIX  ERE  (extended  regular+       expression)  that  also  supports GNU word boundaries (\b, \B, \<, \>),        and  nothing  else.   If  you  have  trouble,  be  sure  to  check  our        https://hledger.org/hledger.html#regular-expressions doc. -       Important note: the record that is matched is not the original  record,-       but  a synthetic one, with any enclosing double quotes (but not enclos-+       Important  note: the record that is matched is not the original record,+       but a synthetic one, with any enclosing double quotes (but not  enclos-        ing whitespace) removed, and always comma-separated (which means that a-       field  containing  a  comma  will  appear like two fields).  Eg, if the+       field containing a comma will appear like  two  fields).   Eg,  if  the        original record is 2020-01-01; "Acme, Inc.";  1,000, the REGEX will ac-        tually see 2020-01-01,Acme, Inc.,  1,000). @@ -509,14 +511,14 @@                %CSVFIELD REGEX -       which  matches just the content of a particular CSV field.  CSVFIELD is-       a percent sign followed by the field's  name  or  column  number,  like+       which matches just the content of a particular CSV field.  CSVFIELD  is+       a  percent  sign  followed  by  the field's name or column number, like        %date or %1.     Combining matchers        A single matcher can be written on the same line as the "if"; or multi-        ple matchers can be written on the following lines, non-indented.  Mul--       tiple  matchers are OR'd (any one of them can match), unless one begins+       tiple matchers are OR'd (any one of them can match), unless one  begins        with an & symbol, in which case it is AND'ed with the previous matcher.                if@@ -525,8 +527,8 @@                RULE     Rules applied on successful match-       After the patterns there should be one or more rules to apply, all  in--       dented  by at least one space.  Three kinds of rule are allowed in con-+       After  the patterns there should be one or more rules to apply, all in-+       dented by at least one space.  Three kinds of rule are allowed in  con-        ditional blocks:         o field assignments (to set a hledger field)@@ -556,11 +558,11 @@               MATCHER3,VALUE31,VALUE32,...,VALUE3n               <empty line> -       Conditional tables ("if tables") are  a  different  syntax  to  specify-       field  assignments that will be applied only to CSV records which match+       Conditional  tables  ("if  tables")  are  a different syntax to specify+       field assignments that will be applied only to CSV records which  match        certain patterns. -       MATCHER could be either field or record matcher,  as  described  above.+       MATCHER  could  be  either field or record matcher, as described above.        When MATCHER matches, values from that row would be assigned to the CSV        fields named on the if line, in the same order. @@ -584,17 +586,17 @@                 ...                 CSVFIELDNAMEn VALUE3n -       Each line starting with MATCHER should contain enough (possibly  empty)+       Each  line starting with MATCHER should contain enough (possibly empty)        values for all the listed fields. -       Rules  would be checked and applied in the order they are listed in the+       Rules would be checked and applied in the order they are listed in  the        table and, like with if blocks, later rules (in the same or another ta-        ble) or if blocks could override the effect of any rule. -       Instead  of ',' you can use a variety of other non-alphanumeric charac-+       Instead of ',' you can use a variety of other non-alphanumeric  charac-        ters as a separator.  First character after if is taken to be the sepa--       rator  for the rest of the table.  It is the responsibility of the user-       to ensure that separator does not occur inside MATCHERs  and  values  -+       rator for the rest of the table.  It is the responsibility of the  user+       to  ensure  that  separator does not occur inside MATCHERs and values -        there is no way to escape separator.         Example:@@ -605,7 +607,7 @@               2020/01/12.*Plumbing LLC,expenses:house:upkeep,emergency plumbing call-out     end-       This  rule  can  be  used inside if blocks (only), to make hledger stop+       This rule can be used inside if blocks (only),  to  make  hledger  stop        reading this CSV file and move on to the next input file, or to command        execution.  Eg: @@ -616,10 +618,10 @@    date-format               date-format DATEFMT -       This  is  a  helper for the date (and date2) fields.  If your CSV dates-       are not formatted like YYYY-MM-DD,  YYYY/MM/DD  or  YYYY.MM.DD,  you'll-       need  to  add  a  date-format rule describing them with a strptime date-       parsing pattern, which must parse the CSV date value completely.   Some+       This is a helper for the date (and date2) fields.  If  your  CSV  dates+       are  not  formatted  like  YYYY-MM-DD, YYYY/MM/DD or YYYY.MM.DD, you'll+       need to add a date-format rule describing them  with  a  strptime  date+       parsing  pattern, which must parse the CSV date value completely.  Some        examples:                # MM/DD/YY@@ -640,16 +642,29 @@        https://hackage.haskell.org/package/time/docs/Data-Time-For-        mat.html#v:formatTime +   decimal-mark+              decimal-mark .++       or:++              decimal-mark ,++       hledger  automatically accepts either period or comma as a decimal mark+       when parsing numbers (cf Amounts).  However if any numbers in  the  CSV+       contain  digit  group  marks,  such  as thousand-separating commas, you+       should declare the decimal mark explicitly with  this  rule,  to  avoid+       misparsed numbers.+    newest-first-       hledger always sorts the generated transactions by date.   Transactions-       on  the same date should appear in the same order as their CSV records,-       as hledger can usually auto-detect whether the CSV's  normal  order  is+       hledger  always sorts the generated transactions by date.  Transactions+       on the same date should appear in the same order as their CSV  records,+       as  hledger  can  usually auto-detect whether the CSV's normal order is        oldest first or newest first.  But if all of the following are true: -       o the  CSV  might  sometimes  contain just one day of data (all records+       o the CSV might sometimes contain just one day  of  data  (all  records          having the same date) -       o the CSV records are normally in reverse chronological  order  (newest+       o the  CSV  records are normally in reverse chronological order (newest          at the top)         o and you care about preserving the order of same-day transactions@@ -662,9 +677,9 @@    include               include RULESFILE -       This  includes  the  contents  of another CSV rules file at this point.-       RULESFILE is an absolute file path or a path relative  to  the  current-       file's  directory.  This can be useful for sharing common rules between+       This includes the contents of another CSV rules  file  at  this  point.+       RULESFILE  is  an  absolute file path or a path relative to the current+       file's directory.  This can be useful for sharing common rules  between        several rules files, eg:                # someaccount.csv.rules@@ -679,10 +694,10 @@     balance-type        Balance assertions generated by assigning to balanceN are of the simple-       =  type  by  default, which is a single-commodity, subaccount-excluding+       = type by default, which is  a  single-commodity,  subaccount-excluding        assertion.  You may find the subaccount-including variants more useful,-       eg  if  you  have  created some virtual subaccounts of checking to help-       with budgeting.  You can select a different type of assertion with  the+       eg if you have created some virtual subaccounts  of  checking  to  help+       with  budgeting.  You can select a different type of assertion with the        balance-type rule:                # balance assertions will consider all commodities and all subaccounts@@ -697,19 +712,19 @@  TIPS    Rapid feedback-       It's  a  good idea to get rapid feedback while creating/troubleshooting+       It's a good idea to get rapid feedback  while  creating/troubleshooting        CSV rules.  Here's a good way, using entr from http://eradman.com/entr-        project :                $ ls foo.csv* | entr bash -c 'echo ----; hledger -f foo.csv print desc:SOMEDESC' -       A  desc:  query (eg) is used to select just one, or a few, transactions-       of interest.  "bash -c" is used to run multiple  commands,  so  we  can-       echo  a  separator  each  time the command re-runs, making it easier to+       A desc: query (eg) is used to select just one, or a  few,  transactions+       of  interest.   "bash  -c"  is used to run multiple commands, so we can+       echo a separator each time the command re-runs,  making  it  easier  to        read the output.     Valid CSV-       hledger accepts CSV conforming to RFC 4180.  When CSV  values  are  en-+       hledger  accepts  CSV  conforming to RFC 4180.  When CSV values are en-        closed in quotes, note:         o they must be double quotes (not single quotes)@@ -717,9 +732,9 @@        o spaces outside the quotes are not allowed     File Extension-       To  help hledger identify the format and show the right error messages,-       CSV/SSV/TSV files should normally be named with a .csv,  .ssv  or  .tsv-       filename  extension.   Or,  the file path should be prefixed with csv:,+       To help hledger identify the format and show the right error  messages,+       CSV/SSV/TSV  files  should  normally be named with a .csv, .ssv or .tsv+       filename extension.  Or, the file path should be  prefixed  with  csv:,        ssv: or tsv:.  Eg:                $ hledger -f foo.ssv print@@ -728,48 +743,48 @@                $ cat foo | hledger -f ssv:- foo -       You can override the file extension with a separator  rule  if  needed.+       You  can  override  the file extension with a separator rule if needed.        See also: Input files in the hledger manual.     Reading multiple CSV files-       If  you  use  multiple  -f  options to read multiple CSV files at once,-       hledger will look for a correspondingly-named rules file for  each  CSV-       file.   But if you use the --rules-file option, that rules file will be+       If you use multiple -f options to read  multiple  CSV  files  at  once,+       hledger  will  look for a correspondingly-named rules file for each CSV+       file.  But if you use the --rules-file option, that rules file will  be        used for all the CSV files.     Valid transactions        After reading a CSV file, hledger post-processes and validates the gen-        erated journal entries as it would for a journal file - balancing them,-       applying balance assignments, and canonicalising  amount  styles.   Any-       errors  at this stage will be reported in the usual way, displaying the+       applying  balance  assignments,  and canonicalising amount styles.  Any+       errors at this stage will be reported in the usual way, displaying  the        problem entry.         There is one exception: balance assertions, if you have generated them,-       will  not  be checked, since normally these will work only when the CSV-       data is part of the main journal.  If you do need to check balance  as-+       will not be checked, since normally these will work only when  the  CSV+       data  is part of the main journal.  If you do need to check balance as-        sertions generated from CSV right away, pipe into another hledger:                $ hledger -f file.csv print | hledger -f- print     Deduplicating, importing-       When  you  download a CSV file periodically, eg to get your latest bank-       transactions, the new file may overlap with  the  old  one,  containing+       When you download a CSV file periodically, eg to get your  latest  bank+       transactions,  the  new  file  may overlap with the old one, containing        some of the same records.         The import command will (a) detect the new transactions, and (b) append        just those transactions to your main journal.  It is idempotent, so you-       don't  have to remember how many times you ran it or with which version-       of the CSV.  (It keeps state in a hidden .latest.FILE.csv  file.)  This+       don't have to remember how many times you ran it or with which  version+       of  the  CSV.  (It keeps state in a hidden .latest.FILE.csv file.) This        is the easiest way to import CSV data.  Eg:                # download the latest CSV files, then run this command.               # Note, no -f flags needed here.               $ hledger import *.csv [--dry] -       This  method  works  for  most CSV files.  (Where records have a stable+       This method works for most CSV files.  (Where  records  have  a  stable        chronological order, and new records appear only at the new end.) -       A number of other tools and workflows, hledger-specific and  otherwise,+       A  number of other tools and workflows, hledger-specific and otherwise,        exist for converting, deduplicating, classifying and managing CSV data.        See: @@ -780,49 +795,71 @@    Setting amounts        A posting amount can be set in one of these ways: -       o by assigning (with a fields list  or  field  assignment)  to  amountN+       o by  assigning  (with  a  fields  list or field assignment) to amountN          (posting N's amount) or amount (posting 1's amount) -       o by  assigning to amountN-in and amountN-out (or amount-in and amount--         out).  For each CSV record, whichever of these has a  non-zero  value-         will  be  used,  with  appropriate  sign.  If both contain a non-zero+       o by assigning to amountN-in and amountN-out (or amount-in and  amount-+         out).   For  each CSV record, whichever of these has a non-zero value+         will be used, with appropriate sign.   If  both  contain  a  non-zero          value, this may not work. -       o by assigning to balanceN (or balance) instead of the  above,  setting-         the  amount  indirectly via a balance assignment.  If you do this the+       o by  assigning  to balanceN (or balance) instead of the above, setting+         the amount indirectly via a balance assignment.  If you do  this  the          default account name may be wrong, so you should set that explicitly.         There is some special handling for an amount's sign: -       o If an amount value is parenthesised, it will be de-parenthesised  and+       o If  an amount value is parenthesised, it will be de-parenthesised and          sign-flipped. -       o If  an amount value begins with a double minus sign, those cancel out+       o If an amount value begins with a double minus sign, those cancel  out          and are removed.         o If an amount value begins with a plus sign, that will be removed     Setting currency/commodity-       If the currency/commodity  symbol  is  included  in  the  CSV's  amount-       field(s), you don't have to do anything special.+       If  the  currency/commodity  symbol  is  included  in  the CSV's amount+       field(s): -       If the currency is provided as a separate CSV field, you can either:+              2020-01-01,foo,$123.00 -       o assign  that  to currency, which adds it to all posting amounts.  The-         symbol will prepended to the amount quantity (on the left side).   If-         you  write  a  trailing space after the symbol, there will be a space-         between symbol and amount  (an  exception  to  the  usual  whitespace-         stripping).+       you don't have to do anything special for the commodity symbol, it will+       be assigned as part of the amount.  Eg: -       o or assign it to currencyN which adds it to posting N's amount only.+              fields date,description,amount -       o or  for  more  control, construct the amount from symbol and quantity-         using field assignment, eg:+              2020-01-01 foo+                  expenses:unknown         $123.00+                  income:unknown          $-123.00 -                fields date,description,currency,quantity-                # add currency symbol on the right:-                amount %quantity %currency+       If the currency is provided as a separate CSV field: +              2020-01-01,foo,USD,123.00++       You can assign that to the currency pseudo-field, which has the special+       effect of prepending itself to every amount in the transaction (on  the+       left, with no separating space):++              fields date,description,currency,amount++              2020-01-01 foo+                  expenses:unknown       USD123.00+                  income:unknown        USD-123.00++       Or,  you  can  use a field assignment to construct the amount yourself,+       with more control.  Eg to put the symbol on the right, and separated by+       a space:++              fields date,description,cur,amt+              amount %amt %cur++              2020-01-01 foo+                  expenses:unknown        123.00 USD+                  income:unknown         -123.00 USD++       Note  we  used a temporary field name (cur) that is not currency - that+       would trigger the prepending effect, which we don't want here.+    Referencing other fields        In field assignments, you can interpolate only CSV fields, not  hledger        fields.   In  the example below, there's both a CSV field and a hledger@@ -922,4 +959,4 @@   -hledger 1.18.99                 September 2020                  hledger_csv(5)+hledger 1.20                     November 2020                  hledger_csv(5)

@@ -1,6 +1,6 @@ .\"t -.TH "hledger_journal" "5" "September 2020" "hledger 1.18.99" "hledger User Manuals"+.TH "hledger_journal" "5" "November 2020" "hledger 1.20" "hledger User Manuals"   @@ -584,40 +584,59 @@ commodity       1 000 000.9455 \f[R] .fi-.SS Amount display style .PP-For each commodity, hledger chooses a consistent format to use when+.SS Commodity display style+.PP+For each commodity, hledger chooses a consistent style to use when displaying amounts. (Except price amounts, which are always displayed as written). The display style is chosen as follows: .IP \[bu] 2 If there is a commodity directive (or default commodity directive) for-the commodity, that format is used (see examples above).+the commodity, its style is used (see examples above). .IP \[bu] 2-Otherwise the format of the first posting amount in that commodity seen-in the journal is used.-But the number of decimal places (\[dq]precision\[dq]) will be the-maximum from all posting amounts in that commodity.+Otherwise the style is inferred from the amounts in that commodity seen+in the journal. .IP \[bu] 2-Or if there are no such amounts in the journal, a default format is used+Or if there are no such amounts in the journal, a default style is used (like \f[C]$1000.00\f[R]). .PP-Transaction prices don\[aq]t affect the amount display style directly,-but occasionally they can do so indirectly (eg when an posting\[aq]s-amount is inferred using a transaction price).+A style is inferred from the journal amounts in a commodity as follows:+.IP \[bu] 2+Use the general style (decimal mark, symbol placement) of the first+amount+.IP \[bu] 2+Use the first-seen digit group style (digit group mark, digit group+sizes), if any+.IP \[bu] 2+Use the maximum number of decimal places of all.+.PP+Transaction price amounts don\[aq]t affect the commodity display style+directly, but occasionally they can do so indirectly (eg when a+posting\[aq]s amount is inferred using a transaction price). If you find this causing problems, use a commodity directive to fix the display style. .PP-In summary: amounts will be displayed much as they appear in your-journal, with the max observed number of decimal places.-If you want to see fewer decimal places in reports, use a commodity-directive to override that.+In summary, each commodity\[aq]s amounts will be normalised to+.IP \[bu] 2+the style declared by a \f[C]commodity\f[R] directive+.IP \[bu] 2+or, the style of the first posting amount in the journal, with the+first-seen digit group style and the maximum-seen number of decimal+places. .PP-hledger uses banker\[aq]s rounding: it rounds to the nearest even+If reports are showing amounts in a way you don\[aq]t like (eg, with too+many decimal places), use a commodity directive to set your preferred+style.+.SS Rounding+.PP+Amounts are stored internally as decimal numbers with up to 255 decimal+places, and displayed with the number of decimal places specified by the+commodity display style.+Note, hledger uses banker\[aq]s rounding: it rounds to the nearest even number, eg 0.5 displayed with zero decimal places is \[dq]0\[dq]).-(Note, prior to hledger 1.17.1 this could vary if hledger happened to be-built with an old version of Decimal (<0.5.1); since 1.17.1 it\[aq]s-guaranteed.)+(Guaranteed since hledger 1.17.1; in older versions this could vary if+hledger was built with Decimal < 0.5.1.) .SS Transaction prices .PP Within a transaction, you can note an amount\[aq]s price in another@@ -950,6 +969,8 @@ Directives\[aq] behaviour and interactions can get a little bit complex, so here is a table summarising the directives and their effects, with links to more detailed docs.+Note part of this table is hidden when viewed in a web browser - scroll+it sideways to see more. .PP .TS tab(@);@@ -984,8 +1005,7 @@ T}@T{ rewrite account names T}@T{-following inline/included entries until end of current file or end-directive+following entries until end of current file or end directive T} T{ \f[C]apply account\f[R]@@ -995,8 +1015,7 @@ T}@T{ prepend a common parent to account names T}@T{-following inline/included entries until end of current file or end-directive+following entries until end of current file or end directive T} T{ \f[C]comment\f[R]@@ -1006,8 +1025,7 @@ T}@T{ ignore part of journal T}@T{-following inline/included entries until end of current file or end-directive+following entries until end of current file or end directive T} T{ \f[C]commodity\f[R]@@ -1017,7 +1035,7 @@ T}@T{ declare a commodity and its number notation & display style T}@T{-number notation: following entries in that commodity in all files;+number notation: following entries in that commodity in all files ; display style: amounts of that commodity in reports T} T{@@ -1057,7 +1075,7 @@ T}@T{ declare a year for yearless dates T}@T{-following inline/included entries until end of current file+following entries until end of current file T} T{ \f[C]=\f[R]@@ -1199,7 +1217,7 @@ (Without this, hledger will parse both \f[C]1,000\f[R] and \f[C]1.000\f[R] as 1). .IP "3." 3-It declares the amount display style to use in output - decimal and+It declares a commodity\[aq]s display style in output - decimal and digit group marks, number of decimal places, symbol placement etc. .PP You are likely to run into one of the problems solved by commodity@@ -1245,7 +1263,14 @@ .PP Note hledger normally uses banker\[aq]s rounding, so 0.5 displayed with zero decimal digits is \[dq]0\[dq].-(More at Amount display style.)+(More at Commodity display style.)+.SS Commodity error checking+.PP+In strict mode, enabled with the \f[C]-s\f[R]/\f[C]--strict\f[R] flag,+hledger will report an error if a commodity symbol is used that has not+been declared by a \f[C]commodity\f[R] directive.+This works similarly to account error checking, see the notes there for+more details. .SS Default commodity .PP The \f[C]D\f[R] directive sets a default commodity, to be used for@@ -1314,15 +1339,13 @@ See Valuation. .SS Declaring accounts .PP-\f[C]account\f[R] directives can be used to pre-declare accounts.-Though not required, they can provide several benefits:+\f[C]account\f[R] directives can be used to declare accounts (ie, the+places that amounts are transferred from and to).+Though not required, these declarations can provide several benefits: .IP \[bu] 2 They can document your intended chart of accounts, providing a reference. .IP \[bu] 2-They can store extra information about accounts (account numbers, notes,-etc.)-.IP \[bu] 2 They can help hledger know your accounts\[aq] types (asset, liability, equity, revenue, expense), useful for reports like balancesheet and incomestatement.@@ -1330,17 +1353,54 @@ They control account display order in reports, allowing non-alphabetic sorting (eg Revenues to appear above Expenses). .IP \[bu] 2+They can store extra information about accounts (account numbers, notes,+etc.)+.IP \[bu] 2 They help with account name completion in the add command, hledger-iadd, hledger-web, ledger-mode etc.+.IP \[bu] 2+In strict mode, they restrict which accounts may be posted to by+transactions, which helps detect typos. .PP The simplest form is just the word \f[C]account\f[R] followed by a-hledger-style account name, eg:+hledger-style account name, eg this account directive declares the+\f[C]assets:bank:checking\f[R] account: .IP .nf \f[C] account assets:bank:checking \f[R] .fi+.SS Account error checking+.PP+By default, accounts come into existence when a transaction references+them by name.+This is convenient, but it means hledger can\[aq]t warn you when you+mis-spell an account name in the journal.+Usually you\[aq]ll find the error later, as an extra account in balance+reports, or an incorrect balance when reconciling.+.PP+In strict mode, enabled with the \f[C]-s\f[R]/\f[C]--strict\f[R] flag,+hledger will report an error if any transaction uses an account name+that has not been declared by an account directive.+Some notes:+.IP \[bu] 2+The declaration is case-sensitive; transactions must use the correct+account name capitalisation.+.IP \[bu] 2+The account directive\[aq]s scope is \[dq]whole file and below\[dq] (see+directives).+This means it affects all of the current file, and any files it+includes, but not parent or sibling files.+The position of account directives within the file does not matter,+though it\[aq]s usual to put them at the top.+.IP \[bu] 2+Accounts can only be declared in \f[C]journal\f[R] files (but will+affect included files in other formats).+.IP \[bu] 2+It\[aq]s currently not possible to declare \[dq]all possible+subaccounts\[dq] with a wildcard; every account posted to must be+declared. .SS Account comments .PP Comments, beginning with a semicolon, can be added:@@ -1941,8 +2001,7 @@ (and also, a goal of depositing $2000 into checking) every month. Goals and actual performance can then be compared in budget reports. .PP-For more details, see: balance: Budget report and Budgeting and-Forecasting.+See also: Budgeting and Forecasting. .PP .SS Auto postings .PP

@@ -4,8 +4,8 @@  File: hledger_journal.info,  Node: Top,  Up: (dir) -hledger_journal(5) hledger 1.18.99-**********************************+hledger_journal(5) hledger 1.20+*******************************  Journal - hledger's default file format, representing a General Journal @@ -487,10 +487,11 @@ * Menu:  * Digit group marks::-* Amount display style::+* Commodity display style::+* Rounding::  -File: hledger_journal.info,  Node: Digit group marks,  Next: Amount display style,  Up: Amounts+File: hledger_journal.info,  Node: Digit group marks,  Next: Commodity display style,  Up: Amounts  1.8.1 Digit group marks -----------------------@@ -523,42 +524,64 @@ commodity       1 000 000.9455  -File: hledger_journal.info,  Node: Amount display style,  Prev: Digit group marks,  Up: Amounts+File: hledger_journal.info,  Node: Commodity display style,  Next: Rounding,  Prev: Digit group marks,  Up: Amounts -1.8.2 Amount display style---------------------------+1.8.2 Commodity display style+----------------------------- -For each commodity, hledger chooses a consistent format to use when+For each commodity, hledger chooses a consistent style to use when displaying amounts.  (Except price amounts, which are always displayed as written).  The display style is chosen as follows:     * If there is a commodity directive (or default commodity directive)-     for the commodity, that format is used (see examples above).+     for the commodity, its style is used (see examples above). -   * Otherwise the format of the first posting amount in that commodity-     seen in the journal is used.  But the number of decimal places-     ("precision") will be the maximum from all posting amounts in that-     commodity.+   * Otherwise the style is inferred from the amounts in that commodity+     seen in the journal. -   * Or if there are no such amounts in the journal, a default format is+   * Or if there are no such amounts in the journal, a default style is      used (like '$1000.00'). -   Transaction prices don't affect the amount display style directly,-but occasionally they can do so indirectly (eg when an posting's amount-is inferred using a transaction price).  If you find this causing-problems, use a commodity directive to fix the display style.+   A style is inferred from the journal amounts in a commodity as+follows: -   In summary: amounts will be displayed much as they appear in your-journal, with the max observed number of decimal places.  If you want to-see fewer decimal places in reports, use a commodity directive to-override that.+   * Use the general style (decimal mark, symbol placement) of the first+     amount+   * Use the first-seen digit group style (digit group mark, digit group+     sizes), if any+   * Use the maximum number of decimal places of all. -   hledger uses banker's rounding: it rounds to the nearest even number,-eg 0.5 displayed with zero decimal places is "0").  (Note, prior to-hledger 1.17.1 this could vary if hledger happened to be built with an-old version of Decimal (<0.5.1); since 1.17.1 it's guaranteed.)+   Transaction price amounts don't affect the commodity display style+directly, but occasionally they can do so indirectly (eg when a+posting's amount is inferred using a transaction price).  If you find+this causing problems, use a commodity directive to fix the display+style. +   In summary, each commodity's amounts will be normalised to++   * the style declared by a 'commodity' directive+   * or, the style of the first posting amount in the journal, with the+     first-seen digit group style and the maximum-seen number of decimal+     places.++   If reports are showing amounts in a way you don't like (eg, with too+many decimal places), use a commodity directive to set your preferred+style.+ +File: hledger_journal.info,  Node: Rounding,  Prev: Commodity display style,  Up: Amounts++1.8.3 Rounding+--------------++Amounts are stored internally as decimal numbers with up to 255 decimal+places, and displayed with the number of decimal places specified by the+commodity display style.  Note, hledger uses banker's rounding: it+rounds to the nearest even number, eg 0.5 displayed with zero decimal+places is "0").  (Guaranteed since hledger 1.17.1; in older versions+this could vary if hledger was built with Decimal < 0.5.1.)++ File: hledger_journal.info,  Node: Transaction prices,  Next: Lot prices and lot dates,  Prev: Amounts,  Up: Transactions  1.9 Transaction prices@@ -888,7 +911,8 @@     Directives' behaviour and interactions can get a little bit complex, so here is a table summarising the directives and their effects, with-links to more detailed docs.+links to more detailed docs.  Note part of this table is hidden when+viewed in a web browser - scroll it sideways to see more.  directiveend       subdirectivespurpose                  can affect (as of          directive                                       2018/06)@@ -896,25 +920,22 @@ 'account'          any     document account names,       all entries in                    text    declare account types &       all files, before                            display order                 or after-'alias'  'end              rewrite account names         following-         aliases'                                        inline/included-                                                         entries until end-                                                         of current file-                                                         or end directive-'apply   'end              prepend a common parent to    following-account' apply             account names                 inline/included-         account'                                        entries until end-                                                         of current file-                                                         or end directive-'comment''end              ignore part of journal        following-         comment'                                        inline/included-                                                         entries until end-                                                         of current file-                                                         or end directive+'alias'  'end              rewrite account names         following entries+         aliases'                                        until end of+                                                         current file or+                                                         end directive+'apply   'end              prepend a common parent to    following entries+account' apply             account names                 until end of+         account'                                        current file or+                                                         end directive+'comment''end              ignore part of journal        following entries+         comment'                                        until end of+                                                         current file or+                                                         end directive 'commodity'        'format'declare a commodity and its   number notation:                            number notation & display     following entries                            style                         in that commodity-                                                         in all files;+                                                         in all files ;                                                          display style:                                                          amounts of that                                                          commodity in@@ -940,10 +961,9 @@                            a commodity                   commodity in                                                          reports, when -V                                                          is used-'Y'                        declare a year for yearless   following-                           dates                         inline/included-                                                         entries until end-                                                         of current file+'Y'                        declare a year for yearless   following entries+                           dates                         until end of+                                                         current file '='                        declare an auto posting       all entries in                            rule, adding postings to      parent/current/child                            other transactions            files (but not@@ -1082,7 +1102,7 @@      formats in your data.  (Without this, hledger will parse both      '1,000' and '1.000' as 1). -  3. It declares the amount display style to use in output - decimal and+  3. It declares a commodity's display style in output - decimal and      digit group marks, number of decimal places, symbol placement etc.     You are likely to run into one of the problems solved by commodity@@ -1117,9 +1137,24 @@ a comma, followed by 0 or more decimal digits.     Note hledger normally uses banker's rounding, so 0.5 displayed with-zero decimal digits is "0".  (More at Amount display style.)+zero decimal digits is "0".  (More at Commodity display style.) +* Menu:++* Commodity error checking::+ +File: hledger_journal.info,  Node: Commodity error checking,  Up: Declaring commodities++1.13.5.1 Commodity error checking+.................................++In strict mode, enabled with the '-s'/'--strict' flag, hledger will+report an error if a commodity symbol is used that has not been declared+by a 'commodity' directive.  This works similarly to account error+checking, see the notes there for more details.++ File: hledger_journal.info,  Node: Default commodity,  Next: Declaring market prices,  Prev: Declaring commodities,  Up: Directives  1.13.6 Default commodity@@ -1182,37 +1217,70 @@ 1.13.8 Declaring accounts ------------------------- -'account' directives can be used to pre-declare accounts.  Though not-required, they can provide several benefits:+'account' directives can be used to declare accounts (ie, the places+that amounts are transferred from and to).  Though not required, these+declarations can provide several benefits:     * They can document your intended chart of accounts, providing a      reference.-   * They can store extra information about accounts (account numbers,-     notes, etc.)    * They can help hledger know your accounts' types (asset, liability,      equity, revenue, expense), useful for reports like balancesheet and      incomestatement.    * They control account display order in reports, allowing      non-alphabetic sorting (eg Revenues to appear above Expenses).+   * They can store extra information about accounts (account numbers,+     notes, etc.)    * They help with account name completion in the add command,      hledger-iadd, hledger-web, ledger-mode etc.+   * In strict mode, they restrict which accounts may be posted to by+     transactions, which helps detect typos.     The simplest form is just the word 'account' followed by a-hledger-style account name, eg:+hledger-style account name, eg this account directive declares the+'assets:bank:checking' account:  account assets:bank:checking  * Menu: +* Account error checking:: * Account comments:: * Account subdirectives:: * Account types:: * Account display order::  -File: hledger_journal.info,  Node: Account comments,  Next: Account subdirectives,  Up: Declaring accounts+File: hledger_journal.info,  Node: Account error checking,  Next: Account comments,  Up: Declaring accounts -1.13.8.1 Account comments+1.13.8.1 Account error checking+...............................++By default, accounts come into existence when a transaction references+them by name.  This is convenient, but it means hledger can't warn you+when you mis-spell an account name in the journal.  Usually you'll find+the error later, as an extra account in balance reports, or an incorrect+balance when reconciling.++   In strict mode, enabled with the '-s'/'--strict' flag, hledger will+report an error if any transaction uses an account name that has not+been declared by an account directive.  Some notes:++   * The declaration is case-sensitive; transactions must use the+     correct account name capitalisation.+   * The account directive's scope is "whole file and below" (see+     directives).  This means it affects all of the current file, and+     any files it includes, but not parent or sibling files.  The+     position of account directives within the file does not matter,+     though it's usual to put them at the top.+   * Accounts can only be declared in 'journal' files (but will affect+     included files in other formats).+   * It's currently not possible to declare "all possible subaccounts"+     with a wildcard; every account posted to must be declared.+++File: hledger_journal.info,  Node: Account comments,  Next: Account subdirectives,  Prev: Account error checking,  Up: Declaring accounts++1.13.8.2 Account comments .........................  Comments, beginning with a semicolon, can be added:@@ -1232,7 +1300,7 @@  File: hledger_journal.info,  Node: Account subdirectives,  Next: Account types,  Prev: Account comments,  Up: Declaring accounts -1.13.8.2 Account subdirectives+1.13.8.3 Account subdirectives ..............................  We also allow (and ignore) Ledger-style indented subdirectives, just for@@ -1250,7 +1318,7 @@  File: hledger_journal.info,  Node: Account types,  Next: Account display order,  Prev: Account subdirectives,  Up: Declaring accounts -1.13.8.3 Account types+1.13.8.4 Account types ......................  hledger recognises five main types of account, corresponding to the@@ -1329,7 +1397,7 @@  File: hledger_journal.info,  Node: Account display order,  Prev: Account types,  Up: Declaring accounts -1.13.8.4 Account display order+1.13.8.5 Account display order ..............................  Account directives also set the order in which accounts are displayed,@@ -1723,8 +1791,7 @@ checking) every month.  Goals and actual performance can then be compared in budget reports. -   For more details, see: balance: Budget report and Budgeting and-Forecasting.+   See also: Budgeting and Forecasting.   File: hledger_journal.info,  Node: Auto postings,  Prev: Periodic transactions,  Up: Transactions@@ -1865,124 +1932,130 @@  Tag Table: Node: Top76-Node: Transactions1875-Ref: #transactions1967-Node: Dates3251-Ref: #dates3350-Node: Simple dates3415-Ref: #simple-dates3541-Node: Secondary dates4050-Ref: #secondary-dates4204-Node: Posting dates5540-Ref: #posting-dates5669-Node: Status7041-Ref: #status7162-Node: Description8870-Ref: #description9004-Node: Payee and note9324-Ref: #payee-and-note9438-Node: Comments9773-Ref: #comments9899-Node: Tags11093-Ref: #tags11208-Node: Postings12601-Ref: #postings12729-Node: Virtual postings13755-Ref: #virtual-postings13872-Node: Account names15177-Ref: #account-names15318-Node: Amounts15805-Ref: #amounts15944-Node: Digit group marks17052-Ref: #digit-group-marks17200-Node: Amount display style18138-Ref: #amount-display-style18292-Node: Transaction prices19729-Ref: #transaction-prices19901-Node: Lot prices and lot dates22332-Ref: #lot-prices-and-lot-dates22529-Node: Balance assertions23017-Ref: #balance-assertions23203-Node: Assertions and ordering24236-Ref: #assertions-and-ordering24424-Node: Assertions and included files25124-Ref: #assertions-and-included-files25367-Node: Assertions and multiple -f options25700-Ref: #assertions-and-multiple--f-options25956-Node: Assertions and commodities26088-Ref: #assertions-and-commodities26320-Node: Assertions and prices27477-Ref: #assertions-and-prices27691-Node: Assertions and subaccounts28131-Ref: #assertions-and-subaccounts28360-Node: Assertions and virtual postings28684-Ref: #assertions-and-virtual-postings28926-Node: Assertions and precision29068-Ref: #assertions-and-precision29261-Node: Balance assignments29528-Ref: #balance-assignments29702-Node: Balance assignments and prices30866-Ref: #balance-assignments-and-prices31038-Node: Directives31262-Ref: #directives31421-Node: Directives and multiple files37112-Ref: #directives-and-multiple-files37295-Node: Comment blocks37959-Ref: #comment-blocks38142-Node: Including other files38318-Ref: #including-other-files38498-Node: Default year39422-Ref: #default-year39591-Node: Declaring commodities39998-Ref: #declaring-commodities40181-Node: Default commodity41987-Ref: #default-commodity42173-Node: Declaring market prices43062-Ref: #declaring-market-prices43257-Node: Declaring accounts44114-Ref: #declaring-accounts44300-Node: Account comments45225-Ref: #account-comments45388-Node: Account subdirectives45812-Ref: #account-subdirectives46007-Node: Account types46320-Ref: #account-types46504-Node: Account display order49550-Ref: #account-display-order49720-Node: Rewriting accounts50871-Ref: #rewriting-accounts51056-Node: Basic aliases51813-Ref: #basic-aliases51959-Node: Regex aliases52663-Ref: #regex-aliases52835-Node: Combining aliases53554-Ref: #combining-aliases53747-Node: Aliases and multiple files55023-Ref: #aliases-and-multiple-files55232-Node: end aliases55811-Ref: #end-aliases55968-Node: Default parent account56069-Ref: #default-parent-account56237-Node: Periodic transactions57121-Ref: #periodic-transactions57296-Node: Periodic rule syntax59168-Ref: #periodic-rule-syntax59374-Node: Two spaces between period expression and description!60078-Ref: #two-spaces-between-period-expression-and-description60397-Node: Forecasting with periodic transactions61081-Ref: #forecasting-with-periodic-transactions61386-Node: Budgeting with periodic transactions63441-Ref: #budgeting-with-periodic-transactions63680-Node: Auto postings64129-Ref: #auto-postings64269-Node: Auto postings and multiple files66448-Ref: #auto-postings-and-multiple-files66652-Node: Auto postings and dates66861-Ref: #auto-postings-and-dates67135-Node: Auto postings and transaction balancing / inferred amounts / balance assertions67310-Ref: #auto-postings-and-transaction-balancing-inferred-amounts-balance-assertions67661-Node: Auto posting tags68003-Ref: #auto-posting-tags68218+Node: Transactions1869+Ref: #transactions1961+Node: Dates3245+Ref: #dates3344+Node: Simple dates3409+Ref: #simple-dates3535+Node: Secondary dates4044+Ref: #secondary-dates4198+Node: Posting dates5534+Ref: #posting-dates5663+Node: Status7035+Ref: #status7156+Node: Description8864+Ref: #description8998+Node: Payee and note9318+Ref: #payee-and-note9432+Node: Comments9767+Ref: #comments9893+Node: Tags11087+Ref: #tags11202+Node: Postings12595+Ref: #postings12723+Node: Virtual postings13749+Ref: #virtual-postings13866+Node: Account names15171+Ref: #account-names15312+Node: Amounts15799+Ref: #amounts15938+Node: Digit group marks17062+Ref: #digit-group-marks17213+Node: Commodity display style18151+Ref: #commodity-display-style18331+Node: Rounding19874+Ref: #rounding19998+Node: Transaction prices20410+Ref: #transaction-prices20582+Node: Lot prices and lot dates23013+Ref: #lot-prices-and-lot-dates23210+Node: Balance assertions23698+Ref: #balance-assertions23884+Node: Assertions and ordering24917+Ref: #assertions-and-ordering25105+Node: Assertions and included files25805+Ref: #assertions-and-included-files26048+Node: Assertions and multiple -f options26381+Ref: #assertions-and-multiple--f-options26637+Node: Assertions and commodities26769+Ref: #assertions-and-commodities27001+Node: Assertions and prices28158+Ref: #assertions-and-prices28372+Node: Assertions and subaccounts28812+Ref: #assertions-and-subaccounts29041+Node: Assertions and virtual postings29365+Ref: #assertions-and-virtual-postings29607+Node: Assertions and precision29749+Ref: #assertions-and-precision29942+Node: Balance assignments30209+Ref: #balance-assignments30383+Node: Balance assignments and prices31547+Ref: #balance-assignments-and-prices31719+Node: Directives31943+Ref: #directives32102+Node: Directives and multiple files37600+Ref: #directives-and-multiple-files37783+Node: Comment blocks38447+Ref: #comment-blocks38630+Node: Including other files38806+Ref: #including-other-files38986+Node: Default year39910+Ref: #default-year40079+Node: Declaring commodities40486+Ref: #declaring-commodities40669+Node: Commodity error checking42513+Ref: #commodity-error-checking42673+Node: Default commodity42930+Ref: #default-commodity43116+Node: Declaring market prices44005+Ref: #declaring-market-prices44200+Node: Declaring accounts45057+Ref: #declaring-accounts45243+Node: Account error checking46445+Ref: #account-error-checking46621+Node: Account comments47800+Ref: #account-comments47994+Node: Account subdirectives48418+Ref: #account-subdirectives48613+Node: Account types48926+Ref: #account-types49110+Node: Account display order52156+Ref: #account-display-order52326+Node: Rewriting accounts53477+Ref: #rewriting-accounts53662+Node: Basic aliases54419+Ref: #basic-aliases54565+Node: Regex aliases55269+Ref: #regex-aliases55441+Node: Combining aliases56160+Ref: #combining-aliases56353+Node: Aliases and multiple files57629+Ref: #aliases-and-multiple-files57838+Node: end aliases58417+Ref: #end-aliases58574+Node: Default parent account58675+Ref: #default-parent-account58843+Node: Periodic transactions59727+Ref: #periodic-transactions59902+Node: Periodic rule syntax61774+Ref: #periodic-rule-syntax61980+Node: Two spaces between period expression and description!62684+Ref: #two-spaces-between-period-expression-and-description63003+Node: Forecasting with periodic transactions63687+Ref: #forecasting-with-periodic-transactions63992+Node: Budgeting with periodic transactions66047+Ref: #budgeting-with-periodic-transactions66286+Node: Auto postings66695+Ref: #auto-postings66835+Node: Auto postings and multiple files69014+Ref: #auto-postings-and-multiple-files69218+Node: Auto postings and dates69427+Ref: #auto-postings-and-dates69701+Node: Auto postings and transaction balancing / inferred amounts / balance assertions69876+Ref: #auto-postings-and-transaction-balancing-inferred-amounts-balance-assertions70227+Node: Auto posting tags70569+Ref: #auto-posting-tags70784  End Tag Table 

@@ -411,42 +411,60 @@               commodity INR 9,99,99,999.00               commodity       1 000 000.9455 -   Amount display style-       For  each  commodity,  hledger  chooses a consistent format to use when-       displaying amounts.  (Except price amounts, which are always  displayed-       as written).  The display style is chosen as follows:+   Commodity display style+       For each commodity, hledger chooses a consistent style to use when dis-+       playing amounts.  (Except price amounts, which are always displayed  as+       written).  The display style is chosen as follows:         o If  there  is  a commodity directive (or default commodity directive)-         for the commodity, that format is used (see examples above).+         for the commodity, its style is used (see examples above). -       o Otherwise the format of the first posting amount  in  that  commodity-         seen in the journal is used.  But the number of decimal places ("pre--         cision") will be the maximum from all posting amounts in that commod--         ity.+       o Otherwise the style is inferred from the amounts  in  that  commodity+         seen in the journal. -       o Or  if  there are no such amounts in the journal, a default format is+       o Or  if  there  are no such amounts in the journal, a default style is          used (like $1000.00). -       Transaction prices don't affect the amount display style directly,  but-       occasionally  they can do so indirectly (eg when an posting's amount is-       inferred using a transaction price).  If you find  this  causing  prob--       lems, use a commodity directive to fix the display style.+       A style is inferred from the journal amounts in a commodity as follows: -       In summary: amounts will be displayed much as they appear in your jour--       nal, with the max observed number of decimal places.  If  you  want  to-       see fewer decimal places in reports, use a commodity directive to over--       ride that.+       o Use the general style (decimal mark, symbol placement) of  the  first+         amount -       hledger uses banker's rounding: it rounds to the nearest  even  number,-       eg  0.5  displayed  with  zero decimal places is "0").  (Note, prior to-       hledger 1.17.1 this could vary if hledger happened to be built with  an-       old version of Decimal (<0.5.1); since 1.17.1 it's guaranteed.)+       o Use  the  first-seen digit group style (digit group mark, digit group+         sizes), if any +       o Use the maximum number of decimal places of all.++       Transaction price amounts don't affect the commodity display style  di-+       rectly, but occasionally they can do so indirectly (eg when a posting's+       amount is inferred using a transaction price).  If you find this  caus-+       ing problems, use a commodity directive to fix the display style.++       In summary, each commodity's amounts will be normalised to++       o the style declared by a commodity directive++       o or,  the  style  of the first posting amount in the journal, with the+         first-seen digit group style and the maximum-seen number  of  decimal+         places.++       If  reports  are  showing amounts in a way you don't like (eg, with too+       many decimal places), use a commodity directive to set  your  preferred+       style.++   Rounding+       Amounts are stored internally as decimal numbers with up to 255 decimal+       places, and displayed with the number of decimal  places  specified  by+       the  commodity display style.  Note, hledger uses banker's rounding: it+       rounds to the nearest even number, eg 0.5 displayed with  zero  decimal+       places  is  "0").   (Guaranteed since hledger 1.17.1; in older versions+       this could vary if hledger was built with Decimal < 0.5.1.)+    Transaction prices        Within a transaction, you can note an amount's price in another commod--       ity.  This can be used to document the cost (in a purchase) or  selling-       price  (in  a  sale).   For  example,  transaction prices are useful to-       record purchases of a foreign currency.  Note  transaction  prices  are+       ity.   This can be used to document the cost (in a purchase) or selling+       price (in a sale).  For  example,  transaction  prices  are  useful  to+       record  purchases  of  a foreign currency.  Note transaction prices are        fixed at the time of the transaction, and do not change over time.  See        also market prices, which represent prevailing exchange rates on a cer-        tain date.@@ -472,14 +490,14 @@                     assets:euros     EUR100          ; one hundred euros purchased                     assets:dollars  $-135          ; for $135 -       4. Like 1, but the @ is parenthesised, i.e.  (@); this is for  compati--          bility  with Ledger journals (Virtual posting costs), and is equiva-+       4. Like  1, but the @ is parenthesised, i.e.  (@); this is for compati-+          bility with Ledger journals (Virtual posting costs), and is  equiva-           lent to 1 in hledger.         5. Like 2, but as in 4 the @@ is parenthesised, i.e.  (@@); in hledger,           this is equivalent to 2. -       Use  the -B/--cost flag to convert amounts to their transaction price's+       Use the -B/--cost flag to convert amounts to their transaction  price's        commodity, if any.  (mnemonic: "B" is from "cost Basis", as in Ledger).        Eg here is how -B affects the balance report for the example above: @@ -490,8 +508,8 @@                              $-135  assets:dollars                               $135  assets:euros    # <- the euros' cost -       Note  -B is sensitive to the order of postings when a transaction price-       is inferred: the inferred price will be in the commodity  of  the  last+       Note -B is sensitive to the order of postings when a transaction  price+       is  inferred:  the  inferred price will be in the commodity of the last        amount.  So if example 3's postings are reversed, while the transaction        is equivalent, -B shows something different: @@ -504,18 +522,18 @@                               EUR100  assets:euros     Lot prices and lot dates-       Ledger allows another kind of price, lot price (four  variants:  {UNIT-+       Ledger  allows  another kind of price, lot price (four variants: {UNIT-        PRICE},   {{TOTALPRICE}},   {=FIXEDUNITPRICE},   {{=FIXEDTOTALPRICE}}),        and/or a lot date ([DATE]) to be specified.  These are normally used to-       select  a  lot when selling investments.  hledger will parse these, for-       compatibility with Ledger journals,  but  currently  ignores  them.   A-       transaction  price,  lot price and/or lot date may appear in any order,+       select a lot when selling investments.  hledger will parse  these,  for+       compatibility  with  Ledger  journals,  but  currently ignores them.  A+       transaction price, lot price and/or lot date may appear in  any  order,        after the posting amount and before the balance assertion if any.     Balance assertions-       hledger supports Ledger-style  balance  assertions  in  journal  files.-       These  look  like, for example, = EXPECTEDBALANCE following a posting's-       amount.  Eg here we assert the expected dollar balance  in  accounts  a+       hledger  supports  Ledger-style  balance  assertions  in journal files.+       These look like, for example, = EXPECTEDBALANCE following  a  posting's+       amount.   Eg  here  we assert the expected dollar balance in accounts a        and b after each posting:                2013/1/1@@ -527,32 +545,32 @@                 b  $-1  =$-2         After reading a journal file, hledger will check all balance assertions-       and report an error if any of them fail.  Balance assertions  can  pro--       tect  you  from, eg, inadvertently disrupting reconciled balances while-       cleaning up old entries.  You can disable  them  temporarily  with  the+       and  report  an error if any of them fail.  Balance assertions can pro-+       tect you from, eg, inadvertently disrupting reconciled  balances  while+       cleaning  up  old  entries.   You can disable them temporarily with the        -I/--ignore-assertions flag, which can be useful for troubleshooting or-       for reading Ledger files.  (Note: this flag currently does not  disable+       for  reading Ledger files.  (Note: this flag currently does not disable        balance assignments, below).     Assertions and ordering-       hledger  sorts  an  account's postings and assertions first by date and-       then (for postings on the same day) by parse order.  Note this is  dif-+       hledger sorts an account's postings and assertions first  by  date  and+       then  (for postings on the same day) by parse order.  Note this is dif-        ferent from Ledger, which sorts assertions only by parse order.  (Also,-       Ledger assertions do not see the accumulated effect of  repeated  post-+       Ledger  assertions  do not see the accumulated effect of repeated post-        ings to the same account within a transaction.)         So, hledger balance assertions keep working if you reorder differently--       dated transactions within the journal.  But if you  reorder  same-dated-       transactions  or postings, assertions might break and require updating.+       dated  transactions  within the journal.  But if you reorder same-dated+       transactions or postings, assertions might break and require  updating.        This order dependence does bring an advantage: precise control over the        order of postings and assertions within a day, so you can assert intra-        day balances.     Assertions and included files-       With included files, things are a little more  complicated.   Including-       preserves  the ordering of postings and assertions.  If you have multi--       ple postings to an account on the  same  day,  split  across  different-       files,  and  you  also want to assert the account's balance on the same+       With  included  files, things are a little more complicated.  Including+       preserves the ordering of postings and assertions.  If you have  multi-+       ple  postings  to  an  account  on the same day, split across different+       files, and you also want to assert the account's balance  on  the  same        day, you'll have to put the assertion in the right file.     Assertions and multiple -f options@@ -560,15 +578,15 @@        -f options.  Use include or concatenate the files instead.     Assertions and commodities-       The  asserted  balance must be a simple single-commodity amount, and in-       fact the assertion checks only  this  commodity's  balance  within  the-       (possibly  multi-commodity)  account  balance.   This is how assertions+       The asserted balance must be a simple single-commodity amount,  and  in+       fact  the  assertion  checks  only  this commodity's balance within the+       (possibly multi-commodity) account balance.   This  is  how  assertions        work in Ledger also.  We could call this a "partial" balance assertion.         To assert the balance of more than one commodity in an account, you can        write multiple postings, each asserting one commodity's balance. -       You  can  make a stronger "total" balance assertion by writing a double+       You can make a stronger "total" balance assertion by writing  a  double        equals sign (== EXPECTEDBALANCE).  This asserts that there are no other        unasserted commodities in the account (or, that their balance is 0). @@ -588,7 +606,7 @@                 a    0 ==  $1         It's not yet possible to make a complete assertion about a balance that-       has multiple commodities.  One workaround is to isolate each  commodity+       has  multiple commodities.  One workaround is to isolate each commodity        into its own subaccount:                2013/1/1@@ -602,21 +620,21 @@                 a:euro   0 ==  1EUR     Assertions and prices-       Balance  assertions  ignore  transaction prices, and should normally be+       Balance assertions ignore transaction prices, and  should  normally  be        written without one:                2019/1/1                 (a)     $1 @ EUR1 = $1 -       We do allow prices to be written there, however, and print shows  them,-       even  though  they  don't affect whether the assertion passes or fails.-       This is for backward compatibility (hledger's  close  command  used  to-       generate  balance  assertions with prices), and because balance assign-+       We  do allow prices to be written there, however, and print shows them,+       even though they don't affect whether the assertion  passes  or  fails.+       This  is  for  backward  compatibility (hledger's close command used to+       generate balance assertions with prices), and because  balance  assign-        ments do use them (see below).     Assertions and subaccounts-       The balance assertions above (= and ==) do not count the  balance  from-       subaccounts;  they check the account's exclusive balance only.  You can+       The  balance  assertions above (= and ==) do not count the balance from+       subaccounts; they check the account's exclusive balance only.  You  can        assert the balance including subaccounts by writing =* or ==*, eg:                2019/1/1@@ -630,16 +648,16 @@        tual.  They are not affected by the --real/-R flag or real: query.     Assertions and precision-       Balance  assertions  compare  the exactly calculated amounts, which are-       not always what is shown by reports.   Eg  a  commodity  directive  may-       limit  the  display  precision, but this will not affect balance asser-+       Balance assertions compare the exactly calculated  amounts,  which  are+       not  always  what  is  shown  by reports.  Eg a commodity directive may+       limit the display precision, but this will not  affect  balance  asser-        tions.  Balance assertion failure messages show exact amounts.     Balance assignments-       Ledger-style balance assignments are also supported.   These  are  like-       balance  assertions, but with no posting amount on the left side of the-       equals sign; instead it is calculated automatically so  as  to  satisfy-       the  assertion.   This  can be a convenience during data entry, eg when+       Ledger-style  balance  assignments  are also supported.  These are like+       balance assertions, but with no posting amount on the left side of  the+       equals  sign;  instead  it is calculated automatically so as to satisfy+       the assertion.  This can be a convenience during data  entry,  eg  when        setting opening balances:                ; starting a new journal, set asset account balances@@ -657,14 +675,14 @@                 expenses:misc         The calculated amount depends on the account's balance in the commodity-       at  that  point  (which depends on the previously-dated postings of the-       commodity to that account since the last balance assertion  or  assign-+       at that point (which depends on the previously-dated  postings  of  the+       commodity  to  that account since the last balance assertion or assign-        ment).  Note that using balance assignments makes your journal a little        less explicit; to know the exact amount posted, you have to run hledger        or do the calculations yourself, instead of just reading it.     Balance assignments and prices-       A  transaction  price in a balance assignment will cause the calculated+       A transaction price in a balance assignment will cause  the  calculated        amount to have that price attached:                2019/1/1@@ -675,14 +693,15 @@                   (a)         $1 @ EUR2 = $1 @ EUR2     Directives-       A directive is a line in the journal beginning with a special  keyword,+       A  directive is a line in the journal beginning with a special keyword,        that influences how the journal is processed.  hledger's directives are        based on a subset of Ledger's, but there are many differences (and also        some differences between hledger versions).         Directives' behaviour and interactions can get a little bit complex, so-       here is a table summarising the  directives  and  their  effects,  with-       links to more detailed docs.+       here  is  a  table  summarising  the directives and their effects, with+       links to more detailed docs.  Note part of this table  is  hidden  when+       viewed in a web browser - scroll it sideways to see more.         direc-     end   di-   subdi-    purpose                        can  affect  (as of        tive       rective     rec-                                     2018/06)@@ -691,54 +710,48 @@        account                any       document  account names, de-   all  entries in all                               text      clare account types  &  dis-   files,  before   or                                         play order                     after-       alias      end                   rewrite account names          following       in--                  aliases                                              line/included   en--                                                                       tries until end  of-                                                                       current file or end+       alias      end                   rewrite account names          following   entries+                  aliases                                              until  end  of cur-+                                                                       rent  file  or  end                                                                        directive-       apply      end apply             prepend a common  parent  to   following       in--       account    account               account names                  line/included   en--                                                                       tries  until end of-                                                                       current file or end+       apply      end apply             prepend  a  common parent to   following   entries+       account    account               account names                  until  end  of cur-+                                                                       rent  file  or  end                                                                        directive-       comment    end  com-             ignore part of journal         following       in--                  ment                                                 line/included   en--                                                                       tries until end  of-                                                                       current file or end+       comment    end  com-             ignore part of journal         following   entries+                  ment                                                 until  end  of cur-+                                                                       rent  file  or  end                                                                        directive-       commod-                format    declare  a commodity and its   number    notation:+       commod-                format    declare a commodity and  its   number    notation:        ity                              number  notation  &  display   following   entries                                         style                          in  that  commodity-                                                                       in all files;  dis-+                                                                       in all files ; dis-                                                                        play style: amounts                                                                        of  that  commodity                                                                        in reports-       D                                declare  a  commodity  to be   default  commodity:+++       D                                declare a  commodity  to  be   default  commodity:                                         used    for    commodityless   following   commod--                                        amounts,  and its number no-   ityless entries un--                                        tation & display style         til  end of current-                                                                       file; number  nota-+                                        amounts, and its number  no-   ityless entries un-+                                        tation & display style         til end of  current+                                                                       file;  number nota-                                                                        tion: following en--                                                                       tries in that  com-+                                                                       tries  in that com-                                                                        modity until end of-                                                                       current file;  dis-+                                                                       current  file; dis-                                                                        play style: amounts                                                                        of  that  commodity                                                                        in reports        include                          include   entries/directives   what  the  included                                         from another file              directives affect        P                                declare a market price for a   amounts   of   that-                                        commodity                      commodity   in  re--                                                                       ports, when  -V  is+                                        commodity                      commodity  in   re-+                                                                       ports,  when  -V is                                                                        used-----       Y                                declare  a year for yearless   following       in--                                        dates                          line/included   en--                                                                       tries until end  of-                                                                       current file+       Y                                declare a year for  yearless   following   entries+                                        dates                          until end  of  cur-+                                                                       rent file        =                                declare   an   auto  posting   all entries in par-                                         rule,  adding  postings   to   ent/current/child                                         other transactions             files (but not sib-@@ -842,7 +855,7 @@           formats in your data.  (Without this, hledger will parse both  1,000           and 1.000 as 1). -       3. It  declares the amount display style to use in output - decimal and+       3. It  declares  a  commodity's  display  style in output - decimal and           digit group marks, number of decimal places, symbol placement etc.         You are likely to run into one of the problems solved by commodity  di-@@ -877,18 +890,24 @@        comma, followed by 0 or more decimal digits.         Note  hledger  normally  uses  banker's rounding, so 0.5 displayed with-       zero decimal digits is "0".  (More at Amount display style.)+       zero decimal digits is "0".  (More at Commodity display style.) +   Commodity error checking+       In strict mode, enabled with the -s/--strict flag, hledger will  report+       an  error if a commodity symbol is used that has not been declared by a+       commodity directive.  This works similarly to account  error  checking,+       see the notes there for more details.+    Default commodity-       The D directive sets a default commodity, to be used for amounts  with-+       The  D directive sets a default commodity, to be used for amounts with-        out a commodity symbol (ie, plain numbers).  This commodity will be ap-        plied to all subsequent commodity-less amounts, or until the next D di-        rective.  (Note, this is different from Ledger's D.) -       For  compatibility/historical reasons, D also acts like a commodity di-+       For compatibility/historical reasons, D also acts like a commodity  di-        rective, setting the commodity's display style (for output) and decimal        mark (for parsing input).  As with commodity, the amount must always be-       written with a decimal mark (period or comma).  If both directives  are+       written  with a decimal mark (period or comma).  If both directives are        used, commodity's style takes precedence.         The syntax is D AMOUNT.  Eg:@@ -902,9 +921,9 @@                 b     Declaring market prices-       The  P directive declares a market price, which is an exchange rate be--       tween two commodities on a certain date.  (In Ledger, they  are  called-       "historical  prices".)  These are often obtained from a stock exchange,+       The P directive declares a market price, which is an exchange rate  be-+       tween  two  commodities on a certain date.  (In Ledger, they are called+       "historical prices".) These are often obtained from a  stock  exchange,        cryptocurrency exchange, or the foreign exchange market.         Here is the format:@@ -915,47 +934,78 @@         o COMMODITYA is the symbol of the commodity being priced -       o COMMODITYBAMOUNT is an amount (symbol and quantity) in a second  com-+       o COMMODITYBAMOUNT  is an amount (symbol and quantity) in a second com-          modity, giving the price in commodity B of one unit of commodity A. -       These  two  market price directives say that one euro was worth 1.35 US+       These two market price directives say that one euro was worth  1.35  US        dollars during 2009, and $1.40 from 2010 onward:                P 2009/1/1 EUR $1.35               P 2010/1/1 EUR $1.40 -       The -V, -X and --value flags use these market  prices  to  show  amount+       The  -V,  -X  and  --value flags use these market prices to show amount        values in another commodity.  See Valuation.     Declaring accounts-       account directives can be used to pre-declare accounts.  Though not re--       quired, they can provide several benefits:+       account directives can be used to declare accounts (ie, the places that+       amounts  are transferred from and to).  Though not required, these dec-+       larations can provide several benefits:         o They can document your intended chart of accounts, providing a refer-          ence. -       o They  can  store  extra  information about accounts (account numbers,-         notes, etc.)--       o They can help hledger know your accounts'  types  (asset,  liability,-         equity,  revenue,  expense), useful for reports like balancesheet and+       o They  can  help  hledger know your accounts' types (asset, liability,+         equity, revenue, expense), useful for reports like  balancesheet  and          incomestatement. -       o They control account display order in  reports,  allowing  non-alpha-+       o They  control  account  display order in reports, allowing non-alpha-          betic sorting (eg Revenues to appear above Expenses). +       o They can store extra information  about  accounts  (account  numbers,+         notes, etc.)+        o They  help  with account name completion in the add command, hledger-          iadd, hledger-web, ledger-mode etc. -       The simplest form is just the word account followed by a  hledger-style-       account name, eg:+       o In strict mode, they restrict which accounts  may  be  posted  to  by+         transactions, which helps detect typos. +       The  simplest form is just the word account followed by a hledger-style+       account name, eg this account directive declares the assets:bank:check-+       ing account:+               account assets:bank:checking +   Account error checking+       By  default, accounts come into existence when a transaction references+       them by name.  This is convenient, but it means hledger can't warn  you+       when you mis-spell an account name in the journal.  Usually you'll find+       the error later, as an extra account in balance reports, or  an  incor-+       rect balance when reconciling.++       In  strict mode, enabled with the -s/--strict flag, hledger will report+       an error if any transaction uses an account name that has not been  de-+       clared by an account directive.  Some notes:++       o The  declaration is case-sensitive; transactions must use the correct+         account name capitalisation.++       o The account directive's scope is "whole file and below"  (see  direc-+         tives).  This means it affects all of the current file, and any files+         it includes, but not parent or sibling files.  The  position  of  ac-+         count  directives  within the file does not matter, though it's usual+         to put them at the top.++       o Accounts can only be declared in journal files (but will  affect  in-+         cluded files in other formats).++       o It's  currently  not  possible  to declare "all possible subaccounts"+         with a wildcard; every account posted to must be declared.+    Account comments        Comments, beginning with a semicolon, can be added: -       o on  the  same line, after two or more spaces (because ; is allowed in+       o on the same line, after two or more spaces (because ; is  allowed  in          account names)         o on the next lines, indented@@ -969,7 +1019,7 @@        Same-line comments are not supported by Ledger, or hledger <1.13.     Account subdirectives-       We also allow (and ignore) Ledger-style  indented  subdirectives,  just+       We  also  allow  (and ignore) Ledger-style indented subdirectives, just        for compatibility.:                account assets:bank:checking@@ -988,21 +1038,21 @@        Asset, Liability, Equity, Revenue, Expense.         These account types are important for controlling which accounts appear-       in  the  balancesheet, balancesheetequity, incomestatement reports (and+       in the balancesheet, balancesheetequity, incomestatement  reports  (and        probably for other things in future). -       Additionally, we recognise the Cash type, which is also an  Asset,  and-       which  causes  accounts to appear in the cashflow report.  ("Cash" here-       means liquid assets, eg bank balances but typically not investments  or+       Additionally,  we  recognise the Cash type, which is also an Asset, and+       which causes accounts to appear in the cashflow report.   ("Cash"  here+       means  liquid assets, eg bank balances but typically not investments or        receivables.)     Declaring account types        Generally, to make these reports work you should declare your top-level        accounts and their types, using account directives with type: tags. -       The tag's value should be one of: Asset,  Liability,  Equity,  Revenue,-       Expense,  Cash,  A,  L, E, R, X, C (all case insensitive).  The type is-       inherited by all subaccounts except where they override it.   Here's  a+       The  tag's  value  should be one of: Asset, Liability, Equity, Revenue,+       Expense, Cash, A, L, E, R, X, C (all case insensitive).   The  type  is+       inherited  by  all subaccounts except where they override it.  Here's a        complete example:                account assets       ; type: Asset@@ -1014,8 +1064,8 @@               account expenses     ; type: Expense     Auto-detected account types-       If  you  happen  to use common english top-level account names, you may-       not need to declare account types, as they will be  detected  automati-+       If you happen to use common english top-level account  names,  you  may+       not  need  to declare account types, as they will be detected automati-        cally using the following rules:         If  name  matches  regular   account type is:@@ -1025,10 +1075,11 @@        ^(debts?|lia-                Liability        bilit(y|ies))(:|$)        ^equity(:|$)                 Equity+        ^(income|revenue)s?(:|$)     Revenue        ^expenses?(:|$)              Expense -       If  account type is Asset and name does not contain regu-   account  type+       If account type is Asset and name does not contain  regu-   account  type        lar expression:                                             is:        --------------------------------------------------------------------------        (investment|receivable|:A/R|:fixed)                         Cash@@ -1038,9 +1089,9 @@     Interference from auto-detected account types        If you assign any account type, it's a good idea to assign all of them,-       to  prevent any confusion from mixing declared and auto-detected types.-       Although it's unlikely to happen in real life, here's an example:  with-       the  following  journal, balancesheetequity shows "liabilities" in both+       to prevent any confusion from mixing declared and auto-detected  types.+       Although  it's unlikely to happen in real life, here's an example: with+       the following journal, balancesheetequity shows "liabilities"  in  both        Liabilities and Equity sections.  Declaring another account as type:Li-        ability would fix it: @@ -1052,8 +1103,8 @@                 equity       -2     Old account type syntax-       In  some  hledger  journals  you might instead see this old syntax (the-       letters ALERX, separated from the account name by two or more  spaces);+       In some hledger journals you might instead see  this  old  syntax  (the+       letters  ALERX, separated from the account name by two or more spaces);        this is deprecated and may be removed soon:                account assets       A@@ -1063,8 +1114,8 @@               account expenses     X     Account display order-       Account  directives also set the order in which accounts are displayed,-       eg in reports, the hledger-ui  accounts  screen,  and  the  hledger-web+       Account directives also set the order in which accounts are  displayed,+       eg  in  reports,  the  hledger-ui  accounts screen, and the hledger-web        sidebar.  By default accounts are listed in alphabetical order.  But if        you have these account directives in the journal: @@ -1086,20 +1137,20 @@         Undeclared accounts, if any, are displayed last, in alphabetical order. -       Note  that  sorting  is  done at each level of the account tree (within-       each group of sibling accounts under the same parent).  And  currently,+       Note that sorting is done at each level of  the  account  tree  (within+       each  group of sibling accounts under the same parent).  And currently,        this directive:                account other:zoo -       would  influence the position of zoo among other's subaccounts, but not+       would influence the position of zoo among other's subaccounts, but  not        the position of other among the top-level accounts.  This means: -       o you will sometimes declare parent accounts (eg account  other  above)+       o you  will  sometimes declare parent accounts (eg account other above)          that you don't intend to post to, just to customize their display or-          der -       o sibling accounts stay together (you couldn't display x:y  in  between+       o sibling  accounts  stay together (you couldn't display x:y in between          a:b and a:c).     Rewriting accounts@@ -1117,14 +1168,14 @@        o customising reports         Account aliases also rewrite account names in account directives.  They-       do  not  affect account names being entered via hledger add or hledger-+       do not affect account names being entered via hledger add  or  hledger-        web.         See also Rewrite account names.     Basic aliases-       To set an account alias, use the alias directive in your journal  file.-       This  affects all subsequent journal entries in the current file or its+       To  set an account alias, use the alias directive in your journal file.+       This affects all subsequent journal entries in the current file or  its        included files.  The spaces around the = are optional:                alias OLD = NEW@@ -1132,49 +1183,49 @@        Or, you can use the --alias 'OLD=NEW' option on the command line.  This        affects all entries.  It's useful for trying out aliases interactively. -       OLD  and  NEW  are case sensitive full account names.  hledger will re--       place any occurrence of the old account name with the new one.   Subac-+       OLD and NEW are case sensitive full account names.   hledger  will  re-+       place  any occurrence of the old account name with the new one.  Subac-        counts are also affected.  Eg:                alias checking = assets:bank:wells fargo:checking               ; rewrites "checking" to "assets:bank:wells fargo:checking", or "checking:a" to "assets:bank:wells fargo:checking:a"     Regex aliases-       There  is  also a more powerful variant that uses a regular expression,+       There is also a more powerful variant that uses a  regular  expression,        indicated by the forward slashes:                alias /REGEX/ = REPLACEMENT         or --alias '/REGEX/=REPLACEMENT'. -       REGEX is a case-insensitive regular expression.   Anywhere  it  matches-       inside  an  account name, the matched part will be replaced by REPLACE--       MENT.  If REGEX contains parenthesised match groups, these can be  ref-+       REGEX  is  a  case-insensitive regular expression.  Anywhere it matches+       inside an account name, the matched part will be replaced  by  REPLACE-+       MENT.   If REGEX contains parenthesised match groups, these can be ref-        erenced by the usual numeric backreferences in REPLACEMENT.  Eg:                alias /^(.+):bank:([^:]+):(.*)/ = \1:\2 \3               ; rewrites "assets:bank:wells fargo:checking" to  "assets:wells fargo checking" -       Also  note that REPLACEMENT continues to the end of line (or on command-       line, to end of option argument), so it  can  contain  trailing  white-+       Also note that REPLACEMENT continues to the end of line (or on  command+       line,  to  end  of  option argument), so it can contain trailing white-        space.     Combining aliases-       You  can  define  as many aliases as you like, using journal directives+       You can define as many aliases as you like,  using  journal  directives        and/or command line options. -       Recursive aliases - where an account name is rewritten  by  one  alias,-       then  by  another  alias, and so on - are allowed.  Each alias sees the+       Recursive  aliases  -  where an account name is rewritten by one alias,+       then by another alias, and so on - are allowed.  Each  alias  sees  the        effect of previously applied aliases. -       In such cases it can be important to understand which aliases  will  be-       applied  and  in  which order.  For (each account name in) each journal+       In  such  cases it can be important to understand which aliases will be+       applied and in which order.  For (each account name  in)  each  journal        entry, we apply: -       1. alias directives preceding the journal entry, most  recently  parsed+       1. alias  directives  preceding the journal entry, most recently parsed           first (ie, reading upward from the journal entry, bottom to top) -       2. --alias  options,  in  the  order  they appeared on the command line+       2. --alias options, in the order they  appeared  on  the  command  line           (left to right).         In other words, for (an account name in) a given journal entry:@@ -1185,20 +1236,20 @@         o aliases defined after/below the entry do not affect it. -       This gives nearby aliases precedence over distant ones, and helps  pro--       vide  semantic stability - aliases will keep working the same way inde-+       This  gives nearby aliases precedence over distant ones, and helps pro-+       vide semantic stability - aliases will keep working the same way  inde-        pendent of which files are being read and in which order. -       In case of trouble, adding --debug=6 to  the  command  line  will  show+       In  case  of  trouble,  adding  --debug=6 to the command line will show        which aliases are being applied when.     Aliases and multiple files-       As  explained at Directives and multiple files, alias directives do not+       As explained at Directives and multiple files, alias directives do  not        affect parent or sibling files.  Eg in this command,                hledger -f a.aliases -f b.journal -       account aliases defined in a.aliases will not  affect  b.journal.   In-+       account  aliases  defined  in a.aliases will not affect b.journal.  In-        cluding the aliases doesn't work either:                include a.aliases@@ -1220,14 +1271,14 @@               include c.journal  ; also affected     end aliases-       You can clear (forget) all  currently  defined  aliases  with  the  end+       You  can  clear  (forget)  all  currently  defined aliases with the end        aliases directive:                end aliases     Default parent account-       You  can  specify  a  parent account which will be prepended to all ac--       counts within a section of the journal.  Use the apply account and  end+       You can specify a parent account which will be  prepended  to  all  ac-+       counts  within a section of the journal.  Use the apply account and end        apply account directives like so:                apply account home@@ -1244,7 +1295,7 @@                   home:food           $10                   home:cash          $-10 -       If  end  apply  account  is omitted, the effect lasts to the end of the+       If end apply account is omitted, the effect lasts to  the  end  of  the        file.  Included files are also affected, eg:                apply account business@@ -1253,50 +1304,50 @@               apply account personal               include personal.journal -       Prior to hledger 1.0, legacy account and end spellings were  also  sup-+       Prior  to  hledger 1.0, legacy account and end spellings were also sup-        ported. -       A  default parent account also affects account directives.  It does not-       affect account names being entered via hledger add or hledger-web.   If-       account  aliases are present, they are applied after the default parent+       A default parent account also affects account directives.  It does  not+       affect  account names being entered via hledger add or hledger-web.  If+       account aliases are present, they are applied after the default  parent        account.     Periodic transactions-       Periodic transaction rules describe transactions that recur.  They  al--       low  hledger  to  generate  temporary  future transactions to help with-       forecasting, so you don't have to write out each one  in  the  journal,-       and  it's easy to try out different forecasts.  Secondly, they are also+       Periodic  transaction rules describe transactions that recur.  They al-+       low hledger to generate temporary  future  transactions  to  help  with+       forecasting,  so  you  don't have to write out each one in the journal,+       and it's easy to try out different forecasts.  Secondly, they are  also        used to define the budgets shown in budget reports. -       Periodic transactions can be a little tricky, so before you  use  them,+       Periodic  transactions  can be a little tricky, so before you use them,        read this whole section - or at least these tips: -       1. Two  spaces  accidentally  added or omitted will cause you trouble -+       1. Two spaces accidentally added or omitted will cause  you  trouble  -           read about this below. -       2. For troubleshooting, show the generated  transactions  with  hledger-          print   --forecast  tag:generated  or  hledger  register  --forecast+       2. For  troubleshooting,  show  the generated transactions with hledger+          print  --forecast  tag:generated  or  hledger  register   --forecast           tag:generated. -       3. Forecasted transactions will begin only  after  the  last  non-fore-+       3. Forecasted  transactions  will  begin  only after the last non-fore-           casted transaction's date. -       4. Forecasted  transactions  will  end 6 months from today, by default.+       4. Forecasted transactions will end 6 months from  today,  by  default.           See below for the exact start/end rules. -       5. period expressions can be tricky.   Their  documentation  needs  im-+       5. period  expressions  can  be  tricky.  Their documentation needs im-           provement, but is worth studying. -       6. Some  period  expressions  with a repeating interval must begin on a-          natural boundary of that interval.  Eg in  weekly  from  DATE,  DATE-          must  be a monday.  ~ weekly from 2019/10/1 (a tuesday) will give an+       6. Some period expressions with a repeating interval must  begin  on  a+          natural  boundary  of  that  interval.  Eg in weekly from DATE, DATE+          must be a monday.  ~ weekly from 2019/10/1 (a tuesday) will give  an           error.         7. Other period expressions with an interval are automatically expanded-          to  cover a whole number of that interval.  (This is done to improve+          to cover a whole number of that interval.  (This is done to  improve           reports, but it also affects periodic transactions.  Yes, it's a bit-          inconsistent  with  the  above.)  Eg: ~ every 10th day of month from-          2020/01, which is equivalent to ~  every  10th  day  of  month  from+          inconsistent with the above.) Eg: ~ every 10th  day  of  month  from+          2020/01,  which  is  equivalent  to  ~  every 10th day of month from           2020/01/01, will be adjusted to start on 2019/12/10.     Periodic rule syntax@@ -1308,17 +1359,17 @@                   expenses:rent          $2000                   assets:bank:checking -       There  is  an additional constraint on the period expression: the start-       date must fall on a natural boundary of the interval.  Eg monthly  from+       There is an additional constraint on the period expression:  the  start+       date  must fall on a natural boundary of the interval.  Eg monthly from        2018/1/1 is valid, but monthly from 2018/1/15 is not. -       Partial  or  relative dates (M/D, D, tomorrow, last week) in the period-       expression can work (useful or not).  They will be relative to  today's-       date,  unless  a  Y  default year directive is in effect, in which case+       Partial or relative dates (M/D, D, tomorrow, last week) in  the  period+       expression  can work (useful or not).  They will be relative to today's+       date, unless a Y default year directive is in  effect,  in  which  case        they will be relative to Y/1/1.     Two spaces between period expression and description!-       If the period expression is  followed  by  a  transaction  description,+       If  the  period  expression  is  followed by a transaction description,        these must be separated by two or more spaces.  This helps hledger know        where the period expression ends, so that descriptions can not acciden-        tally alter their meaning, as in this example:@@ -1332,64 +1383,63 @@         So, -       o Do  write two spaces between your period expression and your transac-+       o Do write two spaces between your period expression and your  transac-          tion description, if any. -       o Don't accidentally write two spaces in the middle of your period  ex-+       o Don't  accidentally write two spaces in the middle of your period ex-          pression.     Forecasting with periodic transactions-       The  --forecast  flag  activates  any periodic transaction rules in the-       journal.  They will generate temporary  recurring  transactions,  which-       are  not  saved  in  the  journal,  but  will appear in all reports (eg+       The --forecast flag activates any periodic  transaction  rules  in  the+       journal.   They  will  generate temporary recurring transactions, which+       are not saved in the journal,  but  will  appear  in  all  reports  (eg        print).  This can be useful for estimating balances into the future, or-       experimenting  with  different scenarios.  Or, it can be used as a data+       experimenting with different scenarios.  Or, it can be used as  a  data        entry aid: describe recurring transactions, and every so often copy the        output of print --forecast into the journal. -       These  transactions  will  have  an extra tag indicating which periodic+       These transactions will have an extra  tag  indicating  which  periodic        rule generated them: generated-transaction:~ PERIODICEXPR.  And a simi--       lar,  hidden  tag  (beginning  with  an underscore) which, because it's-       never displayed by print, can be used to match  transactions  generated+       lar, hidden tag (beginning with  an  underscore)  which,  because  it's+       never  displayed  by print, can be used to match transactions generated        "just now": _generated-transaction:~ PERIODICEXPR. -       Periodic  transactions  are  generated within some forecast period.  By+       Periodic transactions are generated within some  forecast  period.   By        default, this         o begins on the later of           o the report start date if specified with -b/-p/date: -         o the day after the latest normal (non-periodic) transaction  in  the+         o the  day  after the latest normal (non-periodic) transaction in the            journal, or today if there are no normal transactions. -       o ends  on  the  report  end  date  if specified with -e/-p/date:, or 6+       o ends on the report end date  if  specified  with  -e/-p/date:,  or  6          months (180 days) from today. -       This means that periodic transactions will begin only after the  latest-       recorded  transaction.   And a recorded transaction dated in the future-       can prevent generation of periodic transactions.  (You can  avoid  that+       This  means that periodic transactions will begin only after the latest+       recorded transaction.  And a recorded transaction dated in  the  future+       can  prevent  generation of periodic transactions.  (You can avoid that        by writing the future transaction as a one-time periodic rule instead -        put tilde before the date, eg ~ YYYY-MM-DD ...).         Or, you can set your own arbitrary "forecast period", which can overlap-       recorded  transactions,  and need not be in the future, by providing an-       option argument, like --forecast=PERIODEXPR.  Note the equals  sign  is+       recorded transactions, and need not be in the future, by  providing  an+       option  argument,  like --forecast=PERIODEXPR.  Note the equals sign is        required, a space won't work.  PERIODEXPR is a period expression, which-       can specify the start date, end date, or both, like in a  date:  query.-       (See  also  hledger.1  ->  Report  start  &  end date).  Some examples:+       can  specify  the start date, end date, or both, like in a date: query.+       (See also hledger.1 ->  Report  start  &  end  date).   Some  examples:        --forecast=202001-202004, --forecast=jan-, --forecast=2020.     Budgeting with periodic transactions-       With the --budget flag, currently supported  by  the  balance  command,-       each  periodic transaction rule declares recurring budget goals for the-       specified accounts.  Eg the first example  above  declares  a  goal  of-       spending  $2000  on  rent  (and  also,  a goal of depositing $2000 into-       checking) every month.  Goals and actual performance can then  be  com-+       With  the  --budget  flag,  currently supported by the balance command,+       each periodic transaction rule declares recurring budget goals for  the+       specified  accounts.   Eg  the  first  example above declares a goal of+       spending $2000 on rent (and also,  a  goal  of  depositing  $2000  into+       checking)  every  month.  Goals and actual performance can then be com-        pared in budget reports. -       For  more  details, see: balance: Budget report and Budgeting and Fore--       casting.+       See also: Budgeting and Forecasting.     Auto postings        "Automated postings" or "auto postings" are extra  postings  which  get@@ -1526,4 +1576,4 @@   -hledger 1.18.99                 September 2020              hledger_journal(5)+hledger 1.20                     November 2020              hledger_journal(5)