diff --git a/arbtt.cabal b/arbtt.cabal --- a/arbtt.cabal +++ b/arbtt.cabal @@ -1,5 +1,5 @@ name: arbtt -version: 0.8.1.4 +version: 0.9 license: GPL license-file: LICENSE category: Desktop @@ -22,6 +22,7 @@ you understand the consequences of a full-time logger and be careful with this data. homepage: http://arbtt.nomeata.de/ +bug-reports: https://bitbucket.org/nomeata/arbtt/issues extra-source-files: categorize.cfg, arbtt-capture.desktop, README.md, @@ -48,10 +49,10 @@ TimeLog UpgradeLog1 LeftFold - + ghc-options: -rtsopts - if os(windows) + if os(windows) extra-libraries: psapi cpp-options: -DWIN32 ghc-options: -optl-mwindows @@ -98,7 +99,7 @@ Text.Regex.PCRE.Light.Text TermSize ghc-options: -rtsopts - if os(windows) + if os(windows) cpp-options: -DWIN32 else other-modules: @@ -127,7 +128,7 @@ DumpFormat Data.List.TakeR ghc-options: -rtsopts - if os(windows) + if os(windows) cpp-options: -DWIN32 else other-modules: @@ -151,7 +152,7 @@ CommonStartup TimeLog ghc-options: -rtsopts - if os(windows) + if os(windows) cpp-options: -DWIN32 else other-modules: @@ -174,7 +175,7 @@ CommonStartup TimeLog ghc-options: -rtsopts - if os(windows) + if os(windows) cpp-options: -DWIN32 else other-modules: @@ -195,7 +196,7 @@ , tasty-golden >= 2.2.0.2 && <= 2.3 , tasty-hunit >= 0.2 && < 0.11 , HUnit == 1.2.* - , process-extras == 0.2.* + , process-extras == 0.2.* || == 0.3.* , deepseq , binary >= 0.5 , bytestring diff --git a/doc/arbtt.xml b/doc/arbtt.xml --- a/doc/arbtt.xml +++ b/doc/arbtt.xml @@ -330,16 +330,16 @@
The formal grammar of <filename>categorize.cfg</filename> - - - Rules - - [ ] - ( (, - )* | ( ; - )* ) - - + + + Rules + + [ ] + ( (, + )* | ( ; + )* ) + + AliasSpec @@ -353,9 +353,9 @@ Literal -> Literal - - Rule - { + + Rule + { } @@ -368,11 +368,11 @@ tag - + - - Cond - ( + + Cond + ( ) ! | @@ -381,67 +381,67 @@ || - $active + $active - + - [ + [ ] =~ - + =~ - [ + [ ] - + - + - + current window any window - + - - String - $title - $program - $desktop - format - " string literal " - + + String + $title + $program + $desktop + format + " string literal " + - - ListOfString - " string literal " - " string literal " , - + + ListOfString + " string literal " + " string literal " , + - - Number - $idle - day of week - day of month - month - year - number literal - + + Number + $idle + day of week + day of month + month + year + number literal + - - Date - $date - - + + Date + $date + + - - TimeDiff - $time - $sampleage - + + TimeDiff + $time + $sampleage + ( Digit )* Digit : Digit Digit - + Tag @@ -457,11 +457,11 @@ character. - - ListOfRegex - " " - " " , - + + ListOfRegex + " " + " " , + CmpOp @@ -469,7 +469,7 @@ | > | >= - +
A String refers to a double-quoted string of @@ -507,9 +507,524 @@ expressions. - + + Effective Use of Arbtt + + Now that the syntax has been described and the toolbox laid out, + how do you practically go about using and configuring arbtt? + + + Enabling data collection + + After installing arbtt, you need to configure it to run. There + are many ways you can run the arbtt-capture + daemon. One standard way is to include the command + +arbtt-capture & + + in your desktop environments startup script, e.g. + ~/.xinitrc or similar. + + + Another trick is add it as a cron + job. To do so, edit your crontab file (crontab -e) and + add a line like this: + + +DISPLAY=:0 +@reboot arbtt-capture --logfile=/home/username/doc/arbtt/capture.log + + + At boot, arbtt-capture will be run in the + background and will capture a snapshot of the X metadata for + active windows every 60 seconds (the default). If you want more + fine-grained time data at the expense of doubling storage use, + you could increase the sampling rate with an option like + --sample-rate=30. To be resilient to any errors + or segfaults, you could also wrap it in an infinite loop to restart + the daemon should it ever crash, with a command like + + +DISPLAY=:0 +@reboot while true; do arbtt-capture --sample-rate=30; sleep 1m; done + + + + Checking data availability + + arbtt tracks X properties like window title, class, and running + program, and you write rules to classify those strings as + you wish; but this assumes that the necessary data is present in + those properties. + + + For some programs, this is the case. For example, web browsers + like Firefox typically set the X title to the + HTML <title> element of the web page in the + currently-focused tab, which is enough for classification. + + + Some programs do not set titles or class, and all arbtt sees is + empty strings like ""; or they may set the title/class + to a constant like "Liferea", which may be acceptable if + that program is used for only one purpose, but if it is used for + many purposes, then you cannot write a rule matching it without + producing highly-misleading time analyses. (For example, a web + browser may be used for countless purposes, ranging from work to + research to music to writing to programming; but if the web + browser's title/class were always just "Web browser", + how would you classify 5 hours spent using the web browser? If the + 5 hours are classified as any or all of those purposes, then the + results will be misleading garbage - you probably did not spend 5 + hours just listening to music, but a mixture of those purposes, + which changes from day to day.) + + + You should check for such problematic programs upon starting using + arbtt. It would be unfortunate if you were to log for a few + months, go back for a detailed report for some reason, and + discover that the necessary data was never available for + arbtt to log! + + + These programs can sometimes be customized internally, a bug + report filed with the maintainers, or their titles can be + externally set by + wmctrl + or + xprop. + + + <literal>xprop</literal> + + You can check the X properties of a running window by running + the command + xprop + and clicking on the window; xprop will print + out all the relevant X information. For example, the output for + Emacs might look like this + + +$ xprop | tail -5 +WM_CLASS(STRING) = "emacs", "Emacs" +WM_ICON_NAME(STRING) = "emacs@elan" +_NET_WM_ICON_NAME(UTF8_STRING) = "emacs@elan" +WM_NAME(STRING) = "emacs@elan" +_NET_WM_NAME(UTF8_STRING) = "emacs@elan" + + + This is not very helpful: it does not tell us the filename being + edited, the mode being used, or anything. You could classify + time spent in Emacs as "programming" or + "writing", but this would be imperfect, especially if + you do both activities regularly. However, Emacs can be + customized by editing ~/.emacs, and after + some searching with queries like "setting Emacs window + title", the + Emacs + wiki and + manual + advise us to put something like this Elisp in our + .emacs file: + + +(setq frame-title-format "%f") + + + Now the output looks different: + + +$ xprop | tail -5 +WM_CLASS(STRING) = "emacs", "Emacs" +WM_ICON_NAME(STRING) = "/home/gwern/arbtt.page" +_NET_WM_ICON_NAME(UTF8_STRING) = "/home/gwern/arbtt.page" +WM_NAME(STRING) = "/home/gwern/arbtt.page" +_NET_WM_NAME(UTF8_STRING) = "/home/gwern/arbtt.page" + + + With this, we can usefully classify all such time samples as + being “writing”: + + +current window $title == "/home/gwern/arbtt.page" ==> tag Writing, + + + Another common gap is terminals/shells: they often do not + include information in the title like the current working + directory or last shell command. For example, urxvt/Bash: + + +WM_COMMAND(STRING) = { "urxvt" } +_NET_WM_ICON_NAME(UTF8_STRING) = "urxvt" +WM_ICON_NAME(STRING) = "urxvt" +_NET_WM_NAME(UTF8_STRING) = "urxvt" +WM_NAME(STRING) = "urxvt" + + + Programmers may spend many hours in the shell doing a variety of + things (like Emacs), so this is a problem. Fortunately, this is + also solvable by customizing one's .bashrc to + set the prompt to emit an escape code interpreted by the + terminal (baroque, but it works). The following will include the + working directory, a timestamp, and the last command: + + +trap 'echo -ne "\033]2;$(pwd); $(history 1 | sed "s/^[ ]*[0-9]*[ ]*//g")\007"' DEBUG + + + Now the urxvt samples are useful: + + +_NET_WM_NAME(UTF8_STRING) = "/home/gwern/wiki; 2014-09-03 13:39:32 arbtt-stats --help" + + + Some distributions (e.g. Debian) already provide the relevant + configuration for this to happen. If it does not work for you, you can try to add + . /etc/profile.d/vte.sh + to your ~/.bashrc. + + + A rule could classify based on the directory you are working in, + the command one ran, or both. Other shells like zsh can be fixed + this way too but the exact command may differ; you will need to + research and experiment. + + + Some programs can be tricky to set. The + X image viewer + feh has a --title option but it + cannot be set in the configuration file, + .config/feh/themes, because it needs to be + specified dynamically; so you need to set up a shell alias or + script to wrap the command like + feh --title "$(pwd) / %f / %n". + + + + Raw samples + + xprop can be tedious to use on every running + window and you may forget to check seldomly used programs. A better + approach is to use arbtt-stats’s + --dump-samples option: this option will print + out the collected data for specified time periods, allowing you + to examine the X properties en masse. This option can be used + with the --exclude= + option to print the samples for samples not matched + by existing rules as well, which is indispensable for + improving coverage and suggesting ideas for new rules. A good + way to figure out what customizations to make is to run arbtt as + a daemon for a day or so, and then begin examining the raw + samples for problems. + + + An initial configuration session + + An example: suppose I create a simple category file named + foo with just the line + + +$idle > 30 ==> tag inactive + + + I can then dump all my arbtt samples for the past day with a + command like this: + + +arbtt-stats --categorizefile=foo --m=0 --filter='$sampleage <24:00' --dump-samples + + + Because there are so many open windows, this produces a large + amount (26586 lines) of hard-to-read output: + + +... +( ) Navigator: /r/Touhou's Favorite Arranges! Part 71: Retribution for the Eternal Night ~ Imperishable Night : touhou - Iceweasel +( ) Navigator: Configuring the arbtt categorizer (arbtt-stats) - Iceweasel +( ) evince: ATTACHMENT02 +( ) evince: 2009-geisler.pdf — Heart rate variability predicts self-control in goal pursuit +( ) urxvt: /home/gwern; arbtt-stats --categorizefile=foo --m=0 --filter='$sampleage <24:00' --dump-samples +( ) mnemosyne: Mnemosyne +( ) urxvt: /home/gwern; 2014-09-03 13:11:45 xprop +( ) urxvt: /home/gwern; 2014-09-03 13:42:17 history 1 | cut --delimiter=' ' --fields=5- +( ) urxvt: /home/gwern; 2014-09-03 13:12:21 git log -p .emacs +(*) emacs: emacs@elan +( ) urxvt: /home/gwern; 2014-09-01 14:50:30 while true; do cd ~/ && getmail_fetch --ssl pop.gmail.com gwern0 'ugaozoumbhwcijxb' ./mail/; done +( ) urxvt: /home/gwern/blackmarket-mirrors/silkroad2-forums; 2014-08-31 23:20:10 mv /home/gwern/cookies.txt ./; http_proxy="localhost:8118" wget... +( ) urxvt: /home/gwern/blackmarket-mirrors/agora; 2014-08-31 23:15:50 mv /home/gwern/cookies.txt ./; http_proxy="localhost:8118" wget --mirror ... +( ) urxvt: /home/gwern/blackmarket-mirrors/evolution-forums; 2014-08-31 23:04:10 mv ~/cookies.txt ./; http_proxy="localhost:8118" wget --mirror ... +( ) puddletag: puddletag: /home/gwern/music + + + Active windows are denoted by an asterisk, so I can focus & + simplify by adding a pipe like | fgrep '(*)', + producing more manageable output like + + +(*) urxvt: irssi +(*) urxvt: irssi +(*) urxvt: irssi +(*) Navigator: Pyramid of Technology - NextNature.net - Iceweasel +(*) Navigator: Search results - gwern0@gmail.com - Gmail - Iceweasel +(*) Navigator: [New comment] The Wrong Path - gwern0@gmail.com - Gmail - Iceweasel +(*) Navigator: Iceweasel +(*) Navigator: Litecoin Exchange Rate - $4.83 USD - litecoinexchangerate.org - Iceweasel +(*) Navigator: PredictionBook: LiteCoin will trade at >=10 USD per ltc in 2 years, - Iceweasel +(*) urxvt: irssi +(*) Navigator: Bug#691547 closed by Mikhail Gusarov <dottedmag@dottedmag.net> (Re: s3cmd: Man page: --default-mime-type documentation incomplete...) +(*) Navigator: Bug#691547 closed by Mikhail Gusarov <dottedmag@dottedmag.net> (Re: s3cmd: Man page: --default-mime-type documentation incomplete...) +(*) Navigator: Bug#691547 closed by Mikhail Gusarov <dottedmag@dottedmag.net> (Re: s3cmd: Man page: --default-mime-type documentation incomplete...) +(*) urxvt: /home/gwern; 2014-09-02 14:25:17 man s3cmd +(*) evince: bayesiancausality.pdf +(*) evince: bayesiancausality.pdf +(*) puddletag: puddletag: /home/gwern/music +(*) puddletag: puddletag: /home/gwern/music +(*) evince: bayesiancausality.pdf +(*) Navigator: ▶ Umineko no Naku Koro ni Music Box 4 - オルガン小曲 第2億番 ハ短調 - YouTube - Iceweasel +... + + + This is better. We can see a few things: the windows all now + produce enough information to be usefully classified (Gmail can + be classified under email, irssi can be classified as IRC, the + urxvt usage can clearly be classified as programming, the PDF + being read is statistics, etc) in part because of customizations + to bash/urxvt. The duplication still impedes focus, and we don't + know what's most common. We can use another pipeline to sort, + count duplicates, and sort by number of duplicates + (| sort | uniq --count | sort --general-numeric-sort), + yielding: + + + ... + 14 (*) Navigator: A Bluer Shade of White Chapter 4, a frozen fanfic | FanFiction - Iceweasel + 14 (*) Navigator: Iceweasel + 15 (*) evince: 2009-geisler.pdf — Heart rate variability predicts self-control in goal pursuit + 15 (*) Navigator: Tool use by animals - Wikipedia, the free encyclopedia - Iceweasel + 16 (*) Navigator: Hacker News | Add Comment - Iceweasel + 17 (*) evince: bayesiancausality.pdf + 17 (*) Navigator: Comments - Less Wrong Discussion - Iceweasel + 17 (*) Navigator: Keith Gessen · Why not kill them all?: In Donetsk · LRB 11 September 2014 - Iceweasel + 17 (*) Navigator: Notes on the Celebrity Data Theft | Hacker News - Iceweasel + 18 (*) Navigator: A Bluer Shade of White Chapter 1, a frozen fanfic | FanFiction - Iceweasel + 19 (*) gl: mplayer2 + 19 (*) Navigator: Neural networks and deep learning - Iceweasel + 20 (*) Navigator: Harry Potter and the Philosopher's Zombie, a harry potter fanfic | FanFiction - Iceweasel + 20 (*) Navigator: [OBNYC] Time tracking app - gwern0@gmail.com - Gmail - Iceweasel + 25 (*) evince: ps2007.pdf — untitled + 35 (*) emacs: /home/gwern/arbtt.page + 43 (*) Navigator: CCC comments on The Octopus, the Dolphin and Us: a Great Filter tale - Less Wrong - Iceweasel + 62 (*) evince: The physics of information processing superobjects - Anders Sandberg - 1999.pdf — Brains2 + 69 (*) liferea: Liferea + 82 (*) evince: BMS_raftery.pdf — untitled + 84 (*) emacs: emacs@elan + 87 (*) Navigator: overview for gwern - Iceweasel +109 (*) puddletag: puddletag: /home/gwern/music +150 (*) urxvt: irssi + + + Put this way, we can see what rules we should write to + categorize: we could categorize the activities here into a few + categories of "recreational", "statistics", + "music", "email", "IRC", + "research", and "writing"; and add to the + categorize.cfg some rules like thus: + + +$idle > 30 ==> tag inactive, + +current window $title =~ [/.*Hacker News.*/, /.*Less Wrong.*/, /.*overview for gwern.*/, /.*[fF]an[fF]ic.*/, /.* LRB .*/] + || current window $program == "liferea" ==> tag Recreation, +current window $title =~ [/.*puddletag.*/, /.*mplayer2.*/] ==> tag Music, +current window $title =~ [/.*[bB]ayesian.*/, /.*[nN]eural [nN]etworks.*/, /.*ps2007.pdf.*/, /.*[Rr]aftery.*/] ==> tag Statistics, +current window $title =~ [/.*Wikipedia.*/, /.*Heart rate variability.*/, /.*Anders Sandberg.*/] ==> tag Research, +current window $title =~ [/.*Gmail.*/] ==> tag Email, +current window $title =~ [/.*arbtt.*/] ==> tag Writing, +current window $title == "irssi" ==> tag IRC, + + + If we reran the command, we'd see the same output, so we need to + leverage our new rules and exclude any + samples matching our current tags, so now we run a command like: + + +arbtt-stats --categorizefile=foo --filter='$sampleage <24:00' --dump-samples --exclude=Recreation --exclude=Music --exclude=Statistics + --exclude=Research --exclude=Email --exclude=Writing --exclude=IRC | + fgrep '(*)' | sort | uniq --count | sort --general-numeric-sort + + + Now the previous samples disappear, leaving us with a fresh + batch of unclassified samples to work with: + + + 9 (*) Navigator: New Web Order > Nik Cubrilovic - - Notes on the Celebrity Data Theft - Iceweasel + 9 ( ) urxvt: /home/gwern; arbtt-stats --categorizefile=foo --filter='$sampleage <24:00' --dump-samples | fgrep '(*)' | less + 10 (*) evince: ATTACHMENT02 + 10 (*) Navigator: These Giant Copper Orbs Show Just How Much Metal Comes From a Mine | Design | WIRED - Iceweasel + 12 (*) evince: [Jon_Elster]_Alchemies_of_the_Mind_Rationality_an(BookFi.org).pdf — Alchemies of the mind + 12 (*) Navigator: Morality Quiz/Test your Morals, Values & Ethics - YourMorals.Org - Iceweasel + 33 ( ) urxvt: /home/gwern; arbtt-stats --categorizefile=foo --filter='$sampleage <24:00' --dump-samples | fgrep '(*)'... + + + We can add rules categorizing these as 'Recreational', + 'Writing', 'Research', 'Recreational', 'Research', 'Writing', + and 'Writing' respectively; and we might decide at this point + that 'Writing' is starting to become overloaded, so we'll split + it into two tags, 'Writing' and 'Programming'. And then after + tossing another --exclude=Programming into + our rules, we can repeat the process. + + + As we refine our rules, we will quickly spot instances where the + title/class/program are insufficient to allow accurate + classification, and we will figure out the best collection of + tags for our particular purposes. A few iterations is enough for + most purposes. + + + + + + Categorizing advice + + When building up rules, a few rules of thumb should be kept in + mind: + + + + Categorize by purpose, not by program + + + This leads to misleading time reports. Avoid, for example, + lumping all web browser time into a single category named + 'Internet'; this is more misleading than helpful. Good + categories describe an activity or goal, such as 'Work' or + 'Recreation', not a tool, like 'Emacs' or 'Vim'. + + + + + When in doubt, write narrow rules and generalize later + + + Regexps are tricky and it can be easy to write rules far + broader than one intended. The --exclude + filters mean that one will never see samples which are matched + accidentally. If one is in doubt, it can be helpful to take a + specific sample one wants to match and several similar strings + and look at how well one's regexp rule works in Emacs's + regexp-builder + or online regexp-testers like + regexpal. + + + + + Don't try to classify everything + + + You will never classify 100% of samples because sometimes + programs do not include useful X properties and cannot be + fixed, you have samples from before you fixed them, or they + are too transient (like popups and dialogues) to be worth + fixing. It is not necessary to classify 100% of your time, + since as long as the most common programs and, say, + 80% + of your time is classified, then you have most of the value. + It is easy to waste more time tweaking arbtt than one gains + from increased accuracy or more finely-grained tags. + + + + + Avoid large and microscopic tags + + + If a tag takes up more than a third or so of your time, it is + probably too large, masks variation, and can be broken down into more + meaningful tags. Conversely, a tag too narrow to show up regularly in + reports (because it is below the default 1% filter) may not be + helpful because it is usually tiny, and can be combined with the most + similar tag to yield more compact and easily interpreted reports. + + + + + Long-term storage + + Each halving of the sampling rate doubles the number of samples + taken and hence the storage requirement; sampling rates below 20s + are probably wasteful. But even the default 60s can accumulate + into a nontrivial amount of data over a year. A + constantly-changing binary file can interact poorly with backup + systems, may make arbtt analyses slower, and if one's system + occasionally crashes or experiences other problems, cause some + corruption of the log and be a nuisance in having to run + arbtt-recover. + + + Thus it may be a good idea to archive one's + capture.log on an annual basis. If one needs to + query the historical data, the particular log file can be + specified as an option like + --logfile=/home/gwern/doc/arbtt/2013-2014.log + + + + External processing of arbtt statistics + + arbtt supports CSV export of time by category in various levels of + granularity in a 'long' format (multiple rows for each day, with + n row specifying a category's value for that + day). These CSV exports can be imported into statistical programs + like R or Excel and manipulated as desired. + + + R users may prefer to have their time data in a 'wide' format + (each row is 1 day, with n columns for each + possible category); this can be done with the + reshape default library. After reading in the + CSV, the time-intervals can be converted to counts and the data to + a wide data-frame with R code like the following: + + +arbtt <- read.csv("arbtt.csv") +interval <- function(x) { if (!is.na(x)) { if (grepl(" s",x)) as.integer(sub(" s","",x)) + else { y <- unlist(strsplit(x, ":")); + as.integer(y[[1]])*3600 + as.integer(y[[2]])*60 + as.integer(y[[3]]); } + } + else NA + } +arbtt$Time <- sapply(as.character(arbtt$Time), interval) +library(reshape) +arbtt <- reshape(arbtt, v.names="Time", timevar="Tag", idvar="Day", direction="wide") + + + + + Contributed tools + + Due to the export facilities of arbtt (as explained in ), tools analyzing arbtt’s data can be + developed independently. This section lists those that we are aware of. If + you create a tool of your own, or find one somewhere, pleas tell us on the + mailing list (see ). + + + arbtt-graph + + Jayesh Kumar Gupta created a nice d3-based visualization of your arbtt + data, including daily pie charts and barcode graphs. + + + You can find his tool on https://github.com/rejuvyesh/arbtt-graph. + + + Program references arbtt consists of a few command line tools, the most important one is @@ -747,9 +1262,9 @@ PERIOD This is not a report of its own, but causes the selected report to be executed for each of the given PERIOD (which can be - day, month or year) where there exist selected samples. All the reports - will then be printed one after another or, in the case of CSV output, - with an additional column. + minute, hour,day, month or year) where there exist selected + samples. All the reports will then be printed one after another or, + in the case of CSV output, with an additional column. Note that if this option is enabled, samples that are filtered out are completely ignored (to avoid empty reports for periods with @@ -913,7 +1428,7 @@ shows the version number, and exists. - + FILE FILE @@ -923,7 +1438,7 @@ FORMAT FORMAT dumping format to use, where FORMAT is one of human (the default), show or JSON. Case in-sensitive. - + NUMBER NUMBER @@ -1045,7 +1560,7 @@ shows the version number, and exists. - + FILE FILE @@ -1111,7 +1626,7 @@ shows the version number, and exists. - + @@ -1194,14 +1709,32 @@ The version history with changes relevant for the user is documented here. + + Version 0.9 + + + + The option of + arbtt-stats now supports grouping results by + minute or hour. + + + + + Gwern Branwen contributed the . + + + + + Version 0.8.1 - - The syntax now allows for time differences larger than 99:99. + + The syntax now allows for time differences larger than 99:99. (issue #14) - + @@ -1210,12 +1743,12 @@ Version 0.8 - - arbtt-dump can now show the data in other formats - as well, as suggested by Waldir Pimenta (option - ). This includes a human-readale output and - JSON. - + + arbtt-dump can now show the data in other formats + as well, as suggested by Waldir Pimenta (option + ). This includes a human-readale output and + JSON. + New option of arbtt-dump. @@ -1272,9 +1805,9 @@ Version 0.6.4.1 - - Added missing module to the packages. - + + Added missing module to the packages. + @@ -1357,7 +1890,7 @@ Version 0.5 (The ZuriHac-Release) - New command arbtt-import, which imports the output from arbtt-dump. + New command arbtt-import, which imports the output from arbtt-dump. @@ -1376,14 +1909,14 @@ - Unicode is handled correctly in regular expressions. - + Unicode is handled correctly in regular expressions. + - Improved date-handling functions for categorize.cfg. + Improved date-handling functions for categorize.cfg. () - + @@ -1393,7 +1926,7 @@ Bugfix: Added missing modules to the cabal file. - + @@ -1403,7 +1936,7 @@ Implement a custom compression method greatly reduce the file size of the log file. Run arbtt-capture to compress the previous samples as well. - + @@ -1438,11 +1971,11 @@ arbtt-dump. - New command arbtt-recover to rescue data from - a proken data log file. + New command arbtt-recover to rescue data from + a proken data log file. - Actually include this documentation in the released tarball. + Actually include this documentation in the released tarball. diff --git a/src/Data/MyText.hs b/src/Data/MyText.hs --- a/src/Data/MyText.hs +++ b/src/Data/MyText.hs @@ -1,6 +1,7 @@ module Data.MyText where import qualified Data.ByteString.UTF8 as BSU +import qualified Data.ByteString.Lazy.UTF8 as BLU import qualified Data.ByteString.Lazy as LBS import qualified Data.ByteString as BS import Data.Binary.Get @@ -27,17 +28,22 @@ -- Binary instance compatible with Binary String instance Binary Text where put = put . unpack + get = pack <$> get -- The following code exploits that the Binary Char instance uses UTF8 as well -- The downside is that it quietly suceeds for broken input + -- Unfortunately, with binary-0.7, it is no longer possible to implement + -- this nice and lazily, so go via String :-( + {- get = do n <- get :: Get Int - r <- remaining - bs <- lookAhead (getByteString (min (fromIntegral r) (4*n))) -- safe approximation - let utf8bs = BSU.take n bs - unless (BSU.length utf8bs == n) $ + bs <- lookAhead $ getRemainingLazyByteString + let utf8bs = BLU.take (fromIntegral n) bs + unless (BLU.length utf8bs == n) $ fail $ "Coult not parse the expected " ++ show n ++ " utf8 characters." - skip (BS.length utf8bs) - return $ Text utf8bs + let sbs = LBS.toStrict utf8bs + skip $ BS.length sbs + return $ Text sbs + -} {- Possible speedup with a version of binary that provides access to the internals, as the Char instance is actually UTF8, but the length bit is diff --git a/src/Stats.hs b/src/Stats.hs --- a/src/Stats.hs +++ b/src/Stats.hs @@ -1,4 +1,4 @@ -{-# LANGUAGE RecordWildCards, NamedFieldPuns, TypeOperators, TupleSections #-} +{-# LANGUAGE RecordWildCards, NamedFieldPuns, TypeOperators, TupleSections, GADTSyntax, ExistentialQuantification #-} module Stats ( Report(..), ReportOptions(..), @@ -57,7 +57,7 @@ data ActivityFilter = ExcludeActivity ActivityMatcher | OnlyActivity ActivityMatcher deriving (Show, Eq) -data Repeater = ByDay | ByMonth | ByYear +data Repeater = ByMinute | ByHour | ByDay | ByMonth | ByYear deriving (Show, Eq) -- Supported report output formats: text, comma-separated values and @@ -174,19 +174,40 @@ let go' m act = M.insertWith' (+) act (fromInteger (tlRate tl)/1000) m in foldl' go' m (snd (tlData tl))) id -processRepeater :: Repeater -> LeftFold (Bool :!: TimeLogEntry (Ctx, ActivityData)) ReportResults -> LeftFold (Bool :!: TimeLogEntry (Ctx, ActivityData)) ReportResults -processRepeater ByDay rep = - filterElems (\(b :!: _) -> b) $ - pure (RepeatedReportResults "Day" . map (\(d,rr) -> (showGregorian d, rr)) . M.toList) <*> - multiplex (utctDay . tlTime . Strict.snd) rep -processRepeater ByMonth rep = - filterElems (\(b :!: _) -> b) $ - pure (RepeatedReportResults "Month" . map (\((y,m),rr) -> (show y ++ "-" ++ show m, rr)) . M.toList) <*> - multiplex ((\(y,m,_) -> (y, m)). toGregorian . utctDay . tlTime . Strict.snd) rep -processRepeater ByYear rep = - filterElems (\(b :!: _) -> b) $ - pure (RepeatedReportResults "Year" . map (\(y,rr) -> (show y, rr)) . M.toList) <*> - multiplex ((\(y,_,_) -> y). toGregorian . utctDay . tlTime . Strict.snd) rep +processRepeater :: TimeZone -> Repeater -> LeftFold (Bool :!: TimeLogEntry (Ctx, ActivityData)) ReportResults -> LeftFold (Bool :!: TimeLogEntry (Ctx, ActivityData)) ReportResults +processRepeater tz r rep = case repeaterImpl r of + RepeaterImpl catR showR -> + filterElems (\(b :!: _) -> b) $ + pure (RepeatedReportResults (repeaterTitle r) . map (first showR) . M.toList) <*> + multiplex (catR . utcToLocalTime tz . tlTime . Strict.snd) rep + +data RepeaterImpl where + RepeaterImpl :: Ord r => (LocalTime -> r) -> (r -> String) -> RepeaterImpl + +repeaterTitle :: Repeater -> String +repeaterTitle ByMinute = "Minute" +repeaterTitle ByHour = "Hour" +repeaterTitle ByDay = "Day" +repeaterTitle ByMonth = "Month" +repeaterTitle ByYear = "Year" + +repeaterImpl :: Repeater -> RepeaterImpl +repeaterImpl ByMinute = RepeaterImpl + -- a somewhat lazy implementations, using strings... + (formatTime defaultTimeLocale "%F %H:%M") + id +repeaterImpl ByHour = RepeaterImpl + (formatTime defaultTimeLocale "%F %H:00") + id +repeaterImpl ByDay = RepeaterImpl + localDay + showGregorian +repeaterImpl ByMonth = RepeaterImpl + ((\(y,m,_) -> (y, m)) . toGregorian . localDay) + (\(y,m) -> show y ++ "-" ++ show m) +repeaterImpl ByYear = RepeaterImpl + ((\(y,_,_) -> y) . toGregorian . localDay) + show processReport :: ReportOptions -> Report -> LeftFold (Bool :!: TimeLogEntry (Ctx, ActivityData)) ReportResults processReport opts GeneralInfos = diff --git a/src/TimeLog.hs b/src/TimeLog.hs --- a/src/TimeLog.hs +++ b/src/TimeLog.hs @@ -105,13 +105,13 @@ return $ parseTimeLog content parseTimeLog :: (NFData a, ListOfStringable a) => BS.ByteString -> TimeLog a -parseTimeLog input = +parseTimeLog input = if startString == magic then go Nothing rest off else error $ "Timelog starts with unknown marker " ++ show (map (chr.fromIntegral) (BS.unpack startString)) - where + where (startString, rest, off) = runGetState (getLazyByteString (BS.length magic)) input 0 go prev input off = let (v, rest, off') = runGetState (ls_get strs) input off diff --git a/src/stats-main.hs b/src/stats-main.hs --- a/src/stats-main.hs +++ b/src/stats-main.hs @@ -18,6 +18,7 @@ import System.ProgressBar import TermSize import qualified Data.MyText as T +import Data.Time.LocalTime import TimeLog import Categorize @@ -48,7 +49,7 @@ , optLogFile = dir "capture.log" , optCategorizeFile = dir "categorize.cfg" } - + versionStr, header :: String versionStr = "arbtt-stats " ++ showVersion version header = "Usage: arbtt-stats [OPTIONS...]" @@ -91,7 +92,7 @@ "only consider samples matching the condition" , Option "m" ["min-percentage"] (ReqArg (\arg opt -> let ro = (optReportOptions opt) { roMinPercentage = read arg} - in return opt { optReportOptions = ro }) "COND") + in return opt { optReportOptions = ro }) "PERC") "do not show tags with a percentage lower than PERC% (default: 1)" , Option "" ["output-exclude"] (ReqArg (\arg opt -> let filters = ExcludeActivity (parseActivityMatcher arg) : roActivityFilter (optReportOptions opt) @@ -141,10 +142,12 @@ readRepeater :: String -> Repeater readRepeater arg = case map toLower arg of - "day" -> ByDay - "month" -> ByMonth - "year" -> ByYear - _ -> error ("Unsupported parameter to --for-each: '" ++ arg ++ "'") + "minute" -> ByMinute + "hour" -> ByHour + "day" -> ByDay + "month" -> ByMonth + "year" -> ByYear + _ -> error ("Unsupported parameter to --for-each: '" ++ arg ++ "'") readReportFormat :: String -> ReportFormat readReportFormat arg = @@ -163,6 +166,7 @@ (_,_,errs) -> do hPutStr stderr (concat errs ++ usageInfo header options) exitFailure + tz <- getCurrentTimeZone dir <- getAppUserDataDirectory "arbtt" flags <- foldl (>>=) (return (defaultOptions dir)) actions @@ -175,12 +179,12 @@ categorizer <- readCategorizer (optCategorizeFile flags) timelog <- BS.readFile (optLogFile flags) - size <- fileSize <$> getFileStatus (optLogFile flags) isTerm <- hIsTerminalDevice stderr trackedTimelog <- case isTerm of True -> do hSetBuffering stderr NoBuffering + size <- fileSize <$> getFileStatus (optLogFile flags) trackProgressWithChunkSize (fromIntegral size `div` 100) (\_ b -> do (_height, width) <- getTermSize hPutChar stderr '\r' @@ -195,28 +199,26 @@ let captures = parseTimeLog trackedTimelog let allTags = categorizer captures + when (null allTags) $ do putStrLn "Nothing recorded yet" exitFailure - + let filters = (if optAlsoInactive flags then id else (defaultFilter:)) $ optFilters flags let rep = case optReports flags of [] -> TotalTime [x] -> x _ -> error "Please specify exactly one report to generate" - let repeater = foldr (.) id $ map processRepeater (optRepeater flags) + let repeater = foldr (.) id $ map (processRepeater tz) (optRepeater flags) - -- These are defined here, but of course only evaluated when any report - -- refers to them. Some are needed by more than one report, which is then - -- advantageous. let opts = optReportOptions flags let fold = filterPredicate filters `adjoin` repeater (processReport opts rep) let result = runLeftFold fold allTags -- Force the results a bit, to ensure the progress bar to be shown before the title result `seq` return () - + renderReport opts result {-