day_6__polish.mdwn

Last night, I polished the `git annex watch` command.

First, I fixed the double commits problem. There's still some extra
committing going on in the `git-annex` branch that I don't understand. It
seems like a monad shutdown event is somehow being triggered whenever
a git command is run by the commit thread.

I also made `git annex watch` run as a proper daemon,
and `git annex watch --stop` stop it.

---

Then I managed to greatly increase its startup speed. At startup, it
generates "add" events for every symlink in the tree. This is necessary
because it doesn't really know if a symlink is already added, or was
manually added before it starter, or indeed was added while it started up.
Problem was that these events were causing a lot of work staging the
symlinks -- most of which were already correctly staged.

You'd think it could just check if the same symlink was in the index.
But it can't, because the index is in a constant state of flux. The
symlinks might have just been deleted and re-added, or changed, and
the index still have the old value.

Instead, I got creative. :) We can't trust what the index says about the
symlink, but if the index happens to contian a symlink that looks right,
we can trust that the SHA1 of its blob is the right SHA1, and reuse it
when re-staging the symlink. Wham! Massive speedup!

---

Then I started running `git annex watch` on my own real git annex repos,
and noticed some problems.. Like it turns normal files already checked into
git into symlinks. And it leaks memory scanning a big tree. Oops..