seqaid 0.1.2 → 0.1.3
raw patch · 6 files changed
+40/−25 lines, 6 files
Files
- HTML/extra.html +10/−10
- HTML/seqaid.html +10/−10
- HTML/style.css +1/−0
- HTML/style2.css +1/−0
- Seqaid/Demo.hs +12/−4
- seqaid.cabal +6/−1
HTML/extra.html view
@@ -16,8 +16,8 @@ Some words may be in order about where <tt>seqaid</tt> and <tt>leaky</tt> came from, and how this all fits together. <tt>seqaid</tt> was the first of this group of projects, started in June (2014). At that time, it was a GHC API application, and there's been a fair amount of water under the bridge since then.-In July I conceived of <tt>deepseq-bounded</tt> based on considerations while pushing <tt>seqaid</tt>.-August I was on hiatus, then finished <tt>deepseq-bounded</tt>, but wanted to see if I could plug it in to <tt>seqaid</tt> and maybe release them together.+In July I conceived of <tt><span class="nowrap">deepseq-bounded</span></tt> based on considerations while pushing <tt>seqaid</tt>.+August I was on hiatus, then finished <tt><span class="nowrap">deepseq-bounded</span></tt>, but wanted to see if I could plug it in to <tt>seqaid</tt> and maybe release them together. <p> Up till then, I'd been using <tt>Anatomy</tt> as my main space leak example, but it stopped leaking in GHC 7.8.1 (if not sooner).@@ -53,11 +53,11 @@ (It still leaks in 7.8.3.) <p>-During that battle of wits with the compiler, I wasn't really thinking about <tt>deepseq-bounded</tt> or <tt>seqaid</tt>, so the leak example was not contrived with these tools in mind (although they already existed, particularly <tt>deepseq-bounded</tt> which was finished).+During that battle of wits with the compiler, I wasn't really thinking about <tt><span class="nowrap">deepseq-bounded</span></tt> or <tt>seqaid</tt>, so the leak example was not contrived with these tools in mind (although they already existed, particularly <tt><span class="nowrap">deepseq-bounded</span></tt> which was finished). <p> Now that I had a leak example, I was ready to test my leak-plugging tools.-I added a deep list to the state, so that while <tt>NFData.force</tt> could be used to plug the leak, it incurred an arbitrarily-large performance hit.+I added a deep list to the state, so that while <tt>NFData</tt>/<tt>force</tt> could be used to plug the leak, it incurred an arbitrarily-large performance hit. In this respect, there was some contrivance, as I wanted to show how <tt>NFDataN</tt> could outperform <tt>NFData</tt> when used in a similar manner. Then of course I wanted to show how <tt>NFDataP</tt> could outperform them both. I adorned the state with some large strict blobs, which <tt>NFDataN</tt> cannot avoid, but <tt>NFDataP</tt> can.@@ -65,18 +65,18 @@ To date, the optimiser part of <tt>seqaid</tt> is planned but still unimplemented. <p>-I think the example is valid (realistic), notwithstanding these "contrivances".+I think the example is valid (realistic), notwithstanding these contrivances. There hasn't been time yet to put the new tools to work on my real projects. <tt>seqaid</tt> in particular is just breaking out of its shell. I'll be reporting progress as it becomes feasible. <p> The following shows the output of <tt>seqaid</tt> with <tt>leaky</tt>.-It is also a wee bit contrived, as I sweep <tt>NFDataN</tt> <tt>n</tt> value to a fixed depth, and then the fixed (hand-optimised) pattern is developed by replaying iterated <tt>shrinkPat</tt> in reverse.+It is also a wee bit contrived, as I sweep <tt>NFDataN</tt> <tt><em>N</em></tt> value to a fixed depth, and then the fixed (hand-optimised) pattern is developed by replaying iterated <tt>shrinkPat</tt> in reverse. But it does summarise the results nicely. <p>-Using <tt>NFDataN.forcen</tt> <em>N</em>:+Using <tt>NFDataN.forcen <em>N</em></tt>: <pre> live alloc type N 0 357828 3350236 TA@@ -90,7 +90,7 @@ N 8 129440 53405380 TA </pre> <p>-Using <tt>NFDataP.forcep</tt> <em>P</em>:+Using <tt>NFDataP.forcep <em>P</em></tt>: <pre> live alloc type P . 457296 3341600 TA@@ -108,11 +108,11 @@ <p> A few remarks: <ul>-<li>You can see the space leak as a steady, substantial growth in the heap size, from N=0 through N=5, and again in the first five pattern lines.+<li>You can see the space leak as a steady, substantial growth in the heap size, from <tt><em>N</em></tt> equal 0 through 5, and again in the first five pattern lines. <li>You can see the first large, strict blobs get hit at a depth of 5 (with <tt>forcen</tt>). <li>Unfortunately :) it is not until a depth of 6 that the leak can be plugged. (Well, you can plug it without forcing the spine, using let-lifting, but <tt>seqaid</tt> doesn't do that yet.) So <tt>NFDataN</tt> is not a feasible solution for this leak. <li><tt>NFDataP.forcep</tt> successfully plugs the leak without hitting the big strict blobs.-<li>This output doesn't show how <tt>NFData</tt> would perform worse, but for large N there is additional penalty due to the presence of some long lists (not strict, but deep) in the state.+<li>This output doesn't show how <tt>NFData</tt> would perform worse, but for large <tt><em>N</em></tt> there is additional penalty due to the presence of some long lists (not strict, but deep) in the state. <!--<li>It may appear there is still a slight leak in the heap, with the final pattern, but if the run is extended, this space is reclaimed periodically. I don't quite understand this, since I am using <tt>performGC</tt> before each stats line is printed, but the fact is there is no longitudinal net growth. (Oops, I don't see it in this paste anyways...).-->
HTML/seqaid.html view
@@ -33,7 +33,7 @@ <p> -The key contribution of <!--a href="http://hackage.haskell.org/package/seqaid"-->seqaid<!--/a--> is runtime strictness optimisation, through dynamic, principled forcing, as supported by <a href="http://hackage.haskell.org/package/deepseq-bounded">deepseq-bounded</a>.+The key contribution of <!--a href="http://hackage.haskell.org/package/seqaid"-->seqaid<!--/a--> is runtime strictness optimisation, through dynamic, principled forcing, as supported by <a href="http://hackage.haskell.org/package/deepseq-bounded"><span class="nowrap">deepseq-bounded</span></a>. <p> @@ -107,7 +107,7 @@ <h3>Dynamically Configurable Parallelism</h3> -With recent upgrades to <tt>deepseq-bounded</tt>, (the) dynamically-configurable parallelisation harness now comes for free, and should also prove useful.+With recent upgrades to <tt><span class="nowrap">deepseq-bounded</span></tt>, (the) dynamically-configurable parallelisation harness now comes for free, and should also prove useful. The strictness harness machinery of seqaid will work without change for this purpose, including the optimiser, and promises some interesting experiments! <h3>How to Use Seqaid</h3>@@ -157,7 +157,7 @@ Manual instrumentation is also supported, should you prefer. Simply wrap any expressions you'd like in harness with a <tt>seqaid</tt> application. <tt>seqaid</tt> is not an actual function, it's a pattern the preprocessor will recognise.-For technical reasons <span style="font-size: 80%">[no good reason really...]</span>, this presently also requires adding a <a href="http://hackage.haskell.org/package/seqaid/docs/Seqaid-Ann.html#SeqaidAnnManual"><tt>SeqaidAnnManual</tt></a> annotation for each bind you edit.+For technical reasons <span style="font-size: 80%">[no good reason really...]</span>, this presently also requires adding a <a href="http://hackage.haskell.org/package/seqaid/docs/Seqaid-Ann.html#t:SeqaidAnnManual"><tt>SeqaidAnnManual</tt></a> annotation for each bind you edit. <!--span style="font-size: 80%;">[That document is still not written, but <tt>leaky</tt> at <tt>FORCING_STRATEGY=4</tt> is a demonstation.]</span--> <h3><tt>leaky</tt> Contrived to Showcase <tt>seqaid</tt>?</h3>@@ -180,11 +180,11 @@ Most all the work up till now has been to get the basic infrastructure in place. <p>-I hustled this preliminary version of seqaid together in the wake of writing the <a href="http://hackage.haskell.org/package/deepseq-bounded">deepseq-bounded</a> library, just as proof of concept.+I hustled this preliminary version of seqaid together in the wake of writing the <a href="http://hackage.haskell.org/package/deepseq-bounded"><span class="nowrap">deepseq-bounded</span></a> library, just as proof of concept. Unfortunately, auto-instrumentation turned out to be a nightmare, particularly as I had no TH or Core programming experience, setting me back by more than a month... <p>-So I've determined to release <a href="http://hackage.haskell.org/package/deepseq-bounded">leaky</a>, <a href="http://hackage.haskell.org/package/deepseq-bounded">deepseq-bounded</a>, and <a href="http://hackage.haskell.org/package/seqaid">seqaid</a> without further delay.+So I've determined to release <a href="http://hackage.haskell.org/package/leaky">leaky</a>, <a href="http://hackage.haskell.org/package/deepseq-bounded"><span class="nowrap">deepseq-bounded</span></a>, and <a href="http://hackage.haskell.org/package/seqaid">seqaid</a> without further delay. <p> After a much-needed vacation from these topics (which are not my interest; I'm an applications programmer!...), there are a few more things I'm intending to do.@@ -230,7 +230,7 @@ Working on automating for polymorphic types... This has become less of an issue with subexpression injection. <li>-It is possible to transform away the deepseq-bounded code generated by seqaid, leaving only <tt>let</tt> and <tt>seq</tt>.+It is possible to transform away the <tt><span class="nowrap">deepseq-bounded</span></tt> code generated by seqaid, leaving only <tt>let</tt> and <tt>seq</tt>. If you were to accept a (constant) strictification offered by seqaid, you could emit it as a static <tt>let</tt>-and-<tt>seq</tt> code transformation. The point is, if your code was Haskell 98 to begin with, it will still be Haskell 98 afterwards, and also be significantly more performant than the high-level forcing harness of seqaid. <br />@@ -238,13 +238,13 @@ <a id="dig03s" class="dig-toggle-show" href='javascript:toggle("dig03");' style="margin: 10px 100px 10px 30px;">Show</a> <div id="dig03" class="digression" style="margin: 10px 100px 10px 30px;"> <a id="dig03h" class="dig-toggle-hide" href='javascript:toggle("dig03");'>Hide</a><p>-There is nothing intrinsically non-H98 about deepseq-bounded, but for generic deriving support GHC is needed.+There is nothing intrinsically non-H98 about <tt><span class="nowrap">deepseq-bounded</span></tt>, but for generic deriving support GHC is needed. You could write your own NFDataP and superclass instances, and be H98 all the way.-deepseq-bounded can be built in H98 mode with the flag HASKELL98_FRAGMENT...+<tt><span class="nowrap">deepseq-bounded</span></tt> can be built in H98 mode with the flag HASKELL98_FRAGMENT... </div> <li> There is a whole program algebra around manipulating strictness (see for instance-<a href="http://www.fremissant.net/deepseq-bounded/deepseq-bounded.html#fusion">fusion rules</a> in <tt>deepseq-bounded</tt>, and the <a href="http://hackage.haskell.org/package/deepseq-bounded/docs/Control-DeepSeq-Bounded-PatAlg.html">PatAlg</a> API; or, for more theoretical depth, the many articles published over the last 50 years on abstract interpretation, demand propagation, strictness analysis, evaluation transformers...).+<a href="http://www.fremissant.net/deepseq-bounded/deepseq-bounded.html#fusion">fusion rules</a> in <tt><span class="nowrap">deepseq-bounded</span></tt>, and the <a href="http://hackage.haskell.org/package/deepseq-bounded/docs/Control-DeepSeq-Bounded-PatAlg.html">PatAlg</a> API; or, for more theoretical depth, the many articles published over the last 50 years on abstract interpretation, demand propagation, strictness analysis, evaluation transformers...). Whether before or after transforming to <tt>let</tt>-and-<tt>seq</tt>, there are opportunities for simplifying the forcing harness, while preserving strictness and value semantics. </ul> @@ -259,7 +259,7 @@ it is a virtual machine (mostly written in C) which evaluates your code, so implementing seqaid in the RTS will probably be quite a departure from the above. <p>-The new <tt>Seqable</tt> module of <tt>deepseq-bounded</tt> is a step in the right direction, but I've not had time to really investigate it let alone document it.+The new <tt>Seqable</tt> module of <tt><span class="nowrap">deepseq-bounded</span></tt> is a step in the right direction, but I've not had time to really investigate it let alone document it. <p> And anyway, it turns out <tt>seq</tt> is completely desugared by the time it hits the simplifier, so we're dealing with Core <tt>let</tt> versus <tt>case</tt> techniques, and not even function applications.
HTML/style.css view
@@ -26,3 +26,4 @@ tr { text-align: top; vertical-align: top; } td { text-align: top; vertical-align: top; padding-left: 8px; padding-top: 16px; } li { margin-top: 4px; }+.nowrap { white-space: nowrap; }
HTML/style2.css view
@@ -26,3 +26,4 @@ tr { text-align: top; vertical-align: top; } td { text-align: top; vertical-align: top; padding-left: 8px; padding-top: 16px; } li { margin-top: 4px; }+.nowrap { white-space: nowrap; }
Seqaid/Demo.hs view
@@ -23,17 +23,25 @@ import System.Directory import System.Process + leaky_version = "0.1.0.3"+--leaky_version = "0.1.0.0"+ main = do tdir <- createTempDirectory "." "leaky_" -- XXX Note that "cabal get" already extracts the tarball for you.- let seqaid_init_bash = "\-\cabal get leaky-0.1.0.0\n\-\cd leaky-0.1.0.0\n\+#if IS_WINDOWS+ let bang_lines = ""+#else+ let bang_lines = "#!/bin/bash\nset -e\n"+#endif+ let seqaid_init_bash = bang_lines ++ "\+\cabal get leaky-" ++ leaky_version ++ "\n\+\cd leaky-" ++ leaky_version ++ "\n\ \cabal configure\n\ \cabal build\n\ \cabal run 123\n\ \echo\n\-\echo \"(Please see " ++ tdir ++ "/leaky-0.1.0.0/README for the interpretation.)\"\n\+\echo \"(Please see " ++ tdir ++ "/leaky-" ++ leaky_version ++ "/README for the interpretation.)\"\n\ \echo\n" setCurrentDirectory tdir writeFile "seqaidinit.sh" seqaid_init_bash
seqaid.cabal view
@@ -1,6 +1,6 @@ name: seqaid-version: 0.1.2+version: 0.1.3 synopsis: Dynamic strictness control, including space leak repair description: Seqaid is a GHC plugin for non-invasive auto-instrumentation of dynamic strictness (and parallelism) control, shortly to include optimisation for automated space leak relief using minimal strictification. [The optimiser is still in development however.] .@@ -275,6 +275,11 @@ -- can start at the left margin! ghc-options: -pgmP cpphs -optP --cpp ghc-options: -optP-Wundef -fno-warn-overlapping-patterns++ if os(windows)+ cpp-options: -DIS_WINDOWS=1+ else+ cpp-options: -DIS_WINDOWS=0 default-extensions: CPP default-language: Haskell2010