pandoc-pyplot 1.0.3.0 → 1.1.0.0
raw patch · 7 files changed
+327/−140 lines, 7 filesdep +hspecdep +hspec-expectationsdep +tastydep ~basedep ~pandoc-typesPVP ok
version bump matches the API change (PVP)
Dependencies added: hspec, hspec-expectations, tasty, tasty-hspec, tasty-hunit
Dependency ranges changed: base, pandoc-types
API changes (from Hackage documentation)
- Text.Pandoc.Filter.Pyplot: showError :: PandocPyplotError -> String
+ Text.Pandoc.Filter.Pyplot: instance GHC.Classes.Eq Text.Pandoc.Filter.Pyplot.PandocPyplotError
+ Text.Pandoc.Filter.Pyplot: instance GHC.Show.Show Text.Pandoc.Filter.Pyplot.PandocPyplotError
Files
- CHANGELOG.md +43/−4
- README.md +39/−9
- package.yaml +0/−50
- pandoc-pyplot.cabal +51/−40
- src/Text/Pandoc/Filter/Pyplot.hs +66/−37
- test/Main.hs +126/−0
- test/fixtures/include.py +2/−0
CHANGELOG.md view
@@ -1,7 +1,46 @@ # Change log -pandoc-pyplot uses [Semantic Versioning][].-The change log is available through the [releases on GitHub][].+pandoc-pyplot uses [Semantic Versioning](http://semver.org/spec/v2.0.0.html) -[Semantic Versioning]: http://semver.org/spec/v2.0.0.html-[releases on GitHub]: https://github.com/githubuser/pandoc-pyplot/releases+Release 1.1.0.0+---------------++* Added the ability to include Python files before code using the `plot_include=script.py` attribute.+* Added a test suite.++Release 1.0.3.0+---------------++* Fixed an issue where `pandoc-pyplot` would not build with base < 4.9 (#1)++Release 1.0.2.0+---------------++* Added support for captions using the `plot_alt=...` attribute. For example:++ ```markdown+ ```{plot_target=test.png plot_alt="This is a caption"}+ import matplotlib.pyplot as plt+ plt.figure()+ plt.plot([1,2,3,4,5],[1,2,3,4,5])+ ```+ ```++Release 1.0.1.0+---------------++* Added `plotTransform :: Pandoc -> IO Pandoc` function to transform entire documents. This makes it easier to integrate `pandoc-pyplot` into Hakyll-based sites!++Release 1.0.0.1+---------------++* Updated README with fixes and warnings+* Added top-level package documentation compatible with Haddock+* Added Unsafe language extension, as this filter will run arbitrary Python scripts.++Release 1.0.0.0+---------------++Initial release.++See documentation on [Hackage](https://hackage.haskell.org/package/pandoc-pyplot)
README.md view
@@ -1,6 +1,6 @@ # pandoc-pyplot -[](http://hackage.haskell.org/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) +[](http://hackage.haskell.org/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) ## A Pandoc filter for generating figures with Matplotlib from code directly in documents @@ -51,7 +51,7 @@ You can also specify a caption for your image. This is done using the optional `plot_alt` parameter: ```markdown - ```{plot_target=my_figure.jpg, plot_alt="This is a simple figure"} + ```{plot_target=my_figure.jpg plot_alt="This is a simple figure"} import matplotlib.pyplot as plt plt.figure() @@ -60,6 +60,40 @@ ``` ``` +### Including scripts + +If you find yourself always repeating some steps, inclusion of scripts is possible using the `plot_include` parameter. For example, if you want all plots to have the [`ggplot`](https://matplotlib.org/tutorials/introductory/customizing.html#sphx-glr-tutorials-introductory-customizing-py) style, you can write a very short preamble `style.py` like so: + +```python +import matplotlib.pyplot as plt +plt.style.use('ggplot') +``` + +and include it in your document as follows: + +```markdown + ```{plot_target=my_figure.jpg plot_include=style.py} + plt.figure() + plt.plot([0,1,2,3,4], [1,2,3,4,5]) + plt.title('This is an example figure') + ``` +``` + +Which is equivalent to writing the following markdown: + +```markdown + ```{plot_target=my_figure.jpg} + import matplotlib.pyplot as plt + plt.style.use('ggplot') + + plt.figure() + plt.plot([0,1,2,3,4], [1,2,3,4,5]) + plt.title('This is an example figure') + ``` +``` + +This `plot_include` parameter is perfect for longer documents with many plots. Simply define the style you want in a separate script! You can also import packages this way, or define functions you often use. + ## Installation ### Binaries @@ -101,13 +135,13 @@ The filter program must be in your `PATH`. In case it is, you can use the filter with Pandoc as follows: ```bash -pandoc --filter pandoc-pyplot input.md output.html +pandoc --filter pandoc-pyplot input.md --output output.html ``` -Another example with PDF output: +In which case, the output is HTML. Another example with PDF output: ```bash -pandoc --filter pandoc-pyplot input.md output.pdf +pandoc --filter pandoc-pyplot input.md --output output.pdf ``` Python exceptions will be printed to screen in case of a problem. @@ -144,7 +178,3 @@ ## Warning Do not run this filter on unknown documents. There is nothing in `pandoc-pyplot` that can stop a Python script from performing evil actions. This is the reason this package is deemed __unsafe__ in the parlance of [Safe Haskell](https://ghc.haskell.org/trac/ghc/wiki/SafeHaskell). - -## Aknowledgements - -This package is inspired from [`pandoc-include-code`](https://github.com/owickstrom/pandoc-include-code).
− package.yaml
@@ -1,50 +0,0 @@-name: pandoc-pyplot -version: '1.0.3.0' -github: "LaurentRDC/pandoc-pyplot" -license: MIT -author: "Laurent P. René de Cotret" -maintainer: "Laurent P. René de Cotret" -synopsis: A Pandoc filter for including figures generated from Matplotlib -description: - A pandoc filter for including figures generated from Matplotlib. - Keep the document and Python code in the same location. Output from Matplotlib - is captured and included as a figure. -category: Documentation - -license-file: LICENSE.md - -extra-source-files: -- CHANGELOG.md -- LICENSE.md -- package.yaml -- README.md -- stack.yaml - -ghc-options: -Wall -Wcompat - -library: - dependencies: - - base > 4 && < 5 - - directory - - filepath - - pandoc-types > 1.12 && < 2 - - temporary - - typed-process - - containers - source-dirs: src - exposed-modules: - - Text.Pandoc.Filter.Pyplot - - Text.Pandoc.Filter.Scripting - -executables: - pandoc-pyplot: - source-dirs: executable - main: Main.hs - dependencies: - - base > 4 && < 5 - - pandoc-pyplot - - pandoc-types > 1.12 && < 2 - ghc-options: - - -rtsopts - - -threaded - - -with-rtsopts=-N
pandoc-pyplot.cabal view
@@ -1,13 +1,6 @@-cabal-version: 1.12 ---- This file has been generated from package.yaml by hpack version 0.30.0.------ see: https://github.com/sol/hpack------ hash: 25fd8ea89e42939262b70218fe5deda1f7e6afceb3614082738bee06ed427a51- name: pandoc-pyplot-version: 1.0.3.0+version: 1.1.0.0+cabal-version: >= 1.12 synopsis: A Pandoc filter for including figures generated from Matplotlib description: A pandoc filter for including figures generated from Matplotlib. Keep the document and Python code in the same location. Output from Matplotlib is captured and included as a figure. category: Documentation@@ -21,42 +14,60 @@ extra-source-files: CHANGELOG.md LICENSE.md- package.yaml README.md stack.yaml+ test/fixtures/*.py source-repository head- type: git- location: https://github.com/LaurentRDC/pandoc-pyplot+ type: git+ location: https://github.com/LaurentRDC/pandoc-pyplot library- exposed-modules:- Text.Pandoc.Filter.Pyplot- Text.Pandoc.Filter.Scripting- other-modules:- Paths_pandoc_pyplot- hs-source-dirs:- src- ghc-options: -Wall -Wcompat- build-depends:- base >4 && <5- , containers- , directory- , filepath- , pandoc-types >1.12 && <2- , temporary- , typed-process- default-language: Haskell2010+ exposed-modules:+ Text.Pandoc.Filter.Pyplot+ Text.Pandoc.Filter.Scripting+ other-modules:+ Paths_pandoc_pyplot+ hs-source-dirs:+ src+ ghc-options: -Wall -Wcompat+ build-depends:+ base >=4 && <5+ , containers+ , directory+ , filepath+ , pandoc-types >1.12 && <2+ , temporary+ , typed-process+ default-language: Haskell2010 executable pandoc-pyplot- main-is: Main.hs- other-modules:- Paths_pandoc_pyplot- hs-source-dirs:- executable- ghc-options: -Wall -Wcompat -rtsopts -threaded -with-rtsopts=-N- build-depends:- base >4 && <5- , pandoc-pyplot- , pandoc-types >1.12 && <2- default-language: Haskell2010+ main-is: Main.hs+ other-modules:+ Paths_pandoc_pyplot+ hs-source-dirs:+ executable+ ghc-options: -Wall -Wcompat -rtsopts -threaded -with-rtsopts=-N+ build-depends:+ base >=4 && <5+ , pandoc-pyplot+ , pandoc-types >1.12 && <2+ default-language: Haskell2010++test-suite tests+ type: exitcode-stdio-1.0+ hs-source-dirs: test+ main-is: Main.hs+ build-depends: base >= 4 && < 5+ , directory+ , filepath+ , pandoc-types >= 1.12 && <= 2+ , pandoc-pyplot+ , tasty+ , tasty-hunit+ , tasty-hspec+ , temporary+ , hspec+ , hspec-expectations+ default-language: Haskell2010+
src/Text/Pandoc/Filter/Pyplot.hs view
@@ -12,17 +12,42 @@ This module defines a Pandoc filter @makePlot@ that can be used to walk over a Pandoc document and generate figures from Python code blocks. + +The syntax for code blocks is simple, Code blocks with the @plot_target=...@ +attribute will trigger the filter. The code block will be reworked into a Python +script and the output figure will be captured. + +Here are the possible attributes what pandoc-pyplot understands: + + * @plot_target=...@ (_required_): Filepath where the resulting figure should be saved. + * @plot_alt="..."@ (_optional_): Specify a plot caption (or alternate text). + * @plot_include=...@ (_optional_): Path to a Python script to include before the code block. + Ideal to avoid repetition over many figures. + +Here are some example blocks in Markdown: + +@ +This is a paragraph + +```{plot_target=my_figure.jpg plot_alt="This is a caption."} +import matplotlib.pyplot as plt + +plt.figure() +plt.plot([0,1,2,3,4], [1,2,3,4,5]) +plt.title('This is an example figure') +``` +@ -} module Text.Pandoc.Filter.Pyplot ( makePlot - , makePlot' + , makePlot' -- For testing , plotTransform , PandocPyplotError(..) - , showError ) where import Control.Monad ((>=>)) import qualified Data.Map.Strict as M +import Data.Maybe (fromMaybe) import Data.Monoid ((<>)) import System.Directory (doesDirectoryExist) import System.FilePath (isValid, replaceExtension, takeDirectory) @@ -37,33 +62,39 @@ | InvalidTargetError FilePath -- ^ Invalid figure path | MissingDirectoryError FilePath -- ^ Directory where to save figure does not exist | BlockingCallError -- ^ Python script contains a block call to 'show()' + deriving Eq +instance Show PandocPyplotError where + -- | Translate filter error to an error message + show (ScriptError exitcode) = "Script error: plot could not be generated. Exit code " <> (show exitcode) + show (InvalidTargetError fname) = "Target filename " <> fname <> " is not valid." + show (MissingDirectoryError dirname) = "Target directory " <> dirname <> " does not exist." + show BlockingCallError = "Script contains a blocking call to show, like 'plt.show()'" + + -- | Datatype containing all parameters required -- to run pandoc-pyplot data FigureSpec = FigureSpec - { target :: FilePath -- ^ filepath where generated figure will be saved - , alt :: String -- ^ Alternate text for the figure (optional) - , script :: PythonScript -- ^ Source code for the figure - , blockAttrs :: Attr -- ^ Attributes not related to @pandoc-pyplot@ will be propagated + { target :: FilePath -- ^ filepath where generated figure will be saved. + , alt :: String -- ^ Alternate text for the figure (optional). + , script :: PythonScript -- ^ Source code for the figure. + , includePath :: Maybe FilePath -- ^ Path to a Python to be included before the script. + , blockAttrs :: Attr -- ^ Attributes not related to @pandoc-pyplot@ will be propagated. } --- | Get the source code for a script including provisions to capture --- the output. -scriptWithCapture :: FigureSpec -> PythonScript -scriptWithCapture spec = addPlotCapture (target spec) (script spec) - --- | Determine where to save the script source based on plot target -scriptSourcePath :: FigureSpec -> FilePath -scriptSourcePath spec = replaceExtension (target spec) ".txt" - --- | Get the source code for a figure script in a presentable way -presentableScript :: FigureSpec -> PythonScript -presentableScript spec = mconcat [ "# Source code for ", target spec, "\n", script spec ] +-- | Use figure specification to render a full plot script, including everything except plot capture +renderScript :: FigureSpec -> IO PythonScript +renderScript spec = do + includeScript <- fromMaybe (return "") $ readFile <$> (includePath spec) + return $ mconcat [ "# Source code for ", target spec, "\n" + , "# Generated by pandoc-pyplot\n" + , includeScript, "\n", script spec] -- Keys that pandoc-pyplot will look for in code blocks -targetKey, altTextKey :: String -targetKey = "plot_target" -altTextKey = "plot_alt" +targetKey, altTextKey, includePathKey :: String +targetKey = "plot_target" +altTextKey = "plot_alt" +includePathKey = "plot_include" -- | Determine inclusion specifications from Block attributes. -- Note that the target key is required, but all other parameters are optional @@ -78,6 +109,7 @@ { target = fname , alt = M.findWithDefault "Figure generated by pandoc-pyplot" altTextKey attrs' , script = content + , includePath = M.lookup includePathKey attrs' -- Propagate attributes that are not related to pandoc-pyplot , blockAttrs = (id', cls, filteredAttrs) } @@ -95,21 +127,25 @@ Nothing -> return $ Right block -- Could parse : run the script and capture output Just spec -> do + + -- Rendered script, including possible inclusions and other additions + -- except the plot capture. + rendered <- renderScript spec + let figurePath = target spec figureDir = takeDirectory figurePath - scriptSource = script spec -- Check that the directory in which to save the figure exists - validDirectory <- doesDirectoryExist figureDir + validDirectory <- doesDirectoryExist $ takeDirectory figurePath if | not (isValid figurePath) -> return $ Left $ InvalidTargetError figurePath | not validDirectory -> return $ Left $ MissingDirectoryError figureDir - | hasBlockingShowCall scriptSource -> return $ Left $ BlockingCallError + | hasBlockingShowCall rendered -> return $ Left $ BlockingCallError | otherwise -> do - -- Running the script happens on the next line - -- Note that the script is slightly modified to be able to capture the output - result <- runTempPythonScript (scriptWithCapture spec) + -- Running the script + -- A plot capture (plt.savefig(...)) is added as well + result <- runTempPythonScript $ addPlotCapture (target spec) rendered case result of ScriptFailure code -> return $ Left $ ScriptError code @@ -118,8 +154,8 @@ -- so it can be inspected -- Note : using a .txt file allows to view source directly -- in the browser, in the case of HTML output - let sourcePath = scriptSourcePath spec - writeFile sourcePath (presentableScript spec) + let sourcePath = replaceExtension figurePath ".txt" + writeFile sourcePath rendered -- Propagate attributes that are not related to pandoc-pyplot let relevantAttrs = blockAttrs spec @@ -132,18 +168,11 @@ return $ Right $ Para $ [image] --- | Translate filter error to an error message -showError :: PandocPyplotError -> String -showError (ScriptError exitcode) = "Script error: plot could not be generated. Exit code " <> (show exitcode) -showError (InvalidTargetError fname) = "Target filename " <> fname <> " is not valid." -showError (MissingDirectoryError dirname) = "Target directory " <> dirname <> " does not exist." -showError BlockingCallError = "Script contains a blocking call to show, like 'plt.show()'" - -- | Highest-level function that can be walked over a Pandoc tree. -- All code blocks that have the 'plot_target' parameter will be considered -- figures. makePlot :: Block -> IO Block -makePlot = makePlot' >=> either (fail . showError) return +makePlot = makePlot' >=> either (fail . show) return -- | Walk over an entire Pandoc document, changing appropriate code blocks -- into figures.
+ test/Main.hs view
@@ -0,0 +1,126 @@+ +module Main where + +import Control.Monad (unless) +import Data.List (isInfixOf) + +import Test.Tasty +import Test.Tasty.HUnit + +import qualified Text.Pandoc.Filter.Pyplot as Filter +import qualified Text.Pandoc.Filter.Scripting as Filter +import Text.Pandoc.JSON + +import System.Directory (doesFileExist) +import System.FilePath ((</>)) +import System.IO.Temp (getCanonicalTemporaryDirectory) + +main :: IO () +main = defaultMain $ + testGroup "Text.Pandoc.Filter.Pyplot" + [ testFileCreation + , testFileInclusion + , testBlockingCallError + ] + +-- | Create a code block with the right attributes to trigger pandoc-pyplot +mkPlotCodeBlock :: FilePath -- ^ Plot target + -> String -- ^ Plot alt description + -> String -- ^ Plot caption + -> Maybe FilePath -- ^ Possible inclusion + -> Filter.PythonScript -- ^ Script + -> Block +mkPlotCodeBlock target alt caption include script = CodeBlock attrs script + where + attrs = case include of + Nothing -> ( mempty , mempty + , [ ("plot_target", target) + , ("plot_alt", alt) + , ("plot_caption", caption) + ] + ) + Just includePath -> ( mempty , mempty + , [ ("plot_target", target) + , ("plot_alt", alt) + , ("plot_caption", caption) + , ("plot_include", includePath) + ] + ) + +-- | Assert that a file exists +assertFileExists :: HasCallStack + => FilePath + -> Assertion +assertFileExists filepath = do + fileExists <- doesFileExist filepath + unless fileExists (assertFailure msg) + where + msg = mconcat ["File ", filepath, " does not exist."] + +-- | Assert that a list first list is contained, +-- wholly and intact, anywhere within the second. +assertIsInfix :: (Eq a, Show a, HasCallStack) + => [a] + -> [a] + -> Assertion +assertIsInfix xs ys = + unless (xs `isInfixOf` ys) (assertFailure msg) + where + msg = mconcat ["Expected ", show xs, " to be an infix of ", show ys] + +------------------------------------------------------------------------------- +-- Test that plot files and source files are created when the filter is run +testFileCreation :: TestTree +testFileCreation = testCase "writes output and source files" $ do + tempDir <- getCanonicalTemporaryDirectory + let codeBlock = mkPlotCodeBlock + (tempDir </> "test.png") + mempty + mempty + Nothing + "import matplotlib.pyplot as plt\n" + _ <- Filter.makePlot' codeBlock + assertFileExists (tempDir </> "test.png") + assertFileExists (tempDir </> "test.txt") + +------------------------------------------------------------------------------- +-- Test that included files are found within the source +testFileInclusion :: TestTree +testFileInclusion = testCase "includes plot inclusions" $ do + tempDir <- getCanonicalTemporaryDirectory + + let codeBlock = mkPlotCodeBlock + (tempDir </> "test.png") + mempty + mempty + (Just "test/fixtures/include.py") + "import matplotlib.pyplot as plt\n" + _ <- Filter.makePlot' codeBlock + + inclusion <- readFile "test/fixtures/include.py" + src <- readFile (tempDir </> "test.txt") + assertIsInfix inclusion src + +------------------------------------------------------------------------------- +-- Test that a script containing a blockign call to matplotlib.pyplot.show +-- returns the appropriate error +testBlockingCallError :: TestTree +testBlockingCallError = testCase "raises an exception for blocking calls" $ do + tempDir <- getCanonicalTemporaryDirectory + + let codeBlock = mkPlotCodeBlock + (tempDir </> "test.png") + mempty + mempty + Nothing + "import matplotlib.pyplot as plt\nplt.show()" + + result <- Filter.makePlot' codeBlock + case result of + Right block -> assertFailure "did not catch the expected blocking call" + Left error -> + if error == Filter.BlockingCallError + then pure () + else assertFailure "did not catch the expected blocking call" + +-------------------------------------------------------------------------------
+ test/fixtures/include.py view
@@ -0,0 +1,2 @@+import matplotlib.pyplot as plt +plt.style.use('ggplot')