diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,46 @@ # Change log -pandoc-pyplot uses [Semantic Versioning][]. -The change log is available through the [releases on GitHub][]. +pandoc-pyplot uses [Semantic Versioning](http://semver.org/spec/v2.0.0.html) -[Semantic Versioning]: http://semver.org/spec/v2.0.0.html -[releases on GitHub]: https://github.com/githubuser/pandoc-pyplot/releases +Release 1.1.0.0 +--------------- + +* Added the ability to include Python files before code using the `plot_include=script.py` attribute. +* Added a test suite. + +Release 1.0.3.0 +--------------- + +* Fixed an issue where `pandoc-pyplot` would not build with base < 4.9 (#1) + +Release 1.0.2.0 +--------------- + +* Added support for captions using the `plot_alt=...` attribute. For example: + + ```markdown + ```{plot_target=test.png plot_alt="This is a caption"} + import matplotlib.pyplot as plt + plt.figure() + plt.plot([1,2,3,4,5],[1,2,3,4,5]) + ``` + ``` + +Release 1.0.1.0 +--------------- + +* Added `plotTransform :: Pandoc -> IO Pandoc` function to transform entire documents. This makes it easier to integrate `pandoc-pyplot` into Hakyll-based sites! + +Release 1.0.0.1 +--------------- + +* Updated README with fixes and warnings +* Added top-level package documentation compatible with Haddock +* Added Unsafe language extension, as this filter will run arbitrary Python scripts. + +Release 1.0.0.0 +--------------- + +Initial release. + +See documentation on [Hackage](https://hackage.haskell.org/package/pandoc-pyplot) diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # pandoc-pyplot -[![Hackage version](https://img.shields.io/hackage/v/pandoc-pyplot.svg)](http://hackage.haskell.org/package/pandoc-pyplot) [![Stackage version](http://stackage.org/package/pandoc-pyplot/badge/nightly)](http://stackage.org/nightly/package/pandoc-pyplot) [![Build status](https://ci.appveyor.com/api/projects/status/qbmq9cyks5jup48e?svg=true)](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) +[![Hackage version](https://img.shields.io/hackage/v/pandoc-pyplot.svg)](http://hackage.haskell.org/package/pandoc-pyplot) [![Stackage version (LTS)](http://stackage.org/package/pandoc-pyplot/badge/lts)](http://stackage.org/nightly/package/pandoc-pyplot) [![Stackage version (nightly)](http://stackage.org/package/pandoc-pyplot/badge/nightly)](http://stackage.org/nightly/package/pandoc-pyplot) [![Build status](https://ci.appveyor.com/api/projects/status/qbmq9cyks5jup48e?svg=true)](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) ## A Pandoc filter for generating figures with Matplotlib from code directly in documents @@ -51,7 +51,7 @@ You can also specify a caption for your image. This is done using the optional `plot_alt` parameter: ```markdown - ```{plot_target=my_figure.jpg, plot_alt="This is a simple figure"} + ```{plot_target=my_figure.jpg plot_alt="This is a simple figure"} import matplotlib.pyplot as plt plt.figure() @@ -60,6 +60,40 @@ ``` ``` +### Including scripts + +If you find yourself always repeating some steps, inclusion of scripts is possible using the `plot_include` parameter. For example, if you want all plots to have the [`ggplot`](https://matplotlib.org/tutorials/introductory/customizing.html#sphx-glr-tutorials-introductory-customizing-py) style, you can write a very short preamble `style.py` like so: + +```python +import matplotlib.pyplot as plt +plt.style.use('ggplot') +``` + +and include it in your document as follows: + +```markdown + ```{plot_target=my_figure.jpg plot_include=style.py} + plt.figure() + plt.plot([0,1,2,3,4], [1,2,3,4,5]) + plt.title('This is an example figure') + ``` +``` + +Which is equivalent to writing the following markdown: + +```markdown + ```{plot_target=my_figure.jpg} + import matplotlib.pyplot as plt + plt.style.use('ggplot') + + plt.figure() + plt.plot([0,1,2,3,4], [1,2,3,4,5]) + plt.title('This is an example figure') + ``` +``` + +This `plot_include` parameter is perfect for longer documents with many plots. Simply define the style you want in a separate script! You can also import packages this way, or define functions you often use. + ## Installation ### Binaries @@ -101,13 +135,13 @@ The filter program must be in your `PATH`. In case it is, you can use the filter with Pandoc as follows: ```bash -pandoc --filter pandoc-pyplot input.md output.html +pandoc --filter pandoc-pyplot input.md --output output.html ``` -Another example with PDF output: +In which case, the output is HTML. Another example with PDF output: ```bash -pandoc --filter pandoc-pyplot input.md output.pdf +pandoc --filter pandoc-pyplot input.md --output output.pdf ``` Python exceptions will be printed to screen in case of a problem. @@ -144,7 +178,3 @@ ## Warning Do not run this filter on unknown documents. There is nothing in `pandoc-pyplot` that can stop a Python script from performing evil actions. This is the reason this package is deemed __unsafe__ in the parlance of [Safe Haskell](https://ghc.haskell.org/trac/ghc/wiki/SafeHaskell). - -## Aknowledgements - -This package is inspired from [`pandoc-include-code`](https://github.com/owickstrom/pandoc-include-code). diff --git a/package.yaml b/package.yaml deleted file mode 100644 --- a/package.yaml +++ /dev/null @@ -1,50 +0,0 @@ -name: pandoc-pyplot -version: '1.0.3.0' -github: "LaurentRDC/pandoc-pyplot" -license: MIT -author: "Laurent P. René de Cotret" -maintainer: "Laurent P. René de Cotret" -synopsis: A Pandoc filter for including figures generated from Matplotlib -description: - A pandoc filter for including figures generated from Matplotlib. - Keep the document and Python code in the same location. Output from Matplotlib - is captured and included as a figure. -category: Documentation - -license-file: LICENSE.md - -extra-source-files: -- CHANGELOG.md -- LICENSE.md -- package.yaml -- README.md -- stack.yaml - -ghc-options: -Wall -Wcompat - -library: - dependencies: - - base > 4 && < 5 - - directory - - filepath - - pandoc-types > 1.12 && < 2 - - temporary - - typed-process - - containers - source-dirs: src - exposed-modules: - - Text.Pandoc.Filter.Pyplot - - Text.Pandoc.Filter.Scripting - -executables: - pandoc-pyplot: - source-dirs: executable - main: Main.hs - dependencies: - - base > 4 && < 5 - - pandoc-pyplot - - pandoc-types > 1.12 && < 2 - ghc-options: - - -rtsopts - - -threaded - - -with-rtsopts=-N diff --git a/pandoc-pyplot.cabal b/pandoc-pyplot.cabal --- a/pandoc-pyplot.cabal +++ b/pandoc-pyplot.cabal @@ -1,13 +1,6 @@ -cabal-version: 1.12 - --- This file has been generated from package.yaml by hpack version 0.30.0. --- --- see: https://github.com/sol/hpack --- --- hash: 25fd8ea89e42939262b70218fe5deda1f7e6afceb3614082738bee06ed427a51 - name: pandoc-pyplot -version: 1.0.3.0 +version: 1.1.0.0 +cabal-version: >= 1.12 synopsis: A Pandoc filter for including figures generated from Matplotlib description: A pandoc filter for including figures generated from Matplotlib. Keep the document and Python code in the same location. Output from Matplotlib is captured and included as a figure. category: Documentation @@ -21,42 +14,60 @@ extra-source-files: CHANGELOG.md LICENSE.md - package.yaml README.md stack.yaml + test/fixtures/*.py source-repository head - type: git - location: https://github.com/LaurentRDC/pandoc-pyplot + type: git + location: https://github.com/LaurentRDC/pandoc-pyplot library - exposed-modules: - Text.Pandoc.Filter.Pyplot - Text.Pandoc.Filter.Scripting - other-modules: - Paths_pandoc_pyplot - hs-source-dirs: - src - ghc-options: -Wall -Wcompat - build-depends: - base >4 && <5 - , containers - , directory - , filepath - , pandoc-types >1.12 && <2 - , temporary - , typed-process - default-language: Haskell2010 + exposed-modules: + Text.Pandoc.Filter.Pyplot + Text.Pandoc.Filter.Scripting + other-modules: + Paths_pandoc_pyplot + hs-source-dirs: + src + ghc-options: -Wall -Wcompat + build-depends: + base >=4 && <5 + , containers + , directory + , filepath + , pandoc-types >1.12 && <2 + , temporary + , typed-process + default-language: Haskell2010 executable pandoc-pyplot - main-is: Main.hs - other-modules: - Paths_pandoc_pyplot - hs-source-dirs: - executable - ghc-options: -Wall -Wcompat -rtsopts -threaded -with-rtsopts=-N - build-depends: - base >4 && <5 - , pandoc-pyplot - , pandoc-types >1.12 && <2 - default-language: Haskell2010 + main-is: Main.hs + other-modules: + Paths_pandoc_pyplot + hs-source-dirs: + executable + ghc-options: -Wall -Wcompat -rtsopts -threaded -with-rtsopts=-N + build-depends: + base >=4 && <5 + , pandoc-pyplot + , pandoc-types >1.12 && <2 + default-language: Haskell2010 + +test-suite tests + type: exitcode-stdio-1.0 + hs-source-dirs: test + main-is: Main.hs + build-depends: base >= 4 && < 5 + , directory + , filepath + , pandoc-types >= 1.12 && <= 2 + , pandoc-pyplot + , tasty + , tasty-hunit + , tasty-hspec + , temporary + , hspec + , hspec-expectations + default-language: Haskell2010 + diff --git a/src/Text/Pandoc/Filter/Pyplot.hs b/src/Text/Pandoc/Filter/Pyplot.hs --- a/src/Text/Pandoc/Filter/Pyplot.hs +++ b/src/Text/Pandoc/Filter/Pyplot.hs @@ -12,17 +12,42 @@ This module defines a Pandoc filter @makePlot@ that can be used to walk over a Pandoc document and generate figures from Python code blocks. + +The syntax for code blocks is simple, Code blocks with the @plot_target=...@ +attribute will trigger the filter. The code block will be reworked into a Python +script and the output figure will be captured. + +Here are the possible attributes what pandoc-pyplot understands: + + * @plot_target=...@ (_required_): Filepath where the resulting figure should be saved. + * @plot_alt="..."@ (_optional_): Specify a plot caption (or alternate text). + * @plot_include=...@ (_optional_): Path to a Python script to include before the code block. + Ideal to avoid repetition over many figures. + +Here are some example blocks in Markdown: + +@ +This is a paragraph + +```{plot_target=my_figure.jpg plot_alt="This is a caption."} +import matplotlib.pyplot as plt + +plt.figure() +plt.plot([0,1,2,3,4], [1,2,3,4,5]) +plt.title('This is an example figure') +``` +@ -} module Text.Pandoc.Filter.Pyplot ( makePlot - , makePlot' + , makePlot' -- For testing , plotTransform , PandocPyplotError(..) - , showError ) where import Control.Monad ((>=>)) import qualified Data.Map.Strict as M +import Data.Maybe (fromMaybe) import Data.Monoid ((<>)) import System.Directory (doesDirectoryExist) import System.FilePath (isValid, replaceExtension, takeDirectory) @@ -37,33 +62,39 @@ | InvalidTargetError FilePath -- ^ Invalid figure path | MissingDirectoryError FilePath -- ^ Directory where to save figure does not exist | BlockingCallError -- ^ Python script contains a block call to 'show()' + deriving Eq +instance Show PandocPyplotError where + -- | Translate filter error to an error message + show (ScriptError exitcode) = "Script error: plot could not be generated. Exit code " <> (show exitcode) + show (InvalidTargetError fname) = "Target filename " <> fname <> " is not valid." + show (MissingDirectoryError dirname) = "Target directory " <> dirname <> " does not exist." + show BlockingCallError = "Script contains a blocking call to show, like 'plt.show()'" + + -- | Datatype containing all parameters required -- to run pandoc-pyplot data FigureSpec = FigureSpec - { target :: FilePath -- ^ filepath where generated figure will be saved - , alt :: String -- ^ Alternate text for the figure (optional) - , script :: PythonScript -- ^ Source code for the figure - , blockAttrs :: Attr -- ^ Attributes not related to @pandoc-pyplot@ will be propagated + { target :: FilePath -- ^ filepath where generated figure will be saved. + , alt :: String -- ^ Alternate text for the figure (optional). + , script :: PythonScript -- ^ Source code for the figure. + , includePath :: Maybe FilePath -- ^ Path to a Python to be included before the script. + , blockAttrs :: Attr -- ^ Attributes not related to @pandoc-pyplot@ will be propagated. } --- | Get the source code for a script including provisions to capture --- the output. -scriptWithCapture :: FigureSpec -> PythonScript -scriptWithCapture spec = addPlotCapture (target spec) (script spec) - --- | Determine where to save the script source based on plot target -scriptSourcePath :: FigureSpec -> FilePath -scriptSourcePath spec = replaceExtension (target spec) ".txt" - --- | Get the source code for a figure script in a presentable way -presentableScript :: FigureSpec -> PythonScript -presentableScript spec = mconcat [ "# Source code for ", target spec, "\n", script spec ] +-- | Use figure specification to render a full plot script, including everything except plot capture +renderScript :: FigureSpec -> IO PythonScript +renderScript spec = do + includeScript <- fromMaybe (return "") $ readFile <$> (includePath spec) + return $ mconcat [ "# Source code for ", target spec, "\n" + , "# Generated by pandoc-pyplot\n" + , includeScript, "\n", script spec] -- Keys that pandoc-pyplot will look for in code blocks -targetKey, altTextKey :: String -targetKey = "plot_target" -altTextKey = "plot_alt" +targetKey, altTextKey, includePathKey :: String +targetKey = "plot_target" +altTextKey = "plot_alt" +includePathKey = "plot_include" -- | Determine inclusion specifications from Block attributes. -- Note that the target key is required, but all other parameters are optional @@ -78,6 +109,7 @@ { target = fname , alt = M.findWithDefault "Figure generated by pandoc-pyplot" altTextKey attrs' , script = content + , includePath = M.lookup includePathKey attrs' -- Propagate attributes that are not related to pandoc-pyplot , blockAttrs = (id', cls, filteredAttrs) } @@ -95,21 +127,25 @@ Nothing -> return $ Right block -- Could parse : run the script and capture output Just spec -> do + + -- Rendered script, including possible inclusions and other additions + -- except the plot capture. + rendered <- renderScript spec + let figurePath = target spec figureDir = takeDirectory figurePath - scriptSource = script spec -- Check that the directory in which to save the figure exists - validDirectory <- doesDirectoryExist figureDir + validDirectory <- doesDirectoryExist $ takeDirectory figurePath if | not (isValid figurePath) -> return $ Left $ InvalidTargetError figurePath | not validDirectory -> return $ Left $ MissingDirectoryError figureDir - | hasBlockingShowCall scriptSource -> return $ Left $ BlockingCallError + | hasBlockingShowCall rendered -> return $ Left $ BlockingCallError | otherwise -> do - -- Running the script happens on the next line - -- Note that the script is slightly modified to be able to capture the output - result <- runTempPythonScript (scriptWithCapture spec) + -- Running the script + -- A plot capture (plt.savefig(...)) is added as well + result <- runTempPythonScript $ addPlotCapture (target spec) rendered case result of ScriptFailure code -> return $ Left $ ScriptError code @@ -118,8 +154,8 @@ -- so it can be inspected -- Note : using a .txt file allows to view source directly -- in the browser, in the case of HTML output - let sourcePath = scriptSourcePath spec - writeFile sourcePath (presentableScript spec) + let sourcePath = replaceExtension figurePath ".txt" + writeFile sourcePath rendered -- Propagate attributes that are not related to pandoc-pyplot let relevantAttrs = blockAttrs spec @@ -132,18 +168,11 @@ return $ Right $ Para $ [image] --- | Translate filter error to an error message -showError :: PandocPyplotError -> String -showError (ScriptError exitcode) = "Script error: plot could not be generated. Exit code " <> (show exitcode) -showError (InvalidTargetError fname) = "Target filename " <> fname <> " is not valid." -showError (MissingDirectoryError dirname) = "Target directory " <> dirname <> " does not exist." -showError BlockingCallError = "Script contains a blocking call to show, like 'plt.show()'" - -- | Highest-level function that can be walked over a Pandoc tree. -- All code blocks that have the 'plot_target' parameter will be considered -- figures. makePlot :: Block -> IO Block -makePlot = makePlot' >=> either (fail . showError) return +makePlot = makePlot' >=> either (fail . show) return -- | Walk over an entire Pandoc document, changing appropriate code blocks -- into figures. diff --git a/test/Main.hs b/test/Main.hs new file mode 100644 --- /dev/null +++ b/test/Main.hs @@ -0,0 +1,126 @@ + +module Main where + +import Control.Monad (unless) +import Data.List (isInfixOf) + +import Test.Tasty +import Test.Tasty.HUnit + +import qualified Text.Pandoc.Filter.Pyplot as Filter +import qualified Text.Pandoc.Filter.Scripting as Filter +import Text.Pandoc.JSON + +import System.Directory (doesFileExist) +import System.FilePath (()) +import System.IO.Temp (getCanonicalTemporaryDirectory) + +main :: IO () +main = defaultMain $ + testGroup "Text.Pandoc.Filter.Pyplot" + [ testFileCreation + , testFileInclusion + , testBlockingCallError + ] + +-- | Create a code block with the right attributes to trigger pandoc-pyplot +mkPlotCodeBlock :: FilePath -- ^ Plot target + -> String -- ^ Plot alt description + -> String -- ^ Plot caption + -> Maybe FilePath -- ^ Possible inclusion + -> Filter.PythonScript -- ^ Script + -> Block +mkPlotCodeBlock target alt caption include script = CodeBlock attrs script + where + attrs = case include of + Nothing -> ( mempty , mempty + , [ ("plot_target", target) + , ("plot_alt", alt) + , ("plot_caption", caption) + ] + ) + Just includePath -> ( mempty , mempty + , [ ("plot_target", target) + , ("plot_alt", alt) + , ("plot_caption", caption) + , ("plot_include", includePath) + ] + ) + +-- | Assert that a file exists +assertFileExists :: HasCallStack + => FilePath + -> Assertion +assertFileExists filepath = do + fileExists <- doesFileExist filepath + unless fileExists (assertFailure msg) + where + msg = mconcat ["File ", filepath, " does not exist."] + +-- | Assert that a list first list is contained, +-- wholly and intact, anywhere within the second. +assertIsInfix :: (Eq a, Show a, HasCallStack) + => [a] + -> [a] + -> Assertion +assertIsInfix xs ys = + unless (xs `isInfixOf` ys) (assertFailure msg) + where + msg = mconcat ["Expected ", show xs, " to be an infix of ", show ys] + +------------------------------------------------------------------------------- +-- Test that plot files and source files are created when the filter is run +testFileCreation :: TestTree +testFileCreation = testCase "writes output and source files" $ do + tempDir <- getCanonicalTemporaryDirectory + let codeBlock = mkPlotCodeBlock + (tempDir "test.png") + mempty + mempty + Nothing + "import matplotlib.pyplot as plt\n" + _ <- Filter.makePlot' codeBlock + assertFileExists (tempDir "test.png") + assertFileExists (tempDir "test.txt") + +------------------------------------------------------------------------------- +-- Test that included files are found within the source +testFileInclusion :: TestTree +testFileInclusion = testCase "includes plot inclusions" $ do + tempDir <- getCanonicalTemporaryDirectory + + let codeBlock = mkPlotCodeBlock + (tempDir "test.png") + mempty + mempty + (Just "test/fixtures/include.py") + "import matplotlib.pyplot as plt\n" + _ <- Filter.makePlot' codeBlock + + inclusion <- readFile "test/fixtures/include.py" + src <- readFile (tempDir "test.txt") + assertIsInfix inclusion src + +------------------------------------------------------------------------------- +-- Test that a script containing a blockign call to matplotlib.pyplot.show +-- returns the appropriate error +testBlockingCallError :: TestTree +testBlockingCallError = testCase "raises an exception for blocking calls" $ do + tempDir <- getCanonicalTemporaryDirectory + + let codeBlock = mkPlotCodeBlock + (tempDir "test.png") + mempty + mempty + Nothing + "import matplotlib.pyplot as plt\nplt.show()" + + result <- Filter.makePlot' codeBlock + case result of + Right block -> assertFailure "did not catch the expected blocking call" + Left error -> + if error == Filter.BlockingCallError + then pure () + else assertFailure "did not catch the expected blocking call" + +------------------------------------------------------------------------------- diff --git a/test/fixtures/include.py b/test/fixtures/include.py new file mode 100644 --- /dev/null +++ b/test/fixtures/include.py @@ -0,0 +1,2 @@ +import matplotlib.pyplot as plt +plt.style.use('ggplot')