diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,11 @@ pandoc-pyplot uses [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +Release 2.0.1.0 +--------------- + +* Support for Markdown formatting in figure captions, including LaTeX math. + Release 2.0.0.0 --------------- @@ -11,7 +16,7 @@ * Removed the ability to control the filename and format directly using the `plot_target=...` attribute. * Added the ability to control the directory in which figures will be saved using the `directory=...` attribute. * Added the possibility to control the figures dots-per-inch (i.e. pixel density) with the `dpi=...` attribute. -* Added the ability to control the figure format with the `format=...` attribute. Possible values are currently `"png"`, `"svg"`, `"pdf"`, and `"jpg"`/`"jpeg"`. +* Added the ability to control the figure format with the `format=...` attribute. Possible values are currently `"png"`, `"svg"`, `"pdf"`, `"jpg"`/`"jpeg"` and `"eps"`. * The confusing `plot_alt=...` attribute has been renamed to `caption=...` for obvious reasons. * The `plot_include=...` attribute has been renamed to `include=...`. * Added the generation of a higher resolution figure for every figure `pandoc-pyplot` understands. diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -1,17 +1,13 @@ -# pandoc-pyplot - -[![Hackage version](https://img.shields.io/hackage/v/pandoc-pyplot.svg)](http://hackage.haskell.org/package/pandoc-pyplot) [![Stackage version (LTS)](http://stackage.org/package/pandoc-pyplot/badge/lts)](http://stackage.org/nightly/package/pandoc-pyplot) [![Stackage version (nightly)](http://stackage.org/package/pandoc-pyplot/badge/nightly)](http://stackage.org/nightly/package/pandoc-pyplot) [![Build status](https://ci.appveyor.com/api/projects/status/qbmq9cyks5jup48e?svg=true)](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) +# pandoc-pyplot - A Pandoc filter to generate Matplotlib figures directly in documents -## A Pandoc filter for generating figures with Matplotlib from code directly in documents +[![Hackage version](https://img.shields.io/hackage/v/pandoc-pyplot.svg)](http://hackage.haskell.org/package/pandoc-pyplot) [![Stackage version (LTS)](http://stackage.org/package/pandoc-pyplot/badge/lts)](http://stackage.org/nightly/package/pandoc-pyplot) [![Stackage version (nightly)](http://stackage.org/package/pandoc-pyplot/badge/nightly)](http://stackage.org/nightly/package/pandoc-pyplot) [![Build status](https://ci.appveyor.com/api/projects/status/qbmq9cyks5jup48e?svg=true)](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) ![](https://img.shields.io/badge/license-MIT-green.svg) -Inspired by [sphinx](https://sphinxdoc.org)'s `plot_directive`, `pandoc-pyplot` helps turn Python code present in your documents to embedded Matplotlib figures. +`pandoc-pyplot` turns Python code present in your documents to embedded Matplotlib figures. ## Usage The filter recognizes code blocks with the `pyplot` class present. It will run the script in the associated code block in a Python interpreter and capture the generated Matplotlib figure. -### Basic example - Here is a basic example using the scripting `matplotlib.pyplot` API: ```markdown @@ -40,10 +36,16 @@ There are more examples in the [source repository](https://github.com/LaurentRDC/pandoc-pyplot), in the `\examples` directory. -### Link to source code +## Features -In case of an output format that supports links (e.g. HTML), the embedded image generated by `pandoc-pyplot` will be a link to the source code which was used to generate the file. Therefore, other people can see what Python code was used to create your figures. +### No wasted work +`pandoc-pyplot` minimizes work, only generating figures if it absolutely must. Therefore, you can confidently run the filter on very large documents containing dozens of figures --- like a book or a thesis --- and only the figures which have recently changed will be re-generated. + +### Link to source code and high-resolution figure + +In case of an output format that supports links (e.g. HTML), the embedded image generated by `pandoc-pyplot` will be a link to the source code which was used to generate the file. Therefore, other people can see what Python code was used to create your figures. A high resolution image will be made available in a caption link. + ### Captions You can also specify a caption for your image. This is done using the optional `caption` parameter: @@ -58,6 +60,8 @@ ``` ``` +Caption formatting is either plain text or Markdown. LaTeX-style math is also support in captions (using dollar signs $...$). + ### Including scripts If you find yourself always repeating some steps, inclusion of scripts is possible using the `include` parameter. For example, if you want all plots to have the [`ggplot`](https://matplotlib.org/tutorials/introductory/customizing.html#sphx-glr-tutorials-introductory-customizing-py) style, you can write a very short preamble `style.py` like so: @@ -100,7 +104,7 @@ ### Installers -Starting with `pandoc-pyplot` version 1.1.0.0, Windows installers are also made available thanks to [Inno Setup](http://www.jrsoftware.org/isinfo.php). +Windows installers are made available thanks to [Inno Setup](http://www.jrsoftware.org/isinfo.php). You can download them from the [release page](https://github.com/LaurentRDC/pandoc-pyplot/releases/latest). ### From Hackage/Stackage diff --git a/pandoc-pyplot.cabal b/pandoc-pyplot.cabal --- a/pandoc-pyplot.cabal +++ b/pandoc-pyplot.cabal @@ -1,5 +1,5 @@ name: pandoc-pyplot -version: 2.0.0.0 +version: 2.0.1.0 cabal-version: >= 1.12 synopsis: A Pandoc filter for including figures generated from Matplotlib description: A pandoc filter for including figures generated from Matplotlib. Keep the document and Python code in the same location. Output from Matplotlib is captured and included as a figure. @@ -33,16 +33,16 @@ src ghc-options: -Wall -Wcompat build-depends: - base >=4 && <5 + base >=4 && <5 , containers , directory , filepath , hashable > 1 && < 2 + , pandoc > 2 && < 3 , pandoc-types >1.12 && <2 , temporary , text , typed-process - , random > 1 && < 2 default-language: Haskell2010 executable pandoc-pyplot diff --git a/src/Text/Pandoc/Filter/FigureSpec.hs b/src/Text/Pandoc/Filter/FigureSpec.hs --- a/src/Text/Pandoc/Filter/FigureSpec.hs +++ b/src/Text/Pandoc/Filter/FigureSpec.hs @@ -15,22 +15,55 @@ ( FigureSpec(..) , SaveFormat(..) , saveFormatFromString + , toImage + , sourceCodePath , figurePath - , hiresFigurePath , addPlotCapture -- for testing purposes , extension ) where +import Control.Monad (join) + import Data.Hashable (Hashable, hash, hashWithSalt) +import Data.Maybe (fromMaybe) import qualified Data.Text as T import System.FilePath (FilePath, addExtension, replaceExtension, ()) -import Text.Pandoc.Definition (Attr) +import Text.Pandoc.Definition +import Text.Pandoc.Builder (imageWith, link, para, fromList, toList) import Text.Pandoc.Filter.Scripting (PythonScript) +import Text.Pandoc.Class (runPure) +import Text.Pandoc.Extensions (extensionsFromList, Extension(..)) +import Text.Pandoc.Options (def, ReaderOptions(..)) +import Text.Pandoc.Readers (readMarkdown) + +readerOptions :: ReaderOptions +readerOptions = def + {readerExtensions = + extensionsFromList + [ Ext_tex_math_dollars + , Ext_superscript + , Ext_subscript + ] + } + +-- | Read a figure caption in Markdown format. +captionReader :: String -> Maybe [Inline] +captionReader t = either (const Nothing) (Just . extractFromBlocks) $ runPure $ readMarkdown' (T.pack t) + where + readMarkdown' = readMarkdown readerOptions + + extractFromBlocks (Pandoc _ blocks) = mconcat $ extractInlines <$> blocks + + extractInlines (Plain inlines) = inlines + extractInlines (Para inlines) = inlines + extractInlines (LineBlock multiinlines) = join multiinlines + extractInlines _ = [] + data SaveFormat = PNG | PDF @@ -71,6 +104,21 @@ hashWithSalt salt spec = hashWithSalt salt (caption spec, script spec, directory spec, dpi spec, blockAttrs spec) +-- | Convert a FigureSpec to a Pandoc block component +toImage :: FigureSpec -> Block +toImage spec = head . toList $ para $ imageWith attrs' target' "fig:" caption' + -- To render images as figures with captions, the target title + -- must be "fig:" + -- Janky? yes + where + attrs' = blockAttrs spec + target' = figurePath spec + srcLink = link (replaceExtension target' ".txt") mempty "Source code" + hiresLink = link (hiresFigurePath spec) mempty "high res." + captionText = fromList $ fromMaybe mempty (captionReader $ caption spec) + captionLinks = mconcat [" (", srcLink, ", ", hiresLink, ")"] + caption' = captionText <> captionLinks + -- | Determine the path a figure should have. figurePath :: FigureSpec -> FilePath figurePath spec = (directory spec stem spec) @@ -78,6 +126,10 @@ stem = flip addExtension ext . show . hash ext = extension . saveFormat $ spec +-- | Determine the path to the source code that generated the figure. +sourceCodePath :: FigureSpec -> FilePath +sourceCodePath = flip replaceExtension ".txt" . figurePath + -- | The path to the high-resolution figure. hiresFigurePath :: FigureSpec -> FilePath hiresFigurePath spec = flip replaceExtension (".hires" <> ext) . figurePath $ spec @@ -85,10 +137,9 @@ ext = extension . saveFormat $ spec -- | Modify a Python plotting script to save the figure to a filename. --- An additional file (with extension PNG) will also be captured. -addPlotCapture :: - FigureSpec -- ^ Path where to save the figure - -> PythonScript -- ^ Code block with added capture +-- An additional file will also be captured. +addPlotCapture :: FigureSpec -- ^ Path where to save the figure + -> PythonScript -- ^ Code block with added capture addPlotCapture spec = mconcat [ script spec diff --git a/src/Text/Pandoc/Filter/Pyplot.hs b/src/Text/Pandoc/Filter/Pyplot.hs --- a/src/Text/Pandoc/Filter/Pyplot.hs +++ b/src/Text/Pandoc/Filter/Pyplot.hs @@ -94,80 +94,63 @@ import System.Directory (createDirectoryIfMissing, doesFileExist) -import System.FilePath (isValid, makeValid, - replaceExtension, takeDirectory) +import System.FilePath (makeValid, takeDirectory) import Text.Pandoc.Definition import Text.Pandoc.Walk (walkM) import Text.Pandoc.Filter.FigureSpec (FigureSpec (..), SaveFormat (..), addPlotCapture, - figurePath, hiresFigurePath, - saveFormatFromString) + figurePath, sourceCodePath, saveFormatFromString, + toImage) import Text.Pandoc.Filter.Scripting -- | Possible errors returned by the filter data PandocPyplotError - = ScriptError Int -- ^ Running Python script has yielded an error - | InvalidTargetError FilePath -- ^ Invalid figure path - | BlockingCallError -- ^ Python script contains a block call to 'show()' + = ScriptError Int -- ^ Running Python script has yielded an error + | BlockingCallError -- ^ Python script contains a block call to 'show()' deriving (Eq) --- | Translate filter error to an error message instance Show PandocPyplotError where - show (ScriptError exitcode) = - "Script error: plot could not be generated. Exit code " <> (show exitcode) - show (InvalidTargetError fname) = "Target filename " <> fname <> " is not valid." - show BlockingCallError = "Script contains a blocking call to show, like 'plt.show()'" + show (ScriptError exitcode) = "Script error: plot could not be generated. Exit code " <> (show exitcode) + show BlockingCallError = "Script contains a blocking call to show, like 'plt.show()'" -- | Keys that pandoc-pyplot will look for in code blocks. These are only exported for testing purposes. directoryKey, captionKey, dpiKey, includePathKey, saveFormatKey :: String -directoryKey = "directory" - -captionKey = "caption" - -dpiKey = "dpi" - +directoryKey = "directory" +captionKey = "caption" +dpiKey = "dpi" includePathKey = "include" - -saveFormatKey = "format" +saveFormatKey = "format" -- | list of all keys related to pandoc-pyplot. inclusionKeys :: [String] inclusionKeys = [directoryKey, captionKey, dpiKey, includePathKey, saveFormatKey] -- | Determine inclusion specifications from Block attributes. --- Note that the target key is required, but all other parameters are optional +-- Note that the @".pyplot"@ class is required, but all other parameters are optional parseFigureSpec :: Block -> IO (Maybe FigureSpec) parseFigureSpec (CodeBlock (id', cls, attrs) content) | "pyplot" `elem` cls = Just <$> figureSpec | otherwise = return Nothing where - attrs' = Map.fromList attrs + attrs' = Map.fromList attrs filteredAttrs = filter (\(k, _) -> k `notElem` inclusionKeys) attrs - dir = makeValid $ Map.findWithDefault "generated" directoryKey attrs' - format = fromMaybe (PNG) $ saveFormatFromString $ Map.findWithDefault "png" saveFormatKey attrs' - includePath = Map.lookup includePathKey attrs' + dir = makeValid $ Map.findWithDefault "generated" directoryKey attrs' + format = fromMaybe (PNG) $ saveFormatFromString $ Map.findWithDefault "png" saveFormatKey attrs' + includePath = Map.lookup includePathKey attrs' + figureSpec :: IO FigureSpec figureSpec = do - includeScript <- fromMaybe (return "") $ T.readFile <$> includePath - let header = "# Generated by pandoc-pyplot " <> ((T.pack . showVersion) version) - fullScript = mconcat $ intersperse "\n" [header, includeScript, T.pack content] - caption' = Map.findWithDefault mempty captionKey attrs' - dpi' = read $ Map.findWithDefault "80" dpiKey attrs' + includeScript <- fromMaybe (return mempty) $ T.readFile <$> includePath + let header = "# Generated by pandoc-pyplot " <> ((T.pack . showVersion) version) + fullScript = mconcat $ intersperse "\n" [header, includeScript, T.pack content] + caption' = Map.findWithDefault mempty captionKey attrs' + dpi' = read $ Map.findWithDefault "80" dpiKey attrs' blockAttrs' = (id', filter (/= "pyplot") cls, filteredAttrs) return $ FigureSpec caption' fullScript format dir dpi' blockAttrs' -parseFigureSpec _ = return Nothing --- | Check figure specifications for common mistakes -validateSpec :: FigureSpec -> Maybe PandocPyplotError -validateSpec spec - | not (isValid path) = Just $ InvalidTargetError path - | hasBlockingShowCall rendered = Just $ BlockingCallError - | otherwise = Nothing - where - path = figurePath spec - rendered = script spec +parseFigureSpec _ = return Nothing -- | Run the Python script. In case the file already exists, we can safely assume -- there is no need to re-run it. @@ -175,59 +158,28 @@ runScriptIfNecessary spec = do createDirectoryIfMissing True . takeDirectory $ figurePath spec fileAlreadyExists <- doesFileExist $ figurePath spec - if fileAlreadyExists - then return ScriptSuccess - else do - result <- runTempPythonScript $ addPlotCapture spec - case result of - ScriptFailure code -> return $ ScriptFailure code - ScriptSuccess - -- Save the original script into a separate file - -- so it can be inspected - -- Note : using a .txt file allows to view source directly - -- in the browser, in the case of HTML output - -> do - let sourcePath = replaceExtension (figurePath spec) ".txt" - T.writeFile sourcePath $ script spec - return ScriptSuccess + result <- if fileAlreadyExists + then return ScriptSuccess + else runTempPythonScript $ addPlotCapture spec + case result of + ScriptFailure code -> return $ ScriptFailure code + ScriptSuccess -> T.writeFile (sourceCodePath spec) (script spec) >> return ScriptSuccess -- | Main routine to include Matplotlib plots. -- Code blocks containing the attributes @.pyplot@ are considered -- Python plotting scripts. All other possible blocks are ignored. makePlot' :: Block -> IO (Either PandocPyplotError Block) makePlot' block = do - parsed <- parseFigureSpec block + parsed <- parseFigureSpec block case parsed of - Nothing -> return $ Right block + Nothing -> return $ Right block Just spec -> - case validateSpec spec of - Just err -> return $ Left err - Nothing -> do - result <- runScriptIfNecessary spec - case result of - ScriptFailure code -> return $ Left $ ScriptError code - ScriptSuccess -> do - let relevantAttrs = blockAttrs spec - sourcePath = replaceExtension (figurePath spec) ".txt" - hiresPath = hiresFigurePath spec - srcTarget = Link nullAttr [Str "Source code"] (sourcePath, "") - hiresTarget = Link nullAttr [Str "high res."] (hiresPath, "") - -- TODO: use pandoc-types Builder module - caption' = - [ Str $ caption spec - , Space - , Str "(" - , srcTarget - , Str "," - , Space - , hiresTarget - , Str ")" - ] - -- To render images as figures with captions, the target title - -- must be "fig:" - -- Janky? yes - image = Image relevantAttrs caption' (figurePath spec, "fig:") - return $ Right $ Para $ [image] + if hasBlockingShowCall (script spec) + then return $ Left BlockingCallError + else handleResult spec <$> runScriptIfNecessary spec + where + handleResult _ (ScriptFailure code) = Left $ ScriptError code + handleResult spec ScriptSuccess = Right $ toImage spec -- | Highest-level function that can be walked over a Pandoc tree. -- All code blocks that have the '.pyplot' parameter will be considered diff --git a/test/Main.hs b/test/Main.hs --- a/test/Main.hs +++ b/test/Main.hs @@ -14,7 +14,10 @@ import qualified Text.Pandoc.Filter.FigureSpec as P import qualified Text.Pandoc.Filter.Pyplot as P import qualified Text.Pandoc.Filter.Scripting as P + import Text.Pandoc.JSON +import qualified Text.Pandoc.Builder as B +import qualified Text.Pandoc.Definition as B import System.Directory (createDirectory, createDirectoryIfMissing, @@ -30,7 +33,7 @@ defaultMain $ testGroup "Text.Pandoc.Filter.Pyplot" - [testFileCreation, testFileInclusion, testSaveFormat, testBlockingCallError] + [testFileCreation, testFileInclusion, testSaveFormat, testBlockingCallError, testMarkdownFormattingCaption] plotCodeBlock :: P.PythonScript -> Block plotCodeBlock script = CodeBlock (mempty, ["pyplot"], mempty) (unpack script) @@ -126,12 +129,12 @@ assertEqual "" numberjpgFiles 2 ------------------------------------------------------------------------------- --- Test that a script containing a blockign call to matplotlib.pyplot.show +-- Test that a script containing a blocking call to matplotlib.pyplot.show -- returns the appropriate error testBlockingCallError :: TestTree testBlockingCallError = testCase "raises an exception for blocking calls" $ do - tempDir <- getCanonicalTemporaryDirectory + tempDir <- ( "test-blocking-call-error") <$>getCanonicalTemporaryDirectory let codeBlock = plotCodeBlock "import matplotlib.pyplot as plt\nplt.show()" result <- P.makePlot' codeBlock case result of @@ -140,4 +143,26 @@ if error == P.BlockingCallError then pure () else assertFailure "did not catch the expected blocking call" +------------------------------------------------------------------------------- + +------------------------------------------------------------------------------- +-- Test that Markdown formatting in captions is correctly rendered +testMarkdownFormattingCaption :: TestTree +testMarkdownFormattingCaption = + testCase "appropriately parses Markdown captions" $ do + tempDir <- ( "test-caption-parsing") <$> getCanonicalTemporaryDirectory + -- Note that this test is fragile, in the sense that the expected result must be carefully + -- constructed + let expected = [B.Strong [B.Str "caption"]] + codeBlock = addDirectory tempDir $ addCaption "**caption**" $ plotCodeBlock "import matplotlib.pyplot as plt" + result <- P.makePlot' codeBlock + case result of + Left error -> assertFailure $ "an error occured: " <> show error + Right block -> assertIsInfix expected (extractCaption block) + where + extractCaption (B.Para blocks) = extractImageCaption . head $ blocks + extractCaption _ = mempty + + extractImageCaption (Image _ c _) = c + extractImageCaption _ = mempty -------------------------------------------------------------------------------