pandoc-plot 0.1.0.0 → 0.2.0.0
raw patch · 17 files changed
+400/−125 lines, 17 filesdep +parallel-iodep +turtle
Dependencies added: parallel-io, turtle
Files
- CHANGELOG.md +8/−0
- README.md +20/−20
- example-config.yml +56/−0
- executable/Main.hs +9/−6
- pandoc-plot.cabal +5/−1
- src/Text/Pandoc/Filter/Plot.hs +44/−19
- src/Text/Pandoc/Filter/Plot/Configuration.hs +18/−7
- src/Text/Pandoc/Filter/Plot/Parse.hs +13/−26
- src/Text/Pandoc/Filter/Plot/Renderers.hs +25/−3
- src/Text/Pandoc/Filter/Plot/Renderers/GNUPlot.hs +41/−0
- src/Text/Pandoc/Filter/Plot/Renderers/Mathematica.hs +1/−1
- src/Text/Pandoc/Filter/Plot/Renderers/Matlab.hs +10/−1
- src/Text/Pandoc/Filter/Plot/Renderers/Prelude.hs +9/−1
- src/Text/Pandoc/Filter/Plot/Scripting.hs +12/−5
- src/Text/Pandoc/Filter/Plot/Types.hs +9/−26
- tests/Common.hs +117/−9
- tests/Main.hs +3/−0
CHANGELOG.md view
@@ -2,6 +2,14 @@ pandoc-plot uses [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +Release 0.2.0.0 +--------------- + +* Added support for gnuplot. +* Added more tests for all toolkits. + +* Fixed an issue where the package could not be installed because a source file was not included in the cabal file. + Release 0.1.0.0 ---------------
README.md view
@@ -1,10 +1,10 @@ # pandoc-plot -## A Pandoc filter to generate figures from code blocks in documents using your plotting toolkit of choice +## A Pandoc filter to generate figures from code blocks in documents -[](https://ci.appveyor.com/project/LaurentRDC/pandoc-plot) [](https://dev.azure.com/laurentdecotret/pandoc-plot/_build/latest?definitionId=5&branchName=master)  +[](http://hackage.haskell.org/package/pandoc-plot) [](https://ci.appveyor.com/project/LaurentRDC/pandoc-plot) [](https://dev.azure.com/laurentdecotret/pandoc-plot/_build/latest?definitionId=5&branchName=master)  -`pandoc-plot` turns code blocks present in your documents into embedded figures, using your plotting toolkit of choice, including Matplotlib, ggplot2, MATLAB, Mathematica, and more. +`pandoc-plot` turns code blocks present in your documents (Markdown, LaTeX, etc.) into embedded figures, using your plotting toolkit of choice, including Matplotlib, ggplot2, MATLAB, Mathematica, and more. ## Table of content @@ -47,9 +47,11 @@ pandoc --filter pandoc-plot input.md --output output.html ``` +*Note that pandoc-plot only works with pandoc >= 2.8 because of some breaking changes in pandoc's API.* + ## Supported toolkits -`pandoc-plot` currently supports the following plotting toolkits (installed separately): +`pandoc-plot` currently supports the following plotting toolkits (**installed separately**): * `matplotlib`: plots using the [matplotlib](https://matplotlib.org/) Python library; * `plotly_python` : plots using the [plotly](https://plot.ly/python/) Python library; @@ -57,20 +59,15 @@ * `mathplot` : plots using [Mathematica](https://www.wolfram.com/mathematica/); * `octaveplot`: plots using [GNU Octave](https://www.gnu.org/software/octave/); * `ggplot2`: plots using [ggplot2](https://ggplot2.tidyverse.org/); +* `gnuplot`: plots using [gnuplot](http://www.gnuplot.info/); + To know which toolkits are useable on *your machine* (and which ones are not available), you can check with the `--toolkits/-t` flag: ```bash pandoc-plot --toolkits ``` -### In progress - -Support for the following plotting toolkits is coming: - -* [gnuplot](http://www.gnuplot.info/) -* [Plotly R](https://plot.ly/r/) - **Wish your plotting toolkit of choice was available? Please [raise an issue](https://github.com/LaurentRDC/pandoc-plot/issues)!** ## Features @@ -82,7 +79,7 @@ __Markdown__: ~~~markdown -```{.matlabplot caption="This is a simple figure"} +```{.matlabplot caption="This is a simple figure with a **Markdown** caption"} x = 0: .1 : 2*pi; y1 = cos(x); y2 = sin(x); @@ -95,7 +92,7 @@ __LaTex__: ```latex -\begin{minted}[caption=This is a simple figure]{matlabplot} +\begin{minted}[caption=This is a simple figure with a \LaTeX caption]{matlabplot} x = 0: .1 : 2*pi; y1 = cos(x); y2 = sin(x); @@ -105,7 +102,7 @@ \end{minted} ``` -Caption formatting is either plain text or Markdown. LaTeX-style math is also support in captions (using dollar signs $...$). +Caption formatting should match the document formatting. ### Link to source code @@ -273,17 +270,20 @@ ### Binaries -Windows binaries are available on [GitHub](https://github.com/LaurentRDC/pandoc-plot/releases). Place the executable in a location that is in your PATH to be able to call it. - -If you can show me how to generate binaries for other platform using e.g. Azure Pipelines, let me know! +*Coming soon* ### Installers (Windows) -Windows installers are made available thanks to [Inno Setup](http://www.jrsoftware.org/isinfo.php). You can download them from the [release page](https://github.com/LaurentRDC/pandoc-plot/releases/latest). +*Coming soon* -### From Hackage/Stackage +### From Hackage -*Coming soon* +`pandoc-plot` is available on [Hackage](http://hackage.haskell.org/package/pandoc-plot). Using the [`cabal-install`](https://www.haskell.org/cabal/) tool: + +```bash +cabal update +cabal install pandoc-plot +``` ### From source
+ example-config.yml view
@@ -0,0 +1,56 @@+ +# This is an example configuration. Everything in this file is optional. +# Please refer to the documentation to know about the parameters herein. +# +# The `executable` parameter for all toolkits can be either the +# executable name (if it is present on the PATH), or +# the full path to the executable. +# E.g.: +# executable: python3 +# executable: "C:\Python37\Scripts\python.exe" +# +# Note that this file should be re-named to ".pandoc-plot.yml" before pandoc-plot +# notices it. + +# The following parameters affect all toolkits +directory: plots/ +source: false +dpi: 80 +format: PNG + +# The possible parameters for the Matplotlib toolkit +matplotlib: + # preamble: matplotlib.py + tight_bbox: false + transparent: false + executable: python + +# The possible parameters for the MATLAB toolkit +matlabplot: + # preamble: matlab.m + executable: matlab + +# The possible parameters for the Plotly/Python toolkit +plotly_python: + # preamble: plotly-python.py + executable: python + +# The possible parameters for the Mathematica toolkit +mathplot: + # preamble: mathematica.m + executable: math + +# The possible parameters for the GNU Octave toolkit +octaveplot: + # preamble: octave.m + executable: octave + +# The possible parameters for the ggplot2 toolkit +ggplot2: + # preamble: ggplot2.r + executable: Rscript + +# The possible parameters for the gnuplot toolkit +gnuplot: + # preamble: gnuplot.gp + executable: gnuplot
executable/Main.hs view
@@ -8,7 +8,7 @@ import Control.Monad (join) import Data.Default.Class (def) -import Data.List (intersperse) +import Data.List (intersperse, (\\)) import Data.Monoid ((<>)) import qualified Data.Text as T import qualified Data.Text.IO as T @@ -20,11 +20,10 @@ import System.IO.Temp (writeSystemTempFile) import Text.Pandoc.Filter.Plot (availableToolkits, - plotTransform, - unavailableToolkits) + plotTransform) import Text.Pandoc.Filter.Plot.Internal (Toolkit (..), cls, Configuration(..), supportedSaveFormats, - configuration) + configuration, toolkits) import Text.Pandoc.JSON (toJSONFilter) @@ -106,9 +105,13 @@ go (Just Toolkits) _ = do c <- config putStrLn "\nAVAILABLE TOOLKITS\n" - availableToolkits c >>= mapM_ toolkitInfo + available <- availableToolkits c + return available >>= mapM_ toolkitInfo putStrLn "\nUNAVAILABLE TOOLKITS\n" - unavailableToolkits c >>= mapM_ toolkitInfo + -- We don't use unavailableToolkits because this would force + -- more IO actions + let unavailable = toolkits \\ available + return unavailable >>= mapM_ toolkitInfo go (Just Config) _ = T.writeFile ".example-pandoc-plot.yml" exampleConfig go Nothing _ = toJSONFilterWithConfig
pandoc-plot.cabal view
@@ -1,5 +1,5 @@ name: pandoc-plot -version: 0.1.0.0 +version: 0.2.0.0 cabal-version: >= 1.12 synopsis: A Pandoc filter to include figures generated from code blocks using your plotting toolkit of choice. description: A Pandoc filter to include figures generated from code blocks. Keep the document and code in the same location. Output is captured and included as a figure. @@ -16,6 +16,7 @@ LICENSE README.md stack.yaml + example-config.yml source-repository head type: git @@ -39,6 +40,7 @@ Text.Pandoc.Filter.Plot.Renderers.Mathematica Text.Pandoc.Filter.Plot.Renderers.Octave Text.Pandoc.Filter.Plot.Renderers.GGPlot2 + Text.Pandoc.Filter.Plot.Renderers.GNUPlot hs-source-dirs: src ghc-options: -Wall -Wcompat @@ -51,9 +53,11 @@ , hashable >= 1 && < 2 , pandoc >= 2.8 && < 3 , pandoc-types >= 1.20 && < 2 + , parallel-io >= 0.3.3 && < 0.4 , shakespeare >= 2.0 && < 3 , temporary , text >= 1 && < 2 + , turtle >= 1.5 && < 2 , typed-process >= 0.2.1 && < 1 , yaml >= 0.8 && < 1 , mtl >= 2.2 && < 2.3
src/Text/Pandoc/Filter/Plot.hs view
@@ -54,8 +54,13 @@ , unavailableToolkits ) where +import Control.Monad.IO.Class (liftIO) import Control.Monad.Reader (runReaderT) +import Data.Maybe (fromMaybe) + +import System.IO (hPutStrLn, stderr) + import Text.Pandoc.Definition import Text.Pandoc.Walk (walkM) @@ -65,34 +70,54 @@ -- | Highest-level function that can be walked over a Pandoc tree. -- All code blocks that have the @.plot@ / @.plotly@ class will be considered -- figures. -makePlot :: Configuration -> Block -> IO Block -makePlot conf block = - maybe (return block) (\tk -> make tk conf block) (plotToolkit block) +-- +-- The target document format determines how the figure captions should be parsed. +-- By default (i.e. if Nothing), captions will be parsed as Markdown with LaTeX math @$...$@, +makePlot :: Configuration -- ^ Configuration for default values + -> Maybe Format -- ^ Input document format + -> Block + -> IO Block +makePlot conf mfmt block = + let fmt = fromMaybe (Format "markdown+tex_math_dollars") mfmt + in maybe (return block) (\tk -> make tk conf fmt block) (plotToolkit block) -- | Walk over an entire Pandoc document, changing appropriate code blocks -- into figures. Default configuration is used. -plotTransform :: Configuration -> Pandoc -> IO Pandoc -plotTransform = walkM . makePlot +plotTransform :: Configuration -- ^ Configuration for default values + -> Maybe Format -- ^ Input document format + -> Pandoc + -> IO Pandoc +plotTransform conf mfmt = walkM $ makePlot conf mfmt -- | Force to use a particular toolkit to render appropriate code blocks. -make :: Toolkit -> Configuration -> Block -> IO Block -make tk conf block = do - let runEnv = PlotEnv tk conf - runReaderT (makePlot' block >>= either (fail . show) return) runEnv +-- +-- Failing to render a figure does not stop the filter, so that you may run the filter +-- on documents without having all necessary toolkits installed. In this case, error +-- messages are printed to stderr, and blocks are left unchanged. +make :: Toolkit + -> Configuration -- ^ Configuration for default values + -> Format -- ^ Input document format + -> Block + -> IO Block +make tk conf fmt block = runReaderT (makePlot' block) (PlotEnv tk conf) where - makePlot' blk = do - parsed <- parseFigureSpec blk - maybe - (return $ Right blk) - (\s -> handleResult s <$> runScriptIfNecessary s) - parsed + makePlot' :: Block -> PlotM Block + makePlot' blk + = parseFigureSpec blk + >>= maybe + (return blk) + (\s -> runScriptIfNecessary s >>= handleResult s) where - handleResult _ (ScriptChecksFailed msg) = Left $ ScriptChecksFailedError msg - handleResult _ (ScriptFailure cmd code) = Left $ ScriptError cmd code - handleResult spec ScriptSuccess = Right $ toImage spec - + handleResult spec ScriptSuccess = return $ toImage fmt spec + handleResult _ (ScriptChecksFailed msg) = do + liftIO $ hPutStrLn stderr $ "pandoc-plot: The script check failed with message: " <> msg + return blk + handleResult _ (ScriptFailure _ code) = do + liftIO $ hPutStrLn stderr $ "pandoc-plot: The script failed with exit code " <> show code + return blk + -- | Possible errors returned by the filter data PandocPlotError
src/Text/Pandoc/Filter/Plot/Configuration.hs view
@@ -50,6 +50,7 @@ , _mathematicaPrec :: !MathematicaPrecursor , _octavePrec :: !OctavePrecursor , _ggplot2Prec :: !GGPlot2Precursor + , _gnuplotPrec :: !GNUPlotPrecursor } instance Default ConfigPrecursor where @@ -65,6 +66,7 @@ , _mathematicaPrec = def , _octavePrec = def , _ggplot2Prec = def + , _gnuplotPrec = def } @@ -80,6 +82,7 @@ data MathematicaPrecursor = MathematicaPrecursor {_mathematicaPreamble :: !(Maybe FilePath), _mathematicaExe :: !FilePath} data OctavePrecursor = OctavePrecursor {_octavePreamble :: !(Maybe FilePath), _octaveExe :: !FilePath} data GGPlot2Precursor = GGPlot2Precursor {_ggplot2Preamble :: !(Maybe FilePath), _ggplot2Exe :: !FilePath} +data GNUPlotPrecursor = GNUPlotPrecursor {_gnuplotPreamble :: !(Maybe FilePath), _gnuplotExe :: !FilePath} instance Default MatplotlibPrecursor where @@ -90,37 +93,42 @@ instance Default MathematicaPrecursor where def = MathematicaPrecursor Nothing (mathematicaExe def) instance Default OctavePrecursor where def = OctavePrecursor Nothing (octaveExe def) instance Default GGPlot2Precursor where def = GGPlot2Precursor Nothing (ggplot2Exe def) +instance Default GNUPlotPrecursor where def = GNUPlotPrecursor Nothing (gnuplotExe def) instance FromJSON MatplotlibPrecursor where parseJSON (Object v) = MatplotlibPrecursor - <$> v .:? (tshow MatplotlibPreambleK) + <$> v .:? (tshow PreambleK) <*> v .:? (tshow MatplotlibTightBBoxK) .!= (matplotlibTightBBox def) <*> v .:? (tshow MatplotlibTransparentK) .!= (matplotlibTransparent def) - <*> v .:? (tshow MatplotlibExecutableK) .!= (matplotlibExe def) + <*> v .:? (tshow ExecutableK) .!= (matplotlibExe def) parseJSON _ = fail $ mconcat ["Could not parse ", show Matplotlib, " configuration."] instance FromJSON MatlabPrecursor where - parseJSON (Object v) = MatlabPrecursor <$> v .:? (tshow MatlabPreambleK) <*> v .:? (tshow MatlabExecutableK) .!= (matlabExe def) + parseJSON (Object v) = MatlabPrecursor <$> v .:? (tshow PreambleK) <*> v .:? (tshow ExecutableK) .!= (matlabExe def) parseJSON _ = fail $ mconcat ["Could not parse ", show Matlab, " configuration."] instance FromJSON PlotlyPythonPrecursor where - parseJSON (Object v) = PlotlyPythonPrecursor <$> v .:? (tshow PlotlyPythonPreambleK) <*> v .:? (tshow PlotlyPythonExecutableK) .!= (plotlyPythonExe def) + parseJSON (Object v) = PlotlyPythonPrecursor <$> v .:? (tshow PreambleK) <*> v .:? (tshow ExecutableK) .!= (plotlyPythonExe def) parseJSON _ = fail $ mconcat ["Could not parse ", show PlotlyPython, " configuration."] instance FromJSON MathematicaPrecursor where - parseJSON (Object v) = MathematicaPrecursor <$> v .:? (tshow MathematicaPreambleK) <*> v .:? (tshow MathematicaExecutableK) .!= (mathematicaExe def) + parseJSON (Object v) = MathematicaPrecursor <$> v .:? (tshow PreambleK) <*> v .:? (tshow ExecutableK) .!= (mathematicaExe def) parseJSON _ = fail $ mconcat ["Could not parse ", show Mathematica, " configuration."] instance FromJSON OctavePrecursor where - parseJSON (Object v) = OctavePrecursor <$> v .:? (tshow OctavePreambleK) <*> v .:? (tshow OctaveExecutableK) .!= (octaveExe def) + parseJSON (Object v) = OctavePrecursor <$> v .:? (tshow PreambleK) <*> v .:? (tshow ExecutableK) .!= (octaveExe def) parseJSON _ = fail $ mconcat ["Could not parse ", show Octave, " configuration."] instance FromJSON GGPlot2Precursor where - parseJSON (Object v) = GGPlot2Precursor <$> v .:? (tshow GGPlot2PreambleK) <*> v .:? (tshow GGPlot2ExecutableK) .!= (ggplot2Exe def) + parseJSON (Object v) = GGPlot2Precursor <$> v .:? (tshow PreambleK) <*> v .:? (tshow ExecutableK) .!= (ggplot2Exe def) parseJSON _ = fail $ mconcat ["Could not parse ", show GGPlot2, " configuration."] +instance FromJSON GNUPlotPrecursor where + parseJSON (Object v) = GNUPlotPrecursor <$> v .:? (tshow PreambleK) <*> v .:? (tshow ExecutableK) .!= (ggplot2Exe def) + parseJSON _ = fail $ mconcat ["Could not parse ", show GNUPlot, " configuration."] + instance FromJSON ConfigPrecursor where parseJSON (Null) = return def -- In case of empty file parseJSON (Object v) = do @@ -136,6 +144,7 @@ _mathematicaPrec <- v .:? (cls Mathematica) .!= def _octavePrec <- v .:? (cls Octave) .!= def _ggplot2Prec <- v .:? (cls GGPlot2) .!= def + _gnuplotPrec <- v .:? (cls GNUPlot) .!= def return $ ConfigPrecursor{..} parseJSON _ = fail "Could not parse configuration." @@ -157,6 +166,7 @@ mathematicaExe = _mathematicaExe _mathematicaPrec octaveExe = _octaveExe _octavePrec ggplot2Exe = _ggplot2Exe _ggplot2Prec + gnuplotExe = _gnuplotExe _gnuplotPrec matplotlibPreamble <- readPreamble (_matplotlibPreamble _matplotlibPrec) matlabPreamble <- readPreamble (_matlabPreamble _matlabPrec) @@ -164,6 +174,7 @@ mathematicaPreamble <- readPreamble (_mathematicaPreamble _mathematicaPrec) octavePreamble <- readPreamble (_octavePreamble _octavePrec) ggplot2Preamble <- readPreamble (_ggplot2Preamble _ggplot2Prec) + gnuplotPreamble <- readPreamble (_gnuplotPreamble _gnuplotPrec) return Configuration{..} where
src/Text/Pandoc/Filter/Plot/Parse.hs view
@@ -39,13 +39,11 @@ import System.FilePath (makeValid) import Text.Pandoc.Definition (Block (..), Inline, - Pandoc (..)) + Pandoc (..), Format(..)) import Text.Pandoc.Class (runPure) -import Text.Pandoc.Extensions (Extension (..), - extensionsFromList) import Text.Pandoc.Options (ReaderOptions (..)) -import Text.Pandoc.Readers (readMarkdown) +import Text.Pandoc.Readers (getReader, Reader(..)) import Text.Pandoc.Filter.Plot.Renderers import Text.Pandoc.Filter.Plot.Types @@ -74,13 +72,12 @@ let extraAttrs' = parseExtraAttrs toolkit attrs' header = comment toolkit $ "Generated by pandoc-plot " <> ((pack . showVersion) version) defaultPreamble = preambleSelector toolkit conf - -- Note that the default preamble changes based on the RendererM - -- which is why we use @preambleSelector@ as the default value + includeScript <- fromMaybe (return defaultPreamble) ((liftIO . TIO.readFile) <$> preamblePath) let -- Filtered attributes that are not relevant to pandoc-plot - -- This presumes that inclusionKeys includes ALL possible keys, for all renderers + -- This presumes that inclusionKeys includes ALL possible keys, for all toolkits filteredAttrs = filter (\(k, _) -> k `notElem` (tshow <$> inclusionKeys)) attrs defWithSource = defaultWithSource conf defSaveFmt = defaultSaveFormat conf @@ -112,26 +109,16 @@ plotToolkit _ = Nothing --- | Reader options for captions. -readerOptions :: ReaderOptions -readerOptions = def - {readerExtensions = - extensionsFromList - [ Ext_tex_math_dollars - , Ext_superscript - , Ext_subscript - , Ext_raw_tex - ] - } - - --- | Read a figure caption in Markdown format. LaTeX math @$...$@ is supported, --- as are Markdown subscripts and superscripts. -captionReader :: Text -> Maybe [Inline] -captionReader t = either (const Nothing) (Just . extractFromBlocks) $ runPure $ readMarkdown' t +-- | Reader a caption, based on input document format +captionReader :: Format -> Text -> Maybe [Inline] +captionReader (Format f) t = either (const Nothing) (Just . extractFromBlocks) $ runPure $ do + (reader, exts) <- getReader f + let readerOpts = def {readerExtensions = exts} + -- Assuming no ByteString readers... + case reader of + TextReader fct -> fct readerOpts t + _ -> return mempty where - readMarkdown' = readMarkdown readerOptions - extractFromBlocks (Pandoc _ blocks) = mconcat $ extractInlines <$> blocks extractInlines (Plain inlines) = inlines
src/Text/Pandoc/Filter/Plot/Renderers.hs view
@@ -25,10 +25,11 @@ , unavailableToolkits ) where -import Control.Monad (filterM) +import Control.Concurrent.ParallelIO.Local import Data.List ((\\)) import Data.Map.Strict (Map) +import Data.Maybe (catMaybes) import Data.Text (Text) import Text.Pandoc.Filter.Plot.Renderers.Mathematica @@ -37,6 +38,7 @@ import Text.Pandoc.Filter.Plot.Renderers.Octave import Text.Pandoc.Filter.Plot.Renderers.Plotly import Text.Pandoc.Filter.Plot.Renderers.GGPlot2 +import Text.Pandoc.Filter.Plot.Renderers.GNUPlot import Text.Pandoc.Filter.Plot.Types @@ -49,6 +51,7 @@ scriptExtension Mathematica = ".m" scriptExtension Octave = ".m" scriptExtension GGPlot2 = ".r" +scriptExtension GNUPlot = ".gp" -- Make a string into a comment @@ -59,6 +62,7 @@ comment Mathematica = \t -> mconcat ["(*", t, "*)"] comment Octave = mappend "% " comment GGPlot2 = mappend "# " +comment GNUPlot = mappend "# " -- | The function that maps from configuration to the preamble. @@ -69,6 +73,7 @@ preambleSelector Mathematica = mathematicaPreamble preambleSelector Octave = octavePreamble preambleSelector GGPlot2 = ggplot2Preamble +preambleSelector GNUPlot = gnuplotPreamble -- | Save formats supported by this renderer. @@ -79,6 +84,7 @@ supportedSaveFormats Mathematica = mathematicaSupportedSaveFormats supportedSaveFormats Octave = octaveSupportedSaveFormats supportedSaveFormats GGPlot2 = ggplot2SupportedSaveFormats +supportedSaveFormats GNUPlot = gnuplotSupportedSaveFormats -- Checks to perform before running a script. If ANY check fails, @@ -103,6 +109,7 @@ command Mathematica = mathematicaCommand command Octave = octaveCommand command GGPlot2 = ggplot2Command +command GNUPlot = gnuplotCommand -- | Script fragment required to capture a figure. @@ -113,6 +120,7 @@ capture Mathematica = mathematicaCapture capture Octave = octaveCapture capture GGPlot2 = ggplot2Capture +capture GNUPlot = gnuplotCapture -- | Check if a toolkit is available, based on the current configuration @@ -123,14 +131,28 @@ toolkitAvailable Mathematica = mathematicaAvailable toolkitAvailable Octave = octaveAvailable toolkitAvailable GGPlot2 = ggplot2Available +toolkitAvailable GNUPlot = gnuplotAvailable -- | List of toolkits available on this machine. -- The executables to look for are taken from the configuration. availableToolkits :: Configuration -> IO [Toolkit] -availableToolkits conf = filterM (\tk -> toolkitAvailable tk conf) toolkits - +availableToolkits conf = do + -- Certain toolkits (e.g. Python-based toolkits) + -- may take a long time to startup. + -- Therefore, we check for available of toolkits in parallel. + -- TODO: benchmark. Is this overkill? + maybeToolkits <- withPool (length toolkits) $ + \pool -> parallel pool (maybeToolkit <$> toolkits) + return $ catMaybes maybeToolkits + where + maybeToolkit tk = do + available <- toolkitAvailable tk conf + if available + then return $ Just tk + else return Nothing + -- | List of toolkits not available on this machine. -- The executables to look for are taken from the configuration. unavailableToolkits :: Configuration -> IO [Toolkit]
+ src/Text/Pandoc/Filter/Plot/Renderers/GNUPlot.hs view
@@ -0,0 +1,41 @@+{-# LANGUAGE NoImplicitPrelude #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE QuasiQuotes #-} +{-# LANGUAGE RecordWildCards #-} +{-| +Module : $header$ +Copyright : (c) Laurent P René de Cotret, 2020 +License : GNU GPL, version 2 or above +Maintainer : laurent.decotret@outlook.com +Stability : internal +Portability : portable + +Rendering gnuplot plots code blocks +-} + +module Text.Pandoc.Filter.Plot.Renderers.GNUPlot ( + gnuplotSupportedSaveFormats + , gnuplotCommand + , gnuplotCapture + , gnuplotAvailable +) where + +import Text.Pandoc.Filter.Plot.Renderers.Prelude + +gnuplotSupportedSaveFormats :: [SaveFormat] +gnuplotSupportedSaveFormats = [PNG] -- TODO: support more save formats + + +gnuplotCommand :: Configuration -> FigureSpec -> FilePath -> Text +gnuplotCommand Configuration{..} _ fp = [st|#{gnuplotExe} -c #{fp}|] + + +gnuplotAvailable :: Configuration -> IO Bool +gnuplotAvailable Configuration{..} = commandSuccess [st|#{gnuplotExe} -h|] -- TODO: test this + + +gnuplotCapture :: FigureSpec -> FilePath -> Script +gnuplotCapture FigureSpec{..} fname = [st| +set term png +set output '#{fname}' +|]
src/Text/Pandoc/Filter/Plot/Renderers/Mathematica.hs view
@@ -36,5 +36,5 @@ mathematicaCapture :: FigureSpec -> FilePath -> Script mathematicaCapture FigureSpec{..} fname = [st| -Export["#{fname}", %, show saveFormat] +Export["#{fname}", %, #{show saveFormat}] |]
src/Text/Pandoc/Filter/Plot/Renderers/Matlab.hs view
@@ -31,8 +31,17 @@ matlabCommand Configuration{..} _ fp = [st|#{matlabExe} -batch "run('#{fp}')"|] +-- On Windows at least, "matlab -help" actually returns -1, even though the +-- help text is shown successfully! +-- Therefore, we cannot rely on this behavior to know if matlab is present, +-- like other toolkits. matlabAvailable :: Configuration -> IO Bool -matlabAvailable Configuration{..} = commandSuccess [st|#{matlabExe} -h|] +matlabAvailable Configuration{..} = existsOnPath (matlabExe <> ext) + where + -- The @which@ function from Turtle only works on + -- windows if the executable extension is included. + ext = if isWindows then ".exe" else mempty + matlabCapture :: FigureSpec -> FilePath -> Script matlabCapture FigureSpec{..} fname = [st|
src/Text/Pandoc/Filter/Plot/Renderers/Prelude.hs view
@@ -18,8 +18,10 @@ , st , unpack , commandSuccess + , existsOnPath ) where +import Data.Maybe (isJust) import Data.Text (Text, unpack) import System.Exit (ExitCode(..)) import System.Process.Typed (runProcess, shell, @@ -27,6 +29,7 @@ nullStream) import Text.Shakespeare.Text (st) +import qualified Turtle as Sh import Text.Pandoc.Filter.Plot.Types @@ -39,4 +42,9 @@ $ setStdout nullStream $ setStderr nullStream $ shell (unpack s) - return $ ec == ExitSuccess+ return $ ec == ExitSuccess + + +-- | Checks that an executable is available on path, at all. +existsOnPath :: FilePath -> IO Bool +existsOnPath fp = Sh.which (Sh.fromString fp) >>= fmap isJust . return
src/Text/Pandoc/Filter/Plot/Scripting.hs view
@@ -38,7 +38,7 @@ import Text.Pandoc.Builder (fromList, imageWith, link, para, toList) -import Text.Pandoc.Definition (Block (..)) +import Text.Pandoc.Definition (Block (..), Format) import Text.Pandoc.Filter.Plot.Parse (captionReader) import Text.Pandoc.Filter.Plot.Renderers @@ -84,7 +84,12 @@ -- so that there is never any collision scriptPath <- tempScriptPath spec let captureFragment = (capture tk) spec (figurePath spec) - scriptWithCapture = mconcat [script, "\n", captureFragment] + -- Note: for gnuplot, the capture string must be placed + -- BEFORE plotting happens. Since this is only really an + -- issue for gnuplot, we have a special case. + scriptWithCapture = if (tk == GNUPlot) + then mconcat [captureFragment, "\n", script] + else mconcat [script, "\n", captureFragment] liftIO $ T.writeFile scriptPath scriptWithCapture let command_ = T.unpack $ command tk conf spec scriptPath @@ -100,8 +105,10 @@ -- | Convert a @FigureSpec@ to a Pandoc block component. -- Note that the script to generate figure files must still -- be run in another function. -toImage :: FigureSpec -> Block -toImage spec = head . toList $ para $ imageWith attrs' (T.pack target') "fig:" caption' +toImage :: Format -- ^ text format of the caption + -> FigureSpec + -> Block +toImage fmt spec = head . toList $ para $ imageWith attrs' (T.pack target') "fig:" caption' -- To render images as figures with captions, the target title -- must be "fig:" -- Janky? yes @@ -110,7 +117,7 @@ target' = figurePath spec withSource' = withSource spec srcLink = link (T.pack $ replaceExtension target' ".txt") mempty "Source code" - captionText = fromList $ fromMaybe mempty (captionReader $ caption spec) + captionText = fromList $ fromMaybe mempty (captionReader fmt $ caption spec) captionLinks = mconcat [" (", srcLink, ")"] caption' = if withSource' then captionText <> captionLinks else captionText
src/Text/Pandoc/Filter/Plot/Types.hs view
@@ -58,6 +58,7 @@ | Mathematica | Octave | GGPlot2 + | GNUPlot deriving (Bounded, Eq, Enum, Generic) -- | This instance should only be used to display toolkit names @@ -68,6 +69,7 @@ show Mathematica = "Mathematica" show Octave = "GNU Octave" show GGPlot2 = "ggplot2" + show GNUPlot = "gnuplot" -- | Class name which will trigger the filter cls :: Toolkit -> Text @@ -77,13 +79,14 @@ cls Mathematica = "mathplot" cls Octave = "octaveplot" cls GGPlot2 = "ggplot2" +cls GNUPlot = "gnuplot" type PlotM a = ReaderT PlotEnv IO a data PlotEnv - = PlotEnv { toolkit :: !Toolkit - , config :: !Configuration + = PlotEnv { toolkit :: !Toolkit + , config :: !Configuration } data Configuration = Configuration @@ -98,6 +101,7 @@ , mathematicaPreamble :: !Script , octavePreamble :: !Script , ggplot2Preamble :: !Script + , gnuplotPreamble :: !Script -- Toolkit executables , matplotlibExe :: !FilePath , matlabExe :: !FilePath @@ -105,6 +109,7 @@ , mathematicaExe :: !FilePath , octaveExe :: !FilePath , ggplot2Exe :: !FilePath + , gnuplotExe :: !FilePath -- Toolkit-specific options , matplotlibTightBBox :: !Bool , matplotlibTransparent :: !Bool @@ -123,6 +128,7 @@ , mathematicaPreamble = mempty , octavePreamble = mempty , ggplot2Preamble = mempty + , gnuplotPreamble = mempty -- Toolkit executables -- Default values are executable names as if on the PATH , matplotlibExe = if isWindows then "python" else "python3" @@ -131,6 +137,7 @@ , mathematicaExe = "math" , octaveExe = "octave" , ggplot2Exe = "Rscript" + , gnuplotExe = "gnuplot" -- Toolkit-specific , matplotlibTightBBox = False , matplotlibTransparent = False @@ -164,18 +171,6 @@ | ExecutableK | MatplotlibTightBBoxK | MatplotlibTransparentK - | MatplotlibPreambleK - | MatplotlibExecutableK - | PlotlyPythonPreambleK - | PlotlyPythonExecutableK - | MatlabPreambleK - | MatlabExecutableK - | MathematicaPreambleK - | MathematicaExecutableK - | OctavePreambleK - | OctaveExecutableK - | GGPlot2PreambleK - | GGPlot2ExecutableK deriving (Bounded, Eq, Enum) -- | Keys that pandoc-plot will look for in code blocks. @@ -190,18 +185,6 @@ show ExecutableK = "executable" show MatplotlibTightBBoxK = "tight_bbox" show MatplotlibTransparentK = "transparent" - show MatplotlibPreambleK = show PreambleK - show PlotlyPythonPreambleK = show PreambleK - show MatlabPreambleK = show PreambleK - show MathematicaPreambleK = show PreambleK - show OctavePreambleK = show PreambleK - show GGPlot2PreambleK = show PreambleK - show MatplotlibExecutableK = show ExecutableK - show MatlabExecutableK = show ExecutableK - show PlotlyPythonExecutableK = show ExecutableK - show MathematicaExecutableK = show ExecutableK - show OctaveExecutableK = show ExecutableK - show GGPlot2ExecutableK = show ExecutableK -- | List of all keys related to pandoc-plot that
tests/Common.hs view
@@ -9,6 +9,7 @@ import Data.Default.Class (def) import Data.List (isInfixOf, isSuffixOf) import Data.Monoid ((<>)) +import Data.String (fromString) import Data.Text (Text, pack, unpack) import Test.Tasty @@ -30,17 +31,20 @@ import System.FilePath (takeExtensions, (</>)) import System.IO.Temp (getCanonicalTemporaryDirectory) +-- Default caption format +defFmt = B.Format "markdown+tex_math_dollars" ------------------------------------------------------------------------------- -- Test that plot files and source files are created when the filter is run testFileCreation :: Toolkit -> TestTree testFileCreation tk = testCase "writes output files in appropriate directory" $ do - tempDir <- (</> "test-file-creation") <$> getCanonicalTemporaryDirectory + let postfix = unpack . cls $ tk + tempDir <- (</> "test-file-creation-" <> postfix) <$> getCanonicalTemporaryDirectory ensureDirectoryExistsAndEmpty tempDir let cb = (addDirectory tempDir $ codeBlock tk (trivialContent tk)) - _ <- (make tk) def cb + _ <- (make tk) def defFmt cb filesCreated <- length <$> listDirectory tempDir assertEqual "" 2 filesCreated @@ -49,12 +53,13 @@ testFileInclusion :: Toolkit -> TestTree testFileInclusion tk = testCase "includes plot inclusions" $ do - tempDir <- (</> "test-file-inclusion") <$> getCanonicalTemporaryDirectory + let postfix = unpack . cls $ tk + tempDir <- (</> "test-file-inclusion-" <> postfix) <$> getCanonicalTemporaryDirectory ensureDirectoryExistsAndEmpty tempDir - let cb = (addInclusion (include tk) $ + let cb = (addPreamble (include tk) $ addDirectory tempDir $ codeBlock tk (trivialContent tk)) - _ <- (make tk) def cb + _ <- (make tk) def defFmt cb inclusion <- readFile (include tk) sourcePath <- head . filter (isExtensionOf "txt") <$> listDirectory tempDir src <- readFile (tempDir </> sourcePath) @@ -66,25 +71,120 @@ include Mathematica = "tests/includes/mathplot.m" include Octave = "tests/includes/octave.m" include GGPlot2 = "tests/includes/ggplot2.r" + include GNUPlot = "tests/includes/gnuplot.gp" ------------------------------------------------------------------------------- -- Test that the files are saved in the appropriate format testSaveFormat :: Toolkit -> TestTree testSaveFormat tk = testCase "saves in the appropriate format" $ do - tempDir <- (</> "test-safe-format") <$> getCanonicalTemporaryDirectory + let postfix = unpack . cls $ tk + tempDir <- (</> "test-safe-format-" <> postfix) <$> getCanonicalTemporaryDirectory ensureDirectoryExistsAndEmpty tempDir let fmt = head (supportedSaveFormats tk) cb = (addSaveFormat fmt $ addDirectory tempDir $ codeBlock tk (trivialContent tk)) - _ <- (make tk) def cb + _ <- (make tk) def defFmt cb numberjpgFiles <- length <$> filter (isExtensionOf (extension fmt)) <$> listDirectory tempDir assertEqual "" numberjpgFiles 1 +------------------------------------------------------------------------------- +-- Test that it is possible to not render source links in captions +testWithSource :: Toolkit -> TestTree +testWithSource tk = + testCase "appropriately omits links to source code" $ do + let postfix = unpack . cls $ tk + tempDir <- (</> "test-caption-links-" <> postfix) <$> getCanonicalTemporaryDirectory + ensureDirectoryExistsAndEmpty tempDir + let expected = "caption content" + noSource = addWithSource False + $ addDirectory tempDir + $ addCaption expected + $ codeBlock tk (trivialContent tk) + withSource = addWithSource True + $ addDirectory tempDir + $ addCaption expected + $ codeBlock tk (trivialContent tk) + blockNoSource <- (make tk) def defFmt noSource + blockWithSource <- (make tk) def defFmt withSource + -- In the case where source=false, the caption is used verbatim. + -- Otherwise, links will be appended to the caption; hence, the caption + -- is no longer equal to the initial value + assertEqual "" (B.toList $ fromString expected) (extractCaption blockNoSource) + assertNotEqual "" (B.toList $ fromString expected) (extractCaption blockWithSource) + + where + extractCaption (B.Para blocks) = extractImageCaption . head $ blocks + extractCaption _ = mempty + + extractImageCaption (Image _ c _) = c + extractImageCaption _ = mempty + +------------------------------------------------------------------------------- +-- Test that parameters in code blocks will override the defaults in configuration +testOverrideConfiguration :: Toolkit -> TestTree +testOverrideConfiguration tk = + -- We set the default save format to JPG via the configuration, + -- but set the code block parameter to PNG. + -- Therefore, after the filter has been used, there should be one PNG file and + -- no JPG files. + testCase "code block attributes override configuration defaults" $ do + let postfix = unpack . cls $ tk + tempDir <- (</> "test-caption-links-" <> postfix) <$> getCanonicalTemporaryDirectory + ensureDirectoryExistsAndEmpty tempDir + + let config = (def::Configuration) { defaultDirectory = tempDir + , defaultSaveFormat = JPG} + + -- Not all toolkits support both save formats + when ( JPG `elem` supportedSaveFormats tk + && PNG `elem` supportedSaveFormats tk + ) $ do + + let cb = addDirectory tempDir + $ addSaveFormat PNG + $ codeBlock tk (trivialContent tk) + _ <- (make tk) config defFmt cb + + numberPngFiles <- + length <$> filter (isExtensionOf (extension PNG)) <$> + listDirectory (defaultDirectory config) + numberJpgFiles <- + length <$> filter (isExtensionOf (extension JPG)) <$> + listDirectory (defaultDirectory config) + assertEqual "" numberPngFiles 1 + assertEqual "" numberJpgFiles 0 + +------------------------------------------------------------------------------- +-- Test that Markdown formatting in captions is correctly rendered +testMarkdownFormattingCaption :: Toolkit -> TestTree +testMarkdownFormattingCaption tk = + testCase "appropriately parses captions" $ do + let postfix = unpack . cls $ tk + tempDir <- (</> "test-caption-parsing-" <> postfix) <$> getCanonicalTemporaryDirectory + ensureDirectoryExistsAndEmpty tempDir + + -- Note that this test is fragile, in the sense that the expected result must be carefully + -- constructed + let expected = [B.Strong [B.Str "caption"]] + cb = addDirectory tempDir + $ addCaption "**caption**" + $ codeBlock tk (trivialContent tk) + fmt = B.Format "markdown+tex_math_dollars" + result <- (make tk) def fmt cb + assertIsInfix expected (extractCaption result) + where + extractCaption (B.Para blocks) = extractImageCaption . head $ blocks + extractCaption _ = mempty + + extractImageCaption (Image _ c _) = c + extractImageCaption _ = mempty + + codeBlock :: Toolkit -> Script -> Block codeBlock tk script = CodeBlock (mempty, [cls tk], mempty) script @@ -96,6 +196,7 @@ trivialContent Mathematica = "\n" trivialContent Octave = "figure('visible', 'off')\nplot (-10:0.1:10);" trivialContent GGPlot2 = "library(ggplot2)\nggplot()\n" +trivialContent GNUPlot = "plot sin(x)" addCaption :: String -> Block -> Block @@ -108,8 +209,8 @@ CodeBlock (id', cls, attrs ++ [(tshow DirectoryK, pack dir)]) script -addInclusion :: FilePath -> Block -> Block -addInclusion inclusionPath (CodeBlock (id', cls, attrs) script) = +addPreamble :: FilePath -> Block -> Block +addPreamble inclusionPath (CodeBlock (id', cls, attrs) script) = CodeBlock (id', cls, attrs ++ [(tshow PreambleK, pack inclusionPath)]) script @@ -135,6 +236,13 @@ unless fileExists (assertFailure msg) where msg = mconcat ["File ", filepath, " does not exist."] + + +-- | Assert not equal +assertNotEqual :: (HasCallStack, Eq a, Show a) => String -> a -> a -> Assertion +assertNotEqual msg expected actual = + unless (expected /= actual) + (assertFailure $ mconcat [msg, ": expected ", show expected, " but got ", show actual]) -- | Not available with GHC < 8.4
tests/Main.hs view
@@ -38,6 +38,9 @@ [ testFileCreation , testFileInclusion , testSaveFormat + , testWithSource + , testOverrideConfiguration + , testMarkdownFormattingCaption ] <*> [tk]