pandoc-plot 0.3.0.0 → 0.4.0.0
raw patch · 11 files changed
+461/−135 lines, 11 filesdep +bytestringdep +githash
Dependencies added: bytestring, githash
Files
- CHANGELOG.md +10/−0
- README.md +90/−13
- executable/Main.hs +139/−81
- pandoc-plot.cabal +9/−4
- src/Text/Pandoc/Filter/Plot.hs +35/−27
- src/Text/Pandoc/Filter/Plot/Clean.hs +136/−0
- src/Text/Pandoc/Filter/Plot/Internal.hs +7/−5
- src/Text/Pandoc/Filter/Plot/Renderers.hs +1/−0
- src/Text/Pandoc/Filter/Plot/Scripting.hs +12/−5
- tests/Common.hs +21/−0
- tests/Main.hs +1/−0
CHANGELOG.md view
@@ -2,6 +2,16 @@ pandoc-plot uses [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +Release 0.4.0.0 +--------------- + +* Updated documentation. +* Added a `--full-version` flag to the executable, which includes which version of pandoc/pandoc-types was used, as well as the git revision. +* Added the `clean` command to the executable. This can be used to clean-up output files produced by pandoc-plot. +* Changed the flag `--write-example-config` to the command `write-example-config`. +* Added the top-level function `cleanOutputDir` to clean output of pandoc-plot. This is only accessible if `pandoc-plot` is used as a library. +* Added a distinction between failure to render a figure because of a mistake, and failing to render a figure because the toolkit is not installed. `pandoc-plot` will give better error messages in the latter case. + Release 0.3.0.0 ---------------
README.md view
@@ -8,23 +8,28 @@ ## Table of content -* [Usage](#usage) +* [Overview](#overview) * [Supported toolkits](#supported-toolkits) * [Features](#features) * [Captions](#captions) * [Link to source code](#link-to-source-code) * [Preamble scripts](#preamble-scripts) - * [No wasted work](#no-wasted-work) + * [Performance](#performance) * [Compatibility with pandoc-crossref](#compatibility-with-pandoc-crossref) * [Configuration](#configuration) * [Toolkit-specific options](#toolkit-specific-options) -* [Usage as a Haskell library](#usage-as-a-haskell-library) +* [Detailed usage](#detailed-usage) + * [As a filter](#as-a-filter) + * [Cleaning output](#cleaning-output) + * [Configuration template](#configuration-template) + * [As a Haskell library](#as-a-haskell-library) + * [Usage with Hakyll](#usage-with-hakyll) * [Installation](#installation) * [Warning](#warning) -## Usage +## Overview -This program is a [Pandoc](https://pandoc.org/) filter. It operates on the Pandoc abstract syntax tree, and can therefore be used in the middle of conversion from input format to output format. +This program is a [Pandoc](https://pandoc.org/) filter. It can therefore be used in the middle of conversion from input format to output format, replacing code blocks with figures. The filter recognizes code blocks with classes that match plotting toolkits. For example, using the `matplotlib` toolkit: @@ -171,10 +176,14 @@ This `preamble` parameter is perfect for longer documents with many plots. Simply define the style you want in a separate script! You can also import packages this way, or define functions you often use. -### No wasted work +### Performance -`pandoc-plot` minimizes work, only generating figures if it absolutely must, i.e. if the content has changed. Therefore, you can confidently run the filter on very large documents containing dozens of figures --- like a book or a thesis --- and only the figures which have changed will be re-generated. +`pandoc-plot` minimizes work, only generating figures if it absolutely must, i.e. if the content has changed. +`pandoc-plot` will save the hash of the source code used to generate a figure in its filename. Before generating a figure, `pandoc-plot` will check it this figure already exists based on the hash of its source! This also means that there is no way to directly name figures. + +Therefore, you can confidently run the filter on very large documents containing hundreds of figures --- like a book or a thesis --- and only the figures which have changed will be re-generated. + ### Compatibility with pandoc-crossref [`pandoc-crossref`](https://github.com/lierdakil/pandoc-crossref) is a pandoc filter that makes it effortless to cross-reference objects in Markdown documents. @@ -273,11 +282,82 @@ * `tight_bbox` is a boolean that determines whether to use `bbox_inches="tight"` or not when saving Matplotlib figures. For example, `tight_bbox: true`. See [here](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.savefig.html) for details. * `transparent` is a boolean that determines whether to make Matplotlib figure background transparent or not. This is useful, for example, for displaying a plot on top of a colored background on a web page. High-resolution figures are not affected. For example, `transparent: true`. -## Usage as a Haskell library +## Detailed usage +`pandoc-plot` is a command line executable with a few functions. You can take a look at the help using the `-h`/`--help` flag: + +```bash +$ pandoc-plot --help +pandoc-plot - generate figures directly in documents using your plotting toolkit +of choice. + +Usage: pandoc-plot.exe ([-v|--version] | [--full-version] | [-m|--manual] | + [-t|--toolkits]) [COMMAND] [AST] + This pandoc filter generates plots from code blocks using a multitude of + possible renderers. This allows to keep documentation and figures in perfect + synchronicity. + +Available options: + -v,--version Show version number and exit. + --full-version Show full version information and exit. + -m,--manual Open the manual page in the default web browser and + exit. + -t,--toolkits Show information on toolkits and exit. Executables + from the configuration file will be used, if a + '.pandoc-plot.yml' file is in the current directory. + -h,--help Show this help text + +Available commands: + clean Clean output directories where figures from FILE + might be stored. WARNING: All files in those + directories will be deleted. + write-example-config Write example configuration to a file. + +More information can be found via the manual (pandoc-plot --manual) or the repository README, located at + https://github.com/LaurentRDC/pandoc-plot +``` + +### As a filter + +The most common use for `pandoc-plot` is as a pandoc filter, in which case it should be called without arguments. For example: + +```bash +pandoc --filter pandoc-plot -i input.md -o output.html +``` + +If `pandoc-plot` fails to render a code block into a figure, the filtering will not stop. Your code blocks will stay unchanged. + +You can chain other filters with it (e.g., [`pandoc-crossref`](https://github.com/lierdakil/pandoc-crossref)) like so: + +```bash +pandoc --filter pandoc-plot --filter pandoc-crossref -i input.md -o output.html +``` + +### Cleaning output + +Figures produced by `pandoc-plot` can be placed in a few different locations. You can set a default location in the [Configuration](#configuration), but you can also re-direct specific figures in other directories if you use the `directory=...` argument in code blocks. These figures will build up over time. You can use the `clean` command to scan documents and delete the associated `pandoc-plot` output files. For example, to delete the figures generated from the `input.md` file: + +```bash +pandoc-plot clean input.md +``` + +This sill remove all directories where a figure *could* have been placed. **WARNING**: all files will be removed. + +### Configuration template + +Because `pandoc-plot` supports a few toolkits, there are a lot of configuration options. Don't start from scratch! The `write-example-config` command will create a file for you, which you can then modify: + +```bash +pandoc-plot write-example-config +``` + +You will need to re-name the file to `.pandoc-ploy.yml` to be able to use it, so don't worry about overwriting your own configuration. + +### As a Haskell library + To include the functionality of `pandoc-plot` in a Haskell package, you can use the `makePlot` function (for single blocks) or `plotTransform` function (for entire documents). [Take a look at the documentation on Hackage](https://hackage.haskell.org/package/pandoc-plot). -### Usage with Hakyll +#### Usage with Hakyll In case you want to use the filter with your own Hakyll setup, you can use a transform function that works on entire documents: @@ -294,10 +374,7 @@ pandocCompilerWithTransformM defaultHakyllReaderOptions defaultHakyllWriterOptions - (unsafeCompiler . plotTransform def fmt) - where - config = def -- Default configuration - fmt = Just "markdown" -- Document format, including extensions + (unsafeCompiler . plotTransform def) -- default configuration ``` ## Installation
executable/Main.hs view
@@ -1,11 +1,12 @@-{-# LANGUAGE ApplicativeDo #-} -{-# LANGUAGE LambdaCase #-} -{-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE ApplicativeDo #-} +{-# LANGUAGE LambdaCase #-} +{-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE RecordWildCards #-} module Main where import Control.Applicative ((<|>)) -import Control.Monad (join) +import Control.Monad (join, forM_) import Data.Default.Class (def) import Data.List (intersperse, (\\)) @@ -13,6 +14,8 @@ import qualified Data.Text as T import qualified Data.Text.IO as T +import GitHash as Git + import Options.Applicative import qualified Options.Applicative.Help.Pretty as P @@ -21,10 +24,13 @@ import Text.Pandoc.Filter.Plot (availableToolkits, plotTransform) -import Text.Pandoc.Filter.Plot.Internal (Toolkit (..), cls, Configuration(..), +import Text.Pandoc.Filter.Plot.Internal (cls, Configuration(..), supportedSaveFormats, - configuration, toolkits) + configuration, toolkits, + readDoc, cleanOutputDirs) +import Text.Pandoc (pandocVersion) +import Text.Pandoc.Definition (pandocTypesVersion) import Text.Pandoc.JSON (toJSONFilter) import Web.Browser (openBrowser) @@ -35,17 +41,98 @@ import ManPage (embedManualHtml) import ExampleConfig (embedExampleConfig) +-- It is understood that Opts Nothing Nothing should be used for filtering +data Opts = Opts + { optCommand :: Maybe Command + , optFlag :: Maybe Flag + } + +-- The difference between commands and flags is that commands perform actions, +-- while flags only display information. + +data Command = Clean FilePath + | WriteConfig + +data Flag = Version + | FullVersion + | Manual + | Toolkits + deriving (Eq) + + main :: IO () main = join $ execParser opts - where - opts = info (run <**> helper) + where + opts = info (optparse <**> helper) (fullDesc - <> progDesc "This pandoc filter generates plots from code blocks using a multitude of possible renderers. This allows to keep documentation and figures in perfect synchronicity." + <> progDesc "This pandoc filter generates plots from code blocks using a multitude of possible renderers. \ + \This allows to keep documentation and figures in perfect synchronicity." <> header "pandoc-plot - generate figures directly in documents using your plotting toolkit of choice." <> footerDoc (Just footer') ) + + optparse = do + flag_ <- flagParser + command_ <- commandParser + -- The extra optional input below only serves to show + -- to the user that the last argument is the AST from pandoc + -- The parsed input is never used + input <- optional $ strArgument (metavar "AST") + return $ go flag_ command_ input + + go :: Maybe Flag -> Maybe Command -> Maybe String -> IO () + go (Just Version) _ _ = putStrLn (V.showVersion version) + go (Just FullVersion) _ _ = showFullVersion + go (Just Manual) _ _ = showManPage + go (Just Toolkits) _ _ = showAvailableToolkits + go _ (Just (Clean fp)) _ = clean fp + go _ (Just WriteConfig) _ = T.writeFile ".example-pandoc-plot.yml" exampleConfig + go Nothing Nothing _ = toJSONFilterWithConfig +flagParser :: Parser (Maybe Flag) +flagParser = versionP <|> fullVersionP <|> manualP <|> toolkitsP + where + versionP = flag Nothing (Just Version) (mconcat + [ long "version" + , short 'v' + , help "Show version number and exit." + ]) + + fullVersionP = flag Nothing (Just FullVersion) (mconcat + [ long "full-version" + , help "Show full version information and exit." + ]) + manualP = flag Nothing (Just Manual) (mconcat + [ long "manual" + , short 'm' + , help "Open the manual page in the default web browser and exit." + ]) + + toolkitsP = flag Nothing (Just Toolkits) (mconcat + [ long "toolkits" + , short 't' + , help "Show information on toolkits and exit. Executables from the configuration \ + \file will be used, if a '.pandoc-plot.yml' file is in the current directory." + ]) + +commandParser :: Parser (Maybe Command) +commandParser = optional $ subparser ( + command "clean" ( + info (cleanP <**> helper) ( + progDesc "Clean output directories where figures from FILE might be stored.\ + \ WARNING: All files in those directories will be deleted." + ) + ) + <> command "write-example-config" ( + info (writeConfigP <**> helper) (progDesc "Write example configuration to a file.") + ) + ) + where + cleanP = Clean <$> strArgument (metavar "FILE") + writeConfigP = pure WriteConfig + + toJSONFilterWithConfig :: IO () toJSONFilterWithConfig = do c <- config @@ -60,91 +147,62 @@ else return def -data Flag = Version - | Manual - | Toolkits - | Config - deriving (Eq) +manualHtml :: T.Text +manualHtml = T.pack $(embedManualHtml) -run :: Parser (IO ()) -run = do - versionP <- flag Nothing (Just Version) (mconcat - [ long "version" - , short 'v' - , help "Show version number and exit." - ]) +exampleConfig :: T.Text +exampleConfig = T.pack $(embedExampleConfig) - manualP <- flag Nothing (Just Manual) (mconcat - [ long "manual" - , short 'm' - , help "Open the manual page in the default web browser and exit." - ]) - toolkitsP <- flag Nothing (Just Toolkits) (mconcat - [ long "toolkits" - , short 't' - , help "Show information on toolkits and exit. Executables from the configuration \ - \file will be used, if a '.pandoc-plot.yml' file is in the current directory." - ]) - - configP <- flag Nothing (Just Config) (mconcat - [ long "write-example-config" - , help "Write an example configuration in '.pandoc-plot.yml', \ - \which you can subsequently customize, and exit. If '.pandoc-plot.yml' \ - \already exists, an error will be thrown. "]) - - input <- optional $ strArgument (metavar "AST") - return $ go (versionP <|> manualP <|> toolkitsP <|> configP) input - where - go :: Maybe Flag -> Maybe String -> IO () - go (Just Version) _ = putStrLn (V.showVersion version) - go (Just Manual) _ = writeSystemTempFile "pandoc-plot-manual.html" (T.unpack manualHtml) - >>= \fp -> openBrowser ("file:///" <> fp) - >> return () - go (Just Toolkits) _ = do - c <- config - putStrLn "\nAVAILABLE TOOLKITS\n" - available <- availableToolkits c - return available >>= mapM_ toolkitInfo - putStrLn "\nUNAVAILABLE TOOLKITS\n" - -- We don't use unavailableToolkits because this would force - -- more IO actions - let unavailable = toolkits \\ available - return unavailable >>= mapM_ toolkitInfo - go (Just Config) _ = T.writeFile ".example-pandoc-plot.yml" exampleConfig - - go Nothing _ = toJSONFilterWithConfig +showFullVersion :: IO () +showFullVersion = do + putStrLn $ "pandoc-plot " <> (V.showVersion version) + putStrLn $ "Git revision " <> (Git.giHash $$tGitInfoCwd) + putStrLn $ mconcat [ "Compiled with pandoc " + , (T.unpack pandocVersion) + , " and pandoc-types " + , V.showVersion pandocTypesVersion + ] -manualHtml :: T.Text -manualHtml = T.pack $(embedManualHtml) +showAvailableToolkits :: IO () +showAvailableToolkits = do + c <- config + putStrLn "\nAVAILABLE TOOLKITS\n" + available <- availableToolkits c + return available >>= mapM_ toolkitInfo + putStrLn "\nUNAVAILABLE TOOLKITS\n" + -- We don't use unavailableToolkits because this would force + -- more IO actions + let unavailable = toolkits \\ available + return unavailable >>= mapM_ toolkitInfo + where + toolkitInfo tk = do + putStrLn $ "Toolkit: " <> show tk + putStrLn $ " Code block trigger: " <> (T.unpack . cls $ tk) + putStrLn $ " Supported save formats: " <> (mconcat . intersperse ", " . fmap show $ supportedSaveFormats tk) + putStrLn mempty -exampleConfig :: T.Text -exampleConfig = T.pack $(embedExampleConfig) +clean :: FilePath -> IO () +clean fp = do + conf <- config + putStrLn $ "Cleaning output directories for " <> fp + cleanedDirs <- readDoc fp >>= cleanOutputDirs conf + forM_ cleanedDirs $ \d -> putStrLn $ "Removed directory " <> d -toolkitInfo :: Toolkit -> IO () -toolkitInfo tk = do - putStrLn $ "Toolkit: " <> show tk - putStrLn $ " Code block trigger: " <> (T.unpack . cls $ tk) - putStrLn $ " Supported save formats: " <> (mconcat . intersperse ", " . fmap show $ supportedSaveFormats tk) - putStrLn mempty - +showManPage :: IO () +showManPage = + writeSystemTempFile "pandoc-plot-manual.html" (T.unpack manualHtml) + >>= \fp -> openBrowser ("file:///" <> fp) + >> return () -- | Use Doc type directly because of newline formatting footer' :: P.Doc -footer' = mconcat [ - P.text "Example usage with pandoc:" - , P.line, P.line - , P.indent 4 $ P.string "> pandoc --filter pandoc-plot input.md --output output.html" - , P.line, P.line - , P.text "If you use pandoc-plot in combination with other filters, you probably want to run pandoc-plot first. Here is an example with pandoc-crossref:" - , P.line, P.line - , P.indent 4 $ P.string "> pandoc --filter pandoc-plot --filter pandoc-crossref -i input.md -o output.pdf" - , P.line, P.line - , P.text "More information can be found via the manual (pandoc-plot --manual) or the repository README, located at" +footer' = mconcat + [ P.text "More information can be found via the manual (pandoc-plot --manual) or the repository README, located at" , P.line , P.indent 4 $ P.text "https://github.com/LaurentRDC/pandoc-plot" , P.line
pandoc-plot.cabal view
@@ -1,5 +1,5 @@ name: pandoc-plot -version: 0.3.0.0 +version: 0.4.0.0 cabal-version: >= 1.12 synopsis: A Pandoc filter to include figures generated from code blocks using your plotting toolkit of choice. description: A Pandoc filter to include figures generated from code blocks. Keep the document and code in the same location. Output is captured and included as a figure. @@ -28,6 +28,7 @@ Text.Pandoc.Filter.Plot.Internal other-modules: Paths_pandoc_plot + Text.Pandoc.Filter.Plot.Clean Text.Pandoc.Filter.Plot.Configuration Text.Pandoc.Filter.Plot.Types Text.Pandoc.Filter.Plot.Parse @@ -43,9 +44,12 @@ Text.Pandoc.Filter.Plot.Renderers.GNUPlot hs-source-dirs: src - ghc-options: -Wall -Wcompat + ghc-options: + -Wall + -Wcompat build-depends: base >= 4.11 && <5 + , bytestring , containers , directory , data-default-class >= 0.1.2 && < 0.2 @@ -73,16 +77,17 @@ executable ghc-options: -Wall -Wcompat -rtsopts -threaded -with-rtsopts=-N build-depends: - base >=4.11 && <5 + base >= 4.11 && <5 , directory , data-default-class >= 0.1.2 , deepseq , filepath + , githash >= 0.1.3.0 && < 1 , open-browser >= 0.2.1.0 , optparse-applicative >= 0.14 && < 1 , pandoc , pandoc-plot - , pandoc-types >1.12 && <2 + , pandoc-types > 1.12 && <2 , template-haskell > 2.7 && < 3 , temporary , text
src/Text/Pandoc/Filter/Plot.hs view
@@ -8,11 +8,11 @@ Stability : unstable Portability : portable -This module defines a Pandoc filter @makePlot@ and related functions +This module defines a Pandoc filter @plotTransform@ and related functions that can be used to walk over a Pandoc document and generate figures from -code blocks using a multitude of plotting toolkits. +code blocks, using a multitude of plotting toolkits. -The syntax for code blocks is simple, Code blocks with the appropriate class +The syntax for code blocks is simple. Code blocks with the appropriate class attribute will trigger the filter: * @matplotlib@ for matplotlib-based Python plots; @@ -23,6 +23,17 @@ * @ggplot2@ for ggplot2-based R plots; * @gnuplot@ for gnuplot plots; +For example, in Markdown: + +@ + This is a paragraph. + + ```{.matlabplot} + figure() + plot([1,2,3,4,5], [1,2,3,4,5], '-k) + ``` +@ + The code block will be reworked into a script and the output figure will be captured. Optionally, the source code used to generate the figure will be linked in the caption. @@ -31,7 +42,7 @@ * @directory=...@ : Directory where to save the figure. * @source=true|false@ : Whether or not to link the source code of this figure in the caption. Ideal for web pages, for example. Default is false. * @format=...@: Format of the generated figure. This can be an extension or an acronym, e.g. @format=PNG@. - * @caption="..."@: Specify a plot caption (or alternate text). Captions should be formatted in Markdown, with LaTeX math. + * @caption="..."@: Specify a plot caption (or alternate text). Format for captions is specified in the documentation for the @Configuration@ type. * @dpi=...@: Specify a value for figure resolution, or dots-per-inch. Certain toolkits ignore this. * @preamble=...@: Path to a file to include before the code block. Ideal to avoid repetition over many figures. @@ -41,7 +52,7 @@ Here is an example code block which will render a figure using gnuplot, in Markdown: @ - ```{.gnuplot format=png caption="Sinusoidal function"} + ```{.gnuplot format=png caption="Sinusoidal function" source=true} sin(x) set xlabel "x" @@ -54,13 +65,16 @@ makePlot -- * Operating on whole Pandoc documents , plotTransform + -- * Cleaning output directories + , cleanOutputDirs -- * Runtime configuration , configuration , Configuration(..) , SaveFormat(..) , Script - -- * For testing purposes ONLY + -- * For testing and internal purposes ONLY , make + , readDoc , availableToolkits , unavailableToolkits ) where @@ -77,11 +91,12 @@ -- | Highest-level function that can be walked over a Pandoc tree. --- All code blocks that have the @.plot@ / @.plotly@ class will be considered --- figures. +-- All code blocks that have the appropriate class names will be considered +-- figures, e.g. @.matplotlib@. -- --- The target document format determines how the figure captions should be parsed. --- By default (i.e. if Nothing), captions will be parsed as Markdown with LaTeX math @$...$@, +-- Failing to render a figure does not stop the filter, so that you may run the filter +-- on documents without having all necessary toolkits installed. In this case, error +-- messages are printed to stderr, and blocks are left unchanged. makePlot :: Configuration -- ^ Configuration for default values -> Block -> IO Block @@ -89,7 +104,11 @@ -- | Walk over an entire Pandoc document, changing appropriate code blocks --- into figures. Default configuration is used. +-- into figures. +-- +-- Failing to render a figure does not stop the filter, so that you may run the filter +-- on documents without having all necessary toolkits installed. In this case, error +-- messages are printed to stderr, and blocks are left unchanged. plotTransform :: Configuration -- ^ Configuration for default values -> Pandoc -- ^ Input document -> IO Pandoc @@ -116,22 +135,11 @@ where handleResult spec ScriptSuccess = return $ toImage (captionFormat conf) spec handleResult _ (ScriptChecksFailed msg) = do - liftIO $ hPutStrLn stderr $ "pandoc-plot: The script check failed with message: " <> msg + liftIO $ hPutStrLn stderr $ " ERROR (pandoc-plot) The script check failed with message: " <> msg return blk handleResult _ (ScriptFailure _ code) = do - liftIO $ hPutStrLn stderr $ "pandoc-plot: The script failed with exit code " <> show code + liftIO $ hPutStrLn stderr $ "ERROR (pandoc-plot) The script failed with exit code " <> show code return blk - - --- | Possible errors returned by the filter -data PandocPlotError - = ScriptError String Int -- ^ Running script has yielded an error - | ScriptChecksFailedError String -- ^ Script did not pass all checks - deriving (Eq) - -instance Show PandocPlotError where - show (ScriptError cmd exitcode) = mconcat [ "Script error: plot could not be generated.\n" - , " Command: ", cmd, "\n" - , " Exit code " <> (show exitcode) - ] - show (ScriptChecksFailedError msg) = "Script did not pass all checks: " <> msg+ handleResult _ (ToolkitNotInstalled tk') = do + liftIO $ hPutStrLn stderr $ "ERROR (pandoc-plot) The " <> show tk' <> " toolkit is required but not installed." + return blk
+ src/Text/Pandoc/Filter/Plot/Clean.hs view
@@ -0,0 +1,136 @@+{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE FlexibleContexts #-} + +{-| +Module : $header$ +Copyright : (c) Laurent P René de Cotret, 2020 +License : GNU GPL, version 2 or above +Maintainer : laurent.decotret@outlook.com +Stability : internal +Portability : portable + +Utilities to clean pandoc-plot output directories. +-} + +module Text.Pandoc.Filter.Plot.Clean ( + cleanOutputDirs + , readDoc +) where + + +import Control.Monad.Reader (runReaderT, forM, liftIO) + +import qualified Data.ByteString.Lazy as B +import Data.Char (toLower) +import Data.Default.Class (def) +import Data.List (nub) +import Data.Maybe (fromMaybe, catMaybes) + +import Data.Text (Text) +import qualified Data.Text.IO as Text + +import System.Directory (removePathForcibly) + +import System.FilePath (takeExtension) + +import Text.Pandoc.Class (runIO) +import Text.Pandoc.Definition +import Text.Pandoc.Error (handleError) +import qualified Text.Pandoc.Readers as P +import qualified Text.Pandoc.Options as P +import Text.Pandoc.Walk (query, Walkable) + +import Text.Pandoc.Filter.Plot.Parse +import Text.Pandoc.Filter.Plot.Types + + +-- | Clean all output related to pandoc-plot. This includes output directories specified +-- in the configuration and in the document/block. Note that *all* files in pandoc-plot output +-- directories will be removed. +-- +-- The cleaned directories are returned. +cleanOutputDirs :: Walkable Block b => Configuration -> b -> IO [FilePath] +cleanOutputDirs conf doc = do + directories <- sequence $ query (\b -> [outputDir b]) doc + forM (nub . catMaybes $ directories) removeDir + where + outputDir b = + maybe + (return Nothing) + (\tk -> runReaderT (parseFigureSpec b >>= return . fmap directory) (PlotEnv tk conf)) + (plotToolkit b) + + removeDir :: FilePath -> IO FilePath + removeDir d = removePathForcibly d >> return d + + +-- | Read document, guessing what extensions and reader options are appropriate. If +-- the file cannot be read for any reason, an error is thrown. +readDoc :: FilePath -> IO Pandoc +readDoc fp = handleError =<< (runIO $ do + let fmt = fromMaybe mempty (formatFromFilePath fp) + (reader, exts) <- P.getReader fmt + let readerOpts = def {P.readerExtensions = exts} + case reader of + P.TextReader fct -> do + t <- liftIO $ Text.readFile fp + fct readerOpts t + P.ByteStringReader bst -> do + b <- liftIO $ B.readFile fp + bst readerOpts b) + + +-- Determine format based on file extension +-- Note : this is exactly the heuristic used by pandoc here: +-- https://github.com/jgm/pandoc/blob/master/src/Text/Pandoc/App/FormatHeuristics.hs +-- +-- However, this is not exported, so I must re-define it here. +formatFromFilePath :: FilePath -> Maybe Text +formatFromFilePath x = + case takeExtension (map toLower x) of + ".adoc" -> Just "asciidoc" + ".asciidoc" -> Just "asciidoc" + ".context" -> Just "context" + ".ctx" -> Just "context" + ".db" -> Just "docbook" + ".doc" -> Just "doc" -- so we get an "unknown reader" error + ".docx" -> Just "docx" + ".dokuwiki" -> Just "dokuwiki" + ".epub" -> Just "epub" + ".fb2" -> Just "fb2" + ".htm" -> Just "html" + ".html" -> Just "html" + ".icml" -> Just "icml" + ".json" -> Just "json" + ".latex" -> Just "latex" + ".lhs" -> Just "markdown+lhs" + ".ltx" -> Just "latex" + ".markdown" -> Just "markdown" + ".md" -> Just "markdown" + ".ms" -> Just "ms" + ".muse" -> Just "muse" + ".native" -> Just "native" + ".odt" -> Just "odt" + ".opml" -> Just "opml" + ".org" -> Just "org" + ".pdf" -> Just "pdf" -- so we get an "unknown reader" error + ".pptx" -> Just "pptx" + ".roff" -> Just "ms" + ".rst" -> Just "rst" + ".rtf" -> Just "rtf" + ".s5" -> Just "s5" + ".t2t" -> Just "t2t" + ".tei" -> Just "tei" + ".tei.xml" -> Just "tei" + ".tex" -> Just "latex" + ".texi" -> Just "texinfo" + ".texinfo" -> Just "texinfo" + ".text" -> Just "markdown" + ".textile" -> Just "textile" + ".txt" -> Just "markdown" + ".wiki" -> Just "mediawiki" + ".xhtml" -> Just "html" + ".ipynb" -> Just "ipynb" + ".csv" -> Just "csv" + ['.',y] | y `elem` ['1'..'9'] -> Just "man" + _ -> Nothing
src/Text/Pandoc/Filter/Plot/Internal.hs view
@@ -17,10 +17,12 @@ , module Text.Pandoc.Filter.Plot.Scripting , module Text.Pandoc.Filter.Plot.Parse , module Text.Pandoc.Filter.Plot.Configuration + , module Text.Pandoc.Filter.Plot.Clean ) where -import Text.Pandoc.Filter.Plot.Parse -import Text.Pandoc.Filter.Plot.Renderers -import Text.Pandoc.Filter.Plot.Scripting -import Text.Pandoc.Filter.Plot.Types -import Text.Pandoc.Filter.Plot.Configuration+import Text.Pandoc.Filter.Plot.Parse +import Text.Pandoc.Filter.Plot.Renderers +import Text.Pandoc.Filter.Plot.Scripting +import Text.Pandoc.Filter.Plot.Types +import Text.Pandoc.Filter.Plot.Configuration +import Text.Pandoc.Filter.Plot.Clean
src/Text/Pandoc/Filter/Plot/Renderers.hs view
@@ -21,6 +21,7 @@ , parseExtraAttrs , command , capture + , toolkitAvailable , availableToolkits , unavailableToolkits ) where
src/Text/Pandoc/Filter/Plot/Scripting.hs view
@@ -48,8 +48,9 @@ -- | Possible result of running a script data ScriptResult = ScriptSuccess - | ScriptChecksFailed String -- Message - | ScriptFailure String Int -- Command and exit code + | ScriptChecksFailed String -- Message + | ScriptFailure String Int -- Command and exit code + | ToolkitNotInstalled Toolkit -- Script failed because toolkit is not installed -- Run script as described by the spec, only if necessary runScriptIfNecessary :: FigureSpec -> PlotM ScriptResult @@ -63,8 +64,7 @@ case result of ScriptSuccess -> liftIO $ T.writeFile (sourceCodePath spec) (script spec) >> return ScriptSuccess - ScriptFailure cmd code -> return $ ScriptFailure cmd code - ScriptChecksFailed msg -> return $ ScriptChecksFailed msg + other -> return other -- Run script as described by the spec @@ -99,7 +99,14 @@ $ shell command_ case ec of ExitSuccess -> return ScriptSuccess - ExitFailure code -> return $ ScriptFailure command_ code + ExitFailure code -> do + -- Two possible types of failures: either the script + -- failed because the toolkit was not available, or + -- because of a genuine error + toolkitInstalled <- liftIO $ toolkitAvailable tk conf + if toolkitInstalled + then return $ ScriptFailure command_ code + else return $ ToolkitNotInstalled tk -- | Convert a @FigureSpec@ to a Pandoc block component.
tests/Common.hs view
@@ -207,6 +207,27 @@ extractImageCaption _ = mempty +------------------------------------------------------------------------------- +-- Test that cleanOutpuDirs correctly cleans the output directory specified in a block. +testCleanOutputDirs :: Toolkit -> TestTree +testCleanOutputDirs tk = + testCase "correctly cleans output directories" $ do + let postfix = unpack . cls $ tk + tempDir <- (</> "test-clean-output-dir" <> postfix) <$> getCanonicalTemporaryDirectory + ensureDirectoryExistsAndEmpty tempDir + + let cb = addDirectory tempDir + $ codeBlock tk (trivialContent tk) + + result <- (make tk) def cb + cleanedDirs <- cleanOutputDirs def cb + + assertEqual "" [tempDir] cleanedDirs + + outputDirExists <- doesDirectoryExist tempDir + assertEqual "" outputDirExists False + + codeBlock :: Toolkit -> Script -> Block codeBlock tk script = CodeBlock (mempty, [cls tk], mempty) script
tests/Main.hs view
@@ -49,6 +49,7 @@ , testOverrideConfiguration , testMarkdownFormattingCaption1 , testMarkdownFormattingCaption2 + , testCleanOutputDirs ] <*> [tk]