pandoc-pyplot 2.1.2.0 → 2.1.3.0
raw patch · 10 files changed
+215/−89 lines, 10 filesdep +optparse-applicative
Dependencies added: optparse-applicative
Files
- CHANGELOG.md +16/−0
- README.md +28/−13
- executable/Main.hs +78/−63
- pandoc-pyplot.cabal +7/−6
- src/Text/Pandoc/Filter/Pyplot.hs +10/−2
- src/Text/Pandoc/Filter/Pyplot/Configuration.hs +22/−2
- src/Text/Pandoc/Filter/Pyplot/FigureSpec.hs +2/−1
- src/Text/Pandoc/Filter/Pyplot/Types.hs +21/−1
- stack.yaml +0/−1
- test/Main.hs +31/−0
CHANGELOG.md view
@@ -2,6 +2,22 @@ pandoc-pyplot uses [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +Release 2.1.3.0 +--------------- + +* Switched to using [optparse-applicative](https://github.com/pcapriotti/optparse-applicative#arguments) for command-line argument parsing. +* Added a command-line options, "--write-example-config", which will write a config file ".pandoc-pyplot.yml" to show all available configuration options. +* Links to source code and high-res images can be suppressed using `{.pyplot with-links=false ...}` (or via the configuration file with `with-links: false`). This is to get cleaner output in technical documentation (e.g. PDF). Example: + + ```markdown + ```{.pyplot caption="This is a caption" with-links=false} + import matplotlib.pyplot as plt + plt.figure() + plt.plot([1,2,3,4,5],[1,2,3,4,5]) + ``` + ``` +* Added automated builds on macOS and Linux via Azure-Pipelines. Windows build will stay on Appveyor for now. + Release 2.1.2.0 ---------------
README.md view
@@ -1,6 +1,7 @@ # pandoc-pyplot - A Pandoc filter to generate Matplotlib figures directly in documents -[](http://hackage.haskell.org/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot)  +[](http://hackage.haskell.org/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](http://stackage.org/nightly/package/pandoc-pyplot) [](https://ci.appveyor.com/project/LaurentRDC/pandoc-pyplot) [](https://dev.azure.com/laurentdecotret/pandoc-pyplot/_build/latest?definitionId=2&branchName=master) + `pandoc-pyplot` turns Python code present in your documents into embedded Matplotlib figures. @@ -44,14 +45,6 @@ ## Features -### No wasted work - -`pandoc-pyplot` minimizes work, only generating figures if it absolutely must. Therefore, you can confidently run the filter on very large documents containing dozens of figures --- like a book or a thesis --- and only the figures which have recently changed will be re-generated. - -### Link to source code and high-resolution figure - -In case of an output format that supports links (e.g. HTML), the embedded image generated by `pandoc-pyplot` will be a link to the source code which was used to generate the file. Therefore, other people can see what Python code was used to create your figures. A high resolution image will be made available in a caption link. - ### Captions You can also specify a caption for your image. This is done using the optional `caption` parameter: @@ -68,6 +61,20 @@ Caption formatting is either plain text or Markdown. LaTeX-style math is also support in captions (using dollar signs $...$). +### Link to source code and high-resolution figure + +In case of an output format that supports links (e.g. HTML), the embedded image generated by `pandoc-pyplot` will be a link to the source code which was used to generate the file. Therefore, other people can see what Python code was used to create your figures. A high resolution image will be made available in a caption link. + +(*New in version 2.1.3.0*) For cleaner output (e.g. PDF), you can turn this off via the `with-links=false` key: + +```markdown + ```{.pyplot with-links=false} + ... + ``` +``` + +or via a [configuration file](#Configurable). + ### Including scripts If you find yourself always repeating some steps, inclusion of scripts is possible using the `include` parameter. For example, if you want all plots to have the [`ggplot`](https://matplotlib.org/tutorials/introductory/customizing.html#sphx-glr-tutorials-introductory-customizing-py) style, you can write a very short preamble `style.py` like so: @@ -102,6 +109,10 @@ This `include` parameter is perfect for longer documents with many plots. Simply define the style you want in a separate script! You can also import packages this way, or define functions you often use. +### No wasted work + +`pandoc-pyplot` minimizes work, only generating figures if it absolutely must. Therefore, you can confidently run the filter on very large documents containing dozens of figures --- like a book or a thesis --- and only the figures which have recently changed will be re-generated. + ### Compatibility with pandoc-crossref [`pandoc-crossref`](https://github.com/lierdakil/pandoc-crossref) is a pandoc filter that makes it effortless to cross-reference objects on documents. @@ -124,9 +135,7 @@ ### Configurable -*New in version 2.1.0.0* - -To avoid repetition, `pandoc-pyplot` can be configured using simple YAML syntax. `pandoc-pyplot` will look for a `.pandoc-pyplot.yml` file in the current working directory: +(*New in version 2.1.0.0*) To avoid repetition, `pandoc-pyplot` can be configured using simple YAML files. `pandoc-pyplot` will look for a `.pandoc-pyplot.yml` file in the current working directory: ```yaml # You can specify any or all of the following parameters @@ -134,6 +143,7 @@ directory: mydirectory/ include: mystyle.py format: jpeg +with-links: false dpi: 150 flags: [-O, -Wignore] ``` @@ -147,16 +157,21 @@ flags: [] directory: generated/ format: png +with-links: true dpi: 80 ``` +Using `pandoc-pyplot --write-example-config` will write the default configuration to a file `.pandoc-pyplot.yml`, which you can then customize. + ## Installation ### Binaries Windows binaries are available on [GitHub](https://github.com/LaurentRDC/pandoc-pyplot/releases). Place the executable in a location that is in your PATH to be able to call it. -### Installers +If you can show me how to generate binaries for other platform using e.g. Azure Pipelines, let me know! + +### Installers (Windows) Windows installers are made available thanks to [Inno Setup](http://www.jrsoftware.org/isinfo.php). You can download them from the [release page](https://github.com/LaurentRDC/pandoc-pyplot/releases/latest).
executable/Main.hs view
@@ -1,18 +1,25 @@ {-# LANGUAGE LambdaCase #-} {-# LANGUAGE TemplateHaskell #-} +{-# LANGUAGE ApplicativeDo #-} module Main where +import Control.Applicative ((<|>)) +import Control.Monad (join) + import Data.Default.Class (def) import Data.List (intersperse) import Data.Monoid ((<>)) import qualified Data.Text as T -import System.Environment (getArgs) +import Options.Applicative +import qualified Options.Applicative.Help.Pretty as P + import System.Directory (doesFileExist) import System.IO.Temp (writeSystemTempFile) import Text.Pandoc.Filter.Pyplot (plotTransformWithConfig, configuration, SaveFormat(..)) +import Text.Pandoc.Filter.Pyplot.Internal (writeConfig) import Text.Pandoc.JSON (toJSONFilter) import Web.Browser (openBrowser) @@ -22,75 +29,83 @@ import ManPage (embedManualHtml) -supportedSaveFormats :: [SaveFormat] -supportedSaveFormats = enumFromTo minBound maxBound +main :: IO () +main = join $ execParser opts + where + opts = info (run <**> helper) + (fullDesc + <> progDesc "This pandoc filter generates plots from Python code blocks using Matplotlib. This allows to keep documentation and figures in perfect synchronicity." + <> header "pandoc-pyplot - generate Matplotlib figures directly in documents." + <> footerDoc (Just footer') + ) + -manualHtml :: T.Text -manualHtml = T.pack $(embedManualHtml) +toJSONFilterWithConfig :: IO () +toJSONFilterWithConfig = do + configExists <- doesFileExist ".pandoc-pyplot.yml" + config <- if configExists + then configuration ".pandoc-pyplot.yml" + else def + toJSONFilter (plotTransformWithConfig config) -data Flag = Help - | Version + +data Flag = Version | Formats | Manual + | Config deriving (Eq) -parseFlag :: [String] -> Maybe Flag -parseFlag s - | head s `elem` ["-h", "--help"] = Just Help - | head s `elem` ["-v", "--version"] = Just Version - | head s `elem` ["-f", "--formats"] = Just Formats - | head s `elem` ["-m", "--manual"] = Just Manual - | otherwise = Nothing -- This is the regular input from pandoc -flagAction :: Flag -> IO () -flagAction f - | f == Help = showHelp - | f == Version = showVersion - | f == Formats = showFormats - | f == Manual = showManual - | otherwise = error "Unknown flag" - where - showHelp = putStrLn help - showVersion = putStrLn (V.showVersion version) - showFormats = putStrLn . mconcat . intersperse ", " . fmap show $ supportedSaveFormats - showManual = writeSystemTempFile "pandoc-pyplot-manual.html" (T.unpack manualHtml) - >>= \fp -> openBrowser ("file:///" <> fp) >> return () +run :: Parser (IO ()) +run = do + versionP <- flag Nothing (Just Version) (long "version" <> short 'v' + <> help "Show version number and exit.") -help :: String -help = - "\n\ - \\n\ - \ usage: pandoc-pyplot [-h, --help] [-v, --version] [-f, --formats] [-m, --manual]\n\ - \\n\ - \ This pandoc filter generates plots from Python code blocks using Matplotlib.\n\ - \ This allows to keep documentation and figures in perfect synchronicity.\n\ - \\n\ - \ Optional arguments:\n\ - \ -h, --help Show this help message and exit.\n\ - \ -v, --version Show version number and exit.\n\ - \ -f, --formats Show supported output figure formats and exit.\n\ - \ -m, --manual Open the manual page in the default web browser and exit.\n\ - \\n\ - \ Example usage with pandoc: \n\ - \\n\ - \ > pandoc --filter pandoc-pyplot input.md --output output.html\n\ - \\n\ - \ If you use pandoc-pyplot in combination with other filters, you probably want\n\ - \ to run pandoc-pyplot first. Here is an example with pandoc-crossref: \n\ - \\n\ - \ > pandoc --filter pandoc-pyplot --filter pandoc-crossref -i input.md -o output.pdf\n\ - \\n\ - \ More information can be found via the manual (pandoc-pyplot --manual) or the\n\ - \ repository README, located at \n\ - \ https://github.com/LaurentRDC/pandoc-pyplot\n" + formatsP <- flag Nothing (Just Formats) (long "formats" <> short 'f' + <> help "Show supported output figure formats and exit.") -main :: IO () -main = do - configExists <- doesFileExist ".pandoc-pyplot.yml" - config <- if configExists - then configuration ".pandoc-pyplot.yml" - else def + manualP <- flag Nothing (Just Manual) (long "manual" <> short 'm' + <> help "Open the manual page in the default web browser and exit.") - getArgs >>= \args -> case parseFlag args of - Just f -> flagAction f - Nothing -> toJSONFilter (plotTransformWithConfig config)+ configP <- flag Nothing (Just Config) (long "write-example-config" + <> help "Write the default configuration in '.pandoc-pyplot.yml', \ + \which you can subsequently customize, and exit. If '.pandoc-pyplot.yml' \ + \already exists, an error will be thrown. ") + + input <- optional $ strArgument (metavar "AST") + return $ go (versionP <|> formatsP <|> manualP <|> configP) input + where + go :: Maybe Flag -> Maybe String -> IO () + go (Just Version) _ = putStrLn (V.showVersion version) + go (Just Formats) _ = putStrLn . mconcat . intersperse ", " . fmap show $ supportedSaveFormats + go (Just Manual) _ = writeSystemTempFile "pandoc-pyplot-manual.html" (T.unpack manualHtml) + >>= \fp -> openBrowser ("file:///" <> fp) + >> return () + go (Just Config) _ = writeConfig ".pandoc-pyplot.yml" def + go Nothing _ = toJSONFilterWithConfig + + +supportedSaveFormats :: [SaveFormat] +supportedSaveFormats = enumFromTo minBound maxBound + + +manualHtml :: T.Text +manualHtml = T.pack $(embedManualHtml) + + +-- | Use Doc type directly because of newline formatting +footer' :: P.Doc +footer' = mconcat [ + P.text "Example usage with pandoc:" + , P.line, P.line + , P.indent 4 $ P.string "> pandoc --filter pandoc-pyplot input.md --output output.html" + , P.line, P.line + , P.text "If you use pandoc-pyplot in combination with other filters, you probably want to run pandoc-pyplot first. Here is an example with pandoc-crossref:" + , P.line, P.line + , P.indent 4 $ P.string "> pandoc --filter pandoc-pyplot --filter pandoc-crossref -i input.md -o output.pdf" + , P.line, P.line + , P.text "More information can be found via the manual (pandoc-pyplot --manual) or the repository README, located at" + , P.line + , P.indent 4 $ P.text "https://github.com/LaurentRDC/pandoc-pyplot" + , P.line + ]
pandoc-pyplot.cabal view
@@ -1,5 +1,5 @@ name: pandoc-pyplot -version: 2.1.2.0 +version: 2.1.3.0 cabal-version: >= 1.12 synopsis: A Pandoc filter to include figures generated from Python code blocks description: A Pandoc filter to include figures generated from Python code blocks. Keep the document and Python code in the same location. Output from Matplotlib is captured and included as a figure. @@ -59,16 +59,17 @@ executable ghc-options: -Wall -Wcompat -rtsopts -threaded -with-rtsopts=-N build-depends: - base >=4 && <5 + base >=4 && <5 , directory - , data-default-class >= 0.1.2 + , data-default-class >= 0.1.2 , deepseq , filepath - , open-browser >= 0.2.1.0 + , open-browser >= 0.2.1.0 + , optparse-applicative >= 0.14 && < 1 , pandoc , pandoc-pyplot - , pandoc-types >1.12 && <2 - , template-haskell > 2.7 && < 3 + , pandoc-types >1.12 && <2 + , template-haskell > 2.7 && < 3 , temporary , text default-language: Haskell2010
src/Text/Pandoc/Filter/Pyplot.hs view
@@ -137,6 +137,13 @@ filterClass :: String filterClass = "pyplot" + +-- | Flexible boolean parsing +readBool :: String -> Bool +readBool s | s `elem` ["True", "true", "'True'", "'true'", "1"] = True + | s `elem` ["False", "false", "'False'", "'false'", "0"] = False + | otherwise = error $ mconcat ["Could not parse '", s, "' into a boolean. Please use 'True' or 'False'"] + -- | Determine inclusion specifications from Block attributes. -- Note that the @".pyplot"@ class is required, but all other parameters are optional parseFigureSpec :: Configuration -> Block -> IO (Maybe FigureSpec) @@ -157,9 +164,10 @@ format = fromMaybe (defaultSaveFormat config) $ join $ saveFormatFromString <$> Map.lookup saveFormatKey attrs' dir = makeValid $ Map.findWithDefault (defaultDirectory config) directoryKey attrs' dpi' = fromMaybe (defaultDPI config) $ read <$> Map.lookup dpiKey attrs' + withLinks' = fromMaybe (defaultWithLinks config) $ readBool <$> Map.lookup withLinksKey attrs' blockAttrs' = (id', filter (/= filterClass) cls, filteredAttrs) - return $ FigureSpec caption' fullScript format dir dpi' blockAttrs' - + return $ FigureSpec caption' withLinks' fullScript format dir dpi' blockAttrs' + parseFigureSpec _ _ = return Nothing -- | Main routine to include Matplotlib plots.
src/Text/Pandoc/Filter/Pyplot/Configuration.hs view
@@ -13,12 +13,14 @@ module Text.Pandoc.Filter.Pyplot.Configuration ( configuration -- * For testing and internal purposes only + , writeConfig , inclusionKeys , directoryKey , captionKey , dpiKey , includePathKey , saveFormatKey + , withLinksKey ) where import Data.Maybe (fromMaybe) @@ -28,15 +30,18 @@ import Data.Yaml import Data.Yaml.Config (loadYamlSettings, ignoreEnv) +import System.Directory (doesFileExist) + import Text.Pandoc.Filter.Pyplot.Types -- | Keys that pandoc-pyplot will look for in code blocks. These are only exported for testing purposes. -directoryKey, captionKey, dpiKey, includePathKey, saveFormatKey :: String +directoryKey, captionKey, dpiKey, includePathKey, saveFormatKey, withLinksKey :: String directoryKey = "directory" captionKey = "caption" dpiKey = "dpi" includePathKey = "include" saveFormatKey = "format" +withLinksKey = "with-links" -- | list of all keys related to pandoc-pyplot. inclusionKeys :: [String] @@ -45,6 +50,7 @@ , dpiKey , includePathKey , saveFormatKey + , withLinksKey ] -- A @Configuration@ cannot be directly created from a YAML file @@ -60,6 +66,7 @@ = ConfigPrecursor { defaultDirectory_ :: FilePath , defaultIncludePath_ :: Maybe FilePath + , defaultWithLinks_ :: Bool , defaultSaveFormat_ :: String , defaultDPI_ :: Int , interpreter_ :: String @@ -70,6 +77,7 @@ parseJSON (Object v) = ConfigPrecursor <$> v .:? (T.pack directoryKey) .!= (defaultDirectory def) <*> v .:? (T.pack includePathKey) + <*> v .:? (T.pack withLinksKey) .!= (defaultWithLinks def) <*> v .:? (T.pack saveFormatKey) .!= (extension $ defaultSaveFormat def) <*> v .:? (T.pack dpiKey) .!= (defaultDPI def) <*> v .:? "interpreter" .!= (interpreter def) @@ -84,6 +92,7 @@ return $ Configuration { defaultDirectory = defaultDirectory_ prec , defaultIncludeScript = includeScript , defaultSaveFormat = saveFormat' + , defaultWithLinks = defaultWithLinks_ prec , defaultDPI = defaultDPI_ prec , interpreter = interpreter_ prec , flags = flags_ prec @@ -98,4 +107,15 @@ -- -- @since 2.1.0.0 configuration :: FilePath -> IO Configuration -configuration fp = loadYamlSettings [fp] [] ignoreEnv >>= renderConfiguration+configuration fp = loadYamlSettings [fp] [] ignoreEnv >>= renderConfiguration + + +-- | Write a configuration to file. An exception will be raised in case the file would be overwritten. +-- +-- @since 2.1.3.0 +writeConfig :: FilePath -> Configuration -> IO () +writeConfig fp config = do + fileExists <- doesFileExist fp + if fileExists + then error $ mconcat ["File ", fp, " already exists."] + else encodeFile fp config
src/Text/Pandoc/Filter/Pyplot/FigureSpec.hs view
@@ -76,11 +76,12 @@ where attrs' = blockAttrs spec target' = figurePath spec + withLinks' = withLinks spec srcLink = link (replaceExtension target' ".txt") mempty "Source code" hiresLink = link (hiresFigurePath spec) mempty "high res." captionText = fromList $ fromMaybe mempty (captionReader $ caption spec) captionLinks = mconcat [" (", srcLink, ", ", hiresLink, ")"] - caption' = captionText <> captionLinks + caption' = if withLinks' then captionText <> captionLinks else captionText -- | Determine the path a figure should have. figurePath :: FigureSpec -> FilePath
src/Text/Pandoc/Filter/Pyplot/Types.hs view
@@ -1,4 +1,5 @@-{-# LANGUAGE CPP #-} +{-# LANGUAGE CPP #-} +{-# LANGUAGE OverloadedStrings #-} {-| Module : Text.Pandoc.Filter.Pyplot.Types Copyright : (c) Laurent P René de Cotret, 2019 @@ -17,6 +18,7 @@ import Data.Hashable (Hashable, hashWithSalt) import Data.Semigroup as Sem import Data.Text (Text) +import Data.Yaml import Text.Pandoc.Definition (Attr) @@ -120,6 +122,7 @@ = Configuration { defaultDirectory :: FilePath -- ^ The default directory where figures will be saved. , defaultIncludeScript :: PythonScript -- ^ The default script to run before other instructions. + , defaultWithLinks :: Bool -- ^ The default behavior of whether or not to include links to source code and high-res , defaultSaveFormat :: SaveFormat -- ^ The default save format of generated figures. , defaultDPI :: Int -- ^ The default dots-per-inch value for generated figures. , interpreter :: String -- ^ The name of the interpreter to use to render figures. @@ -131,16 +134,31 @@ def = Configuration { defaultDirectory = "generated/" , defaultIncludeScript = mempty + , defaultWithLinks = True , defaultSaveFormat = PNG , defaultDPI = 80 , interpreter = defaultPlatformInterpreter , flags = mempty } +instance ToJSON Configuration where + toJSON (Configuration dir' _ withLinks' savefmt' dpi' interp' flags') = + -- We ignore the include script as we want to examplify that + -- this is for a filepath + object [ "directory" .= dir' + , "include" .= ("example.py" :: FilePath) + , "with-links" .= withLinks' + , "dpi" .= dpi' + , "format" .= (toLower <$> show savefmt') + , "interpreter" .= interp' + , "flags" .= flags' + ] + -- | Datatype containing all parameters required to run pandoc-pyplot data FigureSpec = FigureSpec { caption :: String -- ^ Figure caption. + , withLinks :: Bool -- ^ Append links to source code and high-dpi figure to caption , script :: PythonScript -- ^ Source code for the figure. , saveFormat :: SaveFormat -- ^ Save format of the figure , directory :: FilePath -- ^ Directory where to save the file @@ -150,6 +168,8 @@ instance Hashable FigureSpec where hashWithSalt salt spec = + -- Some things are not included in the hash because they do not affect the outcome + -- of running scripts, e.g. whether links should be shown or not. hashWithSalt salt ( caption spec , script spec , fromEnum . saveFormat $ spec
stack.yaml view
@@ -38,7 +38,6 @@ # using the same syntax as the packages field. # (e.g., acme-missiles-0.3) extra-deps: -- yaml-config-0.4.0 # Override default flag values for local packages and extra-deps # flags: {}
test/Main.hs view
@@ -39,6 +39,7 @@ , testSaveFormat , testBlockingCallError , testMarkdownFormattingCaption + , testWithLinks , testWithConfiguration , testOverridingConfiguration , testBuildConfiguration @@ -67,6 +68,11 @@ addDPI dpi (CodeBlock (id', cls, attrs) script) = CodeBlock (id', cls, attrs ++ [(dpiKey, show dpi)]) script +addWithLinks :: Bool -> Block -> Block +addWithLinks yn (CodeBlock (id', cls, attrs) script) = + CodeBlock (id', cls, attrs ++ [(withLinksKey, show yn)]) script + + -- | Assert that a file exists assertFileExists :: HasCallStack => FilePath -> Assertion assertFileExists filepath = do @@ -193,6 +199,31 @@ extractImageCaption (Image _ c _) = c extractImageCaption _ = mempty ------------------------------------------------------------------------------- + +------------------------------------------------------------------------------- +-- Test that it is possible to not render links in captions +testWithLinks :: TestTree +testWithLinks = + testCase "appropriately omits links to source code and high-res image" $ do + tempDir <- (</> "test-caption-links") <$> getCanonicalTemporaryDirectory + ensureDirectoryExistsAndEmpty tempDir + + -- Note that this test is fragile, in the sense that the expected result must be carefully + -- constructed + let expected = mempty + codeBlock = addWithLinks False $ addDirectory tempDir $ addCaption mempty $ plotCodeBlock "import matplotlib.pyplot as plt" + result <- makePlot' def codeBlock + case result of + Left error -> assertFailure $ "an error occured: " <> show error + Right block -> assertIsInfix expected (extractCaption block) + where + extractCaption (B.Para blocks) = extractImageCaption . head $ blocks + extractCaption _ = mempty + + extractImageCaption (Image _ c _) = c + extractImageCaption _ = mempty +------------------------------------------------------------------------------- + ------------------------------------------------------------------------------- -- Test with configuration