diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,4 @@ +0.1.0.0 +======= + +* Initial release. diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,30 @@ +Copyright Author name here (c) 2016 + +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + + * Neither the name of Author name here nor the names of other + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,286 @@ +pdf-slave +========= + +Tool that compiles `haskintex` (TeX with embedded Haskell) files into PDF documents. +Templates are described in YAML format and can define dependencies. + +Features: + +* Input JSON file for `htex` file that holds template varying data. + +* Option for packing a template in all-in YAML bundle. + +* Support for template dependencies that include: + - Bibtex files + - Images, listings, other static files. + - Other `.htex` templates that compiles into TeX and includes into parent template. + - Other `.htex` templates that compiles into PDF and includes into parent template. + +Template reference +================== + +Common template consists of several files: + +* `template_input.json` - Input data for template in JSON format. + +``` JSON +{ + "line-width": 2, + "spiral-precision": 0.01, + "spiral-interval": [0,4], + "spiral-a": 0.1, + "spiral-b": 4 +} +``` + +* `template.htex` - TeX/LaTeX with embedded Haskell that reads input data from +file `template.json`. You have to provide code at the beginning of file that reads +the inputs, like that: + +``` Haskell +\begin{document} + +\begin{haskellpragmas} +{-# LANGUAGE OverloadedStrings #-} +\end{haskellpragmas} +\begin{writehaskell} +import Data.Aeson +import System.IO.Unsafe (unsafePerformIO) +import qualified Data.ByteString.Lazy as BS + +data Input = Input { + lineWidth :: Double +, spiralPrecision :: Double +, spiralInterval :: (Double, Double) +, spiralA :: Double +, spiralB :: Double +} + +instance FromJSON Input where + parseJSON (Object o) = Input + <$> o .: "line-width" + <*> o .: "spiral-precision" + <*> o .: "spiral-interval" + <*> o .: "spiral-a" + <*> o .: "spiral-b" + +inpt :: Input +inpt = case unsafePerformIO $ fmap eitherDecode' $ BS.readFile "template_input.json" of + Left e -> error (show e) + Right a -> a + +\end{writehaskell} +``` + +* `template.yaml` - description of template and its dependencies. + +``` YAML +name: template01 # name of template +input: template01_input.json # name of input file +body: template01.htex # name of .htex file +dependencies: {} # dependency tree (see below) +haskintex-opts: [] # additional flags to haskintex +``` + +## Dependencies + +There are 4 different types of dependencies: + +* `other` - static files that don't require any processing. They could be images, +listings etc. Example of YAML configuration: + + ``` YAML + dependencies: + lambda.png: # name of dependency is equal to filename relative to parent template + type: other # marks type of dependency + code/demo.hs: + type: other + ``` + + See [examples/example02](https://github.com/NCrashed/pdf-slave/tree/master/examples/template02) for full example. + +* `bibtex` - bibliography files that require additional passes with `bibtex`. Example of YAML configuration: + + ``` YAML + dependencies: + biblio.bib: # name of dependency is equal to filename relative to parent template + type: bibtex # marks type of dependency + ``` + + See [examples/example03](https://github.com/NCrashed/pdf-slave/tree/master/examples/template03) for full example. + +* `template` - other `.htex` templates that are included in parent via `\\input{...}` or `\\include{..}`. Example of YAML configuration: + + ``` YAML + dependencies: + dep1: # name of dependency defines subfolder where the output tex file is located + type: template # marks type of dependency + name: dep1 + input: dep1_input.json + body: dep1.htex + dep2: + type: template + name: dep2 + body: dep2.htex + dependencies: + lambda.png: + type: other + code/demo.hs: + type: other + ``` + + See [examples/example04](https://github.com/NCrashed/pdf-slave/tree/master/examples/template04) for full example. + + **Note that `code/demo.hs` subdependency should be included as `dep2/code/demo.hs` in `dep2.htex` as `dep2.tex` is inlined into parent.** + +* `template_pdf` - other `.htex` templates that are included as PDFs into parent template. Example of YAML configuration: + + ``` YAML + dependencies: + template01: # name of dependency defines subfolder where the output tex file is located + type: template_pdf # marks type of dependency + name: template01 + input: template01_input.json + body: template01.htex + template02: + type: template_pdf + name: template02 + body: template02.htex + dependencies: + lambda.png: + type: other + code/demo.hs: + type: other + ``` + + See [examples/example05](https://github.com/NCrashed/pdf-slave/tree/master/examples/template05) for full example. + +### Input propagation + +When you work with dependencies you have two options how to handle inputs: + +* Define dependency inputs in its own file: + + ``` YAML + dependencies: + dep1: + type: template + name: dep1 + input: dep1_input.json # private input file + body: dep1.htex + ``` + +* Define dependency inputs in parent file: + + ``` YAML + name: template06 + input: template06_input.json # contains inputs for dep1 + body: template06.htex + dependencies: + dep1: + type: template + name: dep1 + body: dep1.htex + ``` + + Contents of `template06_input.json`: + + ``` JSON + { + "dep1": { + "line-width": 2, + "spiral-precision": 0.01, + "spiral-interval": [0,4], + "spiral-a": 0.1, + "spiral-b": 4 + } + } + ``` + + Note that key of subsection must be equal to name of dependency. + + See [examples/example06](https://github.com/NCrashed/pdf-slave/tree/master/examples/template06) for full example. + +## Making bundles + +One can pack all `.htex`, `.json`, `.yaml` and all dependencies in single YAML +bundle that can be easily distributed, transmitted between services and stored: + +``` bash +cd examples/template01 +pdf-slave --template template01.yaml --output template01_bundle.yaml pack +``` + +As modification of such bundles isn't handy, one can unpack bundle: + +``` bash +pdf-slave --template template01_bundle.yaml --output template01_directory unpack +``` + +Rendering of bundles is handled with the same command that is used for ordinary +templates. + +Compilation +=========== + +You need: + +* LaTeX distribution (for instance, texlive or miktex) + +* [stack](https://docs.haskellstack.org/en/stable/README/) or system wide GHC 8.0.1 + Cabal 1.24.0. + +Compilation with stack: +``` bash +git clone https://github.com/NCrashed/pdf-slave.git +cd pdf-slave +stack install +``` + +Compilation with cabal: +``` bash +git clone https://github.com/NCrashed/pdf-slave.git +cd pdf-slave +cabal sandbox init +cabal install --dependencies-only +cabal install +``` + +Running examples +================ + +You need: + +* LaTeX distribution (for instance, texlive or miktex) + +* stack or GHC+Cabal (yes, you need GHC to evaluate templates at runtime) + +* `pdf-slave` and `haskintex` executables in PATH + +Stack users: +``` bash +cd examples/template01 +stack install aeson HaTeX +stack exec -- pdf-slave --template template01.yaml --output output.pdf pdf && xdg-open output.pdf +``` + +Cabal users: +``` bash +cd examples/template01 +cabal sandbox init +cabal install aeson HaTeX +pdf-slave --template template01.yaml --output output.pdf pdf && xdg-open output.pdf +``` + +Docker build +============ + +1. Run `cook_doocker.sh` script. This will build two images, one `pdf-slave-build` for compilation of binaries and the second one `pdf-slave` for production usage. + +2. Usage: + + ``` bash + docker run -it --rm -v $(pwd)/examples/template01:/data/examples pdf-slave pdf --template examples/template01.yaml --output examples/output.pdf + xdg-open examples/template01/output.pdf + ``` + +3. Or download [precompiled container](https://hub.docker.com/r/ncrashed/pdf-slave/) from Docker Hub. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/app/Main.hs b/app/Main.hs new file mode 100644 --- /dev/null +++ b/app/Main.hs @@ -0,0 +1,149 @@ +module Main where + +import Control.Monad.IO.Class +import Data.Maybe (fromMaybe) +import Data.Monoid +import Data.Text (pack) +import Data.Yaml (decodeEither', encode) +import Filesystem.Path.CurrentOS (directory) +import Options.Applicative as O +import Prelude hiding (FilePath) +import Shelly + +import qualified Data.Aeson as A +import qualified Data.ByteString as BS +import qualified Data.ByteString.Lazy as BZ + +import Text.PDF.Slave.Template +import Text.PDF.Slave.Render + +-- | Define actions that CLI can do +data Command = + -- | Usual mode, render PDF for template + GeneratePDF + -- | Make a all-in bundle from template distributed over files + | PackBundle + -- | Make a template distributed over files from all-in bundle + | UnpackBundle + +-- | CLI options +data Options = Options { + -- | Path to template file, if missing, stdin is used + templatePath :: Maybe FilePath + -- | Path to output pdf file, if missing, stdout is used +, pdfOutputPath :: Maybe FilePath + -- | Path to JSON input file for overwritting input data of bundle +, inputOverwritePath :: Maybe FilePath + -- | CLI action +, cliCommand :: Command +} + +-- | Same as 'strOption' but parses Text +filePathOption :: Mod OptionFields String -> Parser FilePath +filePathOption m = fmap (fromText . pack) $ strOption m + +-- | Parser of CLI options +optionsParser :: Parser Options +optionsParser = Options + <$> optional (filePathOption $ + long "template" + <> help "Path to template or bundle YAML description, if missing stdin is used" + <> metavar "TEMPLATE_YAML_PATH" + ) + <*> optional (filePathOption $ + long "output" + <> help "Path to output PDF file, if missing stdout is used" + <> metavar "OUTPUT_PDF_PATH" + ) + <*> optional (filePathOption $ + long "input" + <> help ("You can specify JSON file as input that will overwrite default input" + <> " for bundle or template input file" ) + <> metavar "INPUT_JSON_PATH" + ) + <*> commandParser + +-- | Parser of CLI commands +commandParser :: Parser Command +commandParser = subparser $ + (O.command "pdf" $ info (pure GeneratePDF) $ progDesc "Generate PDF from template") + <> (O.command "pack" $ info (pure PackBundle) $ progDesc "Construct all-in bundle from template distributed over files") + <> (O.command "unpack" $ info (pure UnpackBundle) $ progDesc "Deconstruct all-in bundle to several files") + +-- | Execute PDF slave +pdfSlave :: Options -> IO () +pdfSlave Options{..} = shelly $ do + -- define base directory + baseDir <- case templatePath of + Nothing -> pwd + Just p -> canonic $ directory p + -- read template + (templateContent, contentFilename) <- case templatePath of + Nothing -> do + cnt <- liftIO $ BS.getContents + return (cnt, fromText "") + Just p -> (,p) <$> readBinary p + -- read optional input overwrite + optInput <- case inputOverwritePath of + Nothing -> return Nothing + Just p -> do + cnt <- readBinary p + case A.eitherDecode' . BZ.fromStrict $ cnt of + Left e -> fail $ "Failed to read input overwrite file: " <> show e + Right i -> return $ Just i + -- process particular CLI action + case cliCommand of + -- Generate PDF from template or bundle + GeneratePDF -> do + -- parse template contents + res <- parseBundleOrTemplate contentFilename templateContent + -- render + bs <- case res of + Left bundle -> do + let bundle' = fromMaybe bundle $ (\i -> bundle { templateInput = i }) <$> optInput + renderBundleToPDF bundle' baseDir + Right template -> do + let template' = fromMaybe template $ + (const $ template { templateFileInput = inputOverwritePath}) <$> optInput + renderTemplateToPDF template' baseDir + -- output results + case pdfOutputPath of + Nothing -> liftIO $ BS.putStr bs + Just outputPath -> writeBinary outputPath bs + + -- Pack bundle from distrubuted template format + PackBundle -> case decodeEither' templateContent of + Left e -> fail $ "Failed to parse template: " <> show e + Right tf -> do + res <- loadTemplateInMemory tf + case res of + Left e -> fail $ "Failed to pack bundle: " <> show e + Right t -> do + let bs = encode t + case pdfOutputPath of + Nothing -> liftIO $ BS.putStr bs + Just outputPath -> do + writeBinary outputPath bs + outputPath' <- toTextWarn outputPath + echo $ "Bundle is written to " <> outputPath' + + -- Unpack bundle to distributed template format + UnpackBundle -> case decodeEither' templateContent of + Left e -> fail $ "Failed to parse template: " <> show e + Right t -> case pdfOutputPath of + Nothing -> fail "Expecting --output parameter as destination folder" + Just outputFolder -> do + mkdir_p outputFolder + tf <- storeTemplateInFiles t outputFolder + let templateFilename = outputFolder templateName t <.> "yaml" + writeBinary templateFilename $ encode tf + outputFolder' <- toTextWarn outputFolder + echo $ "Bundle is unpacked to " <> outputFolder' + +main :: IO () +main = execParser opts >>= pdfSlave + where + opts = info (helper <*> optionsParser) + ( fullDesc + <> progDesc "CLI interface for PDF slave." + <> header "pdf-slave - tool for building PDF documents from HTex templates" ) diff --git a/pdf-slave.cabal b/pdf-slave.cabal new file mode 100644 --- /dev/null +++ b/pdf-slave.cabal @@ -0,0 +1,76 @@ +name: pdf-slave +version: 1.0.0.0 +synopsis: Tool to generate PDF from haskintex templates and YAML input +description: Please see README.md +homepage: https://github.com/NCrashed/pdf-slave#readme +license: BSD3 +license-file: LICENSE +author: Anton Gushcha +maintainer: ncrashed@gmail.com +copyright: 2016 Anton Gushcha +category: Web +build-type: Simple +cabal-version: >=1.10 +extra-source-files: + README.md + CHANGELOG.md + stack.yaml + +library + hs-source-dirs: src + exposed-modules: + Text.PDF.Slave + Text.PDF.Slave.Render + Text.PDF.Slave.Template + default-language: Haskell2010 + build-depends: + base >= 4.7 && < 5 + , aeson >= 0.11 && < 0.12 + , base64-bytestring >= 1.0 && < 1.1 + , bytestring >= 0.10 && < 0.11 + , containers >= 0.5 && < 0.6 + , exceptions >= 0.8 && < 0.9 + , haskintex >= 0.6 && < 0.8 + , HaTeX >= 3.16 && < 3.18 + , shelly >= 1.6 && < 1.7 + , system-filepath >= 0.4 && < 0.5 + , text >= 1.2 && < 1.3 + , unordered-containers >= 0.2 && < 0.3 + , yaml >= 0.8 && < 0.9 + + default-extensions: + BangPatterns + DeriveGeneric + FlexibleContexts + FlexibleInstances + FunctionalDependencies + GADTs + GeneralizedNewtypeDeriving + LambdaCase + MultiParamTypeClasses + OverloadedStrings + RecordWildCards + ScopedTypeVariables + StandaloneDeriving + TupleSections + TypeFamilies + +executable pdf-slave + hs-source-dirs: app + main-is: Main.hs + default-language: Haskell2010 + build-depends: + base >= 4.7 && < 5 + , aeson >= 0.11 && < 0.12 + , bytestring >= 0.10 && < 0.11 + , optparse-applicative >= 0.12 && < 0.13 + , pdf-slave + , shelly >= 1.6 && < 1.7 + , system-filepath >= 0.4 && < 0.5 + , text >= 1.2 && < 1.3 + , yaml >= 0.8 && < 0.9 + + default-extensions: + OverloadedStrings + RecordWildCards + TupleSections diff --git a/src/Text/PDF/Slave.hs b/src/Text/PDF/Slave.hs new file mode 100644 --- /dev/null +++ b/src/Text/PDF/Slave.hs @@ -0,0 +1,23 @@ +-- | Reexports +module Text.PDF.Slave( + -- * Template definition + TemplateName + , TemplateInput + , TemplateBody + , TemplateBibtex + , DependencyBody + , BibTexBody + , TemplateDependency(..) + , Template(..) + , TemplateDependencyFile(..) + , TemplateFile(..) + -- ** Bundle helpers + , loadTemplateInMemory + , storeTemplateInFiles + -- * Template rendering to PDF + , PDFContent + , renderTemplateToPDF + ) where + +import Text.PDF.Slave.Render +import Text.PDF.Slave.Template diff --git a/src/Text/PDF/Slave/Render.hs b/src/Text/PDF/Slave/Render.hs new file mode 100644 --- /dev/null +++ b/src/Text/PDF/Slave/Render.hs @@ -0,0 +1,241 @@ +-- | Rendering of templates +module Text.PDF.Slave.Render( + PDFContent + , PDFRenderException(..) + , displayPDFRenderException + , renderBundleOrTemplateFromFile + , renderFromFileBundleToPDF + , renderFromFileToPDF + , renderBundleToPDF + , renderTemplateToPDF + -- * Low-level + , DepFlags + , DepFlag(..) + , renderPdfTemplate + , renderTemplate + , renderTemplateDep + , parseBundleOrTemplate + , parseBundleOrTemplateFromFile + ) where + +import Control.Monad (join) +import Control.Monad.Catch +import Data.Aeson (Value(..)) +import Data.ByteString (ByteString) +import Data.Maybe (fromMaybe) +import Data.Monoid +import Data.Set (Set) +import Data.Yaml (ParseException, decodeEither') +import Filesystem.Path (dropExtension, directory) +import GHC.Generics +import Prelude hiding (FilePath) +import Shelly + +import qualified Data.Aeson as A +import qualified Data.ByteString.Lazy as BZ +import qualified Data.Foldable as F +import qualified Data.HashMap.Strict as H +import qualified Data.Map.Strict as M +import qualified Data.Set as S + +import Text.PDF.Slave.Template + +-- | Contents of PDF file +type PDFContent = ByteString + +-- | Errors that are thrown by rendering functions +data PDFRenderException = + TemplateFormatError FilePath ParseException -- ^ Failed to parse template YAML + | BundleFormatError FilePath ParseException -- ^ Failed to parse template bundle YAML + -- | Failed to parse file in both formats: bundle and template file. + | BundleOrTemplateFormatError FilePath ParseException ParseException + | InputFileFormatError FilePath String -- ^ Failed to parse JSON input + deriving (Generic, Show) + +instance Exception PDFRenderException + +-- | Convert PDF rendering exception to user readable format +displayPDFRenderException :: PDFRenderException -> String +displayPDFRenderException e = case e of + TemplateFormatError f pe -> "Failed to parse template file " <> show f + <> ", reason: " <> show pe + BundleFormatError f pe -> "Failed to parse template bundle file " <> show f + <> ", reason: " <> show pe + BundleOrTemplateFormatError f peBundle peTemplate -> "Failed to parse template file " <> show f <> ". " + <> "\n Tried bundle format: " <> show peBundle + <> "\n Tried template format: " <> show peTemplate + InputFileFormatError f es -> "Failed to parse template input file " <> show f + <> ", reason: " <> show es + +-- | Helper to render either a bundle or distributed template from file to PDF. +renderBundleOrTemplateFromFile :: + FilePath -- ^ Path to either bundle 'Template' or template 'TemplateFile' + -> Maybe Value -- ^ Overwrite of input JSON for bundle + -> Sh PDFContent +renderBundleOrTemplateFromFile filename bundleInput = do + res <- parseBundleOrTemplateFromFile filename + let baseDir = directory filename + case res of + Left bundle -> do + let bundle' = fromMaybe bundle $ fmap (\i -> bundle { templateInput = Just i }) bundleInput + renderBundleToPDF bundle' baseDir + Right template -> renderTemplateToPDF template baseDir + +-- | Try to parse either a bundle or template file +parseBundleOrTemplateFromFile :: FilePath -- ^ Path to either 'Template' or 'TemplateFile' + -> Sh (Either Template TemplateFile) +parseBundleOrTemplateFromFile filename = + parseBundleOrTemplate filename =<< readBinary filename + +-- | Try to parse either a bundle or template file +parseBundleOrTemplate :: FilePath -- ^ Source of data (file or stdin, etc) + -> ByteString -- ^ Contents of either 'Template' or 'TemplateFile' + -> Sh (Either Template TemplateFile) +parseBundleOrTemplate filename cnt = case decodeEither' cnt of + Right bundle -> return $ Left bundle + Left eBundle -> case decodeEither' cnt of + Right template -> return $ Right template + Left eTemplate -> throwM $ BundleOrTemplateFormatError filename eBundle eTemplate + +-- | Helper to render from all-in bundle template +renderFromFileBundleToPDF :: FilePath -- ^ Path to 'Template' all-in bundle + -> Maybe Value -- ^ Overwrite of input JSON for bundle + -> Sh PDFContent +renderFromFileBundleToPDF filename bundleInput = do + cnt <- readBinary filename + case decodeEither' cnt of + Left e -> throwM $ BundleFormatError filename e + Right bundle -> do + let bundle' = fromMaybe bundle $ fmap (\i -> bundle { templateInput = Just i }) bundleInput + renderBundleToPDF bundle' (directory filename) + +-- | Helper to render from template file +renderFromFileToPDF :: FilePath -- ^ Path to 'TemplateFile' + -> Sh PDFContent +renderFromFileToPDF filename = do + cnt <- readBinary filename + case decodeEither' cnt of + Left e -> throwM $ TemplateFormatError filename e + Right template -> renderTemplateToPDF template (directory filename) + +-- | Unpack bundle, render the template, cleanup and return PDF +renderBundleToPDF :: Template -- ^ Input all-in template + -> FilePath -- ^ Base directory + -> Sh PDFContent +renderBundleToPDF bundle baseDir = withTmpDir $ \unpackDir -> do + template <- storeTemplateInFiles bundle unpackDir + renderTemplateToPDF template baseDir + +-- | Render template and return content of resulted PDF file +renderTemplateToPDF :: TemplateFile -- ^ Input template + -> FilePath -- ^ Base directory + -> Sh PDFContent -- ^ Output PDF file +renderTemplateToPDF t@TemplateFile{..} baseDir = withTmpDir $ \outputFolder -> do + -- Parse global input file and pass it as inherited input + minput <- case templateFileInput of + Nothing -> return Nothing + Just inputName -> do + cnt <- readBinary inputName + case A.eitherDecode' . BZ.fromStrict $ cnt of + Left e -> throwM $ InputFileFormatError inputName e + Right a -> return $ Just a + renderPdfTemplate minput t baseDir outputFolder + readBinary (outputFolder templateFileName <.> "pdf") + +-- | Low-level render of template from .htex to .pdf that is recursively used for dependencies +renderPdfTemplate :: Maybe Value -- ^ Inherited input from parent + -> TemplateFile -- ^ Template to render + -> FilePath -- ^ Base directory + -> FilePath -- ^ Output folder + -> Sh () +renderPdfTemplate minput t@TemplateFile{..} baseDir outputFolder = do + flags <- renderTemplate minput t baseDir outputFolder + -- define commands of compilation pipe + let pdflatex = bash "pdflatex" [ + "-synctex=1" + , "-interaction=nonstopmode" + , toTextArg $ templateFileName <.> "tex" ] + bibtex = bash "bibtex" [ + toTextArg $ templateFileName <.> "aux" ] + -- read flags and construct pipe + chdir outputFolder $ do + _ <- if S.member NeedBibtex flags + then pdflatex -|- bibtex -|- pdflatex -|- pdflatex + else pdflatex + return () + +-- | Low-level render of template from .htex to .tex that is recursively used for dependencies +renderTemplate :: Maybe Value -- ^ Inherited input from parent + -> TemplateFile -- ^ Template to render + -> FilePath -- ^ Base directory + -> FilePath -- ^ Output folder + -> Sh DepFlags -- ^ Flags that affects compilation upper in the deptree +renderTemplate minput TemplateFile{..} baseDir outputFolder = do + mkdir_p outputFolder + let renderDepenency = renderTemplateDep minput baseDir outputFolder + depFlags <- M.traverseWithKey renderDepenency templateFileDeps + let + bodyName = dropExtension templateFileBody + haskintex = bash "haskintex" $ [ + "-overwrite" + , "-verbose" + , toTextArg $ baseDir bodyName ] + ++ templateFileHaskintexOpts + -- input file might be missing, if missing we can inject input from parent + case templateFileInput of + Nothing -> whenJust minput $ \input -> do + let filename = outputFolder (templateFileName <> "_input") <.> "json" + writeBinary filename $ BZ.toStrict . A.encode $ input + Just inputName -> do + let inputPath = baseDir inputName + cp inputPath $ outputFolder inputName + _ <- chdir outputFolder haskintex + return $ F.foldMap id depFlags -- merge flags + +-- | Collected dependency markers (for instance, that we need bibtex compilation) +type DepFlags = Set DepFlag + +-- | Dependency marker that is returned from 'renderTemplateDep' +data DepFlag = NeedBibtex -- ^ We need a bibtex compliation + deriving (Generic, Show, Ord, Eq) + +-- | Render template dependency +renderTemplateDep :: Maybe Value -- ^ Inherited input from parent + -> FilePath -- ^ Base directory + -> FilePath -- ^ Output folder + -> TemplateName -- ^ Dependency name + -> TemplateDependencyFile -- ^ Dependency type + -> Sh DepFlags +renderTemplateDep minput baseDir outputFolder name dep = case dep of + BibtexDepFile -> do + let file = fromText name + outputFile = outputFolder file + mkdir_p (directory outputFile) + cp (baseDir file) outputFile + return $ S.singleton NeedBibtex + TemplateDepFile template -> do + let subFolder = baseDir fromText name + outputSubFolder = outputFolder fromText name + renderTemplate minput' template subFolder outputSubFolder + TemplatePdfDepFile template -> do + let subFolder = baseDir fromText name + outputSubFolder = outputFolder fromText name + renderPdfTemplate minput' template subFolder outputSubFolder + return mempty + OtherDepFile -> do + let file = fromText name + outputFile = outputFolder file + mkdir_p (directory outputFile) + cp (baseDir file) outputFile + return mempty + where + -- Try to find subsection in input that refer to the dependency + minput' :: Maybe Value + minput' = join $ flip fmap minput $ \case + Object o -> H.lookup name o + _ -> Nothing + +-- | Same as 'when', but for 'Just' values. +whenJust :: Applicative f => Maybe a -> (a -> f ()) -> f () +whenJust Nothing _ = pure () +whenJust (Just a) f = f a diff --git a/src/Text/PDF/Slave/Template.hs b/src/Text/PDF/Slave/Template.hs new file mode 100644 --- /dev/null +++ b/src/Text/PDF/Slave/Template.hs @@ -0,0 +1,291 @@ +-- | Defines document template +{-# OPTIONS_GHC -fno-warn-orphans #-} +module Text.PDF.Slave.Template( + TemplateName + , TemplateInput + , TemplateBody + , TemplateBibtex + , DependencyBody + , BibTexBody + , TemplateDependency(..) + , Template(..) + , TemplateDependencyFile(..) + , TemplateFile(..) + -- * Helpers + , loadTemplateInMemory + , storeTemplateInFiles + ) where + +import Control.Monad (mzero) +import Data.ByteString (ByteString) +import Data.Monoid +import Data.Text as T +import Data.Yaml +import Filesystem.Path.CurrentOS (directory) +import GHC.Generics +import Prelude hiding (FilePath) +import Shelly as Sh + +import qualified Data.Aeson as A +import qualified Data.ByteString.Base64 as B64 +import qualified Data.ByteString.Lazy as BZ +import qualified Data.Map.Strict as M +import qualified Data.Text.Encoding as T + +-- | Template unique name +type TemplateName = Text + +-- | A template takes simple YAML document as input +type TemplateInput = Value + +-- | Template body is text with .htex content +type TemplateBody = Text + +-- | Template can define additional bibtex database +type TemplateBibtex = Text + +-- | Dependency can be a binary file +type DependencyBody = ByteString + +-- | Content of bibtex file +type BibTexBody = Text + +-- | Template has different types of dependencies, each type of the dependecy +-- has own affect on rendering pipe. +data TemplateDependency = + -- | Bibtex file for references to other documents. Need call to bibtex + BibtexDep BibTexBody + -- | HTex file that need to be compiled to .tex file + | TemplateDep Template + -- | HTex file that need to be compiled to .pdf file + | TemplatePdfDep Template + -- | Any other file that doesn't need a compilation (listings, images, etc) + | OtherDep DependencyBody + deriving (Generic, Show) + +instance FromJSON TemplateDependency where + parseJSON val@(Object o) = do + depType <- o .: "type" + case T.toLower . T.strip $ depType of + "bibtex" -> BibtexDep <$> o .: "body" + "template" -> TemplateDep <$> parseJSON val + "template_pdf" -> TemplatePdfDep <$> parseJSON val + "other" -> do + (t :: Text) <- o .: "body" + either (\e -> fail $ "Cannot decode dependency body (base64): " <> e) (return . OtherDep) $ + B64.decode . T.encodeUtf8 $ t + _ -> fail $ "Unknown template type " <> unpack depType + parseJSON _ = mzero + +instance ToJSON TemplateDependency where + toJSON d = case d of + BibtexDep body -> object [ + "type" .= ("bibtex" :: Text) + , "body" .= body + ] + TemplateDep body -> let + Object o1 = object [ "type" .= ("template" :: Text) ] + Object o2 = toJSON body + in Object (o1 <> o2) + TemplatePdfDep body -> let + Object o1 = object [ "type" .= ("template_pdf" :: Text) ] + Object o2 = toJSON body + in Object (o1 <> o2) + OtherDep bs -> object [ + "type" .= ("other" :: Text) + , "body" .= (T.decodeUtf8 . B64.encode $ bs) + ] + +-- | Description of document template +data Template = Template { + -- | Template has human readable name + templateName :: TemplateName + -- | Template expects input in YAML format + , templateInput :: Maybe TemplateInput + -- | Template contents + , templateBody :: TemplateBody + -- | Template dependencies (bibtex, listings, other htex files) + , templateDeps :: M.Map TemplateName TemplateDependency + -- | Additional flags for `haskintex` + , templateHaskintexOpts :: [Text] + } deriving (Generic, Show) + +instance FromJSON Template where + parseJSON (Object o) = Template + <$> o .: "name" + <*> o .:? "input" + <*> o .: "body" + <*> o .:? "dependencies" .!= mempty + <*> o .:? "haskintex-opts" .!= mempty + parseJSON _ = mzero + +instance ToJSON Template where + toJSON Template{..} = object [ + "name" .= templateName + , "input" .= templateInput + , "body" .= templateBody + , "dependencies" .= templateDeps + , "haskintex-opts" .= templateHaskintexOpts + ] + +-- | Same as 'TemplateDependency' but keeps contents in separate files +data TemplateDependencyFile = + -- | Bibtex file for references to other documents. Need call to bibtex. + -- Name of dependency is a filename with contents. + BibtexDepFile + -- | HTex file that need to be compiled to .tex file + -- Name of dependency defines a subfolder for the template. + | TemplateDepFile TemplateFile + -- | HTex file that need to be compiled to .pdf file + -- Name of dependency deinfes a subfolder for the template. + | TemplatePdfDepFile TemplateFile + -- | Any other file that doesn't need a compilation (listings, images, etc) + -- Name of dependency is a filename with contents. + | OtherDepFile + deriving (Generic, Show) + +instance FromJSON FilePath where + parseJSON (String s) = return $ Sh.fromText s + parseJSON _ = mzero + +instance ToJSON FilePath where + toJSON = String . Sh.toTextIgnore + +instance FromJSON TemplateDependencyFile where + parseJSON val@(Object o) = do + depType <- o .: "type" + case T.toLower . T.strip $ depType of + "bibtex" -> pure BibtexDepFile + "template" -> TemplateDepFile <$> parseJSON val + "template_pdf" -> TemplatePdfDepFile <$> parseJSON val + "other" -> pure OtherDepFile + _ -> fail $ "Unknown template type " <> unpack depType + parseJSON _ = mzero + +instance ToJSON TemplateDependencyFile where + toJSON d = case d of + BibtexDepFile -> object [ + "type" .= ("bibtex" :: Text) + ] + TemplateDepFile body -> let + Object o1 = object [ "type" .= ("template" :: Text) ] + Object o2 = toJSON body + in Object (o1 <> o2) + TemplatePdfDepFile body -> let + Object o1 = object [ "type" .= ("template_pdf" :: Text) ] + Object o2 = toJSON body + in Object (o1 <> o2) + OtherDepFile -> object [ + "type" .= ("other" :: Text) + ] + +-- | Same as 'Template', but holds info about template content and dependencies +-- in other files. +data TemplateFile = TemplateFile { + -- | Template has human readable name + templateFileName :: TemplateName + -- | Template expects input in JSON format. The field contains filename of the YAML file. + , templateFileInput :: Maybe FilePath + -- | Template contents filename. + , templateFileBody :: FilePath + -- | Template dependencies (bibtex, listings, other htex files) + , templateFileDeps :: M.Map TemplateName TemplateDependencyFile + -- | Additional flags for `haskintex` + , templateFileHaskintexOpts :: [Text] + } deriving (Generic, Show) + +instance FromJSON TemplateFile where + parseJSON (Object o) = TemplateFile + <$> o .: "name" + <*> o .:? "input" + <*> o .: "body" + <*> o .:? "dependencies" .!= mempty + <*> o .:? "haskintex-opts" .!= mempty + parseJSON _ = mzero + +instance ToJSON TemplateFile where + toJSON TemplateFile{..} = object [ + "name" .= templateFileName + , "input" .= templateFileInput + , "body" .= templateFileBody + , "dependencies" .= templateFileDeps + , "haskintex-opts" .= templateFileHaskintexOpts + ] + +-- | Load all external references of template into memory +loadTemplateInMemory :: TemplateFile -> Sh (Either String Template) +loadTemplateInMemory TemplateFile{..} = do + inputCnt <- case templateFileInput of + Nothing -> return $ Right Nothing + Just fname -> do + cnt <- readBinary fname + return $ fmap Just . A.eitherDecode' . BZ.fromStrict $ cnt + body <- readfile templateFileBody + deps <- M.traverseWithKey loadDep templateFileDeps + return $ Template + <$> pure templateFileName + <*> inputCnt + <*> pure body + <*> sequence deps + <*> pure templateFileHaskintexOpts + where + loadDep name d = let + filename = fromText name + in case d of + BibtexDepFile -> do + cnt <- readfile filename + return . pure $ BibtexDep cnt + TemplateDepFile body -> do + tmpl <- chdir filename $ loadTemplateInMemory body + return $ TemplateDep <$> tmpl + TemplatePdfDepFile body -> do + tmpl <- chdir filename $ loadTemplateInMemory body + return $ TemplatePdfDep <$> tmpl + OtherDepFile -> do + cnt <- readBinary filename + return . pure $ OtherDep cnt + +-- | Extract all external references of template into file system +storeTemplateInFiles :: Template -> FilePath -> Sh TemplateFile +storeTemplateInFiles Template{..} folder = do + mkdir_p folder + relInputName <- case templateInput of + Nothing -> return Nothing + Just input -> do + let inputName = folder (templateName <> "_input") <.> "json" + writeBinary inputName $ BZ.toStrict $ A.encode input + fmap Just $ relativeTo folder inputName + let bodyName = folder templateName <.> "htex" + mkdir_p $ directory bodyName + writefile bodyName templateBody + relBodyName <- relativeTo folder bodyName + deps <- M.traverseWithKey storeDep templateDeps + return $ TemplateFile { + templateFileName = templateName + , templateFileInput = relInputName + , templateFileBody = relBodyName + , templateFileDeps = deps + , templateFileHaskintexOpts = templateHaskintexOpts + } + where + storeDep name d = case d of + BibtexDep body -> do + let bodyName = folder name + mkdir_p $ directory bodyName + writefile bodyName body + return BibtexDepFile + TemplateDep template -> do + let subfolderName = folder Sh.fromText name + mkdir_p subfolderName + dep <- storeTemplateInFiles template subfolderName + return $ TemplateDepFile dep + TemplatePdfDep template -> do + let subfolderName = folder Sh.fromText name + mkdir_p subfolderName + dep <- storeTemplateInFiles template subfolderName + return $ TemplatePdfDepFile dep + OtherDep body -> do + let bodyName = folder name + mkdir_p $ directory bodyName + writeBinary bodyName body + return OtherDepFile diff --git a/stack.yaml b/stack.yaml new file mode 100644 --- /dev/null +++ b/stack.yaml @@ -0,0 +1,67 @@ +# This file was automatically generated by 'stack init' +# +# Some commonly used options have been documented as comments in this file. +# For advanced use and comprehensive documentation of the format, please see: +# http://docs.haskellstack.org/en/stable/yaml_configuration/ + +# Resolver to choose a 'specific' stackage snapshot or a compiler version. +# A snapshot resolver dictates the compiler version and the set of packages +# to be used for project dependencies. For example: +# +# resolver: lts-3.5 +# resolver: nightly-2015-09-21 +# resolver: ghc-7.10.2 +# resolver: ghcjs-0.1.0_ghc-7.10.2 +# resolver: +# name: custom-snapshot +# location: "./custom-snapshot.yaml" +resolver: lts-7.14 + +# User packages to be built. +# Various formats can be used as shown in the example below. +# +# packages: +# - some-directory +# - https://example.com/foo/bar/baz-0.0.2.tar.gz +# - location: +# git: https://github.com/commercialhaskell/stack.git +# commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a +# - location: https://github.com/commercialhaskell/stack/commit/e7b331f14bcffb8367cd58fbfc8b40ec7642100a +# extra-dep: true +# subdirs: +# - auto-update +# - wai +# +# A package marked 'extra-dep: true' will only be built if demanded by a +# non-dependency (i.e. a user package), and its test suites and benchmarks +# will not be run. This is useful for tweaking upstream packages. +packages: +- '.' + +# Dependency packages to be pulled from upstream that are not in the resolver +# (e.g., acme-missiles-0.3) +extra-deps: [] + +# Override default flag values for local packages and extra-deps +flags: {} + +# Extra package databases containing global packages +extra-package-dbs: [] + +# Control whether we use the GHC we find on the path +# system-ghc: true +# +# Require a specific version of stack, using version ranges +# require-stack-version: -any # Default +# require-stack-version: ">=1.3" +# +# Override the architecture used by stack, especially useful on Windows +# arch: i386 +# arch: x86_64 +# +# Extra directories used by stack for building +# extra-include-dirs: [/path/to/dir] +# extra-lib-dirs: [/path/to/dir] +# +# Allow a newer minor version of GHC than the snapshot specifies +# compiler-check: newer-minor