pandoc 2.3 → 2.3.1

raw patch · 53 files changed

+1791/−1312 lines, 53 filesdep ~hsluadep ~hslua-module-textPVP: major bump suggested

API removals or changes: PVP suggests a major version bump

Dependency ranges changed: hslua, hslua-module-text

API changes (from Hackage documentation)

Files

• AUTHORS.md +4/−0
• MANUAL.txt +11/−11
• changelog +108/−0
• data/translations/ru.yaml +9/−9
• man/pandoc.1 +3/−3
• pandoc.cabal +5/−4
• src/Text/Pandoc/Lua.hs +5/−6
• src/Text/Pandoc/Lua/Filter.hs +76/−57
• src/Text/Pandoc/Lua/Init.hs +9/−4
• src/Text/Pandoc/Lua/Module/MediaBag.hs +1/−1
• src/Text/Pandoc/Lua/Module/Pandoc.hs +14/−14
• src/Text/Pandoc/Lua/Module/Utils.hs +12/−12
• src/Text/Pandoc/Lua/Packages.hs +12/−20
• src/Text/Pandoc/Lua/StackInstances.hs +76/−106
• src/Text/Pandoc/Lua/Util.hs +33/−55
• src/Text/Pandoc/Readers/LaTeX.hs +32/−553
• src/Text/Pandoc/Readers/LaTeX/Parsing.hs +663/−0
• src/Text/Pandoc/Readers/Markdown.hs +7/−5
• src/Text/Pandoc/Readers/Muse.hs +173/−200
• src/Text/Pandoc/Readers/Org/Blocks.hs +1/−4
• src/Text/Pandoc/Readers/Org/Inlines.hs +5/−3
• src/Text/Pandoc/Readers/Org/Shared.hs +4/−0
• src/Text/Pandoc/Readers/RST.hs +2/−2
• src/Text/Pandoc/Writers/ConTeXt.hs +6/−1
• src/Text/Pandoc/Writers/Custom.hs +74/−69
• src/Text/Pandoc/Writers/HTML.hs +143/−14
• src/Text/Pandoc/Writers/JATS.hs +2/−3
• src/Text/Pandoc/Writers/LaTeX.hs +3/−3
• src/Text/Pandoc/Writers/OpenDocument.hs +21/−13
• src/Text/Pandoc/Writers/RTF.hs +4/−2
• src/Text/Pandoc/Writers/Textile.hs +3/−3
• stack.yaml +2/−0
• test/Tests/Lua.hs +7/−7
• test/Tests/Readers/HTML.hs +5/−3
• test/Tests/Readers/Markdown.hs +5/−3
• test/Tests/Readers/Muse.hs +17/−1
• test/Tests/Readers/Org/Inline.hs +7/−0
• test/Tests/Readers/RST.hs +3/−0
• test/command/3716.md +1/−1
• test/command/4885.md +8/−0
• test/command/4913.md +34/−0
• test/command/4919.md +14/−0
• test/command/4928.md +48/−0
• test/markdown-reader-more.native +4/−4
• test/tables.opendocument +92/−92
• test/testsuite.native +4/−4
• test/writer.docbook4 +3/−3
• test/writer.docbook5 +3/−3
• test/writer.html4 +1/−1
• test/writer.html5 +1/−1
• test/writer.jats +6/−7
• test/writer.native +4/−4
• test/writer.textile +1/−1

@@ -28,6 +28,7 @@ - Bjorn Buckwalter - Bradley Kuhn - Brent Yorgey+- Brian Leung - Bryan O'Sullivan - Caleb McDaniel - Calvin Beck@@ -75,6 +76,7 @@ - Hubert Plociniczak - Igor Khorlo - Ilya V. Portnov+- Ivan Trubach - Ivo Clarysse - J. B. Rainsberger - J. Lewis Muir@@ -96,6 +98,7 @@ - John Muccigrosso - John Luke Bentley - Jonas Smedegaard+- Jonas Scholl - Jonathan Daugherty - Jose Luis Duran - José de Mattos Neto@@ -138,6 +141,7 @@ - Nick Bart - Nicolas Kaiser - Nikolay Yakimov+- Nils Carlson - Nokome Bentley - Oliver Matthews - Ophir Lifshitz

@@ -1,6 +1,6 @@ % Pandoc User's Guide % John MacFarlane-% September 16, 2018+% September 27, 2018  Synopsis ========@@ -211,7 +211,7 @@ Options ======= -General options+General options {.options} ---------------  `-f` *FORMAT*, `-r` *FORMAT*, `--from=`*FORMAT*, `--read=`*FORMAT*@@ -459,7 +459,7 @@ [PowerPoint]: https://en.wikipedia.org/wiki/Microsoft_PowerPoint [Vimwiki]: https://vimwiki.github.io -Reader options+Reader options {.options} --------------  `--base-header-level=`*NUMBER*@@ -645,7 +645,7 @@ [perl]: https://metacpan.org/pod/Pandoc::Filter [JavaScript/node.js]: https://github.com/mvhenderson/pandoc-filter-node -General writer options+General writer options {.options} ----------------------  `-s`, `--standalone`@@ -828,7 +828,7 @@     downloaded). If you're behind a proxy, you also need to set     the environment variable `http_proxy` to `http://...`. -Options affecting specific writers+Options affecting specific writers {.options} ----------------------------------  `--self-contained`@@ -1150,7 +1150,7 @@ [Encoding issue with the listings package]:   https://en.wikibooks.org/wiki/LaTeX/Source_Code_Listings#Encoding_issue -Citation rendering+Citation rendering {.options} ------------------  `--bibliography=`*FILE*@@ -1190,7 +1190,7 @@     with the `pandoc-citeproc` filter or with PDF output. It is intended for     use in producing a LaTeX file that can be processed with [`bibtex`] or [`biber`]. -Math rendering in HTML+Math rendering in HTML {.options} ----------------------  The default is to render TeX math as far as possible using Unicode characters.@@ -1251,7 +1251,7 @@ [KaTeX]: https://github.com/Khan/KaTeX [GladTeX]: http://humenda.github.io/GladTeX/ -Options for wrapper scripts+Options for wrapper scripts {.options} ---------------------------  `--dump-args`@@ -4627,9 +4627,9 @@ The `epub:type` attribute ------------------------- -You can mark up the header that corresponds to an EPUB chapter using the-[`epub:type` attribute][epub-type]. For example, to set the attribute-to the value `prologue`, use this markdown:+For `epub3` output, you can mark up the header that corresponds to an EPUB+chapter using the [`epub:type` attribute][epub-type]. For example, to set+the attribute to the value `prologue`, use this markdown:      # My chapter {epub:type=prologue} 

@@ -1,3 +1,108 @@+pandoc (2.3.1)++  * RST reader:++    + Parse RST inlines containing newlines (#4912, Francesco Occhipinti).+      This eliminates a regression introduced after pandoc 2.1.1, which+      caused inline constructions containing newlines not to be recognized.+    + Fix bug with internal link targets (#4919).  They were gobbling up+      indented content underneath.++  * Markdown reader: distinguish autolinks in the AST.  With this change,+    autolinks are parsed as Links with the `uri` class. (The same is true+    for bare links, if the `autolink_bare_uris` extension is enabled.)+    Email autolinks are parsed as Links with the `email` class.  This+    allows the distinction to be represented in the AST.++  * Org reader:++    + Force inline code blocks to honor export options (Brian Leung).+    + Parse empty argument array in inline src blocks (Brian Leung).++  * Muse reader (Alexander Krotov):++    + Added additional tests.+    + Do not allow code markup to be followed by digit.+    + Remove heading level limit.+    + Simplify `<literal>` tag parsers+    + Parse Text instead of String. Benchmark shows 7% improvement.+    + Get rid of HTML parser dependency.+    + Various code improvements.++  * ConTeXt writer: change `\` to `/` in Windows image paths (#4918).+    We do this in the LaTeX writer, and it avoids problems.+    Note that `/` works as a LaTeX path separator on Windows.++  * LaTeX writer:++    + Add support for multiprenote and multipostnote arguments+      with `--biblatex` (Brian Leung, #4930).  The multiprenotes occur before+      the first prefix of a multicite, and the multipostnotes follow the+      last suffix.+    + Fix a use of `last` that might take empty list.  If you ran with+      `--biblatex` and have an empty document (metadata but no blocks),+      pandoc would previously raise an error because of the use of+      `last` on an empty list.++  * RTF writer: Fix build failure with ghc-8.6.1 caused by missing+    MonadFail instance (Jonas Scholl).++  * ODT Writer: Improve table header row style handling (Nils Carlson).+    This changes the way styles for cells in the header row+    and normal rows are handled in ODT tables.+    Previously a new (but identical) style was generated for+    every table, specifying the style of the cells within the table.+    After this change there are two style definitions for table cells,+    one for the cells in the header row, one for all other cells.+    This doesn't change the actual styles, but makes post-processing+    changes to the table styles much simpler as it is no longer+    necessary to introduce new styles for header rows and there are+    now only two styles where there was previously one per table.++  * HTML writer:++    + Don't add `uri` class to presumed autolinks.  Formerly the `uri`+      class was added to autolinks by the HTML writer, but it had to+      guess what was an autolink and could not distinguish+      `[http://example.com](http://example.com)` from+      `<http://example.com>`.  It also incorrectly recognized+      `[pandoc](pandoc)` as an autolink.  Now the HTML writer+      simply passes through the `uri` attribute if it is present,+      but does not add anything.+    + Avoid adding extra section nestings for revealjs.+      Previously revealjs title slides at level (slidelevel - 1)+      were nested under an extra section element, even when+      the section contained no additional (vertical) content.+      That caused problems for some transition effects.+    + Omit unknown attributes in EPUB2 output.  For example,+      `epub:type` attributes should not be passed through,+      or the epub produced will not validate.++  * JATS writer: remove 'role' attribute on 'bold' and 'sc' elements (#4937).+    The JATS spec does not allow these.++  * Textile writer: don't represent `uri` class explicitly+    for autolinks (#4913).++  * Lua filters (Albert Krewinkel):++    + Cleanup filter execution code.+    + Better error on test failure.++  * HTML, Muse reader tests: reduce time taken by round-trip test.++  * Added cabal.project.++  * MANUAL: `epub:type` is only useful for epub3 (Maura Bieg).++  * Use hslua v1.0.0 (Albert Krewinkel).++  * Fix `translations/ru` to use modern Russian orthography+    (Ivan Trubach).++  * Build Windows binary using ghc 8.6.1 and cabal new-build. This+    fixes issues with segfaults in the 32-bit Windows binaries (#4283).+ pandoc (2.3)    * Add `--metadata-file` option (Mauro Bieg, #1960), which allows@@ -53,6 +158,9 @@    * LaTeX reader: +    + Split some general-purpose definitions into a separate, unexported+      module, Text.Pandoc.Readers.LaTeX.Parsing.  This is to avoid+      out-of-memory errors compiling the LaTeX reader.     + Handle parameter patterns for `\def` (#4768, #4771).       For example:  `\def\foo#1[#2]{#1 and #2}`.     + Allow `%` characters in URLs. This affects `\href` and `\url` (#4832).

@@ -1,20 +1,20 @@-Abstract: Аннотація-Appendix: Приложеніе+Abstract: Аннотация+Appendix: Приложение Author: Именной указатель-Bibliography: Библіографія+Bibliography: Литература Cc: исх. Chapter: Глава-Contents: Оглавленіе+Contents: Оглавление Encl: вкл. Figure: Рис.-Index: Предмѣтный указатель-ListOfFigures: Списокъ иллюстрацій-ListOfTables: Списокъ таблицъ+Index: Предметный указатель+ListOfFigures: Список иллюстраций+ListOfTables: Список таблиц Page: с. Part: Часть-Preface: Предисловіе+Preface: Предисловие Proof: Доказательство-References: Примѣчанія+References: Список литературы See: см. SeeAlso: см. также Table: Таблица

@@ -1,5 +1,5 @@ .\"t-.TH PANDOC 1 "September 16, 2018" "pandoc 2.3"+.TH PANDOC 1 "September 27, 2018" "pandoc 2.3.1" .SH NAME pandoc - general markup converter .SH SYNOPSIS@@ -5786,8 +5786,8 @@ .RE .SS The \f[C]epub:type\f[] attribute .PP-You can mark up the header that corresponds to an EPUB chapter using the-\f[C]epub:type\f[] attribute.+For \f[C]epub3\f[] output, you can mark up the header that corresponds+to an EPUB chapter using the \f[C]epub:type\f[] attribute. For example, to set the attribute to the value \f[C]prologue\f[], use this markdown: .IP

@@ -1,5 +1,5 @@ name:            pandoc-version:         2.3+version:         2.3.1 cabal-version:   2.0 build-type:      Custom license:         GPL-2@@ -372,8 +372,8 @@                  blaze-html >= 0.9 && < 0.10,                  blaze-markup >= 0.8 && < 0.9,                  vector >= 0.10 && < 0.13,-                 hslua >= 0.9.5 && < 0.9.6,-                 hslua-module-text >= 0.1.2 && < 0.2,+                 hslua >= 1.0 && < 1.1,+                 hslua-module-text >= 0.2 && < 0.3,                  binary >= 0.5 && < 0.10,                  SHA >= 1.6 && < 1.7,                  haddock-library >= 1.6 && < 1.7,@@ -508,6 +508,7 @@                    Text.Pandoc.Readers.Docx.Util,                    Text.Pandoc.Readers.Docx.StyleMap,                    Text.Pandoc.Readers.Docx.Fields,+                   Text.Pandoc.Readers.LaTeX.Parsing,                    Text.Pandoc.Readers.Odt.Base,                    Text.Pandoc.Readers.Odt.Namespaces,                    Text.Pandoc.Readers.Odt.StyleReader,@@ -615,7 +616,7 @@                   time >= 1.5 && < 1.10,                   directory >= 1 && < 1.4,                   filepath >= 1.1 && < 1.5,-                  hslua >= 0.9.5 && < 0.9.6,+                  hslua >= 1.0 && < 1.1,                   process >= 1.2.3 && < 1.7,                   temporary >= 1.1 && < 1.4,                   Diff >= 0.2 && < 0.4,

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2017–2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de> @@ -16,6 +15,7 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA -}+{-# LANGUAGE NoImplicitPrelude #-} {- |    Module      : Text.Pandoc.Lua    Copyright   : Copyright © 2017–2018 Albert Krewinkel@@ -34,12 +34,11 @@  import Prelude import Control.Monad ((>=>))-import Foreign.Lua (Lua, LuaException (..))+import Foreign.Lua (Lua) import Text.Pandoc.Class (PandocIO) import Text.Pandoc.Definition (Pandoc) import Text.Pandoc.Lua.Filter (LuaFilter, walkMWithLuaFilter)-import Text.Pandoc.Lua.Init (runPandocLua, registerScriptPath)-import Text.Pandoc.Lua.Util (popValue)+import Text.Pandoc.Lua.Init (LuaException (..), runPandocLua, registerScriptPath) import Text.Pandoc.Options (ReaderOptions)  import qualified Foreign.Lua as Lua@@ -61,14 +60,14 @@   top <- Lua.gettop   stat <- Lua.dofile filterPath   if stat /= Lua.OK-    then Lua.throwTopMessageAsError+    then Lua.throwTopMessage     else do       newtop <- Lua.gettop       -- Use the returned filters, or the implicitly defined global filter if       -- nothing was returned.       luaFilters <- if newtop - top >= 1                     then Lua.peek Lua.stackTop-                    else Lua.getglobal "_G" *> fmap (:[]) popValue+                    else Lua.pushglobaltable *> fmap (:[]) Lua.popValue       runAll luaFilters pd  where   registerFormat = do

@@ -1,6 +1,33 @@-{-# LANGUAGE NoImplicitPrelude #-}+{-+Copyright © 2012-2018 John MacFarlane <jgm@berkeley.edu>+            2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>++This program is free software; you can redistribute it and/or modify+it under the terms of the GNU General Public License as published by+the Free Software Foundation; either version 2 of the License, or+(at your option) any later version.++This program is distributed in the hope that it will be useful,+but WITHOUT ANY WARRANTY; without even the implied warranty of+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the+GNU General Public License for more details.++You should have received a copy of the GNU General Public License+along with this program; if not, write to the Free Software+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA+-} {-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE NoImplicitPrelude #-}+{- |+Module      : Text.Pandoc.Lua.Filter+Copyright   : © 2012–2018 John MacFarlane,+              © 2017-2018 Albert Krewinkel+License     : GNU GPL, version 2 or above+Maintainer  : Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>+Stability   : alpha +Types and functions for running Lua filters.+-} module Text.Pandoc.Lua.Filter ( LuaFilterFunction                               , LuaFilter                               , tryFilter@@ -12,62 +39,57 @@                               , inlineElementNames                               ) where import Prelude-import Control.Monad (mplus, unless, when, (>=>))+import Control.Monad (mplus, (>=>)) import Control.Monad.Catch (finally)-import Text.Pandoc.Definition+import Data.Data (Data, DataType, dataTypeConstrs, dataTypeName, dataTypeOf,+                  showConstr, toConstr, tyconUQname) import Data.Foldable (foldrM) import Data.Map (Map)-import qualified Data.Map as Map-import qualified Foreign.Lua as Lua-import Foreign.Lua (FromLuaStack (peek), Lua, StackIndex,-                    Status (OK), ToLuaStack (push))+import Foreign.Lua (Lua, Peekable, Pushable)+import Text.Pandoc.Definition+import Text.Pandoc.Lua.StackInstances () import Text.Pandoc.Walk (walkM, Walkable)-import Data.Data (Data, DataType, dataTypeConstrs, dataTypeName, dataTypeOf,-                  showConstr, toConstr, tyconUQname)-import Text.Pandoc.Lua.StackInstances()-import Text.Pandoc.Lua.Util (typeCheck) -type FunctionMap = Map String LuaFilterFunction--newtype LuaFilterFunction = LuaFilterFunction { functionIndex :: Int }--instance ToLuaStack LuaFilterFunction where-  push = pushFilterFunction+import qualified Data.Map.Strict as Map+import qualified Foreign.Lua as Lua -instance FromLuaStack LuaFilterFunction where-  peek = registerFilterFunction+-- | Filter function stored in the registry+newtype LuaFilterFunction = LuaFilterFunction Lua.Reference -newtype LuaFilter = LuaFilter FunctionMap+-- | Collection of filter functions (at most one function per element+-- constructor)+newtype LuaFilter = LuaFilter (Map String LuaFilterFunction) -instance FromLuaStack LuaFilter where-  peek idx =-    let constrs = metaFilterName : pandocFilterNames-                  ++ blockElementNames-                  ++ inlineElementNames-        fn c acc = do-          Lua.getfield idx c-          filterFn <- Lua.tryLua (peek (-1))-          Lua.pop 1+instance Peekable LuaFilter where+  peek idx = do+    let constrs = metaFilterName+                : pandocFilterNames+                ++ blockElementNames+                ++ inlineElementNames+    let go constr acc = do+          Lua.getfield idx constr+          filterFn <- registerFilterFunction           return $ case filterFn of-            Left _  -> acc-            Right f -> (c, f) : acc-    in LuaFilter . Map.fromList <$> foldrM fn [] constrs+            Nothing -> acc+            Just fn -> Map.insert constr fn acc+    LuaFilter <$> foldrM go Map.empty constrs --- | Push the filter function to the top of the stack.+-- | Register the function at the top of the stack as a filter function in the+-- registry.+registerFilterFunction :: Lua (Maybe LuaFilterFunction)+registerFilterFunction = do+  isFn <- Lua.isfunction Lua.stackTop+  if isFn+    then Just . LuaFilterFunction <$> Lua.ref Lua.registryindex+    else Nothing <$ Lua.pop 1++-- | Retrieve filter function from registry and push it to the top of the stack. pushFilterFunction :: LuaFilterFunction -> Lua ()-pushFilterFunction lf =-  -- The function is stored in a lua registry table, retrieve it from there.-  Lua.rawgeti Lua.registryindex (functionIndex lf)+pushFilterFunction (LuaFilterFunction fnRef) =+  Lua.getref Lua.registryindex fnRef -registerFilterFunction :: StackIndex -> Lua LuaFilterFunction-registerFilterFunction idx = do-  isFn <- Lua.isfunction idx-  unless isFn . Lua.throwLuaError $ "Not a function at index " ++ show idx-  Lua.pushvalue idx-  refIdx <- Lua.ref Lua.registryindex-  return $ LuaFilterFunction refIdx -elementOrList :: FromLuaStack a => a -> Lua [a]+elementOrList :: Peekable a => a -> Lua [a] elementOrList x = do   let topOfStack = Lua.stackTop   elementUnchanged <- Lua.isnil topOfStack@@ -77,12 +99,10 @@        mbres <- Lua.peekEither topOfStack        case mbres of          Right res -> [res] <$ Lua.pop 1-         Left _    -> do-           typeCheck Lua.stackTop Lua.TypeTable-           Lua.toList topOfStack `finally` Lua.pop 1+         Left _    -> Lua.peekList topOfStack `finally` Lua.pop 1  -- | Try running a filter for the given element-tryFilter :: (Data a, FromLuaStack a, ToLuaStack a)+tryFilter :: (Data a, Peekable a, Pushable a)           => LuaFilter -> a -> Lua [a] tryFilter (LuaFilter fnMap) x =   let filterFnName = showConstr (toConstr x)@@ -96,14 +116,13 @@ -- called with given element as argument and is expected to return an element. -- Alternatively, the function can return nothing or nil, in which case the -- element is left unchanged.-runFilterFunction :: ToLuaStack a => LuaFilterFunction -> a -> Lua ()+runFilterFunction :: Pushable a => LuaFilterFunction -> a -> Lua () runFilterFunction lf x = do-  pushFilterFunction lf-  push x-  z <- Lua.pcall 1 1 Nothing-  when (z /= OK) $ do-    let addPrefix = ("Error while running filter function: " ++)-    Lua.throwTopMessageAsError' addPrefix+  let errorPrefix = "Error while running filter function:\n"+  Lua.withExceptionMessage (errorPrefix <>) $ do+    pushFilterFunction lf+    Lua.push x+    Lua.call 1 1  walkMWithLuaFilter :: LuaFilter -> Pandoc -> Lua Pandoc walkMWithLuaFilter f =@@ -156,7 +175,7 @@ pandocFilterNames :: [String] pandocFilterNames = ["Pandoc", "Doc"] -singleElement :: FromLuaStack a => a -> Lua a+singleElement :: Peekable a => a -> Lua a singleElement x = do   elementUnchanged <- Lua.isnil (-1)   if elementUnchanged@@ -167,6 +186,6 @@       Right res -> res <$ Lua.pop 1       Left err  -> do         Lua.pop 1-        Lua.throwLuaError $+        Lua.throwException $           "Error while trying to get a filter's return " ++           "value from lua stack.\n" ++ err

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de> @@ -16,6 +15,7 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA -}+{-# LANGUAGE NoImplicitPrelude #-} {- |    Module      : Text.Pandoc.Lua    Copyright   : Copyright © 2017-2018 Albert Krewinkel@@ -40,7 +40,7 @@ import Data.Data (Data, dataTypeConstrs, dataTypeOf, showConstr) import Data.IORef (newIORef, readIORef) import Data.Version (Version (versionBranch))-import Foreign.Lua (Lua, LuaException (..))+import Foreign.Lua (Lua) import GHC.IO.Encoding (getForeignEncoding, setForeignEncoding, utf8) import Paths_pandoc (version) import Text.Pandoc.Class (PandocIO, getCommonState, getUserDataDir, getMediaBag,@@ -54,17 +54,22 @@ import qualified Foreign.Lua.Module.Text as Lua import qualified Text.Pandoc.Definition as Pandoc +-- | Lua error message+newtype LuaException = LuaException String deriving (Show)+ -- | Run the lua interpreter, using pandoc's default way of environment -- initialization. runPandocLua :: Lua a -> PandocIO (Either LuaException a) runPandocLua luaOp = do   luaPkgParams <- luaPackageParams   enc <- liftIO $ getForeignEncoding <* setForeignEncoding utf8-  res <- liftIO $ Lua.runLuaEither (initLuaState luaPkgParams *> luaOp)+  res <- liftIO $ Lua.runEither (initLuaState luaPkgParams *> luaOp)   liftIO $ setForeignEncoding enc   newMediaBag <- liftIO (readIORef (luaPkgMediaBag luaPkgParams))   setMediaBag newMediaBag-  return res+  return $ case res of+    Left (Lua.Exception msg) -> Left (LuaException msg)+    Right x -> Right x  -- | Generate parameters required to setup pandoc's lua environment. luaPackageParams :: PandocIO LuaPackageParams

@@ -87,7 +87,7 @@   zipWithM_ addEntry [1..] dirContents   return 1  where-  addEntry :: Int -> (FilePath, MimeType, Int) -> Lua ()+  addEntry :: Lua.Integer -> (FilePath, MimeType, Int) -> Lua ()   addEntry idx (fp, mimeType, contentLength) = do     Lua.newtable     Lua.push "path" *> Lua.push fp *> Lua.rawset (-3)

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de> @@ -17,6 +16,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA -} {-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE NoImplicitPrelude #-} {- |    Module      : Text.Pandoc.Lua.Module.Pandoc    Copyright   : Copyright © 2017-2018 Albert Krewinkel@@ -36,13 +36,12 @@ import Data.Default (Default (..)) import Data.Maybe (fromMaybe) import Data.Text (pack)-import Foreign.Lua (ToLuaStack, FromLuaStack, Lua, NumResults, Optional, liftIO)+import Foreign.Lua (Lua, NumResults, Optional, Peekable, Pushable) import System.Exit (ExitCode (..)) import Text.Pandoc.Class (runIO) import Text.Pandoc.Definition (Block, Inline) import Text.Pandoc.Lua.Filter (walkInlines, walkBlocks, LuaFilter) import Text.Pandoc.Lua.StackInstances ()-import Text.Pandoc.Lua.Util (addFunction, loadScriptFromDataDir) import Text.Pandoc.Walk (Walkable) import Text.Pandoc.Options (ReaderOptions (readerExtensions)) import Text.Pandoc.Process (pipeProcess)@@ -57,14 +56,14 @@ -- loaded. pushModule :: Maybe FilePath -> Lua NumResults pushModule datadir = do-  loadScriptFromDataDir datadir "pandoc.lua"-  addFunction "read" readDoc-  addFunction "pipe" pipeFn-  addFunction "walk_block" walkBlock-  addFunction "walk_inline" walkInline+  LuaUtil.loadScriptFromDataDir datadir "pandoc.lua"+  LuaUtil.addFunction "read" readDoc+  LuaUtil.addFunction "pipe" pipeFn+  LuaUtil.addFunction "walk_block" walkBlock+  LuaUtil.addFunction "walk_inline" walkInline   return 1 -walkElement :: (ToLuaStack a, Walkable [Inline] a, Walkable [Block] a)+walkElement :: (Pushable a, Walkable [Inline] a, Walkable [Block] a)             => a -> LuaFilter -> Lua a walkElement x f = walkInlines f x >>= walkBlocks f @@ -82,7 +81,8 @@     Right (reader, es) ->       case reader of         TextReader r -> do-          res <- liftIO $ runIO $ r def{ readerExtensions = es } (pack content)+          res <- Lua.liftIO . runIO $+                 r def{ readerExtensions = es } (pack content)           case res of             Right pd -> (1 :: NumResults) <$ Lua.push pd -- success, push Pandoc             Left s   -> Lua.raiseError (show s)          -- error while reading@@ -94,7 +94,7 @@        -> BL.ByteString        -> Lua NumResults pipeFn command args input = do-  (ec, output) <- liftIO $ pipeProcess Nothing command args input+  (ec, output) <- Lua.liftIO $ pipeProcess Nothing command args input   case ec of     ExitSuccess -> 1 <$ Lua.push output     ExitFailure n -> Lua.raiseError (PipeError command n output)@@ -105,14 +105,14 @@   , pipeErrorOutput :: BL.ByteString   } -instance FromLuaStack PipeError where+instance Peekable PipeError where   peek idx =     PipeError     <$> (Lua.getfield idx "command"    *> Lua.peek (-1) <* Lua.pop 1)     <*> (Lua.getfield idx "error_code" *> Lua.peek (-1) <* Lua.pop 1)     <*> (Lua.getfield idx "output"     *> Lua.peek (-1) <* Lua.pop 1) -instance ToLuaStack PipeError where+instance Pushable PipeError where   push pipeErr = do     Lua.newtable     LuaUtil.addField "command" (pipeErrorCommand pipeErr)@@ -124,7 +124,7 @@         pushPipeErrorMetaTable :: Lua ()         pushPipeErrorMetaTable = do           v <- Lua.newmetatable "pandoc pipe error"-          when v $ addFunction "__tostring" pipeErrorMessage+          when v $ LuaUtil.addFunction "__tostring" pipeErrorMessage          pipeErrorMessage :: PipeError -> Lua BL.ByteString         pipeErrorMessage (PipeError cmd errorCode output) = return $ mconcat

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de> @@ -16,6 +15,7 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA -}+{-# LANGUAGE NoImplicitPrelude #-} {- |    Module      : Text.Pandoc.Lua.Module.Utils    Copyright   : Copyright © 2017-2018 Albert Krewinkel@@ -33,11 +33,11 @@ import Prelude import Control.Applicative ((<|>)) import Data.Default (def)-import Foreign.Lua (FromLuaStack, Lua, LuaInteger, NumResults)+import Foreign.Lua (Peekable, Lua, NumResults) import Text.Pandoc.Class (runIO, setUserDataDir) import Text.Pandoc.Definition (Pandoc, Meta, MetaValue, Block, Inline) import Text.Pandoc.Lua.StackInstances ()-import Text.Pandoc.Lua.Util (addFunction, popValue)+import Text.Pandoc.Lua.Util (addFunction)  import qualified Data.Digest.Pure.SHA as SHA import qualified Data.ByteString.Lazy as BSL@@ -89,7 +89,7 @@             Just x -> return x             Nothing -> do               Lua.getglobal "FORMAT"-              (:[]) <$> popValue+              (:[]) <$> Lua.popValue   filterRes <- Lua.liftIO . runIO $ do     setUserDataDir mbDatadir     JSONFilter.apply def args filterFile doc@@ -121,18 +121,18 @@   | MetaValueElement MetaValue   deriving (Show) -instance FromLuaStack AstElement where+instance Peekable AstElement where   peek idx  = do-    res <- Lua.tryLua $  (PandocElement <$> Lua.peek idx)-                     <|> (InlineElement <$> Lua.peek idx)-                     <|> (BlockElement <$> Lua.peek idx)-                     <|> (MetaElement <$> Lua.peek idx)-                     <|> (MetaValueElement <$> Lua.peek idx)+    res <- Lua.try $  (PandocElement <$> Lua.peek idx)+                  <|> (InlineElement <$> Lua.peek idx)+                  <|> (BlockElement <$> Lua.peek idx)+                  <|> (MetaElement <$> Lua.peek idx)+                  <|> (MetaValueElement <$> Lua.peek idx)     case res of       Right x -> return x-      Left _ -> Lua.throwLuaError+      Left _ -> Lua.throwException         "Expected an AST element, but could not parse value as such."  -- | Convert a number < 4000 to uppercase roman numeral.-toRomanNumeral :: LuaInteger -> Lua String+toRomanNumeral :: Lua.Integer -> Lua String toRomanNumeral = return . Shared.toRomanNumeral . fromIntegral

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de> @@ -16,8 +15,9 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA -}-{-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE NoImplicitPrelude #-}+{-# LANGUAGE ScopedTypeVariables #-} {- |    Module      : Text.Pandoc.Lua.Packages    Copyright   : Copyright © 2017-2018 Albert Krewinkel@@ -35,12 +35,11 @@  import Prelude import Control.Monad (forM_)-import Data.ByteString.Char8 (unpack)+import Data.ByteString (ByteString) import Data.IORef (IORef) import Foreign.Lua (Lua, NumResults, liftIO) import Text.Pandoc.Class (CommonState, readDataFile, runIO, setUserDataDir) import Text.Pandoc.MediaBag (MediaBag)-import Text.Pandoc.Lua.Util (dostring')  import qualified Foreign.Lua as Lua import Text.Pandoc.Lua.Module.Pandoc as Pandoc@@ -57,14 +56,10 @@ -- | Insert pandoc's package loader as the first loader, making it the default. installPandocPackageSearcher :: LuaPackageParams -> Lua () installPandocPackageSearcher luaPkgParams = do-  luaVersion <- Lua.getglobal "_VERSION" *> Lua.peek (-1)-  if luaVersion == "Lua 5.1"-    then Lua.getglobal' "package.loaders"-    else Lua.getglobal' "package.searchers"+  Lua.getglobal' "package.searchers"   shiftArray   Lua.pushHaskellFunction (pandocPackageSearcher luaPkgParams)-  Lua.wrapHaskellFunction-  Lua.rawseti (-2) 1+  Lua.rawseti (Lua.nthFromTop 2) 1   Lua.pop 1           -- remove 'package.searchers' from stack  where   shiftArray = forM_ [4, 3, 2, 1] $ \i -> do@@ -86,7 +81,6 @@  where   pushWrappedHsFun f = do     Lua.pushHaskellFunction f-    Lua.wrapHaskellFunction     return 1   searchPureLuaLoader = do     let filename = pkgName ++ ".lua"@@ -97,21 +91,19 @@         Lua.push ("no file '" ++ filename ++ "' in pandoc's datadir")         return 1 -loadStringAsPackage :: String -> String -> Lua NumResults+loadStringAsPackage :: String -> ByteString -> Lua NumResults loadStringAsPackage pkgName script = do-  status <- dostring' script+  status <- Lua.dostring script   if status == Lua.OK     then return (1 :: NumResults)     else do-      msg <- Lua.peek (-1) <* Lua.pop 1-      Lua.push ("Error while loading ``" ++ pkgName ++ "`.\n" ++ msg)-      Lua.lerror-      return (2 :: NumResults)+      msg <- Lua.popValue+      Lua.raiseError ("Error while loading `" <> pkgName <> "`.\n" <> msg) --- | Get the string representation of the pandoc module-dataDirScript :: Maybe FilePath -> FilePath -> IO (Maybe String)+-- | Get the ByteString representation of the pandoc module.+dataDirScript :: Maybe FilePath -> FilePath -> IO (Maybe ByteString) dataDirScript datadir moduleFile = do   res <- runIO $ setUserDataDir datadir >> readDataFile moduleFile   return $ case res of     Left _ -> Nothing-    Right s -> Just (unpack s)+    Right s -> Just s

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2012-2018 John MacFarlane <jgm@berkeley.edu>             2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>@@ -19,6 +18,7 @@ -} {-# LANGUAGE FlexibleInstances    #-} {-# LANGUAGE LambdaCase           #-}+{-# LANGUAGE NoImplicitPrelude    #-} {-# LANGUAGE ScopedTypeVariables  #-} {-# OPTIONS_GHC -fno-warn-orphans #-} {- |@@ -37,67 +37,59 @@ import Prelude import Control.Applicative ((<|>)) import Control.Monad (when)-import Control.Monad.Catch (finally) import Data.Data (showConstr, toConstr)-import Data.Foldable (forM_)-import Foreign.Lua (FromLuaStack (peek), Lua, LuaInteger, LuaNumber, StackIndex,-                    ToLuaStack (push), Type (..), throwLuaError, tryLua)+import Foreign.Lua (Lua, Peekable, Pushable, StackIndex) import Text.Pandoc.Definition import Text.Pandoc.Extensions (Extensions)-import Text.Pandoc.Lua.Util (pushViaConstructor, typeCheck)+import Text.Pandoc.Lua.Util (defineHowTo, pushViaConstructor) import Text.Pandoc.Options (ReaderOptions (..), TrackChanges)-import Text.Pandoc.Shared (Element (Blk, Sec), safeRead)+import Text.Pandoc.Shared (Element (Blk, Sec)) -import qualified Foreign.Lua as Lua import qualified Data.Set as Set+import qualified Foreign.Lua as Lua import qualified Text.Pandoc.Lua.Util as LuaUtil -defineHowTo :: String -> Lua a -> Lua a-defineHowTo ctx op = op `Lua.modifyLuaError` (("Could not " ++ ctx ++ ": ") ++)--instance ToLuaStack Pandoc where+instance Pushable Pandoc where   push (Pandoc meta blocks) =     pushViaConstructor "Pandoc" blocks meta -instance FromLuaStack Pandoc where+instance Peekable Pandoc where   peek idx = defineHowTo "get Pandoc value" $ do-    typeCheck idx Lua.TypeTable     blocks <- LuaUtil.rawField idx "blocks"-    meta   <- Lua.getfield idx "meta" *> (Lua.peek Lua.stackTop `finally` Lua.pop 1)+    meta   <- LuaUtil.rawField idx "meta"     return $ Pandoc meta blocks -instance ToLuaStack Meta where+instance Pushable Meta where   push (Meta mmap) =     pushViaConstructor "Meta" mmap-instance FromLuaStack Meta where-  peek idx = defineHowTo "get Meta value" $ do-    typeCheck idx Lua.TypeTable-    Meta <$> peek idx+instance Peekable Meta where+  peek idx = defineHowTo "get Meta value" $+    Meta <$> Lua.peek idx -instance ToLuaStack MetaValue where+instance Pushable MetaValue where   push = pushMetaValue-instance FromLuaStack MetaValue where+instance Peekable MetaValue where   peek = peekMetaValue -instance ToLuaStack Block where+instance Pushable Block where   push = pushBlock -instance FromLuaStack Block where+instance Peekable Block where   peek = peekBlock  -- Inline-instance ToLuaStack Inline where+instance Pushable Inline where   push = pushInline -instance FromLuaStack Inline where+instance Peekable Inline where   peek = peekInline  -- Citation-instance ToLuaStack Citation where+instance Pushable Citation where   push (Citation cid prefix suffix mode noteNum hash) =     pushViaConstructor "Citation" cid mode prefix suffix noteNum hash -instance FromLuaStack Citation where+instance Peekable Citation where   peek idx = do     id' <- LuaUtil.rawField idx "id"     prefix <- LuaUtil.rawField idx "prefix"@@ -107,78 +99,63 @@     hash <- LuaUtil.rawField idx "hash"     return $ Citation id' prefix suffix mode num hash -instance ToLuaStack Alignment where-  push = push . show-instance FromLuaStack Alignment where-  peek idx = safeRead' =<< peek idx--instance ToLuaStack CitationMode where-  push = push . show-instance FromLuaStack CitationMode where-  peek idx = safeRead' =<< peek idx--instance ToLuaStack Format where-  push (Format f) = push f-instance FromLuaStack Format where-  peek idx = Format <$> peek idx--instance ToLuaStack ListNumberDelim where-  push = push . show-instance FromLuaStack ListNumberDelim where-  peek idx = safeRead' =<< peek idx+instance Pushable Alignment where+  push = Lua.push . show+instance Peekable Alignment where+  peek = Lua.peekRead -instance ToLuaStack ListNumberStyle where-  push = push . show-instance FromLuaStack ListNumberStyle where-  peek idx = safeRead' =<< peek idx+instance Pushable CitationMode where+  push = Lua.push . show+instance Peekable CitationMode where+  peek = Lua.peekRead -instance ToLuaStack MathType where-  push = push . show-instance FromLuaStack MathType where-  peek idx = safeRead' =<< peek idx+instance Pushable Format where+  push (Format f) = Lua.push f+instance Peekable Format where+  peek idx = Format <$> Lua.peek idx -instance ToLuaStack QuoteType where-  push = push . show-instance FromLuaStack QuoteType where-  peek idx = safeRead' =<< peek idx+instance Pushable ListNumberDelim where+  push = Lua.push . show+instance Peekable ListNumberDelim where+  peek = Lua.peekRead -instance ToLuaStack Double where-  push = push . (realToFrac :: Double -> LuaNumber)-instance FromLuaStack Double where-  peek = fmap (realToFrac :: LuaNumber -> Double) . peek+instance Pushable ListNumberStyle where+  push = Lua.push . show+instance Peekable ListNumberStyle where+  peek = Lua.peekRead -instance ToLuaStack Int where-  push = push . (fromIntegral :: Int -> LuaInteger)-instance FromLuaStack Int where-  peek = fmap (fromIntegral :: LuaInteger-> Int) . peek+instance Pushable MathType where+  push = Lua.push . show+instance Peekable MathType where+  peek = Lua.peekRead -safeRead' :: Read a => String -> Lua a-safeRead' s = case safeRead s of-  Nothing -> throwLuaError ("Could not read: " ++ s)-  Just x  -> return x+instance Pushable QuoteType where+  push = Lua.push . show+instance Peekable QuoteType where+  peek = Lua.peekRead  -- | Push an meta value element to the top of the lua stack. pushMetaValue :: MetaValue -> Lua () pushMetaValue = \case   MetaBlocks blcks  -> pushViaConstructor "MetaBlocks" blcks-  MetaBool bool     -> push bool+  MetaBool bool     -> Lua.push bool   MetaInlines inlns -> pushViaConstructor "MetaInlines" inlns   MetaList metalist -> pushViaConstructor "MetaList" metalist   MetaMap metamap   -> pushViaConstructor "MetaMap" metamap-  MetaString str    -> push str+  MetaString str    -> Lua.push str  -- | Interpret the value at the given stack index as meta value. peekMetaValue :: StackIndex -> Lua MetaValue peekMetaValue idx = defineHowTo "get MetaValue" $ do   -- Get the contents of an AST element.-  let elementContent :: FromLuaStack a => Lua a-      elementContent = peek idx+  let elementContent :: Peekable a => Lua a+      elementContent = Lua.peek idx   luatype <- Lua.ltype idx   case luatype of-    TypeBoolean -> MetaBool <$> peek idx-    TypeString  -> MetaString <$> peek idx-    TypeTable   -> do-      tag <- tryLua $ LuaUtil.getTag idx+    Lua.TypeBoolean -> MetaBool <$> Lua.peek idx+    Lua.TypeString  -> MetaString <$> Lua.peek idx+    Lua.TypeTable   -> do+      tag <- Lua.try $ LuaUtil.getTag idx       case tag of         Right "MetaBlocks"  -> MetaBlocks  <$> elementContent         Right "MetaBool"    -> MetaBool    <$> elementContent@@ -186,16 +163,16 @@         Right "MetaInlines" -> MetaInlines <$> elementContent         Right "MetaList"    -> MetaList    <$> elementContent         Right "MetaString"  -> MetaString  <$> elementContent-        Right t             -> throwLuaError ("Unknown meta tag: " ++ t)+        Right t             -> Lua.throwException ("Unknown meta tag: " <> t)         Left _ -> do           -- no meta value tag given, try to guess.           len <- Lua.rawlen idx           if len <= 0-            then MetaMap <$> peek idx-            else  (MetaInlines <$> peek idx)-                  <|> (MetaBlocks <$> peek idx)-                  <|> (MetaList <$> peek idx)-    _        -> throwLuaError "could not get meta value"+            then MetaMap <$> Lua.peek idx+            else  (MetaInlines <$> Lua.peek idx)+                  <|> (MetaBlocks <$> Lua.peek idx)+                  <|> (MetaList <$> Lua.peek idx)+    _        -> Lua.throwException "could not get meta value"  -- | Push an block element to the top of the lua stack. pushBlock :: Block -> Lua ()@@ -219,7 +196,6 @@ -- | Return the value at the given index as block if possible. peekBlock :: StackIndex -> Lua Block peekBlock idx = defineHowTo "get Block value" $ do-  typeCheck idx Lua.TypeTable   tag <- LuaUtil.getTag idx   case tag of       "BlockQuote"     -> BlockQuote <$> elementContent@@ -239,10 +215,10 @@       "Table"          -> (\(capt, aligns, widths, headers, body) ->                                   Table capt aligns widths headers body)                           <$> elementContent-      _ -> throwLuaError ("Unknown block type: " ++ tag)+      _ -> Lua.throwException ("Unknown block type: " <> tag)  where    -- Get the contents of an AST element.-   elementContent :: FromLuaStack a => Lua a+   elementContent :: Peekable a => Lua a    elementContent = LuaUtil.rawField idx "c"  -- | Push an inline element to the top of the lua stack.@@ -271,7 +247,6 @@ -- | Return the value at the given index as inline if possible. peekInline :: StackIndex -> Lua Inline peekInline idx = defineHowTo "get Inline value" $ do-  typeCheck idx Lua.TypeTable   tag <- LuaUtil.getTag idx   case tag of     "Cite"       -> uncurry Cite <$> elementContent@@ -295,10 +270,10 @@     "Strong"     -> Strong <$> elementContent     "Subscript"  -> Subscript <$> elementContent     "Superscript"-> Superscript <$> elementContent-    _ -> throwLuaError ("Unknown inline type: " ++ tag)+    _ -> Lua.throwException ("Unknown inline type: " <> tag)  where    -- Get the contents of an AST element.-   elementContent :: FromLuaStack a => Lua a+   elementContent :: Peekable a => Lua a    elementContent = LuaUtil.rawField idx "c"  withAttr :: (Attr -> a -> b) -> (LuaAttr, a) -> b@@ -307,18 +282,18 @@ -- | Wrapper for Attr newtype LuaAttr = LuaAttr { fromLuaAttr :: Attr } -instance ToLuaStack LuaAttr where+instance Pushable LuaAttr where   push (LuaAttr (id', classes, kv)) =     pushViaConstructor "Attr" id' classes kv -instance FromLuaStack LuaAttr where-  peek idx = defineHowTo "get Attr value" (LuaAttr <$> peek idx)+instance Peekable LuaAttr where+  peek idx = defineHowTo "get Attr value" (LuaAttr <$> Lua.peek idx)  -- -- Hierarchical elements ---instance ToLuaStack Element where-  push (Blk blk) = push blk+instance Pushable Element where+  push (Blk blk) = Lua.push blk   push (Sec lvl num attr label contents) = do     Lua.newtable     LuaUtil.addField "level" lvl@@ -342,18 +317,13 @@ -- -- Reader Options ---instance ToLuaStack Extensions where-  push exts = push (show exts)--instance ToLuaStack TrackChanges where-  push = push . showConstr . toConstr+instance Pushable Extensions where+  push exts = Lua.push (show exts) -instance ToLuaStack a => ToLuaStack (Set.Set a) where-  push set = do-    Lua.newtable-    forM_ set (`LuaUtil.addValue` True)+instance Pushable TrackChanges where+  push = Lua.push . showConstr . toConstr -instance ToLuaStack ReaderOptions where+instance Pushable ReaderOptions where   push ro = do     let ReaderOptions           (extensions            :: Extensions)

@@ -1,4 +1,3 @@-{-# LANGUAGE NoImplicitPrelude #-} {- Copyright © 2012-2018 John MacFarlane <jgm@berkeley.edu>             2017-2018 Albert Krewinkel <tarleb+pandoc@moltkeplatz.de>@@ -18,6 +17,8 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA -} {-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE LambdaCase #-}+{-# LANGUAGE NoImplicitPrelude #-} {- |    Module      : Text.Pandoc.Lua.Util    Copyright   : © 2012–2018 John MacFarlane,@@ -35,39 +36,35 @@   , addField   , addFunction   , addValue-  , typeCheck-  , popValue-  , PushViaCall-  , pushViaCall   , pushViaConstructor   , loadScriptFromDataDir-  , dostring'+  , defineHowTo+  , throwTopMessageAsError'   ) where  import Prelude-import Control.Monad (when)-import Control.Monad.Catch (finally)-import Data.ByteString.Char8 (unpack)-import Foreign.Lua (FromLuaStack, Lua, NumArgs, StackIndex, Status,-                    ToLuaStack, ToHaskellFunction)+import Control.Monad (unless, when)+import Foreign.Lua ( Lua, NumArgs, Peekable, Pushable, StackIndex+                   , ToHaskellFunction ) import Text.Pandoc.Class (readDataFile, runIOorExplode, setUserDataDir)  import qualified Foreign.Lua as Lua+import qualified Text.Pandoc.UTF8 as UTF8  -- | Get value behind key from table at given index.-rawField :: FromLuaStack a => StackIndex -> String -> Lua a+rawField :: Peekable a => StackIndex -> String -> Lua a rawField idx key = do   absidx <- Lua.absindex idx   Lua.push key   Lua.rawget absidx-  popValue+  Lua.popValue  -- | Add a value to the table at the top of the stack at a string-index.-addField :: ToLuaStack a => String -> a -> Lua ()+addField :: Pushable a => String -> a -> Lua () addField = addValue  -- | Add a key-value pair to the table at the top of the stack.-addValue :: (ToLuaStack a, ToLuaStack b) => a -> b -> Lua ()+addValue :: (Pushable a, Pushable b) => a -> b -> Lua () addValue key value = do   Lua.push key   Lua.push value@@ -78,26 +75,8 @@ addFunction name fn = do   Lua.push name   Lua.pushHaskellFunction fn-  Lua.wrapHaskellFunction   Lua.rawset (-3) -typeCheck :: StackIndex -> Lua.Type -> Lua ()-typeCheck idx expected = do-  actual <- Lua.ltype idx-  when (actual /= expected) $ do-    expName <- Lua.typename expected-    actName <- Lua.typename actual-    Lua.throwLuaError $ "expected " ++ expName ++ " but got " ++ actName ++ "."---- | Get, then pop the value at the top of the stack.-popValue :: FromLuaStack a => Lua a-popValue = do-  resOrError <- Lua.peekEither (-1)-  Lua.pop 1-  case resOrError of-    Left err -> Lua.throwLuaError err-    Right x -> return x- -- | Helper class for pushing a single value to the stack via a lua function. -- See @pushViaCall@. class PushViaCall a where@@ -110,7 +89,7 @@     pushArgs     Lua.call num 1 -instance (ToLuaStack a, PushViaCall b) => PushViaCall (a -> b) where+instance (Pushable a, PushViaCall b) => PushViaCall (a -> b) where   pushViaCall' fn pushArgs num x =     pushViaCall' fn (pushArgs *> Lua.push x) (num + 1) @@ -127,26 +106,11 @@ -- | Load a file from pandoc's data directory. loadScriptFromDataDir :: Maybe FilePath -> FilePath -> Lua () loadScriptFromDataDir datadir scriptFile = do-  script <- fmap unpack . Lua.liftIO . runIOorExplode $+  script <- Lua.liftIO . runIOorExplode $             setUserDataDir datadir >> readDataFile scriptFile-  status <- dostring' script-  when (status /= Lua.OK) .-    Lua.throwTopMessageAsError' $ \msg ->-      "Couldn't load '" ++ scriptFile ++ "'.\n" ++ msg---- | Load a string and immediately perform a full garbage collection. This is--- important to keep the program from hanging: If the program containes a call--- to @require@, then a new loader function is created which then becomes--- garbage. If that function is collected at an inopportune time, i.e. when the--- Lua API is called via a function that doesn't allow calling back into Haskell--- (getraw, setraw, …), then the function's finalizer, and the full program,--- will hang.-dostring' :: String -> Lua Status-dostring' script = do-  loadRes <- Lua.loadstring script-  if loadRes == Lua.OK-    then Lua.pcall 0 1 Nothing <* Lua.gc Lua.GCCOLLECT 0-    else return loadRes+  status <- Lua.dostring script+  when (status /= Lua.OK) $+    throwTopMessageAsError' (("Couldn't load '" ++ scriptFile ++ "'.\n") ++)  -- | Get the tag of a value. This is an optimized and specialized version of -- @Lua.getfield idx "tag"@. It only checks for the field on the table at index@@ -155,7 +119,21 @@ getTag :: StackIndex -> Lua String getTag idx = do   -- push metatable or just the table-  Lua.getmetatable idx >>= \hasMT -> when (not hasMT) (Lua.pushvalue idx)+  Lua.getmetatable idx >>= \hasMT -> unless hasMT (Lua.pushvalue idx)   Lua.push "tag"   Lua.rawget (Lua.nthFromTop 2)-  Lua.peek Lua.stackTop `finally` Lua.pop 2+  Lua.tostring Lua.stackTop <* Lua.pop 2 >>= \case+    Nothing -> Lua.throwException "untagged value"+    Just x -> return (UTF8.toString x)++-- | Modify the message at the top of the stack before throwing it as an+-- Exception.+throwTopMessageAsError' :: (String -> String) -> Lua a+throwTopMessageAsError' modifier = do+  msg <- Lua.tostring' Lua.stackTop+  Lua.pop 2 -- remove error and error string pushed by tostring'+  Lua.throwException (modifier (UTF8.toString msg))+++defineHowTo :: String -> Lua a -> Lua a+defineHowTo ctx = Lua.withExceptionMessage (("Could not " <> ctx <> ": ") <>)

@@ -47,8 +47,7 @@ import Control.Applicative (many, optional, (<|>)) import Control.Monad import Control.Monad.Except (throwError)-import Control.Monad.Trans (lift)-import Data.Char (chr, isAlphaNum, isDigit, isLetter, ord, toLower, toUpper)+import Data.Char (isDigit, isLetter, toLower, toUpper) import Data.Default import Data.List (intercalate, isPrefixOf) import qualified Data.Map as M@@ -63,7 +62,7 @@ import Text.Pandoc.Class (PandocMonad, PandocPure, getResourcePath, lookupEnv,                           readFileFromDirs, report, setResourcePath,                           setTranslations, translateTerm, trace)-import Text.Pandoc.Error (PandocError (PandocMacroLoop, PandocParseError, PandocParsecError))+import Text.Pandoc.Error (PandocError ( PandocParseError, PandocParsecError)) import Text.Pandoc.Highlighting (fromListingsLanguage, languagesByExtension) import Text.Pandoc.ImageSize (numUnit, showFl) import Text.Pandoc.Logging@@ -72,10 +71,10 @@                             optional, space, spaces, withRaw, (<|>)) import Text.Pandoc.Readers.LaTeX.Types (ExpansionPoint (..), Macro (..),                                         ArgSpec (..), Tok (..), TokType (..))+import Text.Pandoc.Readers.LaTeX.Parsing import Text.Pandoc.Shared import qualified Text.Pandoc.Translations as Translations import Text.Pandoc.Walk-import Text.Parsec.Pos import qualified Text.Pandoc.Builder as B  -- for debugging:@@ -137,152 +136,7 @@ --        Left e  -> error (show e) --        Right r -> return r -newtype DottedNum = DottedNum [Int]-  deriving (Show) -renderDottedNum :: DottedNum -> String-renderDottedNum (DottedNum xs) =-  intercalate "." (map show xs)--incrementDottedNum :: Int -> DottedNum -> DottedNum-incrementDottedNum level (DottedNum ns) = DottedNum $-  case reverse (take level (ns ++ repeat 0)) of-       (x:xs) -> reverse (x+1 : xs)-       []     -> []  -- shouldn't happen--data LaTeXState = LaTeXState{ sOptions       :: ReaderOptions-                            , sMeta          :: Meta-                            , sQuoteContext  :: QuoteContext-                            , sMacros        :: M.Map Text Macro-                            , sContainers    :: [String]-                            , sHeaders       :: M.Map Inlines String-                            , sLogMessages   :: [LogMessage]-                            , sIdentifiers   :: Set.Set String-                            , sVerbatimMode  :: Bool-                            , sCaption       :: (Maybe Inlines, Maybe String)-                            , sInListItem    :: Bool-                            , sInTableCell   :: Bool-                            , sLastHeaderNum :: DottedNum-                            , sLastFigureNum :: DottedNum-                            , sLabels        :: M.Map String [Inline]-                            , sHasChapters   :: Bool-                            , sToggles       :: M.Map String Bool-                            }-     deriving Show--defaultLaTeXState :: LaTeXState-defaultLaTeXState = LaTeXState{ sOptions       = def-                              , sMeta          = nullMeta-                              , sQuoteContext  = NoQuote-                              , sMacros        = M.empty-                              , sContainers    = []-                              , sHeaders       = M.empty-                              , sLogMessages   = []-                              , sIdentifiers   = Set.empty-                              , sVerbatimMode  = False-                              , sCaption       = (Nothing, Nothing)-                              , sInListItem    = False-                              , sInTableCell   = False-                              , sLastHeaderNum = DottedNum []-                              , sLastFigureNum = DottedNum []-                              , sLabels        = M.empty-                              , sHasChapters   = False-                              , sToggles       = M.empty-                              }--instance PandocMonad m => HasQuoteContext LaTeXState m where-  getQuoteContext = sQuoteContext <$> getState-  withQuoteContext context parser = do-    oldState <- getState-    let oldQuoteContext = sQuoteContext oldState-    setState oldState { sQuoteContext = context }-    result <- parser-    newState <- getState-    setState newState { sQuoteContext = oldQuoteContext }-    return result--instance HasLogMessages LaTeXState where-  addLogMessage msg st = st{ sLogMessages = msg : sLogMessages st }-  getLogMessages st = reverse $ sLogMessages st--instance HasIdentifierList LaTeXState where-  extractIdentifierList     = sIdentifiers-  updateIdentifierList f st = st{ sIdentifiers = f $ sIdentifiers st }--instance HasIncludeFiles LaTeXState where-  getIncludeFiles = sContainers-  addIncludeFile f s = s{ sContainers = f : sContainers s }-  dropLatestIncludeFile s = s { sContainers = drop 1 $ sContainers s }--instance HasHeaderMap LaTeXState where-  extractHeaderMap     = sHeaders-  updateHeaderMap f st = st{ sHeaders = f $ sHeaders st }--instance HasMacros LaTeXState where-  extractMacros  st  = sMacros st-  updateMacros f st  = st{ sMacros = f (sMacros st) }--instance HasReaderOptions LaTeXState where-  extractReaderOptions = sOptions--instance HasMeta LaTeXState where-  setMeta field val st =-    st{ sMeta = setMeta field val $ sMeta st }-  deleteMeta field st =-    st{ sMeta = deleteMeta field $ sMeta st }--instance Default LaTeXState where-  def = defaultLaTeXState--type LP m = ParserT [Tok] LaTeXState m--withVerbatimMode :: PandocMonad m => LP m a -> LP m a-withVerbatimMode parser = do-  updateState $ \st -> st{ sVerbatimMode = True }-  result <- parser-  updateState $ \st -> st{ sVerbatimMode = False }-  return result--rawLaTeXParser :: (PandocMonad m, HasMacros s, HasReaderOptions s)-               => Bool -> LP m a -> LP m a -> ParserT String s m (a, String)-rawLaTeXParser retokenize parser valParser = do-  inp <- getInput-  let toks = tokenize "source" $ T.pack inp-  pstate <- getState-  let lstate = def{ sOptions = extractReaderOptions pstate }-  let lstate' = lstate { sMacros = extractMacros pstate }-  let rawparser = (,) <$> withRaw valParser <*> getState-  res' <- lift $ runParserT (snd <$> withRaw parser) lstate "chunk" toks-  case res' of-       Left _    -> mzero-       Right toks' -> do-         res <- lift $ runParserT (do when retokenize $ do-                                        -- retokenize, applying macros-                                        doMacros 0-                                        ts <- many (satisfyTok (const True))-                                        setInput ts-                                      rawparser)-                        lstate' "chunk" toks'-         case res of-              Left _    -> mzero-              Right ((val, raw), st) -> do-                updateState (updateMacros (sMacros st <>))-                _ <- takeP (T.length (untokenize toks'))-                return (val, T.unpack (untokenize raw))--applyMacros :: (PandocMonad m, HasMacros s, HasReaderOptions s)-            => String -> ParserT String s m String-applyMacros s = (guardDisabled Ext_latex_macros >> return s) <|>-   do let retokenize = doMacros 0 *>-             (toksToString <$> many (satisfyTok (const True)))-      pstate <- getState-      let lstate = def{ sOptions = extractReaderOptions pstate-                      , sMacros  = extractMacros pstate }-      res <- runParserT retokenize lstate "math" (tokenize "math" (T.pack s))-      case res of-           Left e   -> fail (show e)-           Right s' -> return s'- rawLaTeXBlock :: (PandocMonad m, HasMacros s, HasReaderOptions s)               => ParserT String s m String rawLaTeXBlock = do@@ -326,353 +180,6 @@   lookAhead (try (char '\\' >> letter))   fst <$> rawLaTeXParser True (inlineEnvironment <|> inlineCommand') inlines -tokenize :: SourceName -> Text -> [Tok]-tokenize sourcename = totoks (initialPos sourcename)--totoks :: SourcePos -> Text -> [Tok]-totoks pos t =-  case T.uncons t of-       Nothing        -> []-       Just (c, rest)-         | c == '\n' ->-           Tok pos Newline "\n"-           : totoks (setSourceColumn (incSourceLine pos 1) 1) rest-         | isSpaceOrTab c ->-           let (sps, rest') = T.span isSpaceOrTab t-           in  Tok pos Spaces sps-               : totoks (incSourceColumn pos (T.length sps))-                 rest'-         | isAlphaNum c ->-           let (ws, rest') = T.span isAlphaNum t-           in  Tok pos Word ws-               : totoks (incSourceColumn pos (T.length ws)) rest'-         | c == '%' ->-           let (cs, rest') = T.break (== '\n') rest-           in  Tok pos Comment ("%" <> cs)-               : totoks (incSourceColumn pos (1 + T.length cs)) rest'-         | c == '\\' ->-           case T.uncons rest of-                Nothing -> [Tok pos (CtrlSeq " ") "\\"]-                Just (d, rest')-                  | isLetterOrAt d ->-                      -- \makeatletter is common in macro defs;-                      -- ideally we should make tokenization sensitive-                      -- to \makeatletter and \makeatother, but this is-                      -- probably best for now-                      let (ws, rest'') = T.span isLetterOrAt rest-                          (ss, rest''') = T.span isSpaceOrTab rest''-                      in  Tok pos (CtrlSeq ws) ("\\" <> ws <> ss)-                          : totoks (incSourceColumn pos-                               (1 + T.length ws + T.length ss)) rest'''-                  | isSpaceOrTab d || d == '\n' ->-                      let (w1, r1) = T.span isSpaceOrTab rest-                          (w2, (w3, r3)) = case T.uncons r1 of-                                          Just ('\n', r2)-                                                  -> (T.pack "\n",-                                                        T.span isSpaceOrTab r2)-                                          _ -> (mempty, (mempty, r1))-                          ws = "\\" <> w1 <> w2 <> w3-                      in  case T.uncons r3 of-                               Just ('\n', _) ->-                                 Tok pos (CtrlSeq " ") ("\\" <> w1)-                                 : totoks (incSourceColumn pos (T.length ws))-                                   r1-                               _ ->-                                 Tok pos (CtrlSeq " ") ws-                                 : totoks (incSourceColumn pos (T.length ws))-                                   r3-                  | otherwise  ->-                      Tok pos (CtrlSeq (T.singleton d)) (T.pack [c,d])-                      : totoks (incSourceColumn pos 2) rest'-         | c == '#' ->-           let (t1, t2) = T.span (\d -> d >= '0' && d <= '9') rest-           in  case safeRead (T.unpack t1) of-                    Just i ->-                       Tok pos (Arg i) ("#" <> t1)-                       : totoks (incSourceColumn pos (1 + T.length t1)) t2-                    Nothing ->-                       Tok pos Symbol "#"-                       : totoks (incSourceColumn pos 1) t2-         | c == '^' ->-           case T.uncons rest of-                Just ('^', rest') ->-                  case T.uncons rest' of-                       Just (d, rest'')-                         | isLowerHex d ->-                           case T.uncons rest'' of-                                Just (e, rest''') | isLowerHex e ->-                                  Tok pos Esc2 (T.pack ['^','^',d,e])-                                  : totoks (incSourceColumn pos 4) rest'''-                                _ ->-                                  Tok pos Esc1 (T.pack ['^','^',d])-                                  : totoks (incSourceColumn pos 3) rest''-                         | d < '\128' ->-                                  Tok pos Esc1 (T.pack ['^','^',d])-                                  : totoks (incSourceColumn pos 3) rest''-                       _ -> Tok pos Symbol "^" :-                            Tok (incSourceColumn pos 1) Symbol "^" :-                            totoks (incSourceColumn pos 2) rest'-                _ -> Tok pos Symbol "^"-                     : totoks (incSourceColumn pos 1) rest-         | otherwise ->-           Tok pos Symbol (T.singleton c) : totoks (incSourceColumn pos 1) rest--isSpaceOrTab :: Char -> Bool-isSpaceOrTab ' '  = True-isSpaceOrTab '\t' = True-isSpaceOrTab _    = False--isLetterOrAt :: Char -> Bool-isLetterOrAt '@' = True-isLetterOrAt c   = isLetter c--isLowerHex :: Char -> Bool-isLowerHex x = x >= '0' && x <= '9' || x >= 'a' && x <= 'f'--untokenize :: [Tok] -> Text-untokenize = mconcat . map untoken--untoken :: Tok -> Text-untoken (Tok _ _ t) = t--satisfyTok :: PandocMonad m => (Tok -> Bool) -> LP m Tok-satisfyTok f =-  try $ do-    res <- tokenPrim (T.unpack . untoken) updatePos matcher-    doMacros 0 -- apply macros on remaining input stream-    return res-  where matcher t | f t       = Just t-                  | otherwise = Nothing-        updatePos :: SourcePos -> Tok -> [Tok] -> SourcePos-        updatePos _spos _ (Tok pos _ _ : _) = pos-        updatePos spos _ []                 = incSourceColumn spos 1--doMacros :: PandocMonad m => Int -> LP m ()-doMacros n = do-  verbatimMode <- sVerbatimMode <$> getState-  unless verbatimMode $ do-    inp <- getInput-    case inp of-         Tok spos (CtrlSeq "begin") _ : Tok _ Symbol "{" :-          Tok _ Word name : Tok _ Symbol "}" : ts-            -> handleMacros spos name ts-         Tok spos (CtrlSeq "end") _ : Tok _ Symbol "{" :-          Tok _ Word name : Tok _ Symbol "}" : ts-            -> handleMacros spos ("end" <> name) ts-         Tok _ (CtrlSeq "expandafter") _ : t : ts-            -> do setInput ts-                  doMacros n-                  getInput >>= setInput . combineTok t-         Tok spos (CtrlSeq name) _ : ts-            -> handleMacros spos name ts-         _ -> return ()-  where combineTok (Tok spos (CtrlSeq name) x) (Tok _ Word w : ts)-          | T.all isLetterOrAt w =-            Tok spos (CtrlSeq (name <> w)) (x1 <> w <> x2) : ts-              where (x1, x2) = T.break isSpaceOrTab x-        combineTok t ts = t:ts-        handleMacros spos name ts = do-                macros <- sMacros <$> getState-                case M.lookup name macros of-                     Nothing -> return ()-                     Just (Macro expansionPoint argspecs optarg newtoks) -> do-                       setInput ts-                       let matchTok (Tok _ toktype txt) =-                             satisfyTok (\(Tok _ toktype' txt') ->-                                           toktype == toktype' &&-                                           txt == txt')-                       let matchPattern toks = try $ mapM_ matchTok toks-                       let getargs argmap [] = return argmap-                           getargs argmap (Pattern toks : rest) = try $ do-                              matchPattern toks-                              getargs argmap rest-                           getargs argmap (ArgNum i : Pattern toks : rest) =-                             try $ do-                               x <- mconcat <$> manyTill-                                     (braced <|> ((:[]) <$> anyTok))-                                     (matchPattern toks)-                               getargs (M.insert i x argmap) rest-                           getargs argmap (ArgNum i : rest) = do-                             x <- try $ spaces >> bracedOrToken-                             getargs (M.insert i x argmap) rest-                       args <- case optarg of-                                    Nothing -> getargs M.empty argspecs-                                    Just o  -> do-                                       x <- option o bracketedToks-                                       getargs (M.singleton 1 x) argspecs-                       -- first boolean param is true if we're tokenizing-                       -- an argument (in which case we don't want to-                       -- expand #1 etc.)-                       let addTok False (Tok _ (Arg i) _) acc =-                              case M.lookup i args of-                                   Nothing -> mzero-                                   Just xs -> foldr (addTok True) acc xs-                           -- see #4007-                           addTok _ (Tok _ (CtrlSeq x) txt)-                                  acc@(Tok _ Word _ : _)-                             | not (T.null txt) &&-                               isLetter (T.last txt) =-                               Tok spos (CtrlSeq x) (txt <> " ") : acc-                           addTok _ t acc = setpos spos t : acc-                       ts' <- getInput-                       setInput $ foldr (addTok False) ts' newtoks-                       case expansionPoint of-                            ExpandWhenUsed ->-                              if n > 20  -- detect macro expansion loops-                                 then throwError $ PandocMacroLoop (T.unpack name)-                                 else doMacros (n + 1)-                            ExpandWhenDefined -> return ()---setpos :: SourcePos -> Tok -> Tok-setpos spos (Tok _ tt txt) = Tok spos tt txt--anyControlSeq :: PandocMonad m => LP m Tok-anyControlSeq = satisfyTok isCtrlSeq--isCtrlSeq :: Tok -> Bool-isCtrlSeq (Tok _ (CtrlSeq _) _) = True-isCtrlSeq _                     = False--anySymbol :: PandocMonad m => LP m Tok-anySymbol = satisfyTok isSymbolTok--isSymbolTok :: Tok -> Bool-isSymbolTok (Tok _ Symbol _) = True-isSymbolTok _                = False--spaces :: PandocMonad m => LP m ()-spaces = skipMany (satisfyTok (tokTypeIn [Comment, Spaces, Newline]))--spaces1 :: PandocMonad m => LP m ()-spaces1 = skipMany1 (satisfyTok (tokTypeIn [Comment, Spaces, Newline]))--tokTypeIn :: [TokType] -> Tok -> Bool-tokTypeIn toktypes (Tok _ tt _) = tt `elem` toktypes--controlSeq :: PandocMonad m => Text -> LP m Tok-controlSeq name = satisfyTok isNamed-  where isNamed (Tok _ (CtrlSeq n) _) = n == name-        isNamed _                     = False--symbol :: PandocMonad m => Char -> LP m Tok-symbol c = satisfyTok isc-  where isc (Tok _ Symbol d) = case T.uncons d of-                                    Just (c',_) -> c == c'-                                    _           -> False-        isc _ = False--symbolIn :: PandocMonad m => [Char] -> LP m Tok-symbolIn cs = satisfyTok isInCs-  where isInCs (Tok _ Symbol d) = case T.uncons d of-                                       Just (c,_) -> c `elem` cs-                                       _          -> False-        isInCs _ = False--sp :: PandocMonad m => LP m ()-sp = whitespace <|> endline--whitespace :: PandocMonad m => LP m ()-whitespace = () <$ satisfyTok isSpaceTok--isSpaceTok :: Tok -> Bool-isSpaceTok (Tok _ Spaces _) = True-isSpaceTok _                = False--newlineTok :: PandocMonad m => LP m ()-newlineTok = () <$ satisfyTok isNewlineTok--isNewlineTok :: Tok -> Bool-isNewlineTok (Tok _ Newline _) = True-isNewlineTok _                 = False--comment :: PandocMonad m => LP m ()-comment = () <$ satisfyTok isCommentTok--isCommentTok :: Tok -> Bool-isCommentTok (Tok _ Comment _) = True-isCommentTok _                 = False--anyTok :: PandocMonad m => LP m Tok-anyTok = satisfyTok (const True)--endline :: PandocMonad m => LP m ()-endline = try $ do-  newlineTok-  lookAhead anyTok-  notFollowedBy blankline--blankline :: PandocMonad m => LP m ()-blankline = try $ skipMany whitespace *> newlineTok--primEscape :: PandocMonad m => LP m Char-primEscape = do-  Tok _ toktype t <- satisfyTok (tokTypeIn [Esc1, Esc2])-  case toktype of-       Esc1 -> case T.uncons (T.drop 2 t) of-                    Just (c, _)-                      | c >= '\64' && c <= '\127' -> return (chr (ord c - 64))-                      | otherwise                 -> return (chr (ord c + 64))-                    Nothing -> fail "Empty content of Esc1"-       Esc2 -> case safeRead ('0':'x':T.unpack (T.drop 2 t)) of-                    Just x  -> return (chr x)-                    Nothing -> fail $ "Could not read: " ++ T.unpack t-       _    -> fail "Expected an Esc1 or Esc2 token" -- should not happen--bgroup :: PandocMonad m => LP m Tok-bgroup = try $ do-  skipMany sp-  symbol '{' <|> controlSeq "bgroup" <|> controlSeq "begingroup"--egroup :: PandocMonad m => LP m Tok-egroup = symbol '}' <|> controlSeq "egroup" <|> controlSeq "endgroup"--grouped :: (PandocMonad m,  Monoid a) => LP m a -> LP m a-grouped parser = try $ do-  bgroup-  -- first we check for an inner 'grouped', because-  -- {{a,b}} should be parsed the same as {a,b}-  try (grouped parser <* egroup) <|> (mconcat <$> manyTill parser egroup)--braced' :: PandocMonad m => LP m Tok -> Int -> LP m [Tok]-braced' getTok n =-  handleEgroup <|> handleBgroup <|> handleOther-  where handleEgroup = do-          t <- egroup-          if n == 1-             then return []-             else (t:) <$> braced' getTok (n - 1)-        handleBgroup = do-          t <- bgroup-          (t:) <$> braced' getTok (n + 1)-        handleOther = do-          t <- getTok-          (t:) <$> braced' getTok n--braced :: PandocMonad m => LP m [Tok]-braced = bgroup *> braced' anyTok 1---- URLs require special handling, because they can contain %--- characters.  So we retonenize comments as we go...-bracedUrl :: PandocMonad m => LP m [Tok]-bracedUrl = bgroup *> braced' (retokenizeComment >> anyTok) 1--bracketed :: PandocMonad m => Monoid a => LP m a -> LP m a-bracketed parser = try $ do-  symbol '['-  mconcat <$> manyTill parser (symbol ']')--dimenarg :: PandocMonad m => LP m Text-dimenarg = try $ do-  ch  <- option False $ True <$ symbol '='-  Tok _ _ s <- satisfyTok isWordTok-  guard $ T.take 2 (T.reverse s) `elem`-           ["pt","pc","in","bp","cm","mm","dd","cc","sp"]-  let num = T.take (T.length s - 2) s-  guard $ T.length num > 0-  guard $ T.all isDigit num-  return $ T.pack ['=' | ch] <> s- -- inline elements:  word :: PandocMonad m => LP m Inlines@@ -684,13 +191,6 @@         isRegularSymbol _                = False         isSpecial c = c `Set.member` specialChars -specialChars :: Set.Set Char-specialChars = Set.fromList "#$%&~_^\\{}"--isWordTok :: Tok -> Bool-isWordTok (Tok _ Word _) = True-isWordTok _              = False- inlineGroup :: PandocMonad m => LP m Inlines inlineGroup = do   ils <- grouped inline@@ -1391,9 +891,6 @@ breve 'u' = "ŭ" breve c   = [c] -toksToString :: [Tok] -> String-toksToString = T.unpack . untokenize- mathDisplay :: String -> Inlines mathDisplay = displayMath . trim @@ -1470,7 +967,21 @@ cites :: PandocMonad m => CitationMode -> Bool -> LP m [Citation] cites mode multi = try $ do   cits <- if multi-             then many1 simpleCiteArgs+             then do+               multiprenote <- optionMaybe $ toList <$> paropt+               multipostnote <- optionMaybe $ toList <$> paropt+               let (pre, suf) = case (multiprenote, multipostnote) of+                     (Just s , Nothing) -> (mempty, s)+                     (Nothing , Just t) -> (mempty, t)+                     (Just s , Just t ) -> (s, t)+                     _                  -> (mempty, mempty)+               tempCits <- many1 simpleCiteArgs+               case tempCits of+                 (k:ks) -> case ks of+                             (_:_) -> return $ ((addMprenote pre k):init ks) +++                                                 [addMpostnote suf (last ks)]+                             _ -> return [addMprenote pre (addMpostnote suf k)]+                 _ -> return [[]]              else count 1 simpleCiteArgs   let cs = concat cits   return $ case mode of@@ -1478,6 +989,17 @@                              (c:rest) -> c {citationMode = mode} : rest                              []       -> []         _            -> map (\a -> a {citationMode = mode}) cs+  where mprenote (k:ks) = (k:ks) ++ [Space]+        mprenote _ = mempty+        mpostnote (k:ks) = [Str ",", Space] ++ (k:ks)+        mpostnote _ = mempty+        addMprenote mpn (k:ks) =+          let mpnfinal = case citationPrefix k of+                           (_:_) -> mprenote mpn+                           _ -> mpn+          in addPrefix mpnfinal (k:ks)+        addMprenote _ _ = []+        addMpostnote = addSuffix . mpostnote  citation :: PandocMonad m => String -> CitationMode -> Bool -> LP m Inlines citation name mode multi = do@@ -1532,22 +1054,12 @@           Tok _ _ t <- singleChar           return (str (T.unpack t)) -singleChar :: PandocMonad m => LP m Tok-singleChar = try $ do-  Tok pos toktype t <- satisfyTok (tokTypeIn [Word, Symbol])-  guard $ not $ toktype == Symbol &&-                T.any (`Set.member` specialChars) t-  if T.length t > 1-     then do-       let (t1, t2) = (T.take 1 t, T.drop 1 t)-       inp <- getInput-       setInput $ Tok (incSourceColumn pos 1) toktype t2 : inp-       return $ Tok pos toktype t1-     else return $ Tok pos toktype t- opt :: PandocMonad m => LP m Inlines opt = bracketed inline <|> (str . T.unpack <$> rawopt) +paropt :: PandocMonad m => LP m Inlines+paropt = parenWrapped inline+ rawopt :: PandocMonad m => LP m Text rawopt = do   inner <- untokenize <$> bracketedToks@@ -1578,20 +1090,6 @@                     Tok _ Symbol c     -> c `elem` ["-","+","@","|",":",","]                     _                  -> False) -ignore :: (Monoid a, PandocMonad m) => String -> ParserT s u m a-ignore raw = do-  pos <- getPosition-  report $ SkippedContent raw pos-  return mempty--withRaw :: PandocMonad m => LP m a -> LP m (a, [Tok])-withRaw parser = do-  inp <- getInput-  result <- parser-  nxt <- option (Tok (initialPos "source") Word "") (lookAhead anyTok)-  let raw = takeWhile (/= nxt) inp-  return (result, raw)- inBrackets :: Inlines -> Inlines inBrackets x = str "[" <> x <> str "]" @@ -1601,17 +1099,6 @@ unescapeURL (x:xs) = x:unescapeURL xs unescapeURL [] = "" --- For handling URLs, which allow literal % characters...-retokenizeComment :: PandocMonad m => LP m ()-retokenizeComment = (do-  Tok pos Comment txt <- satisfyTok isCommentTok-  let updPos (Tok pos' toktype' txt') =-        Tok (incSourceColumn (incSourceLine pos' (sourceLine pos - 1))-             (sourceColumn pos)) toktype' txt'-  let newtoks = map updPos $ tokenize (sourceName pos) $ T.tail txt-  getInput >>= setInput . ((Tok pos Symbol "%" : newtoks) ++))-    <|> return ()- mathEnvWith :: PandocMonad m             => (Inlines -> a) -> Maybe Text -> Text -> LP m a mathEnvWith f innerEnv name = f . mathDisplay . inner <$> mathEnv name@@ -2331,9 +1818,6 @@ isArgTok (Tok _ (Arg _) _) = True isArgTok _                 = False -bracedOrToken :: PandocMonad m => LP m [Tok]-bracedOrToken = braced <|> ((:[]) <$> (anyControlSeq <|> singleChar))- newcommand :: PandocMonad m => LP m (Text, Macro) newcommand = do   pos <- getPosition@@ -2383,11 +1867,6 @@          Nothing -> return ()   return (name, Macro ExpandWhenUsed argspecs optarg startcontents,              Macro ExpandWhenUsed [] Nothing endcontents)--bracketedToks :: PandocMonad m => LP m [Tok]-bracketedToks = do-  symbol '['-  mconcat <$> manyTill (braced <|> (:[]) <$> anyTok) (symbol ']')  bracketedNum :: PandocMonad m => LP m Int bracketedNum = do

@@ -0,0 +1,663 @@+{-# LANGUAGE NoImplicitPrelude #-}+{-# LANGUAGE FlexibleInstances     #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE OverloadedStrings     #-}+{-# LANGUAGE ScopedTypeVariables   #-}+{-+Copyright (C) 2006-2018 John MacFarlane <jgm@berkeley.edu>++This program is free software; you can redistribute it and/or modify+it under the terms of the GNU General Public License as published by+the Free Software Foundation; either version 2 of the License, or+(at your option) any later version.++This program is distributed in the hope that it will be useful,+but WITHOUT ANY WARRANTY; without even the implied warranty of+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the+GNU General Public License for more details.++You should have received a copy of the GNU General Public License+along with this program; if not, write to the Free Software+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA+-}++{- |+   Module      : Text.Pandoc.Readers.LaTeX.Parsing+   Copyright   : Copyright (C) 2006-2018 John MacFarlane+   License     : GNU GPL, version 2 or above++   Maintainer  : John MacFarlane <jgm@berkeley.edu>+   Stability   : alpha+   Portability : portable++General parsing types and functions for LaTeX.+-}+module Text.Pandoc.Readers.LaTeX.Parsing+  ( DottedNum(..)+  , renderDottedNum+  , incrementDottedNum+  , LaTeXState(..)+  , defaultLaTeXState+  , LP+  , withVerbatimMode+  , rawLaTeXParser+  , applyMacros+  , tokenize+  , untokenize+  , untoken+  , totoks+  , toksToString+  , satisfyTok+  , doMacros+  , setpos+  , anyControlSeq+  , anySymbol+  , isWordTok+  , isNewlineTok+  , spaces+  , spaces1+  , tokTypeIn+  , controlSeq+  , symbol+  , symbolIn+  , sp+  , whitespace+  , newlineTok+  , comment+  , anyTok+  , singleChar+  , specialChars+  , endline+  , blankline+  , primEscape+  , bgroup+  , egroup+  , grouped+  , braced+  , braced'+  , bracedUrl+  , bracedOrToken+  , bracketed+  , bracketedToks+  , parenWrapped+  , dimenarg+  , ignore+  , withRaw+  ) where++import Prelude+import Control.Applicative (many, (<|>))+import Control.Monad+import Control.Monad.Except (throwError)+import Control.Monad.Trans (lift)+import Data.Char (chr, isAlphaNum, isDigit, isLetter, ord)+import Data.Default+import Data.List (intercalate)+import qualified Data.Map as M+import qualified Data.Set as Set+import Data.Text (Text)+import qualified Data.Text as T+import Text.Pandoc.Builder+import Text.Pandoc.Class (PandocMonad, report)+import Text.Pandoc.Error (PandocError (PandocMacroLoop))+import Text.Pandoc.Logging+import Text.Pandoc.Options+import Text.Pandoc.Parsing hiding (blankline, many, mathDisplay, mathInline,+                            optional, space, spaces, withRaw, (<|>))+import Text.Pandoc.Readers.LaTeX.Types (ExpansionPoint (..), Macro (..),+                                        ArgSpec (..), Tok (..), TokType (..))+import Text.Pandoc.Shared+import Text.Parsec.Pos++newtype DottedNum = DottedNum [Int]+  deriving (Show)++renderDottedNum :: DottedNum -> String+renderDottedNum (DottedNum xs) =+  intercalate "." (map show xs)++incrementDottedNum :: Int -> DottedNum -> DottedNum+incrementDottedNum level (DottedNum ns) = DottedNum $+  case reverse (take level (ns ++ repeat 0)) of+       (x:xs) -> reverse (x+1 : xs)+       []     -> []  -- shouldn't happen++data LaTeXState = LaTeXState{ sOptions       :: ReaderOptions+                            , sMeta          :: Meta+                            , sQuoteContext  :: QuoteContext+                            , sMacros        :: M.Map Text Macro+                            , sContainers    :: [String]+                            , sHeaders       :: M.Map Inlines String+                            , sLogMessages   :: [LogMessage]+                            , sIdentifiers   :: Set.Set String+                            , sVerbatimMode  :: Bool+                            , sCaption       :: (Maybe Inlines, Maybe String)+                            , sInListItem    :: Bool+                            , sInTableCell   :: Bool+                            , sLastHeaderNum :: DottedNum+                            , sLastFigureNum :: DottedNum+                            , sLabels        :: M.Map String [Inline]+                            , sHasChapters   :: Bool+                            , sToggles       :: M.Map String Bool+                            }+     deriving Show++defaultLaTeXState :: LaTeXState+defaultLaTeXState = LaTeXState{ sOptions       = def+                              , sMeta          = nullMeta+                              , sQuoteContext  = NoQuote+                              , sMacros        = M.empty+                              , sContainers    = []+                              , sHeaders       = M.empty+                              , sLogMessages   = []+                              , sIdentifiers   = Set.empty+                              , sVerbatimMode  = False+                              , sCaption       = (Nothing, Nothing)+                              , sInListItem    = False+                              , sInTableCell   = False+                              , sLastHeaderNum = DottedNum []+                              , sLastFigureNum = DottedNum []+                              , sLabels        = M.empty+                              , sHasChapters   = False+                              , sToggles       = M.empty+                              }++instance PandocMonad m => HasQuoteContext LaTeXState m where+  getQuoteContext = sQuoteContext <$> getState+  withQuoteContext context parser = do+    oldState <- getState+    let oldQuoteContext = sQuoteContext oldState+    setState oldState { sQuoteContext = context }+    result <- parser+    newState <- getState+    setState newState { sQuoteContext = oldQuoteContext }+    return result++instance HasLogMessages LaTeXState where+  addLogMessage msg st = st{ sLogMessages = msg : sLogMessages st }+  getLogMessages st = reverse $ sLogMessages st++instance HasIdentifierList LaTeXState where+  extractIdentifierList     = sIdentifiers+  updateIdentifierList f st = st{ sIdentifiers = f $ sIdentifiers st }++instance HasIncludeFiles LaTeXState where+  getIncludeFiles = sContainers+  addIncludeFile f s = s{ sContainers = f : sContainers s }+  dropLatestIncludeFile s = s { sContainers = drop 1 $ sContainers s }++instance HasHeaderMap LaTeXState where+  extractHeaderMap     = sHeaders+  updateHeaderMap f st = st{ sHeaders = f $ sHeaders st }++instance HasMacros LaTeXState where+  extractMacros  st  = sMacros st+  updateMacros f st  = st{ sMacros = f (sMacros st) }++instance HasReaderOptions LaTeXState where+  extractReaderOptions = sOptions++instance HasMeta LaTeXState where+  setMeta field val st =+    st{ sMeta = setMeta field val $ sMeta st }+  deleteMeta field st =+    st{ sMeta = deleteMeta field $ sMeta st }++instance Default LaTeXState where+  def = defaultLaTeXState++type LP m = ParserT [Tok] LaTeXState m++withVerbatimMode :: PandocMonad m => LP m a -> LP m a+withVerbatimMode parser = do+  updateState $ \st -> st{ sVerbatimMode = True }+  result <- parser+  updateState $ \st -> st{ sVerbatimMode = False }+  return result++rawLaTeXParser :: (PandocMonad m, HasMacros s, HasReaderOptions s)+               => Bool -> LP m a -> LP m a -> ParserT String s m (a, String)+rawLaTeXParser retokenize parser valParser = do+  inp <- getInput+  let toks = tokenize "source" $ T.pack inp+  pstate <- getState+  let lstate = def{ sOptions = extractReaderOptions pstate }+  let lstate' = lstate { sMacros = extractMacros pstate }+  let rawparser = (,) <$> withRaw valParser <*> getState+  res' <- lift $ runParserT (snd <$> withRaw parser) lstate "chunk" toks+  case res' of+       Left _    -> mzero+       Right toks' -> do+         res <- lift $ runParserT (do when retokenize $ do+                                        -- retokenize, applying macros+                                        doMacros 0+                                        ts <- many (satisfyTok (const True))+                                        setInput ts+                                      rawparser)+                        lstate' "chunk" toks'+         case res of+              Left _    -> mzero+              Right ((val, raw), st) -> do+                updateState (updateMacros (sMacros st <>))+                _ <- takeP (T.length (untokenize toks'))+                return (val, T.unpack (untokenize raw))++applyMacros :: (PandocMonad m, HasMacros s, HasReaderOptions s)+            => String -> ParserT String s m String+applyMacros s = (guardDisabled Ext_latex_macros >> return s) <|>+   do let retokenize = doMacros 0 *>+             (toksToString <$> many (satisfyTok (const True)))+      pstate <- getState+      let lstate = def{ sOptions = extractReaderOptions pstate+                      , sMacros  = extractMacros pstate }+      res <- runParserT retokenize lstate "math" (tokenize "math" (T.pack s))+      case res of+           Left e   -> fail (show e)+           Right s' -> return s'+tokenize :: SourceName -> Text -> [Tok]+tokenize sourcename = totoks (initialPos sourcename)++totoks :: SourcePos -> Text -> [Tok]+totoks pos t =+  case T.uncons t of+       Nothing        -> []+       Just (c, rest)+         | c == '\n' ->+           Tok pos Newline "\n"+           : totoks (setSourceColumn (incSourceLine pos 1) 1) rest+         | isSpaceOrTab c ->+           let (sps, rest') = T.span isSpaceOrTab t+           in  Tok pos Spaces sps+               : totoks (incSourceColumn pos (T.length sps))+                 rest'+         | isAlphaNum c ->+           let (ws, rest') = T.span isAlphaNum t+           in  Tok pos Word ws+               : totoks (incSourceColumn pos (T.length ws)) rest'+         | c == '%' ->+           let (cs, rest') = T.break (== '\n') rest+           in  Tok pos Comment ("%" <> cs)+               : totoks (incSourceColumn pos (1 + T.length cs)) rest'+         | c == '\\' ->+           case T.uncons rest of+                Nothing -> [Tok pos (CtrlSeq " ") "\\"]+                Just (d, rest')+                  | isLetterOrAt d ->+                      -- \makeatletter is common in macro defs;+                      -- ideally we should make tokenization sensitive+                      -- to \makeatletter and \makeatother, but this is+                      -- probably best for now+                      let (ws, rest'') = T.span isLetterOrAt rest+                          (ss, rest''') = T.span isSpaceOrTab rest''+                      in  Tok pos (CtrlSeq ws) ("\\" <> ws <> ss)+                          : totoks (incSourceColumn pos+                               (1 + T.length ws + T.length ss)) rest'''+                  | isSpaceOrTab d || d == '\n' ->+                      let (w1, r1) = T.span isSpaceOrTab rest+                          (w2, (w3, r3)) = case T.uncons r1 of+                                          Just ('\n', r2)+                                                  -> (T.pack "\n",+                                                        T.span isSpaceOrTab r2)+                                          _ -> (mempty, (mempty, r1))+                          ws = "\\" <> w1 <> w2 <> w3+                      in  case T.uncons r3 of+                               Just ('\n', _) ->+                                 Tok pos (CtrlSeq " ") ("\\" <> w1)+                                 : totoks (incSourceColumn pos (T.length ws))+                                   r1+                               _ ->+                                 Tok pos (CtrlSeq " ") ws+                                 : totoks (incSourceColumn pos (T.length ws))+                                   r3+                  | otherwise  ->+                      Tok pos (CtrlSeq (T.singleton d)) (T.pack [c,d])+                      : totoks (incSourceColumn pos 2) rest'+         | c == '#' ->+           let (t1, t2) = T.span (\d -> d >= '0' && d <= '9') rest+           in  case safeRead (T.unpack t1) of+                    Just i ->+                       Tok pos (Arg i) ("#" <> t1)+                       : totoks (incSourceColumn pos (1 + T.length t1)) t2+                    Nothing ->+                       Tok pos Symbol "#"+                       : totoks (incSourceColumn pos 1) t2+         | c == '^' ->+           case T.uncons rest of+                Just ('^', rest') ->+                  case T.uncons rest' of+                       Just (d, rest'')+                         | isLowerHex d ->+                           case T.uncons rest'' of+                                Just (e, rest''') | isLowerHex e ->+                                  Tok pos Esc2 (T.pack ['^','^',d,e])+                                  : totoks (incSourceColumn pos 4) rest'''+                                _ ->+                                  Tok pos Esc1 (T.pack ['^','^',d])+                                  : totoks (incSourceColumn pos 3) rest''+                         | d < '\128' ->+                                  Tok pos Esc1 (T.pack ['^','^',d])+                                  : totoks (incSourceColumn pos 3) rest''+                       _ -> Tok pos Symbol "^" :+                            Tok (incSourceColumn pos 1) Symbol "^" :+                            totoks (incSourceColumn pos 2) rest'+                _ -> Tok pos Symbol "^"+                     : totoks (incSourceColumn pos 1) rest+         | otherwise ->+           Tok pos Symbol (T.singleton c) : totoks (incSourceColumn pos 1) rest++isSpaceOrTab :: Char -> Bool+isSpaceOrTab ' '  = True+isSpaceOrTab '\t' = True+isSpaceOrTab _    = False++isLetterOrAt :: Char -> Bool+isLetterOrAt '@' = True+isLetterOrAt c   = isLetter c++isLowerHex :: Char -> Bool+isLowerHex x = x >= '0' && x <= '9' || x >= 'a' && x <= 'f'++untokenize :: [Tok] -> Text+untokenize = mconcat . map untoken++untoken :: Tok -> Text+untoken (Tok _ _ t) = t++toksToString :: [Tok] -> String+toksToString = T.unpack . untokenize++satisfyTok :: PandocMonad m => (Tok -> Bool) -> LP m Tok+satisfyTok f =+  try $ do+    res <- tokenPrim (T.unpack . untoken) updatePos matcher+    doMacros 0 -- apply macros on remaining input stream+    return res+  where matcher t | f t       = Just t+                  | otherwise = Nothing+        updatePos :: SourcePos -> Tok -> [Tok] -> SourcePos+        updatePos _spos _ (Tok pos _ _ : _) = pos+        updatePos spos _ []                 = incSourceColumn spos 1++doMacros :: PandocMonad m => Int -> LP m ()+doMacros n = do+  verbatimMode <- sVerbatimMode <$> getState+  unless verbatimMode $ do+    inp <- getInput+    case inp of+         Tok spos (CtrlSeq "begin") _ : Tok _ Symbol "{" :+          Tok _ Word name : Tok _ Symbol "}" : ts+            -> handleMacros spos name ts+         Tok spos (CtrlSeq "end") _ : Tok _ Symbol "{" :+          Tok _ Word name : Tok _ Symbol "}" : ts+            -> handleMacros spos ("end" <> name) ts+         Tok _ (CtrlSeq "expandafter") _ : t : ts+            -> do setInput ts+                  doMacros n+                  getInput >>= setInput . combineTok t+         Tok spos (CtrlSeq name) _ : ts+            -> handleMacros spos name ts+         _ -> return ()+  where combineTok (Tok spos (CtrlSeq name) x) (Tok _ Word w : ts)+          | T.all isLetterOrAt w =+            Tok spos (CtrlSeq (name <> w)) (x1 <> w <> x2) : ts+              where (x1, x2) = T.break isSpaceOrTab x+        combineTok t ts = t:ts+        handleMacros spos name ts = do+                macros <- sMacros <$> getState+                case M.lookup name macros of+                     Nothing -> return ()+                     Just (Macro expansionPoint argspecs optarg newtoks) -> do+                       setInput ts+                       let matchTok (Tok _ toktype txt) =+                             satisfyTok (\(Tok _ toktype' txt') ->+                                           toktype == toktype' &&+                                           txt == txt')+                       let matchPattern toks = try $ mapM_ matchTok toks+                       let getargs argmap [] = return argmap+                           getargs argmap (Pattern toks : rest) = try $ do+                              matchPattern toks+                              getargs argmap rest+                           getargs argmap (ArgNum i : Pattern toks : rest) =+                             try $ do+                               x <- mconcat <$> manyTill+                                     (braced <|> ((:[]) <$> anyTok))+                                     (matchPattern toks)+                               getargs (M.insert i x argmap) rest+                           getargs argmap (ArgNum i : rest) = do+                             x <- try $ spaces >> bracedOrToken+                             getargs (M.insert i x argmap) rest+                       args <- case optarg of+                                    Nothing -> getargs M.empty argspecs+                                    Just o  -> do+                                       x <- option o bracketedToks+                                       getargs (M.singleton 1 x) argspecs+                       -- first boolean param is true if we're tokenizing+                       -- an argument (in which case we don't want to+                       -- expand #1 etc.)+                       let addTok False (Tok _ (Arg i) _) acc =+                              case M.lookup i args of+                                   Nothing -> mzero+                                   Just xs -> foldr (addTok True) acc xs+                           -- see #4007+                           addTok _ (Tok _ (CtrlSeq x) txt)+                                  acc@(Tok _ Word _ : _)+                             | not (T.null txt) &&+                               isLetter (T.last txt) =+                               Tok spos (CtrlSeq x) (txt <> " ") : acc+                           addTok _ t acc = setpos spos t : acc+                       ts' <- getInput+                       setInput $ foldr (addTok False) ts' newtoks+                       case expansionPoint of+                            ExpandWhenUsed ->+                              if n > 20  -- detect macro expansion loops+                                 then throwError $ PandocMacroLoop (T.unpack name)+                                 else doMacros (n + 1)+                            ExpandWhenDefined -> return ()+++setpos :: SourcePos -> Tok -> Tok+setpos spos (Tok _ tt txt) = Tok spos tt txt++anyControlSeq :: PandocMonad m => LP m Tok+anyControlSeq = satisfyTok isCtrlSeq++isCtrlSeq :: Tok -> Bool+isCtrlSeq (Tok _ (CtrlSeq _) _) = True+isCtrlSeq _                     = False++anySymbol :: PandocMonad m => LP m Tok+anySymbol = satisfyTok isSymbolTok++isSymbolTok :: Tok -> Bool+isSymbolTok (Tok _ Symbol _) = True+isSymbolTok _                = False++isWordTok :: Tok -> Bool+isWordTok (Tok _ Word _) = True+isWordTok _              = False++spaces :: PandocMonad m => LP m ()+spaces = skipMany (satisfyTok (tokTypeIn [Comment, Spaces, Newline]))++spaces1 :: PandocMonad m => LP m ()+spaces1 = skipMany1 (satisfyTok (tokTypeIn [Comment, Spaces, Newline]))++tokTypeIn :: [TokType] -> Tok -> Bool+tokTypeIn toktypes (Tok _ tt _) = tt `elem` toktypes++controlSeq :: PandocMonad m => Text -> LP m Tok+controlSeq name = satisfyTok isNamed+  where isNamed (Tok _ (CtrlSeq n) _) = n == name+        isNamed _                     = False++symbol :: PandocMonad m => Char -> LP m Tok+symbol c = satisfyTok isc+  where isc (Tok _ Symbol d) = case T.uncons d of+                                    Just (c',_) -> c == c'+                                    _           -> False+        isc _ = False++symbolIn :: PandocMonad m => [Char] -> LP m Tok+symbolIn cs = satisfyTok isInCs+  where isInCs (Tok _ Symbol d) = case T.uncons d of+                                       Just (c,_) -> c `elem` cs+                                       _          -> False+        isInCs _ = False++sp :: PandocMonad m => LP m ()+sp = whitespace <|> endline++whitespace :: PandocMonad m => LP m ()+whitespace = () <$ satisfyTok isSpaceTok++isSpaceTok :: Tok -> Bool+isSpaceTok (Tok _ Spaces _) = True+isSpaceTok _                = False++newlineTok :: PandocMonad m => LP m ()+newlineTok = () <$ satisfyTok isNewlineTok++isNewlineTok :: Tok -> Bool+isNewlineTok (Tok _ Newline _) = True+isNewlineTok _                 = False++comment :: PandocMonad m => LP m ()+comment = () <$ satisfyTok isCommentTok++isCommentTok :: Tok -> Bool+isCommentTok (Tok _ Comment _) = True+isCommentTok _                 = False++anyTok :: PandocMonad m => LP m Tok+anyTok = satisfyTok (const True)++singleChar :: PandocMonad m => LP m Tok+singleChar = try $ do+  Tok pos toktype t <- satisfyTok (tokTypeIn [Word, Symbol])+  guard $ not $ toktype == Symbol &&+                T.any (`Set.member` specialChars) t+  if T.length t > 1+     then do+       let (t1, t2) = (T.take 1 t, T.drop 1 t)+       inp <- getInput+       setInput $ Tok (incSourceColumn pos 1) toktype t2 : inp+       return $ Tok pos toktype t1+     else return $ Tok pos toktype t++specialChars :: Set.Set Char+specialChars = Set.fromList "#$%&~_^\\{}"++endline :: PandocMonad m => LP m ()+endline = try $ do+  newlineTok+  lookAhead anyTok+  notFollowedBy blankline++blankline :: PandocMonad m => LP m ()+blankline = try $ skipMany whitespace *> newlineTok++primEscape :: PandocMonad m => LP m Char+primEscape = do+  Tok _ toktype t <- satisfyTok (tokTypeIn [Esc1, Esc2])+  case toktype of+       Esc1 -> case T.uncons (T.drop 2 t) of+                    Just (c, _)+                      | c >= '\64' && c <= '\127' -> return (chr (ord c - 64))+                      | otherwise                 -> return (chr (ord c + 64))+                    Nothing -> fail "Empty content of Esc1"+       Esc2 -> case safeRead ('0':'x':T.unpack (T.drop 2 t)) of+                    Just x  -> return (chr x)+                    Nothing -> fail $ "Could not read: " ++ T.unpack t+       _    -> fail "Expected an Esc1 or Esc2 token" -- should not happen++bgroup :: PandocMonad m => LP m Tok+bgroup = try $ do+  skipMany sp+  symbol '{' <|> controlSeq "bgroup" <|> controlSeq "begingroup"++egroup :: PandocMonad m => LP m Tok+egroup = symbol '}' <|> controlSeq "egroup" <|> controlSeq "endgroup"++grouped :: (PandocMonad m,  Monoid a) => LP m a -> LP m a+grouped parser = try $ do+  bgroup+  -- first we check for an inner 'grouped', because+  -- {{a,b}} should be parsed the same as {a,b}+  try (grouped parser <* egroup) <|> (mconcat <$> manyTill parser egroup)++braced' :: PandocMonad m => LP m Tok -> Int -> LP m [Tok]+braced' getTok n =+  handleEgroup <|> handleBgroup <|> handleOther+  where handleEgroup = do+          t <- egroup+          if n == 1+             then return []+             else (t:) <$> braced' getTok (n - 1)+        handleBgroup = do+          t <- bgroup+          (t:) <$> braced' getTok (n + 1)+        handleOther = do+          t <- getTok+          (t:) <$> braced' getTok n++braced :: PandocMonad m => LP m [Tok]+braced = bgroup *> braced' anyTok 1++-- URLs require special handling, because they can contain %+-- characters.  So we retonenize comments as we go...+bracedUrl :: PandocMonad m => LP m [Tok]+bracedUrl = bgroup *> braced' (retokenizeComment >> anyTok) 1++-- For handling URLs, which allow literal % characters...+retokenizeComment :: PandocMonad m => LP m ()+retokenizeComment = (do+  Tok pos Comment txt <- satisfyTok isCommentTok+  let updPos (Tok pos' toktype' txt') =+        Tok (incSourceColumn (incSourceLine pos' (sourceLine pos - 1))+             (sourceColumn pos)) toktype' txt'+  let newtoks = map updPos $ tokenize (sourceName pos) $ T.tail txt+  getInput >>= setInput . ((Tok pos Symbol "%" : newtoks) ++))+    <|> return ()++bracedOrToken :: PandocMonad m => LP m [Tok]+bracedOrToken = braced <|> ((:[]) <$> (anyControlSeq <|> singleChar))++bracketed :: PandocMonad m => Monoid a => LP m a -> LP m a+bracketed parser = try $ do+  symbol '['+  mconcat <$> manyTill parser (symbol ']')++bracketedToks :: PandocMonad m => LP m [Tok]+bracketedToks = do+  symbol '['+  mconcat <$> manyTill (braced <|> (:[]) <$> anyTok) (symbol ']')++parenWrapped :: PandocMonad m => Monoid a => LP m a -> LP m a+parenWrapped parser = try $ do+  symbol '('+  mconcat <$> manyTill parser (symbol ')')++dimenarg :: PandocMonad m => LP m Text+dimenarg = try $ do+  ch  <- option False $ True <$ symbol '='+  Tok _ _ s <- satisfyTok isWordTok+  guard $ T.take 2 (T.reverse s) `elem`+           ["pt","pc","in","bp","cm","mm","dd","cc","sp"]+  let num = T.take (T.length s - 2) s+  guard $ T.length num > 0+  guard $ T.all isDigit num+  return $ T.pack ['=' | ch] <> s++ignore :: (Monoid a, PandocMonad m) => String -> ParserT s u m a+ignore raw = do+  pos <- getPosition+  report $ SkippedContent raw pos+  return mempty++withRaw :: PandocMonad m => LP m a -> LP m (a, [Tok])+withRaw parser = do+  inp <- getInput+  result <- parser+  nxt <- option (Tok (initialPos "source") Word "") (lookAhead anyTok)+  let raw = takeWhile (/= nxt) inp+  return (result, raw)

@@ -1,6 +1,7 @@ {-# LANGUAGE NoImplicitPrelude #-} {-# LANGUAGE RelaxedPolyRec      #-} {-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TupleSections #-}  {- Copyright (C) 2006-2018 John MacFarlane <jgm@berkeley.edu>@@ -1879,23 +1880,24 @@ bareURL = try $ do   guardEnabled Ext_autolink_bare_uris   getState >>= guard . stateAllowLinks-  (orig, src) <- uri <|> emailAddress+  (cls, (orig, src)) <- (("uri",) <$> uri) <|> (("email",) <$> emailAddress)   notFollowedBy $ try $ spaces >> htmlTag (~== TagClose "a")-  return $ return $ B.link src "" (B.str orig)+  return $ return $ B.linkWith ("",[cls],[]) src "" (B.str orig)  autoLink :: PandocMonad m => MarkdownParser m (F Inlines) autoLink = try $ do   getState >>= guard . stateAllowLinks   char '<'-  (orig, src) <- uri <|> emailAddress+  (cls, (orig, src)) <- (("uri",) <$> uri) <|> (("email",) <$> emailAddress)   -- in rare cases, something may remain after the uri parser   -- is finished, because the uri parser tries to avoid parsing   -- final punctuation.  for example:  in `<http://hi---there>`,   -- the URI parser will stop before the dashes.   extra <- fromEntities <$> manyTill nonspaceChar (char '>')-  attr  <- option nullAttr $ try $+  attr  <- option ("", [cls], []) $ try $             guardEnabled Ext_link_attributes >> attributes-  return $ return $ B.linkWith attr (src ++ escapeURI extra) "" (B.str $ orig ++ extra)+  return $ return $ B.linkWith attr (src ++ escapeURI extra) ""+                     (B.str $ orig ++ extra)  image :: PandocMonad m => MarkdownParser m (F Inlines) image = try $ do

@@ -43,7 +43,7 @@ import Control.Monad import Control.Monad.Except (throwError) import Data.Bifunctor-import Data.Char (isLetter, isDigit)+import Data.Char (isAlphaNum) import Data.Default import Data.List (intercalate) import Data.List.Split (splitOn)@@ -56,11 +56,11 @@ import qualified Text.Pandoc.Builder as B import Text.Pandoc.Class (PandocMonad (..)) import Text.Pandoc.Definition+import Text.Pandoc.Error (PandocError (PandocParsecError)) import Text.Pandoc.Logging import Text.Pandoc.Options import Text.Pandoc.Parsing hiding (F, enclosed)-import Text.Pandoc.Readers.HTML (htmlTag)-import Text.Pandoc.Shared (crFilter, underlineSpan)+import Text.Pandoc.Shared (crFilter, underlineSpan, mapLeft)  -- | Read Muse from an input string and return a Pandoc document. readMuse :: PandocMonad m@@ -68,7 +68,8 @@          -> Text          -> m Pandoc readMuse opts s = do-  res <- readWithM parseMuse def{ museOptions = opts } (unpack (crFilter s))+  let input = crFilter s+  res <- mapLeft (PandocParsecError $ unpack input) `liftM` runParserT parseMuse def{ museOptions = opts } "source" input   case res of        Left e  -> throwError e        Right d -> return d@@ -98,7 +99,7 @@                   , museInPara = False                   } -type MuseParser = ParserT String MuseState+type MuseParser = ParserT Text MuseState  instance HasReaderOptions MuseState where   extractReaderOptions = museOptions@@ -125,7 +126,7 @@   many directive   firstSection <- parseBlocks   rest <- many parseSection-  let blocks = mconcat $ (firstSection : rest)+  let blocks = mconcat (firstSection : rest)   st <- getState   let doc = runF (do Pandoc _ bs <- B.doc <$> blocks                      meta <- museMeta st@@ -144,9 +145,8 @@  -- | Trim up to one newline from the beginning of the string. lchop :: String -> String-lchop s = case s of-                    '\n':ss -> ss-                    _       -> s+lchop ('\n':xs) = xs+lchop s = s  -- | Trim up to one newline from the end of the string. rchop :: String -> String@@ -165,6 +165,9 @@   guard $ museLastStrPos st /= Just pos   p +firstColumn :: PandocMonad m => MuseParser m ()+firstColumn = getPosition >>= \pos -> guard (sourceColumn pos == 1)+ -- * Parsers  -- | Parse end-of-line, which can be either a newline or end-of-file.@@ -179,25 +182,18 @@  -- ** HTML parsers --- | Parse HTML tag, returning its attributes and literal contents.-htmlElement :: PandocMonad m-            => String -- ^ Tag name-            -> MuseParser m (Attr, String)-htmlElement tag = try $ do-  (TagOpen _ attr, _) <- htmlTag (~== TagOpen tag [])-  content <- manyTill anyChar endtag-  return (htmlAttrToPandoc attr, content)+openTag :: PandocMonad m => String -> MuseParser m [(String, String)]+openTag tag = try $+  char '<' *> string tag *> manyTill attr (char '>')   where-    endtag = void $ htmlTag (~== TagClose tag)+    attr = try $ (,)+      <$  many1 spaceChar+      <*> many1 (noneOf "=\n")+      <*  string "=\""+      <*> manyTill (noneOf "\"") (char '"') -htmlBlock :: PandocMonad m-          => String -- ^ Tag name-          -> MuseParser m (Attr, String)-htmlBlock tag = try $ do-  many spaceChar-  res <- htmlElement tag-  manyTill spaceChar eol-  return res+closeTag :: PandocMonad m => String -> MuseParser m ()+closeTag tag = try $ string "</" *> string tag *> void (char '>')  -- | Convert HTML attributes to Pandoc 'Attr' htmlAttrToPandoc :: [Attribute String] -> Attr@@ -213,13 +209,11 @@ parseHtmlContent tag = try $ do   many spaceChar   pos <- getPosition-  (TagOpen _ attr, _) <- htmlTag (~== TagOpen tag [])+  attr <- openTag tag   manyTill spaceChar eol-  content <- parseBlocksTill $ try $ count (sourceColumn pos - 1) spaceChar >> endtag+  content <- parseBlocksTill $ try $ count (sourceColumn pos - 1) spaceChar *> closeTag tag   manyTill spaceChar eol -- closing tag must be followed by optional whitespace and newline   return (htmlAttrToPandoc attr, content)-  where-    endtag = void $ htmlTag (~== TagClose tag)  -- ** Directive parsers @@ -228,21 +222,19 @@ parseDirectiveKey = char '#' *> many (letter <|> char '-')  parseEmacsDirective :: PandocMonad m => MuseParser m (String, F Inlines)-parseEmacsDirective = do-  key <- parseDirectiveKey-  spaceChar-  value <- trimInlinesF . mconcat <$> manyTill (choice inlineList) eol-  return (key, value)+parseEmacsDirective = (,)+  <$> parseDirectiveKey+  <*  spaceChar+  <*> (trimInlinesF . mconcat <$> manyTill inline' eol)  parseAmuseDirective :: PandocMonad m => MuseParser m (String, F Inlines)-parseAmuseDirective = do-  key <- parseDirectiveKey-  many1 spaceChar-  value <- trimInlinesF . mconcat <$> many1Till inline endOfDirective-  many blankline-  return (key, value)+parseAmuseDirective = (,)+  <$> parseDirectiveKey+  <*  many1 spaceChar+  <*> (trimInlinesF . mconcat <$> many1Till inline endOfDirective)+  <*  many blankline   where-    endOfDirective = lookAhead $ eof <|> try (newline >> (void blankline <|> void parseDirectiveKey))+    endOfDirective = lookAhead $ eof <|> try (newline *> (void blankline <|> void parseDirectiveKey))  directive :: PandocMonad m => MuseParser m () directive = do@@ -266,8 +258,8 @@   where     nextSection = mempty <$ lookAhead headingStart     parseEnd = mempty <$ eof-    blockStart = ((B.<>) <$> (blockElements <|> emacsNoteBlock)-                         <*> parseBlocks)+    blockStart = (B.<>) <$> (blockElements <|> emacsNoteBlock)+                        <*> parseBlocks     listStart = do       updateState (\st -> st { museInPara = False })       uncurry (B.<>) <$> (anyListUntil parseBlocks <|> amuseNoteBlockUntil parseBlocks)@@ -281,7 +273,7 @@              => MuseParser m (F Blocks) parseSection =   ((B.<>) <$> emacsHeading <*> parseBlocks) <|>-  ((uncurry (B.<>)) <$> amuseHeadingUntil parseBlocks)+  (uncurry (B.<>) <$> amuseHeadingUntil parseBlocks)  parseBlocksTill :: PandocMonad m                 => MuseParser m a@@ -357,30 +349,26 @@  -- | Parse a line comment, starting with @;@ in the first column. comment :: PandocMonad m => MuseParser m (F Blocks)-comment = try $ do-  getPosition >>= \pos -> guard (sourceColumn pos == 1)-  char ';'-  optional (spaceChar >> many (noneOf "\n"))-  eol-  return mempty+comment = try $ mempty+  <$ firstColumn+  <* char ';'+  <* optional (spaceChar *> many (noneOf "\n"))+  <* eol  -- | Parse a horizontal rule, consisting of 4 or more @\'-\'@ characters. separator :: PandocMonad m => MuseParser m (F Blocks)-separator = try $ do-  string "----"-  many $ char '-'-  many spaceChar-  eol-  return $ return B.horizontalRule+separator = try $ pure B.horizontalRule+  <$ string "----"+  <* many (char '-')+  <* many spaceChar+  <* eol  headingStart :: PandocMonad m => MuseParser m (String, Int)-headingStart = try $ do-  anchorId <- option "" $ try (parseAnchor <* manyTill spaceChar eol)-  getPosition >>= \pos -> guard (sourceColumn pos == 1)-  level <- fmap length $ many1 $ char '*'-  guard $ level <= 5-  spaceChar-  return (anchorId, level)+headingStart = try $ (,)+  <$> option "" (try (parseAnchor <* manyTill spaceChar eol))+  <*  firstColumn+  <*> fmap length (many1 $ char '*')+  <*  spaceChar  -- | Parse a single-line heading. emacsHeading :: PandocMonad m => MuseParser m (F Blocks)@@ -406,33 +394,28 @@ -- | Parse an example between @{{{@ and @}}}@. -- It is an Amusewiki extension influenced by Creole wiki, as described in @Text::Amuse@ documentation. example :: PandocMonad m => MuseParser m (F Blocks)-example = try $ do-  string "{{{"-  optional blankline-  contents <- manyTill anyChar $ try (optional blankline >> string "}}}")-  return $ return $ B.codeBlock contents+example = try $ pure . B.codeBlock+  <$  string "{{{"+  <*  optional blankline+  <*> manyTill anyChar (try (optional blankline *> string "}}}"))  -- | Parse an @\<example>@ tag. exampleTag :: PandocMonad m => MuseParser m (F Blocks)-exampleTag = try $ do-  (attr, contents) <- htmlBlock "example"-  return $ return $ B.codeBlockWith attr $ rchop $ intercalate "\n" $ dropSpacePrefix $ splitOn "\n" $ lchop contents+exampleTag = try $ fmap pure $ B.codeBlockWith+  <$  many spaceChar+  <*> (htmlAttrToPandoc <$> openTag "example")+  <*> (rchop . intercalate "\n" . dropSpacePrefix . splitOn "\n" . lchop <$> manyTill anyChar (closeTag "example"))+  <*  manyTill spaceChar eol  -- | Parse a @\<literal>@ tag as a raw block. -- For 'RawInline' @\<literal>@ parser, see 'inlineLiteralTag'. literalTag :: PandocMonad m => MuseParser m (F Blocks)-literalTag = try $ do-  many spaceChar-  (TagOpen _ attr, _) <- htmlTag (~== TagOpen "literal" [])-  manyTill spaceChar eol-  content <- manyTill anyChar endtag-  manyTill spaceChar eol-  return $ return $ rawBlock (htmlAttrToPandoc attr, content)-  where-    endtag = void $ htmlTag (~== TagClose "literal")-    -- FIXME: Emacs Muse inserts <literal> without style into all output formats, but we assume HTML-    format (_, _, kvs)        = fromMaybe "html" $ lookup "style" kvs-    rawBlock (attrs, content) = B.rawBlock (format attrs) $ rchop $ intercalate "\n" $ dropSpacePrefix $ splitOn "\n" $ lchop content+literalTag = try $ fmap pure $ B.rawBlock+  <$  many spaceChar+  <*> (fromMaybe "html" . lookup "style" <$> openTag "literal") -- FIXME: Emacs Muse inserts <literal> without style into all output formats, but we assume HTML+  <*  manyTill spaceChar eol+  <*> (rchop . intercalate "\n" . dropSpacePrefix . splitOn "\n" . lchop <$> manyTill anyChar (closeTag "literal"))+  <*  manyTill spaceChar eol  -- | Parse @\<center>@ tag. -- Currently it is ignored as Pandoc cannot represent centered blocks.@@ -471,8 +454,8 @@  verseLine :: PandocMonad m => MuseParser m (F Inlines) verseLine = do-  indent <- (B.str <$> many1 (char ' ' >> pure '\160')) <|> pure mempty-  rest <- manyTill (choice inlineList) newline+  indent <- (B.str <$> many1 ('\160' <$ char ' ')) <|> pure mempty+  rest <- manyTill inline' newline   return $ trimInlinesF $ mconcat (pure indent : rest)  -- | Parse @\<verse>@ tag.@@ -480,18 +463,20 @@ verseTag = try $ do   many spaceChar   pos <- getPosition-  (TagOpen _ _, _) <- htmlTag (~== TagOpen "verse" [])+  openTag "verse"   manyTill spaceChar eol   let indent = count (sourceColumn pos - 1) spaceChar-  content <- sequence <$> manyTill (indent >> verseLine) (try $ indent >> endtag)+  content <- sequence <$> manyTill (indent *> verseLine) (try $ indent *> closeTag "verse")   manyTill spaceChar eol   return $ B.lineBlock <$> content-  where-    endtag = void $ htmlTag (~== TagClose "verse")  -- | Parse @\<comment>@ tag. commentTag :: PandocMonad m => MuseParser m (F Blocks)-commentTag = htmlBlock "comment" >> return mempty+commentTag = try $ mempty+  <$ many spaceChar+  <* openTag "comment"+  <* manyTill anyChar (closeTag "comment")+  <* manyTill spaceChar eol  -- | Parse paragraph contents. paraContentsUntil :: PandocMonad m@@ -499,7 +484,7 @@                   -> MuseParser m (F Inlines, a) paraContentsUntil end = do   updateState (\st -> st { museInPara = True })-  (l, e) <- someUntil inline $ try (manyTill spaceChar eol >> end)+  (l, e) <- someUntil inline $ try (manyTill spaceChar eol *> end)   updateState (\st -> st { museInPara = False })   return (trimInlinesF $ mconcat l, e) @@ -513,9 +498,10 @@   first (fmap B.para) <$> paraContentsUntil end  noteMarker :: PandocMonad m => MuseParser m String-noteMarker = try $ do-  char '['-  (:) <$> oneOf "123456789" <*> manyTill digit (char ']')+noteMarker = try $ (:)+  <$  char '['+  <*> oneOf "123456789"+  <*> manyTill digit (char ']')  -- Amusewiki version of note -- Parsing is similar to list item, except that note marker is used instead of list marker@@ -558,16 +544,15 @@ lineVerseLine :: PandocMonad m => MuseParser m (F Inlines) lineVerseLine = try $ do   string "> "-  indent <- many (char ' ' >> pure '\160')+  indent <- many ('\160' <$ char ' ')   let indentEl = if null indent then mempty else B.str indent-  rest <- manyTill (choice inlineList) eol+  rest <- manyTill inline' eol   return $ trimInlinesF $ mconcat (pure indentEl : rest)  blanklineVerseLine :: PandocMonad m => MuseParser m (F Inlines)-blanklineVerseLine = try $ do-  char '>'-  blankline-  pure mempty+blanklineVerseLine = try $ mempty+  <$ char '>'+  <* blankline  -- | Parse a line block indicated by @\'>\'@ characters. lineBlock :: PandocMonad m => MuseParser m (F Blocks)@@ -587,7 +572,7 @@   char '-'   void spaceChar <|> lookAhead eol   updateState (\st -> st { museInPara = False })-  (x, (xs, e)) <- listItemContentsUntil (indent + 2) (try (optional blankline >> indentWith indent >> bulletListItemsUntil indent end)) (([],) <$> end)+  (x, (xs, e)) <- listItemContentsUntil (indent + 2) (try (optional blankline *> indentWith indent *> bulletListItemsUntil indent end)) (([],) <$> end)   return (x:xs, e)  -- | Parse a bullet list.@@ -602,13 +587,6 @@   (items, e) <- bulletListItemsUntil indent end   return (B.bulletList <$> sequence items, e) --- | Parses an ordered list marker and returns list attributes.-anyMuseOrderedListMarker :: PandocMonad m => MuseParser m ListAttributes-anyMuseOrderedListMarker = do-  (style, start) <- decimal <|> lowerRoman <|> upperRoman <|> lowerAlpha <|> upperAlpha-  char '.'-  return (start, style, Period)- museOrderedListMarker :: PandocMonad m                       => ListNumberStyle                       -> MuseParser m Int@@ -634,7 +612,7 @@       pos <- getPosition       void spaceChar <|> lookAhead eol       updateState (\st -> st { museInPara = False })-      (x, (xs, e)) <- listItemContentsUntil (sourceColumn pos) (try (optional blankline >> indentWith indent >> museOrderedListMarker style >> continuation)) (([],) <$> end)+      (x, (xs, e)) <- listItemContentsUntil (sourceColumn pos) (try (optional blankline *> indentWith indent *> museOrderedListMarker style *> continuation)) (([],) <$> end)       return (x:xs, e)  -- | Parse an ordered list.@@ -646,10 +624,10 @@   pos <- getPosition   let indent = sourceColumn pos - 1   guard $ indent /= 0-  p@(_, style, _) <- anyMuseOrderedListMarker-  guard $ style `elem` [Decimal, LowerAlpha, UpperAlpha, LowerRoman, UpperRoman]-  (items, e) <- orderedListItemsUntil indent style end-  return (B.orderedListWith p <$> sequence items, e)+  (style, start) <- decimal <|> lowerRoman <|> upperRoman <|> lowerAlpha <|> upperAlpha+  char '.'+  first (fmap (B.orderedListWith (start, style, Period)) . sequence)+    <$> orderedListItemsUntil indent style end  descriptionsUntil :: PandocMonad m                   => Int@@ -658,7 +636,7 @@ descriptionsUntil indent end = do   void spaceChar <|> lookAhead eol   updateState (\st -> st { museInPara = False })-  (x, (xs, e)) <- listItemContentsUntil indent (try (optional blankline >> indentWith indent >> manyTill spaceChar (string "::") >> descriptionsUntil indent end)) (([],) <$> end)+  (x, (xs, e)) <- listItemContentsUntil indent (try (optional blankline *> indentWith indent *> manyTill spaceChar (string "::") *> descriptionsUntil indent end)) (([],) <$> end)   return (x:xs, e)  definitionListItemsUntil :: PandocMonad m@@ -670,8 +648,8 @@   where     continuation = try $ do       pos <- getPosition-      term <- trimInlinesF . mconcat <$> manyTill (choice inlineList) (try $ string "::")-      (x, (xs, e)) <- descriptionsUntil (sourceColumn pos) (try (optional blankline >> indentWith indent >> continuation) <|> (([],) <$> end))+      term <- trimInlinesF . mconcat <$> manyTill inline' (try $ string "::")+      (x, (xs, e)) <- descriptionsUntil (sourceColumn pos) (try (optional blankline *> indentWith indent *> continuation) <|> (([],) <$> end))       let xx = (,) <$> term <*> sequence x       return (xx:xs, e) @@ -727,7 +705,7 @@  tableCell :: PandocMonad m => MuseParser m (F Blocks) tableCell = try $ fmap B.plain . trimInlinesF . mconcat <$> manyTill inline (lookAhead cellEnd)-  where cellEnd = try $ void (many1 spaceChar >> char '|') <|> eol+  where cellEnd = try $ void (many1 spaceChar *> char '|') <|> eol  tableElements :: PandocMonad m => MuseParser m (F [MuseTableElement]) tableElements = sequence <$> (tableParseElement `sepEndBy1` eol)@@ -749,11 +727,10 @@ tableParseRow :: PandocMonad m               => Int -- ^ Number of separator characters               -> MuseParser m (F [Blocks])-tableParseRow n = try $ do-  fields <- tableCell `sepBy2` fieldSep-  return $ sequence fields-    where p `sepBy2` sep = (:) <$> p <*> many1 (sep >> p)-          fieldSep = many1 spaceChar >> count n (char '|') >> (void (many1 spaceChar) <|> void (lookAhead newline))+tableParseRow n = try $+  sequence <$> (tableCell `sepBy2` fieldSep)+    where p `sepBy2` sep = (:) <$> p <*> many1 (sep *> p)+          fieldSep = many1 spaceChar *> count n (char '|') *> (void (many1 spaceChar) <|> void (lookAhead newline))  -- | Parse a table header row. tableParseHeader :: PandocMonad m => MuseParser m (F MuseTableElement)@@ -769,53 +746,51 @@  -- | Parse table caption. tableParseCaption :: PandocMonad m => MuseParser m (F MuseTableElement)-tableParseCaption = try $ do-  many spaceChar-  string "|+"-  fmap MuseCaption <$> (trimInlinesF . mconcat <$> many1Till inline (string "+|"))+tableParseCaption = try $ fmap MuseCaption . trimInlinesF . mconcat+  <$  many spaceChar+  <*  string "|+"+  <*> many1Till inline (string "+|")  -- ** Inline parsers -inlineList :: PandocMonad m => [MuseParser m (F Inlines)]-inlineList = [ whitespace-             , br-             , anchor-             , footnote-             , strong-             , strongTag-             , emph-             , emphTag-             , underlined-             , superscriptTag-             , subscriptTag-             , strikeoutTag-             , verbatimTag-             , classTag-             , nbsp-             , linkOrImage-             , code-             , codeTag-             , mathTag-             , inlineLiteralTag-             , str-             , symbol-             ]+inline' :: PandocMonad m => MuseParser m (F Inlines)+inline' = whitespace+      <|> br+      <|> anchor+      <|> footnote+      <|> strong+      <|> strongTag+      <|> emph+      <|> emphTag+      <|> underlined+      <|> superscriptTag+      <|> subscriptTag+      <|> strikeoutTag+      <|> verbatimTag+      <|> classTag+      <|> nbsp+      <|> linkOrImage+      <|> code+      <|> codeTag+      <|> mathTag+      <|> inlineLiteralTag+      <|> str+      <|> symbol+      <?> "inline"  inline :: PandocMonad m => MuseParser m (F Inlines)-inline = endline <|> choice inlineList <?> "inline"+inline = endline <|> inline'  -- | Parse a soft break. endline :: PandocMonad m => MuseParser m (F Inlines)-endline = try $ do-  newline-  notFollowedBy blankline-  return $ return B.softbreak+endline = try $ pure B.softbreak <$ newline <* notFollowedBy blankline  parseAnchor :: PandocMonad m => MuseParser m String-parseAnchor = try $ do-  getPosition >>= \pos -> guard (sourceColumn pos == 1)-  char '#'-  (:) <$> letter <*> many (letter <|> digit <|> char '-')+parseAnchor = try $ (:)+  <$  firstColumn+  <*  char '#'+  <*> letter+  <*> many (letter <|> digit <|> char '-')  anchor :: PandocMonad m => MuseParser m (F Inlines) anchor = try $ do@@ -839,15 +814,11 @@         return $ B.note contents'  whitespace :: PandocMonad m => MuseParser m (F Inlines)-whitespace = try $ do-  skipMany1 spaceChar-  return $ return B.space+whitespace = try $ pure B.space <$ skipMany1 spaceChar  -- | Parse @\<br>@ tag. br :: PandocMonad m => MuseParser m (F Inlines)-br = try $ do-  string "<br>"-  return $ return B.linebreak+br = try $ pure B.linebreak <$ string "<br>"  emphasisBetween :: (PandocMonad m, Show a) => MuseParser m a -> MuseParser m (F Inlines) emphasisBetween c = try $ enclosedInlines c c@@ -858,22 +829,23 @@          -> ParserT s st m a    -- ^ content parser (to be used repeatedly)          -> ParserT s st m [a] enclosed start end parser = try $-  start >> notFollowedBy spaceChar >> many1Till parser end+  start *> notFollowedBy spaceChar *> many1Till parser end  enclosedInlines :: (PandocMonad m, Show a, Show b)                 => MuseParser m a                 -> MuseParser m b                 -> MuseParser m (F Inlines)-enclosedInlines start end = try $-  trimInlinesF . mconcat <$> (enclosed (atStart start) end inline <* notFollowedBy (satisfy ((||) <$> isLetter <*> isDigit)))+enclosedInlines start end = try $ trimInlinesF . mconcat+  <$> enclosed (atStart start) end inline+  <*  notFollowedBy (satisfy isAlphaNum)  -- | Parse an inline tag, such as @\<em>@ and @\<strong>@. inlineTag :: PandocMonad m           => String -- ^ Tag name           -> MuseParser m (F Inlines)-inlineTag tag = try $ do-  htmlTag (~== TagOpen tag [])-  mconcat <$> manyTill inline (void $ htmlTag (~== TagClose tag))+inlineTag tag = try $ mconcat+  <$  openTag tag+  <*> manyTill inline (closeTag tag)  -- | Parse strong inline markup, indicated by @**@. strong :: PandocMonad m => MuseParser m (F Inlines)@@ -886,9 +858,9 @@ -- | Parse underline inline markup, indicated by @_@. -- Supported only in Emacs Muse mode, not Text::Amuse. underlined :: PandocMonad m => MuseParser m (F Inlines)-underlined = do-  guardDisabled Ext_amuse -- Supported only by Emacs Muse-  fmap underlineSpan <$> emphasisBetween (char '_')+underlined = fmap underlineSpan+  <$  guardDisabled Ext_amuse -- Supported only by Emacs Muse+  <*> emphasisBetween (char '_')  -- | Parse @\<strong>@ tag. strongTag :: PandocMonad m => MuseParser m (F Inlines)@@ -912,21 +884,20 @@  -- | Parse @\<verbatim>@ tag. verbatimTag :: PandocMonad m => MuseParser m (F Inlines)-verbatimTag = return . B.text . snd <$> htmlElement "verbatim"+verbatimTag = return . B.text+  <$  openTag "verbatim"+  <*> manyTill anyChar (closeTag "verbatim")  -- | Parse @\<class>@ tag. classTag :: PandocMonad m => MuseParser m (F Inlines) classTag = do-  (TagOpen _ attrs, _) <- htmlTag (~== TagOpen "class" [])-  res <- manyTill inline (void $ htmlTag (~== TagClose "class"))-  let classes = maybe [] words $ lookup "name" attrs+  classes <- maybe [] words . lookup "name" <$> openTag "class"+  res <- manyTill inline $ closeTag "class"   return $ B.spanWith ("", classes, []) <$> mconcat res  -- | Parse "~~" as nonbreaking space. nbsp :: PandocMonad m => MuseParser m (F Inlines)-nbsp = try $ do-  string "~~"-  return $ return $ B.str "\160"+nbsp = try $ pure (B.str "\160") <$ string "~~"  -- | Parse code markup, indicated by @\'=\'@ characters. code :: PandocMonad m => MuseParser m (F Inlines)@@ -936,26 +907,27 @@   guard $ not $ null contents   guard $ head contents `notElem` " \t\n"   guard $ last contents `notElem` " \t\n"-  notFollowedBy $ satisfy isLetter+  notFollowedBy $ satisfy isAlphaNum   return $ return $ B.code contents  -- | Parse @\<code>@ tag. codeTag :: PandocMonad m => MuseParser m (F Inlines)-codeTag = return . uncurry B.codeWith <$> htmlElement "code"+codeTag = fmap pure $ B.codeWith+  <$> (htmlAttrToPandoc <$> openTag "code")+  <*> manyTill anyChar (closeTag "code")  -- | Parse @\<math>@ tag. -- @\<math>@ tag is an Emacs Muse extension enabled by @(require 'muse-latex2png)@ mathTag :: PandocMonad m => MuseParser m (F Inlines)-mathTag = return . B.math . snd <$> htmlElement "math"+mathTag = return . B.math+  <$  openTag "math"+  <*> manyTill anyChar (closeTag "math")  -- | Parse inline @\<literal>@ tag as a raw inline. inlineLiteralTag :: PandocMonad m => MuseParser m (F Inlines)-inlineLiteralTag =-  (return . rawInline) <$> htmlElement "literal"-  where-    -- FIXME: Emacs Muse inserts <literal> without style into all output formats, but we assume HTML-    format (_, _, kvs)        = fromMaybe "html" $ lookup "style" kvs-    rawInline (attrs, content) = B.rawInline (format attrs) content+inlineLiteralTag = try $ fmap pure $ B.rawInline+  <$> (fromMaybe "html" . lookup "style" <$> openTag "literal") -- FIXME: Emacs Muse inserts <literal> without style into all output formats, but we assume HTML+  <*> manyTill anyChar (closeTag "literal")  str :: PandocMonad m => MuseParser m (F Inlines) str = return . B.str <$> many1 alphaNum <* updateLastStrPos@@ -974,7 +946,9 @@   return res  linkContent :: PandocMonad m => MuseParser m (F Inlines)-linkContent = char '[' >> trimInlinesF . mconcat <$> manyTill inline (char ']')+linkContent = trimInlinesF . mconcat+  <$  char '['+  <*> manyTill inline (char ']')  -- | Parse a link starting with @URL:@ explicitLink :: PandocMonad m => MuseParser m (F Inlines)@@ -989,7 +963,7 @@ image = try $ do   string "[["   (url, (ext, width, align)) <- manyUntil (noneOf "]") (imageExtensionAndOptions <* char ']')-  content <- optionMaybe linkContent+  content <- option mempty linkContent   char ']'   let widthAttr = case align of                     Just 'f' -> [("width", fromMaybe "100" width ++ "%"), ("height", "75%")]@@ -999,7 +973,7 @@                      Just 'l' -> ["align-left"]                      Just 'f' -> []                      _        -> []-  return $ B.imageWith ("", alignClass, widthAttr) (url ++ ext) mempty <$> fromMaybe (return mempty) content+  return $ B.imageWith ("", alignClass, widthAttr) (url ++ ext) mempty <$> content   where -- Taken from muse-image-regexp defined in Emacs Muse file lisp/muse-regexps.el         imageExtensions = [".eps", ".gif", ".jpg", ".jpeg", ".pbm", ".png", ".tiff", ".xbm", ".xpm"]         imageExtension = choice (try . string <$> imageExtensions)@@ -1007,12 +981,11 @@           ext <- imageExtension           (width, align) <- option (Nothing, Nothing) imageAttrs           return (ext, width, align)-        imageAttrs = do-          many1 spaceChar-          width <- optionMaybe (many1 digit)-          many spaceChar-          align <- optionMaybe (oneOf "rlf")-          return (width, align)+        imageAttrs = (,)+          <$  many1 spaceChar+          <*> optionMaybe (many1 digit)+          <*  many spaceChar+          <*> optionMaybe (oneOf "rlf")  link :: PandocMonad m => MuseParser m (F Inlines) link = try $ do

@@ -40,7 +40,7 @@ import Text.Pandoc.Readers.Org.ParserState import Text.Pandoc.Readers.Org.Parsing import Text.Pandoc.Readers.Org.Shared (cleanLinkString, isImageFilename,-                                       originalLang, translateLang)+                                       originalLang, translateLang, exportsCode)  import Text.Pandoc.Builder (Blocks, Inlines) import Text.Pandoc.Class (PandocMonad)@@ -313,9 +313,6 @@     labelledBlock :: F Inlines -> F Blocks    labelledBlock = fmap (B.plain . B.spanWith ("", ["label"], []))--   exportsCode :: [(String, String)] -> Bool-   exportsCode = maybe True (`elem` ["code", "both"]) . lookup "exports"     exportsResults :: [(String, String)] -> Bool    exportsResults = maybe False (`elem` ["results", "both"]) . lookup "exports"

@@ -39,7 +39,7 @@ import Text.Pandoc.Readers.Org.ParserState import Text.Pandoc.Readers.Org.Parsing import Text.Pandoc.Readers.Org.Shared (cleanLinkString, isImageFilename,-                                       originalLang, translateLang)+                                       originalLang, translateLang, exportsCode)  import Text.Pandoc.Builder (Inlines) import qualified Text.Pandoc.Builder as B@@ -525,11 +525,13 @@ inlineCodeBlock = try $ do   string "src_"   lang <- many1 orgArgWordChar-  opts <- option [] $ enclosedByPair '[' ']' inlineBlockOption+  opts <- option [] $ try (enclosedByPair '[' ']' inlineBlockOption)+                        <|> (mempty <$ string "[]")   inlineCode <- enclosedByPair '{' '}' (noneOf "\n\r")   let attrClasses = [translateLang lang]   let attrKeyVal  = originalLang lang <> opts-  returnF $ B.codeWith ("", attrClasses, attrKeyVal) inlineCode+  let codeInlineBlck = B.codeWith ("", attrClasses, attrKeyVal) inlineCode+  returnF $ (if exportsCode opts then codeInlineBlck else mempty)  where    inlineBlockOption :: PandocMonad m => OrgParser m (String, String)    inlineBlockOption = try $ do

@@ -32,6 +32,7 @@   , isImageFilename   , originalLang   , translateLang+  , exportsCode   ) where  import Prelude@@ -96,3 +97,6 @@     "sh"         -> "bash"     "sqlite"     -> "sql"     _            -> cs++exportsCode :: [(String, String)] -> Bool+exportsCode = maybe True (`elem` ["code", "both"]) . lookup "exports"

@@ -1090,7 +1090,7 @@ targetURI :: Monad m => ParserT [Char] st m [Char] targetURI = do   skipSpaces-  optional newline+  optional $ try $ newline >> notFollowedBy blankline   contents <- trim <$>      many1 (satisfy (/='\n')      <|> try (newline >> many1 spaceChar >> noneOf " \t\n"))@@ -1315,7 +1315,6 @@ inline :: PandocMonad m => RSTParser m Inlines inline = choice [ note          -- can start with whitespace, so try before ws                 , link-                , endline                 , strong                 , emph                 , code@@ -1328,6 +1327,7 @@ inlineContent :: PandocMonad m => RSTParser m Inlines inlineContent = choice [ whitespace                        , str+                       , endline                        , smart                        , hyphens                        , escapedChar

@@ -453,7 +453,12 @@       clas = if null cls                 then empty                 else brackets $ text $ toLabel $ head cls-      src' = if isURI src+      -- Use / for path separators on Windows; see #4918+      fixPathSeparators = map $ \c -> case c of+                                           '\\' -> '/'+                                           _    -> c+      src' = fixPathSeparators $+             if isURI src                 then src                 else unEscapeString src   return $ braces $ "\\externalfigure" <> brackets (text src') <> dims <> clas

@@ -1,6 +1,6 @@-{-# LANGUAGE NoImplicitPrelude #-}-{-# LANGUAGE DeriveDataTypeable   #-}-{-# LANGUAGE FlexibleInstances    #-}+{-# LANGUAGE DeriveDataTypeable #-}+{-# LANGUAGE FlexibleInstances  #-}+{-# LANGUAGE NoImplicitPrelude  #-} {- Copyright (C) 2012-2018 John MacFarlane <jgm@berkeley.edu>  This program is free software; you can redistribute it and/or modify@@ -35,25 +35,26 @@ import Control.Arrow ((***)) import Control.Exception import Control.Monad (when)-import Control.Monad.Trans (MonadIO (liftIO)) import Data.Char (toLower) import Data.List (intersperse) import qualified Data.Map as M import Data.Text (Text, pack) import Data.Typeable-import Foreign.Lua (Lua, ToLuaStack (..), callFunc)-import Foreign.Lua.Api+import Foreign.Lua (Lua, Pushable) import Text.Pandoc.Class (PandocIO) import Text.Pandoc.Definition import Text.Pandoc.Error-import Text.Pandoc.Lua.Init (runPandocLua, registerScriptPath)+import Text.Pandoc.Lua.Init (LuaException (LuaException), runPandocLua,+                             registerScriptPath) import Text.Pandoc.Lua.StackInstances ()-import Text.Pandoc.Lua.Util (addField, addValue, dostring')+import Text.Pandoc.Lua.Util (addField) import Text.Pandoc.Options import Text.Pandoc.Templates import qualified Text.Pandoc.UTF8 as UTF8 import Text.Pandoc.Writers.Shared +import qualified Foreign.Lua as Lua+ attrToMap :: Attr -> M.Map String String attrToMap (id',classes,keyvals) = M.fromList     $ ("id", id')@@ -62,26 +63,26 @@  newtype Stringify a = Stringify a -instance ToLuaStack (Stringify Format) where-  push (Stringify (Format f)) = push (map toLower f)+instance Pushable (Stringify Format) where+  push (Stringify (Format f)) = Lua.push (map toLower f) -instance ToLuaStack (Stringify [Inline]) where-  push (Stringify ils) = push =<< inlineListToCustom ils+instance Pushable (Stringify [Inline]) where+  push (Stringify ils) = Lua.push =<< inlineListToCustom ils -instance ToLuaStack (Stringify [Block]) where-  push (Stringify blks) = push =<< blockListToCustom blks+instance Pushable (Stringify [Block]) where+  push (Stringify blks) = Lua.push =<< blockListToCustom blks -instance ToLuaStack (Stringify MetaValue) where-  push (Stringify (MetaMap m))       = push (fmap Stringify m)-  push (Stringify (MetaList xs))     = push (map Stringify xs)-  push (Stringify (MetaBool x))      = push x-  push (Stringify (MetaString s))    = push s-  push (Stringify (MetaInlines ils)) = push (Stringify ils)-  push (Stringify (MetaBlocks bs))   = push (Stringify bs)+instance Pushable (Stringify MetaValue) where+  push (Stringify (MetaMap m))       = Lua.push (fmap Stringify m)+  push (Stringify (MetaList xs))     = Lua.push (map Stringify xs)+  push (Stringify (MetaBool x))      = Lua.push x+  push (Stringify (MetaString s))    = Lua.push s+  push (Stringify (MetaInlines ils)) = Lua.push (Stringify ils)+  push (Stringify (MetaBlocks bs))   = Lua.push (Stringify bs) -instance ToLuaStack (Stringify Citation) where+instance Pushable (Stringify Citation) where   push (Stringify cit) = do-    createtable 6 0+    Lua.createtable 6 0     addField "citationId" $ citationId cit     addField "citationPrefix" . Stringify $ citationPrefix cit     addField "citationSuffix" . Stringify $ citationSuffix cit@@ -93,10 +94,12 @@ -- associated value. newtype KeyValue a b = KeyValue (a, b) -instance (ToLuaStack a, ToLuaStack b) => ToLuaStack (KeyValue a b) where+instance (Pushable a, Pushable b) => Pushable (KeyValue a b) where   push (KeyValue (k, v)) = do-    newtable-    addValue k v+    Lua.newtable+    Lua.push k+    Lua.push v+    Lua.rawset (Lua.nthFromTop 3)  data PandocLuaException = PandocLuaException String     deriving (Show, Typeable)@@ -106,14 +109,13 @@ -- | Convert Pandoc to custom markup. writeCustom :: FilePath -> WriterOptions -> Pandoc -> PandocIO Text writeCustom luaFile opts doc@(Pandoc meta _) = do-  luaScript <- liftIO $ UTF8.readFile luaFile   res <- runPandocLua $ do     registerScriptPath luaFile-    stat <- dostring' luaScript+    stat <- Lua.dofile luaFile     -- check for error in lua script (later we'll change the return type     -- to handle this more gracefully):-    when (stat /= OK) $-      tostring (-1) >>= throw . PandocLuaException . UTF8.toString+    when (stat /= Lua.OK) $+      Lua.tostring' (-1) >>= throw . PandocLuaException . UTF8.toString     -- TODO - call hierarchicalize, so we have that info     rendered <- docToCustom opts doc     context <- metaToJSON opts@@ -122,7 +124,7 @@                meta     return (rendered, context)   let (body, context) = case res of-        Left e -> throw (PandocLuaException (show e))+        Left (LuaException msg) -> throw (PandocLuaException msg)         Right x -> x   case writerTemplate opts of        Nothing  -> return $ pack body@@ -134,7 +136,7 @@ docToCustom :: WriterOptions -> Pandoc -> Lua String docToCustom opts (Pandoc (Meta metamap) blocks) = do   body <- blockListToCustom blocks-  callFunc "Doc" body (fmap Stringify metamap) (writerVariables opts)+  Lua.callFunc "Doc" body (fmap Stringify metamap) (writerVariables opts)  -- | Convert Pandoc block element to Custom. blockToCustom :: Block         -- ^ Block element@@ -142,52 +144,55 @@  blockToCustom Null = return "" -blockToCustom (Plain inlines) = callFunc "Plain" (Stringify inlines)+blockToCustom (Plain inlines) = Lua.callFunc "Plain" (Stringify inlines)  blockToCustom (Para [Image attr txt (src,tit)]) =-  callFunc "CaptionedImage" src tit (Stringify txt) (attrToMap attr)+  Lua.callFunc "CaptionedImage" src tit (Stringify txt) (attrToMap attr) -blockToCustom (Para inlines) = callFunc "Para" (Stringify inlines)+blockToCustom (Para inlines) = Lua.callFunc "Para" (Stringify inlines) -blockToCustom (LineBlock linesList) = callFunc "LineBlock" (map Stringify linesList)+blockToCustom (LineBlock linesList) =+  Lua.callFunc "LineBlock" (map Stringify linesList)  blockToCustom (RawBlock format str) =-  callFunc "RawBlock" (Stringify format) str+  Lua.callFunc "RawBlock" (Stringify format) str -blockToCustom HorizontalRule = callFunc "HorizontalRule"+blockToCustom HorizontalRule = Lua.callFunc "HorizontalRule"  blockToCustom (Header level attr inlines) =-  callFunc "Header" level (Stringify inlines) (attrToMap attr)+  Lua.callFunc "Header" level (Stringify inlines) (attrToMap attr)  blockToCustom (CodeBlock attr str) =-  callFunc "CodeBlock" str (attrToMap attr)+  Lua.callFunc "CodeBlock" str (attrToMap attr) -blockToCustom (BlockQuote blocks) = callFunc "BlockQuote" (Stringify blocks)+blockToCustom (BlockQuote blocks) =+  Lua.callFunc "BlockQuote" (Stringify blocks)  blockToCustom (Table capt aligns widths headers rows) =   let aligns' = map show aligns       capt' = Stringify capt       headers' = map Stringify headers       rows' = map (map Stringify) rows-  in callFunc "Table" capt' aligns' widths headers' rows'+  in Lua.callFunc "Table" capt' aligns' widths headers' rows' -blockToCustom (BulletList items) = callFunc "BulletList" (map Stringify items)+blockToCustom (BulletList items) =+  Lua.callFunc "BulletList" (map Stringify items)  blockToCustom (OrderedList (num,sty,delim) items) =-  callFunc "OrderedList" (map Stringify items) num (show sty) (show delim)+  Lua.callFunc "OrderedList" (map Stringify items) num (show sty) (show delim)  blockToCustom (DefinitionList items) =-  callFunc "DefinitionList"-           (map (KeyValue . (Stringify *** map Stringify)) items)+  Lua.callFunc "DefinitionList"+               (map (KeyValue . (Stringify *** map Stringify)) items)  blockToCustom (Div attr items) =-  callFunc "Div" (Stringify items) (attrToMap attr)+  Lua.callFunc "Div" (Stringify items) (attrToMap attr)  -- | Convert list of Pandoc block elements to Custom. blockListToCustom :: [Block]       -- ^ List of block elements                   -> Lua String blockListToCustom xs = do-  blocksep <- callFunc "Blocksep"+  blocksep <- Lua.callFunc "Blocksep"   bs <- mapM blockToCustom xs   return $ mconcat $ intersperse blocksep bs @@ -200,51 +205,51 @@ -- | Convert Pandoc inline element to Custom. inlineToCustom :: Inline -> Lua String -inlineToCustom (Str str) = callFunc "Str" str+inlineToCustom (Str str) = Lua.callFunc "Str" str -inlineToCustom Space = callFunc "Space"+inlineToCustom Space = Lua.callFunc "Space" -inlineToCustom SoftBreak = callFunc "SoftBreak"+inlineToCustom SoftBreak = Lua.callFunc "SoftBreak" -inlineToCustom (Emph lst) = callFunc "Emph" (Stringify lst)+inlineToCustom (Emph lst) = Lua.callFunc "Emph" (Stringify lst) -inlineToCustom (Strong lst) = callFunc "Strong" (Stringify lst)+inlineToCustom (Strong lst) = Lua.callFunc "Strong" (Stringify lst) -inlineToCustom (Strikeout lst) = callFunc "Strikeout" (Stringify lst)+inlineToCustom (Strikeout lst) = Lua.callFunc "Strikeout" (Stringify lst) -inlineToCustom (Superscript lst) = callFunc "Superscript" (Stringify lst)+inlineToCustom (Superscript lst) = Lua.callFunc "Superscript" (Stringify lst) -inlineToCustom (Subscript lst) = callFunc "Subscript" (Stringify lst)+inlineToCustom (Subscript lst) = Lua.callFunc "Subscript" (Stringify lst) -inlineToCustom (SmallCaps lst) = callFunc "SmallCaps" (Stringify lst)+inlineToCustom (SmallCaps lst) = Lua.callFunc "SmallCaps" (Stringify lst) -inlineToCustom (Quoted SingleQuote lst) = callFunc "SingleQuoted" (Stringify lst)+inlineToCustom (Quoted SingleQuote lst) = Lua.callFunc "SingleQuoted" (Stringify lst) -inlineToCustom (Quoted DoubleQuote lst) = callFunc "DoubleQuoted" (Stringify lst)+inlineToCustom (Quoted DoubleQuote lst) = Lua.callFunc "DoubleQuoted" (Stringify lst) -inlineToCustom (Cite cs lst) = callFunc "Cite" (Stringify lst) (map Stringify cs)+inlineToCustom (Cite cs lst) = Lua.callFunc "Cite" (Stringify lst) (map Stringify cs)  inlineToCustom (Code attr str) =-  callFunc "Code" str (attrToMap attr)+  Lua.callFunc "Code" str (attrToMap attr)  inlineToCustom (Math DisplayMath str) =-  callFunc "DisplayMath" str+  Lua.callFunc "DisplayMath" str  inlineToCustom (Math InlineMath str) =-  callFunc "InlineMath" str+  Lua.callFunc "InlineMath" str  inlineToCustom (RawInline format str) =-  callFunc "RawInline" (Stringify format) str+  Lua.callFunc "RawInline" (Stringify format) str -inlineToCustom LineBreak = callFunc "LineBreak"+inlineToCustom LineBreak = Lua.callFunc "LineBreak"  inlineToCustom (Link attr txt (src,tit)) =-  callFunc "Link" (Stringify txt) src tit (attrToMap attr)+  Lua.callFunc "Link" (Stringify txt) src tit (attrToMap attr)  inlineToCustom (Image attr alt (src,tit)) =-  callFunc "Image" (Stringify alt) src tit (attrToMap attr)+  Lua.callFunc "Image" (Stringify alt) src tit (attrToMap attr) -inlineToCustom (Note contents) = callFunc "Note" (Stringify contents)+inlineToCustom (Note contents) = Lua.callFunc "Note" (Stringify contents)  inlineToCustom (Span attr items) =-  callFunc "Span" (Stringify items) (attrToMap attr)+  Lua.callFunc "Span" (Stringify items) (attrToMap attr)

@@ -50,13 +50,13 @@ import Control.Monad.State.Strict import Data.Char (ord, toLower) import Data.List (intercalate, intersperse, isPrefixOf, partition)-import Data.Maybe (catMaybes, fromMaybe, isJust, isNothing)+import Data.Maybe (catMaybes, fromMaybe, isJust, isNothing, mapMaybe) import qualified Data.Set as Set import Data.String (fromString) import Data.Text (Text) import qualified Data.Text.Lazy as TL import Network.HTTP (urlEncode)-import Network.URI (URI (..), parseURIReference, unEscapeString)+import Network.URI (URI (..), parseURIReference) import Numeric (showHex) import Text.Blaze.Internal (customLeaf, customParent, MarkupM(Empty)) #if MIN_VERSION_blaze_markup(0,6,3)@@ -461,7 +461,7 @@        t <- addAttrs opts attr $                    secttag header'        return $-         (if slideVariant == RevealJsSlides+         (if slideVariant == RevealJsSlides && not (null innerContents)                 then H5.section                 else id) $ mconcat $ t : innerContents      else if writerSectionDivs opts || slide@@ -577,13 +577,23 @@         => [(String, String)] -> StateT WriterState m [Attribute] toAttrs kvs = do   html5 <- gets stHtml5-  return $ map (\(x,y) ->-     customAttribute-        (fromString (if not html5 || x `Set.member` html5Attributes-                                  || "epub:" `isPrefixOf` x-                                  || "data-" `isPrefixOf` x-                        then x-                        else "data-" ++ x)) (toValue y)) kvs+  mbEpubVersion <- gets stEPUBVersion+  return $ mapMaybe (\(x,y) ->+            if html5+               then+                  if x `Set.member` html5Attributes+                     || ':' `elem` x -- e.g. epub: namespace+                     || "data-" `isPrefixOf` x+                     then Just $ customAttribute (fromString x) (toValue y)+                     else Just $ customAttribute (fromString ("data-" ++ x))+                                  (toValue y)+               else+                 if mbEpubVersion == Just EPUB2 &&+                    not (x `Set.member` html4Attributes ||+                         "xml:" `isPrefixOf` x)+                    then Nothing+                    else Just $ customAttribute (fromString x) (toValue y))+            kvs  attrsToHtml :: PandocMonad m             => WriterOptions -> Attr -> StateT WriterState m [Attribute]@@ -1084,10 +1094,7 @@                                              in  '#' : prefix ++ xs                                    _ -> s                         let link = H.a ! A.href (toValue s') $ linkText-                        let attr = if txt == [Str (unEscapeString s)]-                                      then (ident, "uri" : classes, kvs)-                                      else (ident, classes, kvs)-                        link' <- addAttrs opts attr link+                        link' <- addAttrs opts (ident, classes, kvs) link                         return $ if null tit                                     then link'                                     else link' ! A.title (toValue tit)@@ -1421,4 +1428,126 @@   , "width"   , "workertype"   , "wrap"+  ]++html4Attributes :: Set.Set String+html4Attributes = Set.fromList+  [ "abbr"+  , "accept"+  , "accept-charset"+  , "accesskey"+  , "action"+  , "align"+  , "alink"+  , "alt"+  , "archive"+  , "axis"+  , "background"+  , "bgcolor"+  , "border"+  , "cellpadding"+  , "cellspacing"+  , "char"+  , "charoff"+  , "charset"+  , "checked"+  , "cite"+  , "class"+  , "classid"+  , "clear"+  , "code"+  , "codebase"+  , "codetype"+  , "color"+  , "cols"+  , "colspan"+  , "compact"+  , "content"+  , "coords"+  , "data"+  , "datetime"+  , "declare"+  , "defer"+  , "dir"+  , "disabled"+  , "enctype"+  , "face"+  , "for"+  , "frame"+  , "frameborder"+  , "headers"+  , "height"+  , "href"+  , "hreflang"+  , "hspace"+  , "http-equiv"+  , "id"+  , "ismap"+  , "label"+  , "lang"+  , "language"+  , "link"+  , "longdesc"+  , "marginheight"+  , "marginwidth"+  , "maxlength"+  , "media"+  , "method"+  , "multiple"+  , "name"+  , "nohref"+  , "noresize"+  , "noshade"+  , "nowrap"+  , "object"+  , "onblur"+  , "onchange"+  , "onclick"+  , "ondblclick"+  , "onfocus"+  , "onkeydown"+  , "onkeypress"+  , "onkeyup"+  , "onload"+  , "onmousedown"+  , "onmousemove"+  , "onmouseout"+  , "onmouseover"+  , "onmouseup"+  , "onreset"+  , "onselect"+  , "onsubmit"+  , "onunload"+  , "profile"+  , "prompt"+  , "readonly"+  , "rel"+  , "rev"+  , "rows"+  , "rowspan"+  , "rules"+  , "scheme"+  , "scope"+  , "scrolling"+  , "selected"+  , "shape"+  , "size"+  , "span"+  , "src"+  , "standby"+  , "start"+  , "style"+  , "summary"+  , "tabindex"+  , "target"+  , "text"+  , "title"+  , "usemap"+  , "valign"+  , "value"+  , "valuetype"+  , "version"+  , "vlink"+  , "vspace"+  , "width"   ]

@@ -344,7 +344,7 @@ inlineToJATS opts (Emph lst) =   inTagsSimple "italic" <$> inlinesToJATS opts lst inlineToJATS opts (Strong lst) =-  inTags False "bold" [("role", "strong")] <$> inlinesToJATS opts lst+  inTagsSimple "bold" <$> inlinesToJATS opts lst inlineToJATS opts (Strikeout lst) =   inTagsSimple "strike" <$> inlinesToJATS opts lst inlineToJATS opts (Superscript lst) =@@ -352,8 +352,7 @@ inlineToJATS opts (Subscript lst) =   inTagsSimple "sub" <$> inlinesToJATS opts lst inlineToJATS opts (SmallCaps lst) =-  inTags False "sc" [("role", "smallcaps")] <$>-  inlinesToJATS opts lst+  inTagsSimple "sc" <$> inlinesToJATS opts lst inlineToJATS opts (Quoted SingleQuote lst) = do   contents <- inlinesToJATS opts lst   return $ char '‘' <> contents <> char '’'

@@ -176,9 +176,9 @@     modify $ \s -> s{stCsquotes = True}   let (blocks'', lastHeader) = if writerCiteMethod options == Citeproc then                                  (blocks', [])-                               else case last blocks' of-                                 Header 1 _ il -> (init blocks', il)-                                 _             -> (blocks', [])+                               else case reverse blocks' of+                                 Header 1 _ il : _ -> (init blocks', il)+                                 _                 -> (blocks', [])   beamer <- gets stBeamer   blocks''' <- if beamer                   then toSlides blocks''

@@ -403,8 +403,8 @@                       else withParagraphStyle o "Table" [Para c]         th <- if all null h                  then return empty-                 else colHeadsToOpenDocument o name (map fst paraHStyles) h-        tr <- mapM (tableRowToOpenDocument o name (map fst paraStyles)) r+                 else colHeadsToOpenDocument o (map fst paraHStyles) h+        tr <- mapM (tableRowToOpenDocument o (map fst paraStyles)) r         return $ inTags True "table:table" [ ("table:name"      , name)                                            , ("table:style-name", name)                                            ] (vcat columns $$ th $$ vcat tr) $$ captionDoc@@ -416,24 +416,24 @@         return $ imageDoc $$ captionDoc  colHeadsToOpenDocument :: PandocMonad m-                       => WriterOptions -> String -> [String] -> [[Block]]+                       => WriterOptions -> [String] -> [[Block]]                        -> OD m Doc-colHeadsToOpenDocument o tn ns hs =+colHeadsToOpenDocument o ns hs =     inTagsIndented "table:table-header-rows" . inTagsIndented "table:table-row" . vcat <$>-    mapM (tableItemToOpenDocument o tn) (zip ns hs)+    mapM (tableItemToOpenDocument o "TableHeaderRowCell") (zip ns hs)  tableRowToOpenDocument :: PandocMonad m-                       => WriterOptions -> String -> [String] -> [[Block]]+                       => WriterOptions -> [String] -> [[Block]]                        -> OD m Doc-tableRowToOpenDocument o tn ns cs =+tableRowToOpenDocument o ns cs =     inTagsIndented "table:table-row" . vcat <$>-    mapM (tableItemToOpenDocument o tn) (zip ns cs)+    mapM (tableItemToOpenDocument o "TableRowCell") (zip ns cs)  tableItemToOpenDocument :: PandocMonad m                         => WriterOptions -> String -> (String,[Block])                         -> OD m Doc-tableItemToOpenDocument o tn (n,i) =-  let a = [ ("table:style-name" , tn ++ ".A1" )+tableItemToOpenDocument o s (n,i) =+  let a = [ ("table:style-name" , s )           , ("office:value-type", "string"     )           ]   in  inTags True "table:table-cell" a <$>@@ -584,13 +584,21 @@                          , ("style:family", "table-column"       )] $                          selfClosingTag "style:table-column-properties"                          [("style:rel-column-width", printf "%d*" (floor $ w * 65535 :: Integer))]-        cellStyle      = inTags True "style:style"-                         [ ("style:name"  , tableId ++ ".A1")+        headerRowCellStyle = inTags True "style:style"+                         [ ("style:name"  , "TableHeaderRowCell")                          , ("style:family", "table-cell"    )] $                          selfClosingTag "style:table-cell-properties"                          [ ("fo:border", "none")]+        rowCellStyle = inTags True "style:style"+                         [ ("style:name"  , "TableRowCell")+                         , ("style:family", "table-cell"    )] $+                         selfClosingTag "style:table-cell-properties"+                         [ ("fo:border", "none")]+        cellStyles = if num == 0+                     then headerRowCellStyle $$ rowCellStyle+                     else empty         columnStyles   = map colStyle wcs-    in  table $$ vcat columnStyles $$ cellStyle+    in cellStyles $$ table $$ vcat columnStyles  paraStyle :: PandocMonad m => [(String,String)] -> OD m Int paraStyle attrs = do

@@ -341,8 +341,10 @@ listItemToRTF alignment indent marker [] = return $   rtfCompact (indent + listIncrement) (negate listIncrement) alignment              (marker ++ "\\tx" ++ show listIncrement ++ "\\tab ")-listItemToRTF alignment indent marker list = do-  (first:rest) <- mapM (blockToRTF (indent + listIncrement) alignment) list+listItemToRTF alignment indent marker (listFirst:listRest) = do+  let f = blockToRTF (indent + listIncrement) alignment+  first <- f listFirst+  rest <- mapM f listRest   let listMarker = "\\fi" ++ show (negate listIncrement) ++ " " ++ marker ++                    "\\tx" ++ show listIncrement ++ "\\tab"   let insertListMarker ('\\':'f':'i':'-':d:xs) | isDigit d =

@@ -463,15 +463,15 @@ inlineToTextile _ Space = return " "  inlineToTextile opts (Link (_, cls, _) txt (src, _)) = do-  let classes = if null cls-                   then ""-                   else "(" ++ unwords cls ++ ")"   label <- case txt of                 [Code _ s]                  | s == src -> return "$"                 [Str s]                  | s == src -> return "$"                 _           -> inlineListToTextile opts txt+  let classes = if null cls || cls == ["uri"] && label == "$"+                   then ""+                   else "(" ++ unwords cls ++ ")"   return $ "\"" ++ classes ++ label ++ "\":" ++ src  inlineToTextile opts (Image attr@(_, cls, _) alt (source, tit)) = do

@@ -24,6 +24,8 @@ - HsYAML-0.1.1.1 - texmath-0.11.1 - yaml-0.9.0+- hslua-1.0.0+- hslua-module-text-0.2.0 ghc-options:    "$locals": -fhide-source-paths -XNoImplicitPrelude resolver: lts-12.6

@@ -7,7 +7,7 @@ import Data.Version (Version (versionBranch)) import System.FilePath ((</>)) import Test.Tasty (TestTree, localOption)-import Test.Tasty.HUnit (Assertion, assertEqual, testCase)+import Test.Tasty.HUnit (Assertion, assertEqual, assertFailure, testCase) import Test.Tasty.QuickCheck (QuickCheckTests (..), ioProperty, testProperty) import Text.Pandoc.Arbitrary () import Text.Pandoc.Builder (bulletList, divWith, doc, doubleQuoted, emph,@@ -164,11 +164,11 @@    , testCase "informative error messages" . runPandocLua' $ do       Lua.pushboolean True-      err <- Lua.peekEither Lua.stackTop :: Lua.Lua (Either String Pandoc)-      case err of+      err <- Lua.peekEither Lua.stackTop+      case (err :: Either String Pandoc) of         Left msg -> do           let expectedMsg = "Could not get Pandoc value: "-                            ++ "expected table but got boolean."+                            <> "table expected, got boolean"           Lua.liftIO $ assertEqual "unexpected error message" expectedMsg msg         Right _ -> error "Getting a Pandoc element from a bool should fail."   ]@@ -179,13 +179,13 @@     setUserDataDir (Just "../data")     runLuaFilter def ("lua" </> filterPath) [] docIn   case docEither of-    Left _       -> fail "lua filter failed"+    Left exception -> assertFailure (show exception)     Right docRes -> assertEqual msg docExpected docRes -roundtripEqual :: (Eq a, Lua.FromLuaStack a, Lua.ToLuaStack a) => a -> IO Bool+roundtripEqual :: (Eq a, Lua.Peekable a, Lua.Pushable a) => a -> IO Bool roundtripEqual x = (x ==) <$> roundtripped  where-  roundtripped :: (Lua.FromLuaStack a, Lua.ToLuaStack a) => IO a+  roundtripped :: (Lua.Peekable a, Lua.Pushable a) => IO a   roundtripped = runPandocLua' $ do     oldSize <- Lua.gettop     Lua.push x

@@ -31,12 +31,14 @@  roundTrip :: Blocks -> Bool roundTrip b = d'' == d'''-  where d = walk removeRawInlines $ walk makeRoundTrip $ Pandoc nullMeta $ toList b+  where d = walk removeRawInlines $+            walk makeRoundTrip $ Pandoc nullMeta $ toList b         d' = rewrite d         d'' = rewrite d'         d''' = rewrite d''         rewrite = html . T.pack . (++ "\n") . T.unpack .-                  purely (writeHtml5String def { writerWrapText = WrapPreserve })+                  purely (writeHtml5String def+                            { writerWrapText = WrapPreserve })  tests :: [TestTree] tests = [ testGroup "base tag"@@ -75,5 +77,5 @@           , test htmlNativeDivs "<main> followed by text" $ "<main>main content</main>non-main content" =?>             doc (divWith ("", [], [("role", "main")]) (plain (text "main content")) <> plain (text "non-main content"))           ]-        , testProperty "Round trip" roundTrip+        , testProperty "Round trip" (withMaxSuccess 25 roundTrip)         ]

@@ -39,7 +39,7 @@        (unpack inp) (inp, doc $ para ils)  autolink :: String -> Inlines-autolink = autolinkWith nullAttr+autolink = autolinkWith ("",["uri"],[])  autolinkWith :: Attr -> String -> Inlines autolinkWith attr s = linkWith attr s "" (str s)@@ -72,10 +72,12 @@   , ("http://en.wikipedia.org/wiki/Sprite_(computer_graphics)",       autolink "http://en.wikipedia.org/wiki/Sprite_(computer_graphics)")   , ("http://en.wikipedia.org/wiki/Sprite_[computer_graphics]",-      link "http://en.wikipedia.org/wiki/Sprite_%5Bcomputer_graphics%5D" ""+      linkWith ("",["uri"],[])+        "http://en.wikipedia.org/wiki/Sprite_%5Bcomputer_graphics%5D" ""         (str "http://en.wikipedia.org/wiki/Sprite_[computer_graphics]"))   , ("http://en.wikipedia.org/wiki/Sprite_{computer_graphics}",-      link "http://en.wikipedia.org/wiki/Sprite_%7Bcomputer_graphics%7D" ""+      linkWith ("",["uri"],[])+        "http://en.wikipedia.org/wiki/Sprite_%7Bcomputer_graphics%7D" ""         (str "http://en.wikipedia.org/wiki/Sprite_{computer_graphics}"))   , ("http://example.com/Notification_Center-GitHub-20101108-140050.jpg",       autolink "http://example.com/Notification_Center-GitHub-20101108-140050.jpg")

@@ -150,6 +150,10 @@           "Foo =bar=, baz" =?>           para (text "Foo " <> code "bar" <> text ", baz") +        , "Not code if followed by digit" =:+          "Foo =bar=0 baz" =?>+          para (text "Foo =bar=0 baz")+         , "One character code" =: "=c=" =?> para (code "c")          , "Three = characters is not a code" =: "===" =?> para "==="@@ -256,7 +260,7 @@       ]    , testGroup "Blocks"-      [ testProperty "Round trip" roundTrip+      [ testProperty "Round trip" (withMaxSuccess 25 roundTrip)       , "Block elements end paragraphs" =:         T.unlines [ "First paragraph"                   , "----"@@ -758,6 +762,13 @@                       , "    > Baz"                       ] =?>             para ("Foo" <> note (para "Bar" <> lineBlock ["Baz"]))+          , "Footnote ending in self-terminating element and followed by paragraph" =:+            T.unlines [ "Foo[1]"+                      , ""+                      , "[1] > bar"+                      , "baz"+                      ] =?>+            para (str "Foo" <> note (lineBlock ["bar"])) <> para (str "baz")           , test emacsMuse "Emacs multiparagraph footnotes"             (T.unlines               [ "First footnote reference[1] and second footnote reference[2]."@@ -1166,6 +1177,11 @@             ] =?>           bulletList [ lineBlock [ "foo" ] ] <> bulletList [ para "bar" ]         ]+      , "List ending in self-terminating element and followed by paragraph" =:+        T.unlines [ " - > Foo"+                  , "bar"+                  ] =?>+        bulletList [lineBlock ["Foo"]] <> para (str "bar")       -- Test that definition list requires a leading space.       -- Emacs Muse does not require a space, we follow Amusewiki here.       , "Not a definition list" =:

@@ -280,6 +280,13 @@                        )                        "echo 'Hello, World'") +  , "Inline code block with a blank argument array" =:+      "src_sh[]{echo 'Hello, World'}" =?>+    para (codeWith ( ""+                       , [ "bash" ]+                       , [ ("org-language", "sh") ])+                       "echo 'Hello, World'")+   , "Inline code block with toggle" =:       "src_sh[:toggle]{echo $HOME}" =?>     para (codeWith ( ""

@@ -198,5 +198,8 @@           , "bare URI parsing disabled inside emphasis (#4561)" =:             "*http://location*" =?>             para (emph (text "http://location"))+          , "include newlines" =:+            "**before\nafter**" =?>+            para (strong (text "before\nafter"))           ]         ]

@@ -2,5 +2,5 @@ % pandoc <http://example.com>{.foo} ^D-<p><a href="http://example.com" class="uri foo">http://example.com</a></p>+<p><a href="http://example.com" class="foo">http://example.com</a></p> ```

@@ -0,0 +1,8 @@+```+% pandoc -f org -t markdown+This won't show the command.+src_maxima[:exports none :results raw]{tex('integrate(sin((e^x)/pi),x,0,inf));} $$\int_{0}^{\infty }{\sin \left({{e^{x}}\over{\pi}}\right)\;dx}$$+^D+This won\'t show the command.+$$\int_{0}^{\infty }{\sin \left({{e^{x}}\over{\pi}}\right)\;dx}$$+```

@@ -0,0 +1,34 @@+```+% pandoc -f markdown -t html+[https://pandoc.org](https://pandoc.org)+^D+<p><a href="https://pandoc.org">https://pandoc.org</a></p>+```++```+% pandoc -f markdown -t markdown+[https://pandoc.org](https://pandoc.org)+^D+<https://pandoc.org>+```++```+% pandoc -f markdown -t html+<https://pandoc.org>+^D+<p><a href="https://pandoc.org" class="uri">https://pandoc.org</a></p>+```++```+% pandoc -f markdown -t html+<https://pandoc.org>{.foo}+^D+<p><a href="https://pandoc.org" class="foo">https://pandoc.org</a></p>+```++```+% pandoc -f markdown -t html+<me@example.com>+^D+<p><a href="mailto:me@example.com" class="email">me@example.com</a></p>+```

@@ -0,0 +1,14 @@+```+% pandoc -f rst -t native+.. _`tgtmath`:++  .. math::+     :name:++     V = \frac{K}{r^2}+^D+[Div ("tgtmath",[],[])+ [BlockQuote+  [Para [Math DisplayMath "V = \\frac{K}{r^2}"]]]]+```+

@@ -0,0 +1,48 @@+```+% pandoc -f latex -t native+\cites(Multiprenote)(multipostnote)[23][42]{Knu86}[65]{Nie72}+^D+[Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "Multiprenote",Space,Str "23"], citationSuffix = [Str "42"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0},Citation {citationId = "Nie72", citationPrefix = [], citationSuffix = [Str "65",Str ",",Space,Str "multipostnote"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\cites(Multiprenote)(multipostnote)[23][42]{Knu86}[65]{Nie72}"]]]+```++```+% pandoc -f latex -t native+\cites(Multiprenote)()[23][42]{Knu86}[65]{Nie72}+^D+[Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "Multiprenote",Space,Str "23"], citationSuffix = [Str "42"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0},Citation {citationId = "Nie72", citationPrefix = [], citationSuffix = [Str "65"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\cites(Multiprenote)()[23][42]{Knu86}[65]{Nie72}"]]]+```++```+% pandoc -f latex -t native+\cites()(multipostnote)[23][42]{Knu86}[65]{Nie72}+^D+[Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "23"], citationSuffix = [Str "42"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0},Citation {citationId = "Nie72", citationPrefix = [], citationSuffix = [Str "65",Str ",",Space,Str "multipostnote"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\cites()(multipostnote)[23][42]{Knu86}[65]{Nie72}"]]]+```++```+% pandoc -f latex -t native+\cites()()[23][42]{Knu86}[65]{Nie72}+^D+[Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "23"], citationSuffix = [Str "42"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0},Citation {citationId = "Nie72", citationPrefix = [], citationSuffix = [Str "65"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\cites()()[23][42]{Knu86}[65]{Nie72}"]]]+```++```+% pandoc -f latex -t native+\cites(multipostnote)[23][42]{Knu86}[65]{Nie72}+^D+[Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "23"], citationSuffix = [Str "42"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0},Citation {citationId = "Nie72", citationPrefix = [], citationSuffix = [Str "65",Str ",",Space,Str "multipostnote"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\cites(multipostnote)[23][42]{Knu86}[65]{Nie72}"]]]+```++```+% pandoc -f latex -t native+\cites(Multiprenote)(multipostnote){Knu86}+^D+[Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "Multiprenote"], citationSuffix = [Str ",",Space,Str "multipostnote"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\cites(Multiprenote)(multipostnote){Knu86}"]]]+```++```+% pandoc -f latex -t native+\footcites(Multiprenote)(multipostnote)[23][42]{Knu86}[65]{Nie72}+^D+[Para [Note [Para [Cite [Citation {citationId = "Knu86", citationPrefix = [Str "Multiprenote",Space,Str "23"], citationSuffix = [Str "42"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0},Citation {citationId = "Nie72", citationPrefix = [], citationSuffix = [Str "65",Str ",",Space,Str "multipostnote"], citationMode = NormalCitation, citationNoteNum = 0, citationHash = 0}] [RawInline (Format "latex") "\\footcites(Multiprenote)(multipostnote)[23][42]{Knu86}[65]{Nie72}"],Str "."]]]]+```

@@ -45,9 +45,9 @@ ,Para [Str "`hi"] ,Para [Str "there`"] ,Header 2 ("multilingual-urls",[],[]) [Str "Multilingual",Space,Str "URLs"]-,Para [Link ("",[],[]) [Str "http://\27979.com?\27979=\27979"] ("http://\27979.com?\27979=\27979","")]+,Para [Link ("",["uri"],[]) [Str "http://\27979.com?\27979=\27979"] ("http://\27979.com?\27979=\27979","")] ,Para [Link ("",[],[]) [Str "foo"] ("/bar/\27979?x=\27979","title")]-,Para [Link ("",[],[]) [Str "\27979@foo.\27979.baz"] ("mailto:\27979@foo.\27979.baz","")]+,Para [Link ("",["email"],[]) [Str "\27979@foo.\27979.baz"] ("mailto:\27979@foo.\27979.baz","")] ,Header 2 ("numbered-examples",[],[]) [Str "Numbered",Space,Str "examples"] ,OrderedList (1,Example,TwoParens)  [[Plain [Str "First",Space,Str "example."]]@@ -176,8 +176,8 @@   ,[]]] ,Header 2 ("entities-in-links-and-titles",[],[]) [Str "Entities",Space,Str "in",Space,Str "links",Space,Str "and",Space,Str "titles"] ,Para [Link ("",[],[]) [Str "link"] ("/\252rl","\246\246!")]-,Para [Link ("",[],[]) [Str "http://g\246\246gle.com"] ("http://g\246\246gle.com","")]-,Para [Link ("",[],[]) [Str "me@ex\228mple.com"] ("mailto:me@ex\228mple.com","")]+,Para [Link ("",["uri"],[]) [Str "http://g\246\246gle.com"] ("http://g\246\246gle.com","")]+,Para [Link ("",["email"],[]) [Str "me@ex\228mple.com"] ("mailto:me@ex\228mple.com","")] ,Para [Link ("",[],[]) [Str "foobar"] ("/\252rl","\246\246!")] ,Header 2 ("parentheses-in-urls",[],[]) [Str "Parentheses",Space,Str "in",Space,Str "URLs"] ,Para [Link ("",[],[]) [Str "link"] ("/hi(there)","")]

@@ -6,59 +6,59 @@   <table:table-column table:style-name="Table1.D" />   <table:table-header-rows>     <table:table-row>-      <table:table-cell table:style-name="Table1.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P1">Right</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table1.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Left</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table1.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P2">Center</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table1.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Default</text:p>       </table:table-cell>     </table:table-row>   </table:table-header-rows>   <table:table-row>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P3">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P4">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P3">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P4">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P3">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P4">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table1.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>   </table:table-row>@@ -73,59 +73,59 @@   <table:table-column table:style-name="Table2.D" />   <table:table-header-rows>     <table:table-row>-      <table:table-cell table:style-name="Table2.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P5">Right</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table2.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Left</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table2.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P6">Center</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table2.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Default</text:p>       </table:table-cell>     </table:table-row>   </table:table-header-rows>   <table:table-row>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P7">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P8">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P7">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P8">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P7">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P8">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table2.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>   </table:table-row>@@ -139,59 +139,59 @@   <table:table-column table:style-name="Table3.D" />   <table:table-header-rows>     <table:table-row>-      <table:table-cell table:style-name="Table3.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P9">Right</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table3.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Left</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table3.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P10">Center</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table3.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Default</text:p>       </table:table-cell>     </table:table-row>   </table:table-header-rows>   <table:table-row>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P11">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P12">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P11">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P12">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P11">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P12">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table3.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>   </table:table-row>@@ -206,46 +206,46 @@   <table:table-column table:style-name="Table4.D" />   <table:table-header-rows>     <table:table-row>-      <table:table-cell table:style-name="Table4.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P13">Centered Header</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table4.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Left Aligned</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table4.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P14">Right Aligned</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table4.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Default aligned</text:p>       </table:table-cell>     </table:table-row>   </table:table-header-rows>   <table:table-row>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P15">First</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">row</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P16">12.0</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">Example of a row that spans       multiple lines.</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P15">Second</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">row</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P16">5.0</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table4.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">Here’s another one. Note the       blank line between rows.</text:p>     </table:table-cell>@@ -262,46 +262,46 @@   <table:table-column table:style-name="Table5.D" />   <table:table-header-rows>     <table:table-row>-      <table:table-cell table:style-name="Table5.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P17">Centered Header</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table5.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Left Aligned</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table5.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="P18">Right Aligned</text:p>       </table:table-cell>-      <table:table-cell table:style-name="Table5.A1" office:value-type="string">+      <table:table-cell table:style-name="TableHeaderRowCell" office:value-type="string">         <text:p text:style-name="Table_20_Heading">Default aligned</text:p>       </table:table-cell>     </table:table-row>   </table:table-header-rows>   <table:table-row>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P19">First</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">row</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P20">12.0</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">Example of a row that spans       multiple lines.</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P19">Second</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">row</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P20">5.0</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table5.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">Here’s another one. Note the       blank line between rows.</text:p>     </table:table-cell>@@ -315,44 +315,44 @@   <table:table-column table:style-name="Table6.C" />   <table:table-column table:style-name="Table6.D" />   <table:table-row>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P24">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P25">12</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P26">12</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P24">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P25">123</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P26">123</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P24">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P25">1</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table6.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P26">1</text:p>     </table:table-cell>   </table:table-row>@@ -365,31 +365,31 @@   <table:table-column table:style-name="Table7.C" />   <table:table-column table:style-name="Table7.D" />   <table:table-row>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P29">First</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">row</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P30">12.0</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">Example of a row that spans       multiple lines.</text:p>     </table:table-cell>   </table:table-row>   <table:table-row>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P29">Second</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">row</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="P30">5.0</text:p>     </table:table-cell>-    <table:table-cell table:style-name="Table7.A1" office:value-type="string">+    <table:table-cell table:style-name="TableRowCell" office:value-type="string">       <text:p text:style-name="Table_20_Contents">Here’s another one. Note the       blank line between rows.</text:p>     </table:table-cell>

@@ -384,14 +384,14 @@ ,Para [Str "Here\8217s",Space,Str "an",Space,Link ("",[],[]) [Str "inline",Space,Str "link"] ("/script?foo=1&bar=2",""),Str "."] ,Para [Str "Here\8217s",Space,Str "an",Space,Link ("",[],[]) [Str "inline",Space,Str "link",Space,Str "in",Space,Str "pointy",Space,Str "braces"] ("/script?foo=1&bar=2",""),Str "."] ,Header 2 ("autolinks",[],[]) [Str "Autolinks"]-,Para [Str "With",Space,Str "an",Space,Str "ampersand:",Space,Link ("",[],[]) [Str "http://example.com/?foo=1&bar=2"] ("http://example.com/?foo=1&bar=2","")]+,Para [Str "With",Space,Str "an",Space,Str "ampersand:",Space,Link ("",["uri"],[]) [Str "http://example.com/?foo=1&bar=2"] ("http://example.com/?foo=1&bar=2","")] ,BulletList  [[Plain [Str "In",Space,Str "a",Space,Str "list?"]]- ,[Plain [Link ("",[],[]) [Str "http://example.com/"] ("http://example.com/","")]]+ ,[Plain [Link ("",["uri"],[]) [Str "http://example.com/"] ("http://example.com/","")]]  ,[Plain [Str "It",Space,Str "should."]]]-,Para [Str "An",Space,Str "e-mail",Space,Str "address:",Space,Link ("",[],[]) [Str "nobody@nowhere.net"] ("mailto:nobody@nowhere.net","")]+,Para [Str "An",Space,Str "e-mail",Space,Str "address:",Space,Link ("",["email"],[]) [Str "nobody@nowhere.net"] ("mailto:nobody@nowhere.net","")] ,BlockQuote- [Para [Str "Blockquoted:",Space,Link ("",[],[]) [Str "http://example.com/"] ("http://example.com/","")]]+ [Para [Str "Blockquoted:",Space,Link ("",["uri"],[]) [Str "http://example.com/"] ("http://example.com/","")]] ,Para [Str "Auto-links",Space,Str "should",Space,Str "not",Space,Str "occur",Space,Str "here:",Space,Code ("",[],[]) "<http://example.com/>"] ,CodeBlock ("",[],[]) "or here: <http://example.com/>" ,HorizontalRule

@@ -1298,7 +1298,7 @@     <title>Autolinks</title>     <para>       With an ampersand:-      <ulink url="http://example.com/?foo=1&amp;bar=2">http://example.com/?foo=1&amp;bar=2</ulink>+      <ulink url="http://example.com/?foo=1&amp;bar=2" role="uri">http://example.com/?foo=1&amp;bar=2</ulink>     </para>     <itemizedlist spacing="compact">       <listitem>@@ -1308,7 +1308,7 @@       </listitem>       <listitem>         <para>-          <ulink url="http://example.com/">http://example.com/</ulink>+          <ulink url="http://example.com/" role="uri">http://example.com/</ulink>         </para>       </listitem>       <listitem>@@ -1323,7 +1323,7 @@     <blockquote>       <para>         Blockquoted:-        <ulink url="http://example.com/">http://example.com/</ulink>+        <ulink url="http://example.com/" role="uri">http://example.com/</ulink>       </para>     </blockquote>     <para>

@@ -1273,7 +1273,7 @@     <title>Autolinks</title>     <para>       With an ampersand:-      <link xlink:href="http://example.com/?foo=1&amp;bar=2">http://example.com/?foo=1&amp;bar=2</link>+      <link xlink:href="http://example.com/?foo=1&amp;bar=2" role="uri">http://example.com/?foo=1&amp;bar=2</link>     </para>     <itemizedlist spacing="compact">       <listitem>@@ -1283,7 +1283,7 @@       </listitem>       <listitem>         <para>-          <link xlink:href="http://example.com/">http://example.com/</link>+          <link xlink:href="http://example.com/" role="uri">http://example.com/</link>         </para>       </listitem>       <listitem>@@ -1298,7 +1298,7 @@     <blockquote>       <para>         Blockquoted:-        <link xlink:href="http://example.com/">http://example.com/</link>+        <link xlink:href="http://example.com/" role="uri">http://example.com/</link>       </para>     </blockquote>     <para>

@@ -508,7 +508,7 @@ <li><a href="http://example.com/" class="uri">http://example.com/</a></li> <li>It should.</li> </ul>-<p>An e-mail address: <a href="mailto:nobody@nowhere.net">nobody@nowhere.net</a></p>+<p>An e-mail address: <a href="mailto:nobody@nowhere.net" class="email">nobody@nowhere.net</a></p> <blockquote> <p>Blockquoted: <a href="http://example.com/" class="uri">http://example.com/</a></p> </blockquote>

@@ -511,7 +511,7 @@ <li><a href="http://example.com/" class="uri">http://example.com/</a></li> <li>It should.</li> </ul>-<p>An e-mail address: <a href="mailto:nobody@nowhere.net">nobody@nowhere.net</a></p>+<p>An e-mail address: <a href="mailto:nobody@nowhere.net" class="email">nobody@nowhere.net</a></p> <blockquote> <p>Blockquoted: <a href="http://example.com/" class="uri">http://example.com/</a></p> </blockquote>

@@ -583,7 +583,7 @@   </boxed-text>   <p>Interpreted markdown in a table:</p>   <p>This is <italic>emphasized</italic></p>-  <p>And this is <bold role="strong">strong</bold></p>+  <p>And this is <bold>strong</bold></p>   <p>Here’s a simple block:</p>   <boxed-text>     <p>foo</p>@@ -614,14 +614,13 @@ <sec id="inline-markup">   <title>Inline Markup</title>   <p>This is <italic>emphasized</italic>, and so <italic>is this</italic>.</p>-  <p>This is <bold role="strong">strong</bold>, and so <bold role="strong">is-  this</bold>.</p>+  <p>This is <bold>strong</bold>, and so <bold>is this</bold>.</p>   <p>An <italic><ext-link ext-link-type="uri" xlink:href="/url">emphasized   link</ext-link></italic>.</p>-  <p><bold role="strong"><italic>This is strong and em.</italic></bold></p>-  <p>So is <bold role="strong"><italic>this</italic></bold> word.</p>-  <p><bold role="strong"><italic>This is strong and em.</italic></bold></p>-  <p>So is <bold role="strong"><italic>this</italic></bold> word.</p>+  <p><bold><italic>This is strong and em.</italic></bold></p>+  <p>So is <bold><italic>this</italic></bold> word.</p>+  <p><bold><italic>This is strong and em.</italic></bold></p>+  <p>So is <bold><italic>this</italic></bold> word.</p>   <p>This is code: <monospace>&gt;</monospace>, <monospace>$</monospace>,   <monospace>\</monospace>, <monospace>\$</monospace>,   <monospace>&lt;html&gt;</monospace>.</p>

@@ -384,14 +384,14 @@ ,Para [Str "Here\8217s",Space,Str "an",Space,Link ("",[],[]) [Str "inline",Space,Str "link"] ("/script?foo=1&bar=2",""),Str "."] ,Para [Str "Here\8217s",Space,Str "an",Space,Link ("",[],[]) [Str "inline",Space,Str "link",Space,Str "in",Space,Str "pointy",Space,Str "braces"] ("/script?foo=1&bar=2",""),Str "."] ,Header 2 ("autolinks",[],[]) [Str "Autolinks"]-,Para [Str "With",Space,Str "an",Space,Str "ampersand:",Space,Link ("",[],[]) [Str "http://example.com/?foo=1&bar=2"] ("http://example.com/?foo=1&bar=2","")]+,Para [Str "With",Space,Str "an",Space,Str "ampersand:",Space,Link ("",["uri"],[]) [Str "http://example.com/?foo=1&bar=2"] ("http://example.com/?foo=1&bar=2","")] ,BulletList  [[Plain [Str "In",Space,Str "a",Space,Str "list?"]]- ,[Plain [Link ("",[],[]) [Str "http://example.com/"] ("http://example.com/","")]]+ ,[Plain [Link ("",["uri"],[]) [Str "http://example.com/"] ("http://example.com/","")]]  ,[Plain [Str "It",Space,Str "should."]]]-,Para [Str "An",Space,Str "e-mail",Space,Str "address:",Space,Link ("",[],[]) [Str "nobody@nowhere.net"] ("mailto:nobody@nowhere.net","")]+,Para [Str "An",Space,Str "e-mail",Space,Str "address:",Space,Link ("",["email"],[]) [Str "nobody@nowhere.net"] ("mailto:nobody@nowhere.net","")] ,BlockQuote- [Para [Str "Blockquoted:",Space,Link ("",[],[]) [Str "http://example.com/"] ("http://example.com/","")]]+ [Para [Str "Blockquoted:",Space,Link ("",["uri"],[]) [Str "http://example.com/"] ("http://example.com/","")]] ,Para [Str "Auto-links",Space,Str "should",Space,Str "not",Space,Str "occur",Space,Str "here:",Space,Code ("",[],[]) "<http://example.com/>"] ,CodeBlock ("",[],[]) "or here: <http://example.com/>" ,HorizontalRule

@@ -660,7 +660,7 @@ * "$":http://example.com/ * It should. -An e-mail address: "nobody@nowhere.net":mailto:nobody@nowhere.net+An e-mail address: "(email)nobody@nowhere.net":mailto:nobody@nowhere.net  bq. Blockquoted: "$":http://example.com/