diff --git a/AUTHORS.md b/AUTHORS.md --- a/AUTHORS.md +++ b/AUTHORS.md @@ -89,6 +89,7 @@ - Ethan Riley - Étienne Bersac - Ezwal +- Fabián Heredia Montiel - Félix Baylac-Jacqué - Felix Yan - Florian Beeres @@ -242,9 +243,11 @@ - Raniere Silva - Raymond Ehlers - Recai Oktaş +- Rowan Rodrik van der Molen - Roland Hieber - Roman Beránek - RyanGlScott +- S.P.H - Salim B - Samuel Tardieu - Sascha Wilde @@ -341,4 +344,5 @@ - timo-a - vijayphoenix - wiefling +- willj-dev - wuffi diff --git a/MANUAL.txt b/MANUAL.txt --- a/MANUAL.txt +++ b/MANUAL.txt @@ -1,7 +1,7 @@ --- title: Pandoc User's Guide author: John MacFarlane -date: November 02, 2021 +date: November 20, 2021 --- # Synopsis @@ -266,6 +266,7 @@ - `tikiwiki` ([TikiWiki markup]) - `twiki` ([TWiki markup]) - `vimwiki` ([Vimwiki]) + - the path of a custom Lua reader, see [Custom readers and writers] below ::: Extensions can be individually enabled or disabled by @@ -338,7 +339,7 @@ - `tei` ([TEI Simple]) - `xwiki` ([XWiki markup]) - `zimwiki` ([ZimWiki markup]) - - the path of a custom Lua writer, see [Custom writers] below + - the path of a custom Lua writer, see [Custom readers and writers] below ::: Note that `odt`, `docx`, `epub`, and `pdf` output will not be directed @@ -1480,6 +1481,7 @@ Code Error ----- ------------------------------------ + 1 PandocIOError 3 PandocFailOnWarningError 4 PandocAppError 5 PandocTemplateError @@ -1501,6 +1503,7 @@ 66 PandocMakePDFError 67 PandocSyntaxMapError 83 PandocFilterError + 84 PandocLuaError 91 PandocMacroLoop 92 PandocUTF8DecodingError 93 PandocIpynbDecodingError @@ -5331,7 +5334,7 @@ is added; other elements are placed in a surrounding Div or Span elemnet with a `data-pos` attribute. -#### Extension: `short_subsuperscript` #### +#### Extension: `short_subsuperscripts` #### Parse multimarkdown style subscripts and superscripts, which start with a '~' or '^' character, respectively, and include the alphanumeric sequence @@ -6572,20 +6575,36 @@ reference-doc while creating docx output (see below), and maintain the same styles in your input and output files. -# Custom writers +# Custom readers and writers -Pandoc can be extended with custom writers written in [Lua]. (Pandoc -includes a Lua interpreter, so Lua need not be installed separately.) +Pandoc can be extended with custom readers and writers written +in [Lua]. (Pandoc includes a Lua interpreter, so Lua need not +be installed separately.) -To use a custom writer, simply specify the path to the Lua script -in place of the output format. For example: +To use a custom reader or writer, simply specify the path to the +Lua script in place of the input or output format. For example: pandoc -t data/sample.lua + pandoc -f my_custom_markup_language.lua -t latex -s -Creating a custom writer requires writing a Lua function for each -possible element in a pandoc document. To get a documented example -which you can modify according to your needs, do +A custom reader is a Lua script that defines one function, +Reader, which takes a string as input and returns a Pandoc +AST. See the [Lua filters documentation] for documentation +of the functions that are available for creating pandoc +AST elements. For parsing, the [lpeg] parsing library +is available by default. To see a sample custom reader: + pandoc --print-default-data-file creole.lua + +If you want your custom reader to have access to reader options +(e.g. the tab stop setting), you give your Reader function a +second `options` parameter. + +A custom writer is a Lua script that defines a function +that specifies how to render each element in a Pandoc AST. +To see a documented example which you can modify according +to your needs: + pandoc --print-default-data-file sample.lua Note that custom writers have no default template. If you want @@ -6596,6 +6615,7 @@ subdirectory of your user data directory (see [Templates]). [Lua]: https://www.lua.org +[lpeg]: http://www.inf.puc-rio.br/~roberto/lpeg/ # Reproducible builds diff --git a/cabal.project b/cabal.project --- a/cabal.project +++ b/cabal.project @@ -3,3 +3,7 @@ flags: +embed_data_files constraints: aeson >= 2.0.1.0 +source-repository-package + type: git + location: https://github.com/jgm/texmath.git + tag: 674bcbaec03e5550f155623de6662953bd157625 diff --git a/changelog.md b/changelog.md --- a/changelog.md +++ b/changelog.md @@ -1,4 +1,171 @@ -# Revision history for pandoc +## Revision history for pandoc + +## pandoc 2.16.2 (2021-11-21) + + * Add interface for custom readers written in Lua (#7669). + Users can now do `-f myreader.lua` and pandoc will treat the + script `myreader.lua` as a custom reader, which parses an input + string to a pandoc AST, using the pandoc module defined for + Lua filters. A sample custom reader can be found in `data/creole.lua`. + Also see documentation in `doc/custom-readers.md`. + + * New module Text.Pandoc.Readers.Custom, exporting `readCustom` [API change]. + + * Allow `plain` to be used in raw attribute syntax. + + * Accept empty `--metadata-file` (#7675). + This was a regression from 2.15 behavior. + + * Markdown reader: Improve `inlinesInBalancedBrackets`. + This is just a small improvement in terms of performance, but it's simpler + and more direct code. Also, we avoid parsing interparagraph spaces in + balanced brackets, as the original did. + + * BibTeX reader: Properly handle commented lines in BibTeX/BibLaTeX (#7668). + + * RST reader: handle class attribute for for custom roles (#7699, + willj-dev). Previously the class attribute was ignored, and the name + of the role used as the class. + + * DocBook reader: + + + Add `` support (Rowan Rodrik van der Molen). + + Support for `` (#7607, Rowan Rodrik van der Molen). + + * LaTeX reader: + + + Add rudimentary support for `\autoref` (#7693). + + Add 'uri' class when parsing `\url`, for consistency + with treatment of autolinks in other formats (#7672). + + * JATS reader: Capture `alt-text` in figures (#7703, Aner Lucero). + + * MediaWiki writer: use HTML spans for anchors when header has id (#7697). + We need to generate a span when the header's ID doesn't match + the one MediaWiki would generate automatically. Note that MediaWiki's + generation scheme is different from pandoc's (it uses uppercase letters, + and `_` instead of `-`, for example). This means that in going from + markdown to mediawiki, we'll now get spans before almost every heading, + unless explicit identifiers are used that correspond to the ones MediaWiki + auto-generates. This is uglier output but it's necessary for internal + links to work properly. + + * Markdown writer: don't create autolinks when this loses information + (#7692). Previously we sometimes lost attributes when rendering links + as autolinks. + + * Text.Pandoc.Readers.Metadata: allow multiple YAML documents when parsing + YAML for `yamlBsToRefs`. Some people use `---` as the end delimiter in + YAML bibliography files, which causes the `yaml` library to emit an + error unless we explicitly allow multiple YAML documents (and just + consider the first). + + * JATS writer: + + + Ensure figures are wrapped with `

` in list items + (Albert Krewinkel). This prevents the generation of invalid output. + + Add URL to element citation entries (Albert Krewinkel). + The URL of a reference, if present, is added in tag `` to + element-citation entries. + + * HTML writer: Don't create invalid `data-` attribute for empty + attribute key (#7546). + + * LaTeX writer: + + + Babel mappings: use `ancientgreek` for `grc`. + + With `-t latex-smart`, don't generate `\ldots` from ellipsis (#7674). + Instead just use unicode ellipsis. + + * JATS template: fix `equal-contrib` attribute (Albert Krewinkel). + The standard requires the value to be either `yes` or `no`, but is was + set to `true` for authors who contributed equally. + + * reveal.js template: Add `disableLayout` variable (Christophe Dervieux). + + * Text.Pandoc.Error: sort errors in `handleError` by exit code + (Albert Krewinkel). + + * Text.Pandoc.Writers.Shared: Improve toLegacyTable (#7683, + Christian Despres). + + * Lua subsystem: + + + Include lpeg module (#7649, Albert Krewinkel). Compiles the `lpeg` + library (Parsing Expression Grammars For Lua) into + the program. Package maintainers may choose to rely on package + dependencies to make lpeg available, in which case they can compile + the with the constraint `lpeg +rely-on-shared-lpeg-library`. + `lpeg` and `re` are always made available in global variables, + without the need for a `require`. + + + Set `lpeg` and `re` as globals; allow shared lib access via `require`. + The `lpeg` and `re` modules are loaded into globals of the respective + name, but they are not necessarily registered as loaded packages. This + ensures that + + - the built-in library versions are preferred when setting the globals, + - a shared library is used if pandoc has been compiled without `lpeg`, + and + - the `require` mechanism can be used to load the shared library if + available, falling back to the internal version if possible and + necessary. + + + Fix argument order in constructor `pandoc.Cite` (Albert Krewinkel). + This restores the old behavior; argument order had been switched + accidentally in pandoc 2.15. + + + Add Pushable instance for `ReaderOptions` (Albert Krewinkel). + + + Allow to pass custom reader options to `pandoc.read` as an + optional third argument (#7656, Albert Krewinkel). + The object can either be a table or a ReaderOptions value + like `PANDOC_READER_OPTIONS`. Creating new ReaderOptions objects is + possible through the new constructor `pandoc.ReaderOptions`. + + + Display Pandoc values using their native Haskell representation + (Albert Krewinkel). + + + Require latest hslua (2.0.1) (#7661, #7657, Albert Krewinkel). + This fixes issues with + + - misleading error messages when a required function parameter is + omitted; + - absent properties still being listed in the output of `pairs`; and + - alias accessing leading to errors instead of returning `nil`, e.g. + with `(pandoc.Str '').identifier`. + + + Add missing space in "package not found" message (#7658, Albert + Krewinkel). + + * Update build files (#7696, Fabián Heredia Montiel). + Drop old windows 32-bit constraints. + Update cabal `tested-with` field to correspond to `ci.yml` matrix + + * Remove unneeded package dependencies from benchmark target. + + * Require ghc >= 8.6, base >= 4.12. + This allows us to get rid of the old custom prelude and + some crufty cpp. But the primary reason for this is that + conduit has bumped its base lower bound to 4.12, making it + impossible for us to support lower base versions. + + * Require Cabal 2.4. Use wildcards to ensure that all pptx tests are + included (#7677). + + * Update `bash_completion.tpl` (S.P.H.). + + * Add `data/creole.lua` as sample custom reader. + + * Add `doc/custom-readers.md` and `doc/custom-writers.md`. + + * `doc/lua-filters.md`: add section on global modules, including lpeg + (Albert Krewinkel). + + * `MANUAL.txt`: update table of exit codes and corresponding errors + (Albert Krewinkel). + + * Use latest texmath. ## pandoc 2.16.1 (2021-11-02) diff --git a/data/bash_completion.tpl b/data/bash_completion.tpl --- a/data/bash_completion.tpl +++ b/data/bash_completion.tpl @@ -4,7 +4,7 @@ _pandoc() { - local cur prev opts lastc informats outformats datafiles + local cur prev opts lastc informats outformats highlight_styles datafiles COMPREPLY=() cur="${COMP_WORDS[COMP_CWORD]}" prev="${COMP_WORDS[COMP_CWORD-1]}" @@ -57,8 +57,16 @@ COMPREPLY=( $(compgen -W "section chapter part" -- ${cur}) ) return 0 ;; - --highlight-style) + --highlight-style|--print-highlight-style) COMPREPLY=( $(compgen -W "${highlight_styles}" -- ${cur}) ) + return 0 + ;; + --eol) + COMPREPLY=( $(compgen -W "crlf lf native" -- ${cur}) ) + return 0 + ;; + --markdown-headings) + COMPREPLY=( $(compgen -W "setext atx" -- ${cur}) ) return 0 ;; *) diff --git a/data/creole.lua b/data/creole.lua new file mode 100644 --- /dev/null +++ b/data/creole.lua @@ -0,0 +1,190 @@ +-- A sample custom reader for Creole 1.0 (common wiki markup) +-- http://www.wikicreole.org/wiki/CheatSheet + +-- For better performance we put these functions in local variables: +local P, S, R, Cf, Cc, Ct, V, Cs, Cg, Cb, B, C, Cmt = + lpeg.P, lpeg.S, lpeg.R, lpeg.Cf, lpeg.Cc, lpeg.Ct, lpeg.V, + lpeg.Cs, lpeg.Cg, lpeg.Cb, lpeg.B, lpeg.C, lpeg.Cmt + +local whitespacechar = S(" \t\r\n") +local specialchar = S("/*~[]\\{}|") +local wordchar = (1 - (whitespacechar + specialchar)) +local spacechar = S(" \t") +local newline = P"\r"^-1 * P"\n" +local blankline = spacechar^0 * newline +local endline = newline * #-blankline +local endequals = spacechar^0 * P"="^0 * spacechar^0 * newline +local cellsep = spacechar^0 * P"|" + +local function trim(s) + return (s:gsub("^%s*(.-)%s*$", "%1")) +end + +local function ListItem(lev, ch) + local start + if ch == nil then + start = S"*#" + else + start = P(ch) + end + local subitem = function(c) + if lev < 6 then + return ListItem(lev + 1, c) + else + return (1 - 1) -- fails + end + end + local parser = spacechar^0 + * start^lev + * #(- start) + * spacechar^0 + * Ct((V"Inline" - (newline * spacechar^0 * S"*#"))^0) + * newline + * (Ct(subitem("*")^1) / pandoc.BulletList + + + Ct(subitem("#")^1) / pandoc.OrderedList + + + Cc(nil)) + / function (ils, sublist) + return { pandoc.Plain(ils), sublist } + end + return parser +end + +-- Grammar +G = P{ "Doc", + Doc = Ct(V"Block"^0) + / pandoc.Pandoc ; + Block = blankline^0 + * ( V"Header" + + V"HorizontalRule" + + V"CodeBlock" + + V"List" + + V"Table" + + V"Para") ; + Para = Ct(V"Inline"^1) + * newline + / pandoc.Para ; + HorizontalRule = spacechar^0 + * P"----" + * spacechar^0 + * newline + / pandoc.HorizontalRule; + Header = (P("=")^1 / string.len) + * spacechar^1 + * Ct((V"Inline" - endequals)^1) + * endequals + / pandoc.Header; + CodeBlock = P"{{{" + * blankline + * C((1 - (newline * P"}}}"))^0) + * newline + * P"}}}" + / pandoc.CodeBlock; + Placeholder = P"<<<" + * C(P(1) - P">>>")^0 + * P">>>" + / function() return pandoc.Div({}) end; + List = V"BulletList" + + V"OrderedList" ; + BulletList = Ct(ListItem(1,'*')^1) + / pandoc.BulletList ; + OrderedList = Ct(ListItem(1,'#')^1) + / pandoc.OrderedList ; + Table = (V"TableHeader" + Cc{}) + * Ct(V"TableRow"^1) + / function(headrow, bodyrows) + local numcolumns = #(bodyrows[1]) + local aligns = {} + local widths = {} + for i = 1,numcolumns do + aligns[i] = pandoc.AlignDefault + widths[i] = 0 + end + return pandoc.utils.from_simple_table( + pandoc.SimpleTable({}, aligns, widths, headrow, bodyrows)) + end ; + TableHeader = Ct(V"HeaderCell"^1) + * cellsep^-1 + * spacechar^0 + * newline ; + TableRow = Ct(V"BodyCell"^1) + * cellsep^-1 + * spacechar^0 + * newline ; + HeaderCell = cellsep + * P"=" + * spacechar^0 + * Ct((V"Inline" - (newline + cellsep))^0) + / function(ils) return { pandoc.Plain(ils) } end ; + BodyCell = cellsep + * spacechar^0 + * Ct((V"Inline" - (newline + cellsep))^0) + / function(ils) return { pandoc.Plain(ils) } end ; + Inline = V"Emph" + + V"Strong" + + V"LineBreak" + + V"Link" + + V"URL" + + V"Image" + + V"Str" + + V"Space" + + V"SoftBreak" + + V"Escaped" + + V"Placeholder" + + V"Code" + + V"Special" ; + Str = wordchar^1 + / pandoc.Str; + Escaped = P"~" + * C(P(1)) + / pandoc.Str ; + Special = specialchar + / pandoc.Str; + Space = spacechar^1 + / pandoc.Space ; + SoftBreak = endline + * # -(V"HorizontalRule" + V"CodeBlock") + / pandoc.SoftBreak ; + LineBreak = P"\\\\" + / pandoc.LineBreak ; + Code = P"{{{" + * C((1 - P"}}}")^0) + * P"}}}" + / trim / pandoc.Code ; + Link = P"[[" + * C((1 - (P"]]" + P"|"))^0) + * (P"|" * Ct((V"Inline" - P"]]")^1))^-1 * P"]]" + / function(url, desc) + local txt = desc or {pandoc.Str(url)} + return pandoc.Link(txt, url) + end ; + Image = P"{{" + * #-P"{" + * C((1 - (S"}"))^0) + * (P"|" * Ct((V"Inline" - P"}}")^1))^-1 + * P"}}" + / function(url, desc) + local txt = desc or "" + return pandoc.Image(txt, url) + end ; + URL = P"http" + * P"s"^-1 + * P":" + * (1 - (whitespacechar + (S",.?!:;\"'" * #whitespacechar)))^1 + / function(url) + return pandoc.Link(pandoc.Str(url), url) + end ; + Emph = P"//" + * Ct((V"Inline" - P"//")^1) + * P"//" + / pandoc.Emph ; + Strong = P"**" + * Ct((V"Inline" -P"**")^1) + * P"**" + / pandoc.Strong ; +} + +function Reader(input, reader_options) + return lpeg.match(G, input) +end diff --git a/data/templates/article.jats_publishing b/data/templates/article.jats_publishing --- a/data/templates/article.jats_publishing +++ b/data/templates/article.jats_publishing @@ -86,7 +86,7 @@ $if(author)$ $for(author)$ - + $if(author.orcid)$ $author.orcid$ $endif$ diff --git a/data/templates/default.jats_articleauthoring b/data/templates/default.jats_articleauthoring --- a/data/templates/default.jats_articleauthoring +++ b/data/templates/default.jats_articleauthoring @@ -19,7 +19,7 @@ $if(author)$ $for(author)$ - + $if(author.orcid)$ $author.orcid$ $endif$ diff --git a/data/templates/default.revealjs b/data/templates/default.revealjs --- a/data/templates/default.revealjs +++ b/data/templates/default.revealjs @@ -137,7 +137,7 @@ // Disables the default reveal.js slide layout (scaling and centering) // so that you can use custom CSS layout - disableLayout: false, + disableLayout: $disableLayout$, // Vertical centering of slides center: $center$, diff --git a/man/pandoc.1 b/man/pandoc.1 --- a/man/pandoc.1 +++ b/man/pandoc.1 @@ -1,7 +1,7 @@ '\" t -.\" Automatically generated by Pandoc 2.16 +.\" Automatically generated by Pandoc 2.16.1 .\" -.TH "Pandoc User\[cq]s Guide" "" "November 02, 2021" "pandoc 2.16.1" "" +.TH "Pandoc User\[cq]s Guide" "" "November 20, 2021" "pandoc 2.16.2" "" .hy .SH NAME pandoc - general markup converter @@ -304,6 +304,8 @@ \f[C]twiki\f[R] (TWiki markup) .IP \[bu] 2 \f[C]vimwiki\f[R] (Vimwiki) +.IP \[bu] 2 +the path of a custom Lua reader, see Custom readers and writers below .PP Extensions can be individually enabled or disabled by appending \f[C]+EXTENSION\f[R] or \f[C]-EXTENSION\f[R] to the format name. @@ -432,7 +434,7 @@ .IP \[bu] 2 \f[C]zimwiki\f[R] (ZimWiki markup) .IP \[bu] 2 -the path of a custom Lua writer, see Custom writers below +the path of a custom Lua writer, see Custom readers and writers below .PP Note that \f[C]odt\f[R], \f[C]docx\f[R], \f[C]epub\f[R], and \f[C]pdf\f[R] output will not be directed to \f[I]stdout\f[R] unless @@ -1598,6 +1600,11 @@ T} _ T{ +1 +T}@T{ +PandocIOError +T} +T{ 3 T}@T{ PandocFailOnWarningError @@ -1703,6 +1710,11 @@ PandocFilterError T} T{ +84 +T}@T{ +PandocLuaError +T} +T{ 91 T}@T{ PandocMacroLoop @@ -6216,7 +6228,7 @@ For elements that accept attributes, a \f[C]data-pos\f[R] attribute is added; other elements are placed in a surrounding Div or Span elemnet with a \f[C]data-pos\f[R] attribute. -.SS Extension: \f[C]short_subsuperscript\f[R] +.SS Extension: \f[C]short_subsuperscripts\f[R] .PP Parse multimarkdown style subscripts and superscripts, which start with a `\[ti]' or `\[ha]' character, respectively, and include the @@ -7810,26 +7822,44 @@ With these custom styles, you can use your input document as a reference-doc while creating docx output (see below), and maintain the same styles in your input and output files. -.SH CUSTOM WRITERS +.SH CUSTOM READERS AND WRITERS .PP -Pandoc can be extended with custom writers written in Lua. +Pandoc can be extended with custom readers and writers written in Lua. (Pandoc includes a Lua interpreter, so Lua need not be installed separately.) .PP -To use a custom writer, simply specify the path to the Lua script in -place of the output format. +To use a custom reader or writer, simply specify the path to the Lua +script in place of the input or output format. For example: .IP .nf \f[C] pandoc -t data/sample.lua +pandoc -f my_custom_markup_language.lua -t latex -s \f[R] .fi .PP -Creating a custom writer requires writing a Lua function for each -possible element in a pandoc document. -To get a documented example which you can modify according to your -needs, do +A custom reader is a Lua script that defines one function, Reader, which +takes a string as input and returns a Pandoc AST. +See the Lua filters documentation for documentation of the functions +that are available for creating pandoc AST elements. +For parsing, the lpeg parsing library is available by default. +To see a sample custom reader: +.IP +.nf +\f[C] +pandoc --print-default-data-file creole.lua +\f[R] +.fi +.PP +If you want your custom reader to have access to reader options +(e.g.\ the tab stop setting), you give your Reader function a second +\f[C]options\f[R] parameter. +.PP +A custom writer is a Lua script that defines a function that specifies +how to render each element in a Pandoc AST. +To see a documented example which you can modify according to your +needs: .IP .nf \f[C] diff --git a/pandoc.cabal b/pandoc.cabal --- a/pandoc.cabal +++ b/pandoc.cabal @@ -1,6 +1,6 @@ -cabal-version: 2.2 +cabal-version: 2.4 name: pandoc -version: 2.16.1 +version: 2.16.2 build-type: Simple license: GPL-2.0-or-later license-file: COPYING.md @@ -11,8 +11,7 @@ stability: alpha homepage: https://pandoc.org category: Text -tested-with: GHC == 8.2.2, GHC == 8.4.4, GHC == 8.6.5, - GHC == 8.8.4, GHC == 8.10.2, GHC == 9.0.1 +tested-with: GHC == 8.6.5, GHC == 8.8.4, GHC == 8.10.7, GHC == 9.0.1 synopsis: Conversion between markup formats description: Pandoc is a Haskell library for converting from one markup format to another, and a command-line tool that uses @@ -174,6 +173,8 @@ data/abbreviations -- sample lua custom writer data/sample.lua + -- sample lua custom reader + data/creole.lua -- lua init script data/init.lua -- pandoc lua module @@ -380,89 +381,8 @@ test/rtf/*.native test/rtf/*.rtf test/pptx/*.pptx - test/pptx/background-image/input.native - test/pptx/background-image/*.pptx - test/pptx/blanks/just-speaker-notes/input.native - test/pptx/blanks/just-speaker-notes/*.pptx - test/pptx/blanks/nbsp-in-body/input.native - test/pptx/blanks/nbsp-in-body/*.pptx - test/pptx/blanks/nbsp-in-heading/input.native - test/pptx/blanks/nbsp-in-heading/*.pptx - test/pptx/code-custom/*.pptx - test/pptx/code/input.native - test/pptx/code/*.pptx - test/pptx/content-with-caption/heading-text-image/input.native - test/pptx/content-with-caption/heading-text-image/*.pptx - test/pptx/content-with-caption/image-text/input.native - test/pptx/content-with-caption/image-text/*.pptx - test/pptx/content-with-caption/text-image/input.native - test/pptx/content-with-caption/text-image/*.pptx - test/pptx/comparison/both-columns/input.native - test/pptx/comparison/both-columns/*.pptx - test/pptx/comparison/extra-image/input.native - test/pptx/comparison/extra-image/*.pptx - test/pptx/comparison/extra-text/input.native - test/pptx/comparison/extra-text/*.pptx - test/pptx/comparison/non-text-first/input.native - test/pptx/comparison/non-text-first/*.pptx - test/pptx/comparison/one-column/input.native - test/pptx/comparison/one-column/*.pptx - test/pptx/document-properties-short-desc/input.native - test/pptx/document-properties-short-desc/*.pptx - test/pptx/document-properties/input.native - test/pptx/document-properties/*.pptx - test/pptx/endnotes-toc/*.pptx - test/pptx/endnotes/input.native - test/pptx/endnotes/*.pptx - test/pptx/footer/input.native - test/pptx/footer/basic/*.pptx - test/pptx/footer/fixed-date/*.pptx - test/pptx/footer/higher-slide-number/*.pptx - test/pptx/footer/no-title-slide/*.pptx - test/pptx/images/input.native - test/pptx/images/*.pptx - test/pptx/incremental-lists/with-flag/input.native - test/pptx/incremental-lists/with-flag/*.pptx - test/pptx/incremental-lists/without-flag/input.native - test/pptx/incremental-lists/without-flag/*.pptx - test/pptx/inline-formatting/input.native - test/pptx/inline-formatting/*.pptx - test/pptx/lists/input.native - test/pptx/lists/*.pptx - test/pptx/list-level/input.native - test/pptx/list-level/*.pptx - test/pptx/raw-ooxml/input.native - test/pptx/raw-ooxml/*.pptx - test/pptx/remove-empty-slides/input.native - test/pptx/remove-empty-slides/*.pptx - test/pptx/slide-breaks-slide-level-1/*.pptx - test/pptx/slide-breaks-toc/*.pptx - test/pptx/slide-breaks/input.native - test/pptx/slide-breaks/*.pptx - test/pptx/slide-level-0/h1-h2-with-table/input.native - test/pptx/slide-level-0/h1-h2-with-table/*.pptx - test/pptx/slide-level-0/h1-with-image/input.native - test/pptx/slide-level-0/h1-with-image/*.pptx - test/pptx/slide-level-0/h1-with-table/input.native - test/pptx/slide-level-0/h1-with-table/*.pptx - test/pptx/slide-level-0/h2-with-image/input.native - test/pptx/slide-level-0/h2-with-image/*.pptx - test/pptx/speaker-notes-after-metadata/input.native - test/pptx/speaker-notes-after-metadata/*.pptx - test/pptx/speaker-notes-afterheader/input.native - test/pptx/speaker-notes-afterheader/*.pptx - test/pptx/speaker-notes-afterseps/input.native - test/pptx/speaker-notes-afterseps/*.pptx - test/pptx/speaker-notes/input.native - test/pptx/speaker-notes/*.pptx - test/pptx/start-numbering-at/input.native - test/pptx/start-numbering-at/*.pptx - test/pptx/tables/input.native - test/pptx/tables/*.pptx - test/pptx/two-column/all-text/input.native - test/pptx/two-column/all-text/*.pptx - test/pptx/two-column/text-and-image/input.native - test/pptx/two-column/text-and-image/*.pptx + test/pptx/**/*.pptx + test/pptx/**/*.native test/ipynb/*.in.native test/ipynb/*.out.native test/ipynb/*.ipynb @@ -488,31 +408,27 @@ common common-options default-language: Haskell2010 - build-depends: base >= 4.9 && < 5 + build-depends: base >= 4.12 && < 5 ghc-options: -Wall -fno-warn-unused-do-bind -Wincomplete-record-updates -Wnoncanonical-monad-instances + -Wcpp-undef + -Wincomplete-uni-patterns + -Widentities + -Wpartial-fields + -Wmissing-signatures + -fhide-source-paths + -- -Wmissing-export-lists - if impl(ghc < 8.6) - hs-source-dirs: prelude - other-modules: Prelude - build-depends: base-compat >= 0.10.5 - other-extensions: NoImplicitPrelude + if impl(ghc >= 8.10) + ghc-options: -Wunused-packages + if impl(ghc >= 9.0) + ghc-options: -Winvalid-haddock + if os(windows) cpp-options: -D_WINDOWS - -- Later: - -- -Wpartial-fields (currently used in Powerpoint writer) - -- -Wmissing-export-lists (currently some Odt modules violate this) - -- -Wredundant-constraints (problematic if we want to support older base) - if impl(ghc >= 8.2) - ghc-options: -Wcpp-undef - if impl(ghc >= 8.4) - ghc-options: -Wincomplete-uni-patterns - -Widentities - -fhide-source-paths - common common-executable import: common-options build-depends: pandoc @@ -550,8 +466,8 @@ file-embed >= 0.0 && < 0.1, filepath >= 1.1 && < 1.5, haddock-library >= 1.10 && < 1.11, - hslua >= 2.0 && < 2.1, - hslua-marshalling >= 2.0 && < 2.1, + hslua >= 2.0.1 && < 2.1, + hslua-marshalling >= 2.0.1 && < 2.1, hslua-module-path >= 1.0 && < 1.1, hslua-module-system >= 1.0 && < 1.1, hslua-module-text >= 1.0 && < 1.1, @@ -561,6 +477,7 @@ http-types >= 0.8 && < 0.13, ipynb >= 0.1.0.2 && < 0.2, jira-wiki-markup >= 1.4 && < 1.5, + lpeg >= 1.0.1 && < 1.1, mtl >= 2.2 && < 2.3, network >= 2.6, network-uri >= 2.6 && < 2.8, @@ -589,10 +506,6 @@ yaml >= 0.11 && < 0.12, zip-archive >= 0.2.3.4 && < 0.5, zlib >= 0.5 && < 0.7 - if os(windows) && arch(i386) - build-depends: basement >= 0.0.10, - foundation >= 0.0.23 - -- basement 0.0.9 won't build on 32-bit windows. if !os(windows) build-depends: unix >= 2.4 && < 2.8 if flag(embed_data_files) @@ -641,6 +554,7 @@ Text.Pandoc.Readers.Ipynb, Text.Pandoc.Readers.CSV, Text.Pandoc.Readers.RTF, + Text.Pandoc.Readers.Custom, Text.Pandoc.Writers, Text.Pandoc.Writers.Native, Text.Pandoc.Writers.Docbook, @@ -924,12 +838,9 @@ main-is: benchmark-pandoc.hs hs-source-dirs: benchmark build-depends: bytestring, - containers, - -- gauge >= 0.2 && < 0.3, tasty-bench >= 0.2 && <= 0.4, mtl >= 2.2 && < 2.3, text >= 1.1.1.0 && < 1.3, - time, deepseq -- we increase heap size to avoid benchmarking garbage collection: ghc-options: -rtsopts -with-rtsopts=-A8m -threaded diff --git a/prelude/Prelude.hs b/prelude/Prelude.hs deleted file mode 100644 --- a/prelude/Prelude.hs +++ /dev/null @@ -1,15 +0,0 @@ -{-# LANGUAGE NoImplicitPrelude #-} - --- The intent is that this Prelude provide the API of --- the base 4.11 Prelude in a way that is portable for --- all base versions. - -module Prelude -( - module Prelude.Compat -, Semigroup(..) -) -where - -import Prelude.Compat -import Data.Semigroup (Semigroup(..)) -- includes (<>) diff --git a/src/Text/Pandoc/App.hs b/src/Text/Pandoc/App.hs --- a/src/Text/Pandoc/App.hs +++ b/src/Text/Pandoc/App.hs @@ -68,6 +68,7 @@ defaultUserDataDir, tshow) import Text.Pandoc.Writers.Shared (lookupMetaString) import Text.Pandoc.Readers.Markdown (yamlToMeta) +import Text.Pandoc.Readers.Custom (readCustom) import qualified Text.Pandoc.UTF8 as UTF8 #ifndef _WINDOWS import System.Posix.IO (stdOutput) @@ -154,11 +155,13 @@ -> ByteStringReader $ \o t -> sandbox files (r o t) (reader, readerExts) <- - if optSandbox opts - then case runPure (getReader readerName) of - Left e -> throwError e - Right (r, rexts) -> return (makeSandboxed r, rexts) - else getReader readerName + if ".lua" `T.isSuffixOf` readerName + then return (TextReader (readCustom (T.unpack readerName)), mempty) + else if optSandbox opts + then case runPure (getReader readerName) of + Left e -> throwError e + Right (r, rexts) -> return (makeSandboxed r, rexts) + else getReader readerName outputSettings <- optToOutputSettings opts let format = outputFormat outputSettings diff --git a/src/Text/Pandoc/App/CommandLineOptions.hs b/src/Text/Pandoc/App/CommandLineOptions.hs --- a/src/Text/Pandoc/App/CommandLineOptions.hs +++ b/src/Text/Pandoc/App/CommandLineOptions.hs @@ -33,10 +33,8 @@ import Data.Char (toLower) import Data.List (intercalate, sort, foldl') #ifdef _WINDOWS -#if MIN_VERSION_base(4,12,0) import Data.List (isPrefixOf) #endif -#endif import Data.Maybe (fromMaybe, isJust) import Data.Text (Text) import Safe (tailDef) @@ -1078,14 +1076,10 @@ -- beginning with \\ to \\?\UNC\. -- See #5127. normalizePath :: FilePath -> FilePath #ifdef _WINDOWS -#if MIN_VERSION_base(4,12,0) normalizePath fp = if "\\\\" `isPrefixOf` fp && not ("\\\\?\\" `isPrefixOf` fp) then "\\\\?\\UNC\\" ++ drop 2 fp else fp -#else -normalizePath = id -#endif #else normalizePath = id #endif diff --git a/src/Text/Pandoc/Citeproc/BibTeX.hs b/src/Text/Pandoc/Citeproc/BibTeX.hs --- a/src/Text/Pandoc/Citeproc/BibTeX.hs +++ b/src/Text/Pandoc/Citeproc/BibTeX.hs @@ -48,7 +48,7 @@ import qualified Data.Sequence as Seq import Data.Char (isAlphaNum, isDigit, isLetter, isUpper, toLower, toUpper, - isLower, isPunctuation) + isLower, isPunctuation, isSpace) import Data.List (foldl', intercalate, intersperse) import Safe (readMay) import Text.Printf (printf) @@ -804,30 +804,34 @@ skipMany nonEntry many (bibItem <* skipMany nonEntry) where nonEntry = bibSkip <|> + comment <|> try (char '@' >> (bibComment <|> bibPreamble <|> bibString)) bibSkip :: BibParser () -bibSkip = skipMany1 (satisfy (/='@')) +bibSkip = skipMany1 (satisfy (\c -> c /='@' && c /='%')) +comment :: BibParser () +comment = char '%' *> void anyLine + bibComment :: BibParser () bibComment = do cistring "comment" - spaces + spaces' void inBraces <|> bibSkip <|> return () bibPreamble :: BibParser () bibPreamble = do cistring "preamble" - spaces + spaces' void inBraces bibString :: BibParser () bibString = do cistring "string" - spaces + spaces' char '{' - spaces + spaces' (k,v) <- entField char '}' updateState (\(l,m) -> (l, Map.insert k v m)) @@ -841,9 +845,9 @@ char '{' res <- manyTill ( take1WhileP (\c -> c /= '{' && c /= '}' && c /= '\\') - <|> (char '\\' >> ( (char '{' >> return "\\{") - <|> (char '}' >> return "\\}") - <|> return "\\")) + <|> (char '\\' >> (do c <- oneOf "{}" + return $ T.pack ['\\',c]) + <|> return "\\") <|> (braced <$> inBraces) ) (char '}') return $ T.concat res @@ -855,8 +859,9 @@ inQuotes = do char '"' T.concat <$> manyTill - ( take1WhileP (\c -> c /= '{' && c /= '"' && c /= '\\') + ( take1WhileP (\c -> c /= '{' && c /= '"' && c /= '\\' && c /= '%') <|> (char '\\' >> T.cons '\\' . T.singleton <$> anyChar) + <|> ("" <$ (char '%' >> anyLine)) <|> braced <$> inBraces ) (char '"') @@ -869,32 +874,35 @@ isBibtexKeyChar c = isAlphaNum c || c `elem` (".:;?!`'()$/*@_+=-[]*&" :: [Char]) +spaces' :: BibParser () +spaces' = skipMany (void (satisfy isSpace) <|> comment) + bibItem :: BibParser Item bibItem = do char '@' pos <- getPosition enttype <- T.toLower <$> take1WhileP isLetter - spaces + spaces' char '{' - spaces + spaces' entid <- take1WhileP isBibtexKeyChar - spaces + spaces' char ',' - spaces - entfields <- entField `sepEndBy` (char ',' >> spaces) - spaces + spaces' + entfields <- entField `sepEndBy` (char ',' >> spaces') + spaces' char '}' return $ Item entid pos enttype (Map.fromList entfields) entField :: BibParser (Text, Text) entField = do k <- fieldName - spaces + spaces' char '=' - spaces + spaces' vs <- (expandString <|> inQuotes <|> inBraces <|> rawWord) `sepBy` - try (spaces >> char '#' >> spaces) - spaces + try (spaces' >> char '#' >> spaces') + spaces' return (k, T.concat vs) resolveAlias :: Text -> Text diff --git a/src/Text/Pandoc/Error.hs b/src/Text/Pandoc/Error.hs --- a/src/Text/Pandoc/Error.hs +++ b/src/Text/Pandoc/Error.hs @@ -171,34 +171,34 @@ exitCode = case e of PandocIOError{} -> 1 + PandocFailOnWarningError{} -> 3 + PandocAppError{} -> 4 + PandocTemplateError{} -> 5 + PandocOptionError{} -> 6 + PandocUnknownReaderError{} -> 21 + PandocUnknownWriterError{} -> 22 + PandocUnsupportedExtensionError{} -> 23 + PandocCiteprocError{} -> 24 + PandocBibliographyError{} -> 25 + PandocEpubSubdirectoryError{} -> 31 + PandocPDFError{} -> 43 + PandocXMLError{} -> 44 + PandocPDFProgramNotFoundError{} -> 47 PandocHttpError{} -> 61 PandocShouldNeverHappenError{} -> 62 PandocSomeError{} -> 63 PandocParseError{} -> 64 PandocParsecError{} -> 65 PandocMakePDFError{} -> 66 - PandocOptionError{} -> 6 PandocSyntaxMapError{} -> 67 - PandocFailOnWarningError{} -> 3 - PandocPDFProgramNotFoundError{} -> 47 - PandocPDFError{} -> 43 - PandocXMLError{} -> 44 PandocFilterError{} -> 83 PandocLuaError{} -> 84 - PandocCouldNotFindDataFileError{} -> 97 - PandocResourceNotFound{} -> 99 - PandocTemplateError{} -> 5 - PandocAppError{} -> 4 - PandocEpubSubdirectoryError{} -> 31 PandocMacroLoop{} -> 91 PandocUTF8DecodingError{} -> 92 PandocIpynbDecodingError{} -> 93 PandocUnsupportedCharsetError{} -> 94 - PandocUnknownReaderError{} -> 21 - PandocUnknownWriterError{} -> 22 - PandocUnsupportedExtensionError{} -> 23 - PandocCiteprocError{} -> 24 - PandocBibliographyError{} -> 25 + PandocCouldNotFindDataFileError{} -> 97 + PandocResourceNotFound{} -> 99 err :: Int -> Text -> IO a err exitCode msg = do diff --git a/src/Text/Pandoc/Lua/Global.hs b/src/Text/Pandoc/Lua/Global.hs --- a/src/Text/Pandoc/Lua/Global.hs +++ b/src/Text/Pandoc/Lua/Global.hs @@ -22,7 +22,7 @@ import Text.Pandoc.Error (PandocError) import Text.Pandoc.Lua.Marshaling () import Text.Pandoc.Lua.Marshaling.CommonState (pushCommonState) -import Text.Pandoc.Lua.Marshaling.ReaderOptions (pushReaderOptions) +import Text.Pandoc.Lua.Marshaling.ReaderOptions (pushReaderOptionsReadonly) import Text.Pandoc.Options (ReaderOptions) import qualified Data.Text as Text @@ -55,7 +55,7 @@ pushUD typePandocLazy doc Lua.setglobal "PANDOC_DOCUMENT" PANDOC_READER_OPTIONS ropts -> do - pushReaderOptions ropts + pushReaderOptionsReadonly ropts Lua.setglobal "PANDOC_READER_OPTIONS" PANDOC_SCRIPT_FILE filePath -> do Lua.push filePath diff --git a/src/Text/Pandoc/Lua/Init.hs b/src/Text/Pandoc/Lua/Init.hs --- a/src/Text/Pandoc/Lua/Init.hs +++ b/src/Text/Pandoc/Lua/Init.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE LambdaCase #-} {-# LANGUAGE OverloadedStrings #-} {- | Module : Text.Pandoc.Lua @@ -13,10 +14,11 @@ ( runLua ) where -import Control.Monad (when) +import Control.Monad (forM, forM_, when) import Control.Monad.Catch (throwM, try) import Control.Monad.Trans (MonadIO (..)) import Data.Data (Data, dataTypeConstrs, dataTypeOf, showConstr) +import Data.Maybe (catMaybes) import HsLua as Lua hiding (status, try) import GHC.IO.Encoding (getForeignEncoding, setForeignEncoding, utf8) import Text.Pandoc.Class.PandocMonad (PandocMonad, readDataFile) @@ -24,6 +26,7 @@ import Text.Pandoc.Lua.Packages (installPandocPackageSearcher) import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua, runPandocLua) import qualified Data.Text as T +import qualified Lua.LPeg as LPeg import qualified Text.Pandoc.Definition as Pandoc import qualified Text.Pandoc.Lua.Module.Pandoc as ModulePandoc @@ -45,6 +48,8 @@ liftPandocLua Lua.openlibs installPandocPackageSearcher initPandocModule + installLpegSearcher + setGlobalModules loadInitScript "init.lua" where initPandocModule :: PandocLua () @@ -53,8 +58,8 @@ ModulePandoc.pushModule -- register as loaded module liftPandocLua $ do - Lua.pushvalue Lua.top Lua.getfield Lua.registryindex Lua.loaded + Lua.pushvalue (Lua.nth 2) Lua.setfield (Lua.nth 2) "pandoc" Lua.pop 1 -- copy constructors into registry @@ -72,6 +77,51 @@ throwM . PandocLuaError . (prefix <>) $ case err of PandocLuaError msg -> msg _ -> T.pack $ show err + + setGlobalModules :: PandocLua () + setGlobalModules = liftPandocLua $ do + let globalModules = + [ ("lpeg", LPeg.luaopen_lpeg_ptr) -- must be loaded first + , ("re", LPeg.luaopen_re_ptr) -- re depends on lpeg + ] + loadedBuiltInModules <- fmap catMaybes . forM globalModules $ + \(pkgname, luaopen) -> do + Lua.pushcfunction luaopen + usedBuiltIn <- Lua.pcall 0 1 Nothing >>= \case + OK -> do -- all good, loading succeeded + -- register as loaded module so later modules can rely on this + Lua.getfield Lua.registryindex Lua.loaded + Lua.pushvalue (Lua.nth 2) + Lua.setfield (Lua.nth 2) pkgname + Lua.pop 1 -- pop _LOADED + return True + _ -> do -- built-in library failed, load system lib + Lua.pop 1 -- ignore error message + -- Try loading via the normal package loading mechanism. + Lua.getglobal "require" + Lua.pushName pkgname + Lua.call 1 1 -- Throws an exception if loading failed again! + return False + + -- Module on top of stack. Register as global + Lua.setglobal pkgname + return $ if usedBuiltIn then Just pkgname else Nothing + + -- Remove module entry from _LOADED table in registry if we used a + -- built-in library. This ensures that later calls to @require@ will + -- prefer the shared library, if any. + forM_ loadedBuiltInModules $ \pkgname -> do + Lua.getfield Lua.registryindex Lua.loaded + Lua.pushnil + Lua.setfield (Lua.nth 2) pkgname + Lua.pop 1 -- registry + + installLpegSearcher :: PandocLua () + installLpegSearcher = liftPandocLua $ do + Lua.getglobal' "package.searchers" + Lua.pushHaskellFunction $ Lua.state >>= liftIO . LPeg.lpeg_searcher + Lua.rawseti (Lua.nth 2) . (+1) . fromIntegral =<< Lua.rawlen (Lua.nth 2) + Lua.pop 1 -- remove 'package.searchers' from stack -- | AST elements are marshaled via normal constructor functions in the -- @pandoc@ module. However, accessing Lua globals from Haskell is diff --git a/src/Text/Pandoc/Lua/Marshaling/AST.hs b/src/Text/Pandoc/Lua/Marshaling/AST.hs --- a/src/Text/Pandoc/Lua/Marshaling/AST.hs +++ b/src/Text/Pandoc/Lua/Marshaling/AST.hs @@ -85,6 +85,10 @@ <#> parameter (optional . peekPandoc) "doc1" "pandoc" "" <#> parameter (optional . peekPandoc) "doc2" "pandoc" "" =#> functionResult pushBool "boolean" "true iff the two values are equal" + , operation Tostring $ lambda + ### liftPure show + <#> parameter peekPandoc "Pandoc" "doc" "" + =#> functionResult pushString "string" "native Haskell representation" ] [ property "blocks" "list of blocks" (pushPandocList pushBlock, \(Pandoc _ blks) -> blks) diff --git a/src/Text/Pandoc/Lua/Marshaling/ReaderOptions.hs b/src/Text/Pandoc/Lua/Marshaling/ReaderOptions.hs --- a/src/Text/Pandoc/Lua/Marshaling/ReaderOptions.hs +++ b/src/Text/Pandoc/Lua/Marshaling/ReaderOptions.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE LambdaCase #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE ScopedTypeVariables #-} {-# OPTIONS_GHC -fno-warn-orphans #-} @@ -15,8 +16,10 @@ module Text.Pandoc.Lua.Marshaling.ReaderOptions ( peekReaderOptions , pushReaderOptions + , pushReaderOptionsReadonly ) where +import Data.Default (def) import HsLua as Lua import Text.Pandoc.Lua.Marshaling.List (pushPandocList) import Text.Pandoc.Options (ReaderOptions (..)) @@ -25,47 +28,106 @@ -- Reader Options -- +-- | Retrieve a ReaderOptions value, either from a normal ReaderOptions +-- value, from a read-only object, or from a table with the same +-- keys as a ReaderOptions object. peekReaderOptions :: LuaError e => Peeker e ReaderOptions -peekReaderOptions = peekUD typeReaderOptions +peekReaderOptions = retrieving "ReaderOptions" . \idx -> + liftLua (ltype idx) >>= \case + TypeUserdata -> choice [ peekUD typeReaderOptions + , peekUD typeReaderOptionsReadonly + ] + idx + TypeTable -> peekReaderOptionsTable idx + _ -> failPeek =<< + typeMismatchMessage "ReaderOptions userdata or table" idx +-- | Pushes a ReaderOptions value as userdata object. pushReaderOptions :: LuaError e => Pusher e ReaderOptions pushReaderOptions = pushUD typeReaderOptions +-- | Pushes a ReaderOptions object, but makes it read-only. +pushReaderOptionsReadonly :: LuaError e => Pusher e ReaderOptions +pushReaderOptionsReadonly = pushUD typeReaderOptionsReadonly + +-- | ReaderOptions object type for read-only values. +typeReaderOptionsReadonly :: LuaError e => DocumentedType e ReaderOptions +typeReaderOptionsReadonly = deftype "ReaderOptions (read-only)" + [ operation Tostring $ lambda + ### liftPure show + <#> udparam typeReaderOptions "opts" "options to print in native format" + =#> functionResult pushString "string" "Haskell representation" + , operation Newindex $ lambda + ### (failLua "This ReaderOptions value is read-only.") + =?> "Throws an error when called, i.e., an assignment is made." + ] + readerOptionsMembers + +-- | 'ReaderOptions' object type. typeReaderOptions :: LuaError e => DocumentedType e ReaderOptions -typeReaderOptions = deftype "pandoc ReaderOptions" - [ operation Tostring luaShow +typeReaderOptions = deftype "ReaderOptions" + [ operation Tostring $ lambda + ### liftPure show + <#> udparam typeReaderOptions "opts" "options to print in native format" + =#> functionResult pushString "string" "Haskell representation" ] - [ readonly "extensions" "" - ( pushString . show - , readerExtensions) - , readonly "standalone" "" - ( pushBool - , readerStandalone) - , readonly "columns" "" - ( pushIntegral - , readerColumns) - , readonly "tab_stop" "" - ( pushIntegral - , readerTabStop) - , readonly "indented_code_classes" "" - ( pushPandocList pushText - , readerIndentedCodeClasses) - , readonly "abbreviations" "" - ( pushSet pushText - , readerAbbreviations) - , readonly "track_changes" "" - ( pushString . show - , readerTrackChanges) - , readonly "strip_comments" "" - ( pushBool - , readerStripComments) - , readonly "default_image_extension" "" - ( pushText - , readerDefaultImageExtension) + readerOptionsMembers + +-- | Member properties of 'ReaderOptions' Lua values. +readerOptionsMembers :: LuaError e + => [Member e (DocumentedFunction e) ReaderOptions] +readerOptionsMembers = + [ property "abbreviations" "" + (pushSet pushText, readerAbbreviations) + (peekSet peekText, \opts x -> opts{ readerAbbreviations = x }) + , property "columns" "" + (pushIntegral, readerColumns) + (peekIntegral, \opts x -> opts{ readerColumns = x }) + , property "default_image_extension" "" + (pushText, readerDefaultImageExtension) + (peekText, \opts x -> opts{ readerDefaultImageExtension = x }) + , property "extensions" "" + (pushString . show, readerExtensions) + (peekRead, \opts x -> opts{ readerExtensions = x }) + , property "indented_code_classes" "" + (pushPandocList pushText, readerIndentedCodeClasses) + (peekList peekText, \opts x -> opts{ readerIndentedCodeClasses = x }) + , property "strip_comments" "" + (pushBool, readerStripComments) + (peekBool, \opts x -> opts{ readerStripComments = x }) + , property "standalone" "" + (pushBool, readerStandalone) + (peekBool, \opts x -> opts{ readerStandalone = x }) + , property "tab_stop" "" + (pushIntegral, readerTabStop) + (peekIntegral, \opts x -> opts{ readerTabStop = x }) + , property "track_changes" "" + (pushString . show, readerTrackChanges) + (peekRead, \opts x -> opts{ readerTrackChanges = x }) ] -luaShow :: LuaError e => DocumentedFunction e -luaShow = defun "__tostring" - ### liftPure show - <#> udparam typeReaderOptions "state" "object to print in native format" - =#> functionResult pushString "string" "Haskell representation" +-- | Retrieves a 'ReaderOptions' object from a table on the stack, using +-- the default values for all missing fields. +-- +-- Internally, this pushes the default reader options, sets each +-- key/value pair of the table in the userdata value, then retrieves the +-- object again. This will update all fields and complain about unknown +-- keys. +peekReaderOptionsTable :: LuaError e => Peeker e ReaderOptions +peekReaderOptionsTable idx = retrieving "ReaderOptions (table)" $ do + liftLua $ do + absidx <- absindex idx + pushUD typeReaderOptions def + let setFields = do + next absidx >>= \case + False -> return () -- all fields were copied + True -> do + pushvalue (nth 2) *> insert (nth 2) + settable (nth 4) -- set in userdata object + setFields + pushnil -- first key + setFields + peekUD typeReaderOptions top + +instance Pushable ReaderOptions where + push = pushReaderOptions diff --git a/src/Text/Pandoc/Lua/Module/Pandoc.hs b/src/Text/Pandoc/Lua/Module/Pandoc.hs --- a/src/Text/Pandoc/Lua/Module/Pandoc.hs +++ b/src/Text/Pandoc/Lua/Module/Pandoc.hs @@ -42,6 +42,8 @@ import Text.Pandoc.Lua.Marshaling.List (List (..)) import Text.Pandoc.Lua.Marshaling.ListAttributes ( mkListAttributes , peekListAttributes) +import Text.Pandoc.Lua.Marshaling.ReaderOptions ( peekReaderOptions + , pushReaderOptions) import Text.Pandoc.Lua.Marshaling.SimpleTable (mkSimpleTable) import Text.Pandoc.Lua.Module.Utils (sha1) import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua, @@ -133,9 +135,9 @@ inlineConstructors :: LuaError e => [DocumentedFunction e] inlineConstructors = [ defun "Cite" - ### liftPure2 Cite - <#> parameter (peekList peekCitation) "citations" "list of Citations" "" + ### liftPure2 (flip Cite) <#> parameter peekInlinesFuzzy "content" "Inline" "placeholder content" + <#> parameter (peekList peekCitation) "citations" "list of Citations" "" =#> functionResult pushInline "Inline" "cite element" , defun "Code" ### liftPure2 (\text mattr -> Code (fromMaybe nullAttr mattr) text) @@ -355,6 +357,12 @@ , mkAttributeList , mkListAttributes , mkSimpleTable + + , defun "ReaderOptions" + ### liftPure id + <#> parameter peekReaderOptions "ReaderOptions|table" "opts" "reader options" + =#> functionResult pushReaderOptions "ReaderOptions" "new object" + #? "Creates a new ReaderOptions value." ] stringConstants :: [Field e] @@ -405,10 +413,12 @@ =?> "output string, or error triple" , defun "read" - ### (\content mformatspec -> do + ### (\content mformatspec mreaderOptions -> do let formatSpec = fromMaybe "markdown" mformatspec + readerOptions = fromMaybe def mreaderOptions res <- Lua.liftIO . runIO $ getReader formatSpec >>= \case - (TextReader r, es) -> r def{ readerExtensions = es } content + (TextReader r, es) -> r readerOptions{ readerExtensions = es } + content _ -> throwError $ PandocSomeError "Only textual formats are supported" case res of @@ -422,6 +432,8 @@ throwM e) <#> parameter peekText "string" "content" "text to parse" <#> optionalParameter peekText "string" "formatspec" "format and extensions" + <#> optionalParameter peekReaderOptions "ReaderOptions" "reader_options" + "reader options" =#> functionResult pushPandoc "Pandoc" "result document" , sha1 diff --git a/src/Text/Pandoc/Lua/Packages.hs b/src/Text/Pandoc/Lua/Packages.hs --- a/src/Text/Pandoc/Lua/Packages.hs +++ b/src/Text/Pandoc/Lua/Packages.hs @@ -64,5 +64,5 @@ Lua.pushHaskellFunction f return 1 reportPandocSearcherFailure = liftPandocLua $ do - Lua.push ("\n\t" <> pkgName <> "is not one of pandoc's default packages") + Lua.push ("\n\t" <> pkgName <> " is not one of pandoc's default packages") return (Lua.NumResults 1) diff --git a/src/Text/Pandoc/Readers/Custom.hs b/src/Text/Pandoc/Readers/Custom.hs new file mode 100644 --- /dev/null +++ b/src/Text/Pandoc/Readers/Custom.hs @@ -0,0 +1,54 @@ +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE FlexibleInstances #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE ScopedTypeVariables #-} +{-# LANGUAGE TypeApplications #-} +{- | + Module : Text.Pandoc.Readers.Custom + Copyright : Copyright (C) 2021 John MacFarlane + License : GNU GPL, version 2 or above + + Maintainer : John MacFarlane + Stability : alpha + Portability : portable + +Supports custom parsers written in Lua which produce a Pandoc AST. +-} +module Text.Pandoc.Readers.Custom ( readCustom ) where +import Control.Exception +import Control.Monad (when) +import Data.Text (Text) +import HsLua as Lua hiding (Operation (Div), render) +import HsLua.Class.Peekable (PeekError) +import Control.Monad.IO.Class (MonadIO) +import Text.Pandoc.Definition +import Text.Pandoc.Lua (Global (..), runLua, setGlobals) +import Text.Pandoc.Lua.Util (dofileWithTraceback) +import Text.Pandoc.Options +import Text.Pandoc.Class (PandocMonad) +import Text.Pandoc.Sources (ToSources(..), sourcesToText) + +-- | Convert custom markup to Pandoc. +readCustom :: (PandocMonad m, MonadIO m, ToSources s) + => FilePath -> ReaderOptions -> s -> m Pandoc +readCustom luaFile opts sources = do + let input = sourcesToText $ toSources sources + let globals = [ PANDOC_SCRIPT_FILE luaFile ] + res <- runLua $ do + setGlobals globals + stat <- dofileWithTraceback luaFile + -- check for error in lua script (later we'll change the return type + -- to handle this more gracefully): + when (stat /= Lua.OK) + Lua.throwErrorAsException + parseCustom input opts + case res of + Left msg -> throw msg + Right doc -> return doc + +parseCustom :: forall e. PeekError e + => Text + -> ReaderOptions + -> LuaE e Pandoc +parseCustom = invoke @e "Reader" + diff --git a/src/Text/Pandoc/Readers/DocBook.hs b/src/Text/Pandoc/Readers/DocBook.hs --- a/src/Text/Pandoc/Readers/DocBook.hs +++ b/src/Text/Pandoc/Readers/DocBook.hs @@ -19,7 +19,7 @@ import Data.Generics import Data.List (intersperse,elemIndex) import Data.List.NonEmpty (nonEmpty) -import Data.Maybe (fromMaybe,mapMaybe) +import Data.Maybe (catMaybes,fromMaybe,mapMaybe,maybeToList) import Data.Text (Text) import qualified Data.Text as T import qualified Data.Text.Lazy as TL @@ -316,7 +316,7 @@ [ ] postcode - A postal code in an address [x] preface - Introductory matter preceding the first chapter of a book [ ] prefaceinfo - Meta-information for a Preface -[ ] primary - The primary word or phrase under which an index term should be +[x] primary - The primary word or phrase under which an index term should be sorted [ ] primaryie - A primary term in an index entry, not in the text [ ] printhistory - The printing history of a document @@ -385,7 +385,7 @@ [o] screeninfo - Information about how a screen shot was produced [ ] screenshot - A representation of what the user sees or might see on a computer screen -[ ] secondary - A secondary word or phrase in an index term +[x] secondary - A secondary word or phrase in an index term [ ] secondaryie - A secondary term in an index entry, rather than in the text [x] sect1 - A top-level section of document [x] sect1info - Meta-information for a Sect1 @@ -461,7 +461,7 @@ [x] td - A table entry in an HTML table [x] term - The word or phrase being defined or described in a variable list [ ] termdef - An inline term definition -[ ] tertiary - A tertiary word or phrase in an index term +[x] tertiary - A tertiary word or phrase in an index term [ ] tertiaryie - A tertiary term in an index entry, rather than in the text [ ] textdata - Pointer to external text data [ ] textobject - A wrapper for a text description of an object and its @@ -829,7 +829,7 @@ "section" -> gets dbSectionLevel >>= sect . (+1) "simplesect" -> gets dbSectionLevel >>= - sectWith (attrValue "id" e,["unnumbered"],[]) . (+1) + sectWith(attrValue "id" e) ["unnumbered"] [] . (+1) "refsect1" -> sect 1 "refsect2" -> sect 2 "refsect3" -> sect 3 @@ -994,8 +994,8 @@ (TableHead nullAttr $ toHeaderRow headrows) [TableBody nullAttr 0 [] $ map toRow bodyrows] (TableFoot nullAttr []) - sect n = sectWith (attrValue "id" e,[],[]) n - sectWith attr n = do + sect n = sectWith(attrValue "id" e) [] [] n + sectWith elId classes attrs n = do isbook <- gets dbBook let n' = if isbook || n == 0 then n + 1 else n headerText <- case filterChild (named "title") e `mplus` @@ -1006,7 +1006,14 @@ modify $ \st -> st{ dbSectionLevel = n } b <- getBlocks e modify $ \st -> st{ dbSectionLevel = n - 1 } - return $ headerWith attr n' headerText <> b + return $ headerWith (elId, classes, maybeToList titleabbrevElAsAttr++attrs) n' headerText <> b + titleabbrevElAsAttr = do + txt <- case filterChild (named "titleabbrev") e `mplus` + (filterChild (named "info") e >>= + filterChild (named "titleabbrev")) of + Just t -> Just ("titleabbrev", strContentRecursive t) + Nothing -> Nothing + return txt lineItems = mapM getInlines $ filterChildren (named "line") e -- | Admonitions are parsed into a div. Following other Docbook tools that output HTML, -- we parse the optional title as a div with the @title@ class, and give the @@ -1080,6 +1087,17 @@ elementToStr (Elem e') = Text $ CData CDataText (strContentRecursive e') Nothing elementToStr x = x +childElTextAsAttr :: Text -> Element -> Maybe (Text, Text) +childElTextAsAttr n e = case findChild q e of + Nothing -> Nothing + Just childEl -> Just (n, strContentRecursive childEl) + where q = QName n (Just "http://docbook.org/ns/docbook") Nothing + +attrValueAsOptionalAttr :: Text -> Element -> Maybe (Text, Text) +attrValueAsOptionalAttr n e = case attrValue n e of + "" -> Nothing + _ -> Just (n, attrValue n e) + parseInline :: PandocMonad m => Content -> DB m Inlines parseInline (Text (CData _ s _)) = return $ text s parseInline (CRef ref) = @@ -1094,6 +1112,28 @@ if ident /= "" || classes /= [] then innerInlines (spanWith (ident,classes,[])) else innerInlines id + "indexterm" -> do + let ident = attrValue "id" e + let classes = T.words $ attrValue "role" e + let attrs = + -- In DocBook, , , , , and + -- have mixed content models. However, because we're representing these + -- elements in Pandoc's AST as attributes of a phrase, we flatten all + -- the descendant content of these elements. + [ childElTextAsAttr "primary" e + , childElTextAsAttr "secondary" e + , childElTextAsAttr "tertiary" e + , childElTextAsAttr "see" e + , childElTextAsAttr "seealso" e + , attrValueAsOptionalAttr "significance" e + , attrValueAsOptionalAttr "startref" e + , attrValueAsOptionalAttr "scope" e + , attrValueAsOptionalAttr "class" e + -- We don't do anything with the "pagenum" attribute, because these only + -- occur within literal sections, which is not supported by Pandoc, + -- because Pandoc has no concept of pages. + ] + return $ spanWith (ident, ("indexterm" : classes), (catMaybes attrs)) mempty "equation" -> equation e displayMath "informalequation" -> equation e displayMath "inlineequation" -> equation e math diff --git a/src/Text/Pandoc/Readers/JATS.hs b/src/Text/Pandoc/Readers/JATS.hs --- a/src/Text/Pandoc/Readers/JATS.hs +++ b/src/Text/Pandoc/Readers/JATS.hs @@ -35,6 +35,7 @@ import qualified Data.Set as S (fromList, member) import Data.Set ((\\)) import Text.Pandoc.Sources (ToSources(..), sourcesToText) +import qualified Data.Foldable as DF type JATS m = StateT JATSState m @@ -226,9 +227,19 @@ mapM getInlines (filterChildren (const True) t) Nothing -> return mempty - img <- getGraphic (Just (capt, attrValue "id" e)) g - return $ para img + + let figAttributes = DF.toList $ + ("alt", ) . strContent <$> + filterChild (named "alt-text") e + + return $ simpleFigureWith + (attrValue "id" e, [], figAttributes) + capt + (attrValue "href" g) + (attrValue "title" g) + _ -> divWith (attrValue "id" e, ["fig"], []) <$> getBlocks e + parseTable = do let isCaption x = named "title" x || named "caption" x capt <- case filterChild isCaption e of diff --git a/src/Text/Pandoc/Readers/LaTeX.hs b/src/Text/Pandoc/Readers/LaTeX.hs --- a/src/Text/Pandoc/Readers/LaTeX.hs +++ b/src/Text/Pandoc/Readers/LaTeX.hs @@ -390,8 +390,8 @@ unescapeURL . removeDoubleQuotes $ untokenize src) -- hyperref - , ("url", (\url -> link url "" (str url)) . unescapeURL . untokenize <$> - bracedUrl) + , ("url", (\url -> linkWith ("",["uri"],[]) url "" (str url)) + . unescapeURL . untokenize <$> bracedUrl) , ("nolinkurl", code . unescapeURL . untokenize <$> bracedUrl) , ("href", do url <- bracedUrl sp diff --git a/src/Text/Pandoc/Readers/LaTeX/Inline.hs b/src/Text/Pandoc/Readers/LaTeX/Inline.hs --- a/src/Text/Pandoc/Readers/LaTeX/Inline.hs +++ b/src/Text/Pandoc/Readers/LaTeX/Inline.hs @@ -339,6 +339,7 @@ , ("cref", rawInlineOr "cref" $ doref "ref") -- from cleveref.sty , ("vref", rawInlineOr "vref" $ doref "ref+page") -- from varioref.sty , ("eqref", rawInlineOr "eqref" $ doref "eqref") -- from amsmath.sty + , ("autoref", rawInlineOr "autoref" $ doref "autoref") -- from hyperref.sty ] acronymCommands :: PandocMonad m => M.Map Text (LP m Inlines) diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs --- a/src/Text/Pandoc/Readers/Markdown.hs +++ b/src/Text/Pandoc/Readers/Markdown.hs @@ -186,26 +186,18 @@ -- including inlines between balanced pairs of square brackets. inlinesInBalancedBrackets :: PandocMonad m => MarkdownParser m (F Inlines) inlinesInBalancedBrackets = - try $ char '[' >> withRaw (go 1) >>= - parseFromString inlines . stripBracket . snd - where stripBracket t = case T.unsnoc t of - Just (t', ']') -> t' - _ -> t - go :: PandocMonad m => Int -> MarkdownParser m () - go 0 = return () - go openBrackets = - (() <$ (escapedChar <|> - code <|> - math <|> - rawHtmlInline <|> - rawLaTeXInline') >> go openBrackets) - <|> - (do char ']' - Control.Monad.when (openBrackets > 1) $ go (openBrackets - 1)) - <|> - (char '[' >> go (openBrackets + 1)) - <|> - (anyChar >> go openBrackets) + mconcat <$> try (char '[' >> go (1 :: Int)) + where + go n = + (:) <$> (note <|> cite <|> bracketedSpan <|> link) <*> go n + <|> + (char '[' *> ((:) <$> pure (pure (B.str "[")) <*> go (n + 1))) + <|> + (char ']' *> (if n > 1 + then (:) <$> pure (pure (B.str "]")) <*> go (n - 1) + else pure [])) + <|> + (:) <$> inline <*> go n -- -- document structure diff --git a/src/Text/Pandoc/Readers/Metadata.hs b/src/Text/Pandoc/Readers/Metadata.hs --- a/src/Text/Pandoc/Readers/Metadata.hs +++ b/src/Text/Pandoc/Readers/Metadata.hs @@ -42,6 +42,7 @@ yamlBsToMeta pMetaValue bstr = do case Yaml.decodeAllEither' bstr of Right (Object o:_) -> fmap Meta <$> yamlMap pMetaValue o + Right [] -> return . return $ mempty Right [Null] -> return . return $ mempty Right _ -> Prelude.fail "expected YAML object" Left err' -> do @@ -55,8 +56,8 @@ -> B.ByteString -> ParserT Sources st m (Future st [MetaValue]) yamlBsToRefs pMetaValue idpred bstr = - case Yaml.decodeEither' bstr of - Right (Object m) -> do + case Yaml.decodeAllEither' bstr of + Right (Object m : _) -> do let isSelected (String t) = idpred t isSelected _ = False let hasSelectedId (Object o) = diff --git a/src/Text/Pandoc/Readers/RST.hs b/src/Text/Pandoc/Readers/RST.hs --- a/src/Text/Pandoc/Readers/RST.hs +++ b/src/Text/Pandoc/Readers/RST.hs @@ -919,14 +919,22 @@ (baseRole, baseFmt, baseAttr) = getBaseRole (parentRole, Nothing, nullAttr) customRoles fmt = if parentRole == "raw" then lookup "format" fields else baseFmt - annotate :: [Text] -> [Text] - annotate = maybe id (:) $ - if baseRole == "code" - then lookup "language" fields - else Nothing - attr = let (ident, classes, keyValues) = baseAttr - -- nub in case role name & language class are the same - in (ident, nub . (role :) . annotate $ classes, keyValues) + + updateClasses :: [Text] -> [Text] + updateClasses oldClasses = let + + codeLanguageClass = if baseRole == "code" + then maybeToList (lookup "language" fields) + else [] + + -- if no ":class:" field is given, the default is the role name + classFieldClasses = maybe [role] T.words (lookup "class" fields) + + -- nub in case role name & language class are the same + in nub (classFieldClasses ++ codeLanguageClass ++ oldClasses) + + attr = let (ident, baseClasses, keyValues) = baseAttr + in (ident, updateClasses baseClasses, keyValues) -- warn about syntax we ignore forM_ fields $ \(key, _) -> case key of diff --git a/src/Text/Pandoc/Writers/HTML.hs b/src/Text/Pandoc/Writers/HTML.hs --- a/src/Text/Pandoc/Writers/HTML.hs +++ b/src/Text/Pandoc/Writers/HTML.hs @@ -633,6 +633,7 @@ return (keys, attrs) else return (Set.insert k keys, addAttr html5 mbEpubVersion k v attrs) addAttr html5 mbEpubVersion x y + | T.null x = id -- see #7546 | html5 = if x `Set.member` (html5Attributes <> rdfaAttributes) || T.any (== ':') x -- e.g. epub: namespace diff --git a/src/Text/Pandoc/Writers/JATS.hs b/src/Text/Pandoc/Writers/JATS.hs --- a/src/Text/Pandoc/Writers/JATS.hs +++ b/src/Text/Pandoc/Writers/JATS.hs @@ -551,6 +551,7 @@ return $ selfClosingTag "inline-graphic" attr isParaOrList :: Block -> Bool +isParaOrList SimpleFigure{} = False -- implicit figures are not paragraphs isParaOrList Para{} = True isParaOrList Plain{} = True isParaOrList BulletList{} = True diff --git a/src/Text/Pandoc/Writers/JATS/References.hs b/src/Text/Pandoc/Writers/JATS/References.hs --- a/src/Text/Pandoc/Writers/JATS/References.hs +++ b/src/Text/Pandoc/Writers/JATS/References.hs @@ -70,6 +70,7 @@ , "pages" `varInTag` "page-range" , "ISBN" `varInTag` "isbn" , "ISSN" `varInTag` "issn" + , "URL" `varInTag` "uri" , varInTagWith "doi" "pub-id" [("pub-id-type", "doi")] , varInTagWith "pmid" "pub-id" [("pub-id-type", "pmid")] ] diff --git a/src/Text/Pandoc/Writers/LaTeX/Lang.hs b/src/Text/Pandoc/Writers/LaTeX/Lang.hs --- a/src/Text/Pandoc/Writers/LaTeX/Lang.hs +++ b/src/Text/Pandoc/Writers/LaTeX/Lang.hs @@ -43,7 +43,7 @@ toBabel (Lang "fr" _ (Just "CA") _ _ _) = "canadien" toBabel (Lang "fra" _ _ vars _ _) | "aca" `elem` vars = "acadian" -toBabel (Lang "grc" _ _ _ _ _) = "polutonikogreek" +toBabel (Lang "grc" _ _ _ _ _) = "ancientgreek" toBabel (Lang "hsb" _ _ _ _ _) = "uppersorbian" toBabel (Lang "la" _ _ vars _ _) | "x-classic" `elem` vars = "classiclatin" diff --git a/src/Text/Pandoc/Writers/LaTeX/Util.hs b/src/Text/Pandoc/Writers/LaTeX/Util.hs --- a/src/Text/Pandoc/Writers/LaTeX/Util.hs +++ b/src/Text/Pandoc/Writers/LaTeX/Util.hs @@ -124,7 +124,7 @@ '\160' -> emits "~" '\x200B' -> emits "\\hspace{0pt}" -- zero-width space '\x202F' -> emits "\\," - '\x2026' -> emitcseq "\\ldots" + '\x2026' | ligatures -> emitcseq "\\ldots" '\x2018' | ligatures -> emitquote "`" '\x2019' | ligatures -> emitquote "'" '\x201C' | ligatures -> emitquote "``" diff --git a/src/Text/Pandoc/Writers/Markdown.hs b/src/Text/Pandoc/Writers/Markdown.hs --- a/src/Text/Pandoc/Writers/Markdown.hs +++ b/src/Text/Pandoc/Writers/Markdown.hs @@ -399,7 +399,8 @@ (literal "```" <> literal "\n") let renderEmpty = mempty <$ report (BlockNotRendered b) case variant of - PlainText -> renderEmpty + PlainText + | f == "plain" -> return $ literal str <> literal "\n" Commonmark | f `elem` ["gfm", "commonmark", "commonmark_x", "markdown"] -> return $ literal str <> literal "\n" diff --git a/src/Text/Pandoc/Writers/Markdown/Inline.hs b/src/Text/Pandoc/Writers/Markdown/Inline.hs --- a/src/Text/Pandoc/Writers/Markdown/Inline.hs +++ b/src/Text/Pandoc/Writers/Markdown/Inline.hs @@ -459,7 +459,8 @@ literal (T.replicate numticks "`") <> literal "{=" <> literal fmt <> literal "}" let renderEmpty = mempty <$ report (InlineNotRendered il) case variant of - PlainText -> renderEmpty + PlainText + | f == "plain" -> return $ literal str Commonmark | f `elem` ["gfm", "commonmark", "commonmark_x", "markdown"] -> return $ literal str @@ -530,7 +531,7 @@ return $ pdoc <+> r modekey SuppressAuthor = "-" modekey _ = "" -inlineToMarkdown opts lnk@(Link attr txt (src, tit)) = do +inlineToMarkdown opts lnk@(Link attr@(ident,classes,kvs) txt (src, tit)) = do variant <- asks envVariant linktext <- inlineListToMarkdown opts txt let linktitle = if T.null tit @@ -538,6 +539,9 @@ else literal $ " \"" <> tit <> "\"" let srcSuffix = fromMaybe src (T.stripPrefix "mailto:" src) let useAuto = isURI src && + T.null ident && + null kvs && + (null classes || classes == ["uri"] || classes == ["email"]) && case txt of [Str s] | escapeURI s == srcSuffix -> True _ -> False diff --git a/src/Text/Pandoc/Writers/MediaWiki.hs b/src/Text/Pandoc/Writers/MediaWiki.hs --- a/src/Text/Pandoc/Writers/MediaWiki.hs +++ b/src/Text/Pandoc/Writers/MediaWiki.hs @@ -128,10 +128,15 @@ blockToMediaWiki HorizontalRule = return "\n-----\n" -blockToMediaWiki (Header level _ inlines) = do +blockToMediaWiki (Header level (ident,_,_) inlines) = do + let autoId = T.replace " " "_" $ stringify inlines contents <- inlineListToMediaWiki inlines let eqs = T.replicate level "=" - return $ eqs <> " " <> contents <> " " <> eqs <> "\n" + return $ + (if T.null ident || autoId == ident + then "" + else " ident <> "\">\n") + <> eqs <> " " <> contents <> " " <> eqs <> "\n" blockToMediaWiki (CodeBlock (_,classes,keyvals) str) = do let at = Set.fromList classes `Set.intersection` highlightingLangs diff --git a/src/Text/Pandoc/Writers/Shared.hs b/src/Text/Pandoc/Writers/Shared.hs --- a/src/Text/Pandoc/Writers/Shared.hs +++ b/src/Text/Pandoc/Writers/Shared.hs @@ -508,7 +508,7 @@ = let (h, w, cBody) = getComponents c cRowPieces = cBody : replicate (w - 1) mempty cPendingPieces = replicate w $ replicate (h - 1) mempty - pendingPieces' = dropWhile null pendingPieces + pendingPieces' = drop w pendingPieces (pendingPieces'', rowPieces) = placeCutCells pendingPieces' cells' in (cPendingPieces <> pendingPieces'', cRowPieces <> rowPieces) | otherwise = ([], []) diff --git a/stack.yaml b/stack.yaml --- a/stack.yaml +++ b/stack.yaml @@ -12,26 +12,27 @@ - doctemplates-0.10 - emojis-0.1.2 - doclayout-0.3.1.1 -- hslua-2.0.0 +- lpeg-1.0.1 +- hslua-2.0.1 - hslua-classes-2.0.0 -- hslua-core-2.0.0 -- hslua-marshalling-2.0.0 +- hslua-core-2.0.0.2 +- hslua-marshalling-2.0.1 - hslua-module-path-1.0.0 - hslua-module-system-1.0.0 - hslua-module-text-1.0.0 - hslua-module-version-1.0.0 -- hslua-objectorientation-2.0.0 +- hslua-objectorientation-2.0.1 - hslua-packaging-2.0.0 -- lua-2.0.0 +- lua-2.0.1 - tasty-hslua-1.0.0 - tasty-lua-1.0.0 - pandoc-types-1.22.1 -- texmath-0.12.3.2 - commonmark-0.2.1.1 - commonmark-extensions-0.2.2 - citeproc-0.6 - aeson-pretty-0.8.9 - ipynb-0.1.0.2 +- texmath-0.12.3.3 ghc-options: "$locals": -fhide-source-paths -Wno-missing-home-modules resolver: lts-18.10 diff --git a/test/Tests/Lua.hs b/test/Tests/Lua.hs --- a/test/Tests/Lua.hs +++ b/test/Tests/Lua.hs @@ -17,8 +17,8 @@ import Control.Monad (when) import HsLua as Lua hiding (Operation (Div), error) import System.FilePath (()) -import Test.Tasty (TestTree, localOption) -import Test.Tasty.HUnit (Assertion, HasCallStack, assertEqual, testCase) +import Test.Tasty (TestTree, testGroup, localOption) +import Test.Tasty.HUnit ((@=?), Assertion, HasCallStack, assertEqual, testCase) import Test.Tasty.QuickCheck (QuickCheckTests (..), ioProperty, testProperty) import Text.Pandoc.Arbitrary () import Text.Pandoc.Builder (bulletList, definitionList, displayMath, divWith, @@ -210,6 +210,26 @@ Lua.getglobal' "pandoc.system" ty <- Lua.ltype Lua.top Lua.liftIO $ assertEqual "module should be a table" Lua.TypeTable ty + + , testGroup "global modules" + [ testCase "module 'lpeg' is loaded into a global" . runLuaTest $ do + s <- Lua.dostring "assert(type(lpeg)=='table')" + Lua.liftIO $ Lua.OK @=? s + + , testCase "module 're' is loaded into a global" . runLuaTest $ do + s <- Lua.dostring "assert(type(re)=='table')" + Lua.liftIO $ Lua.OK @=? s + + , testCase "module 'lpeg' is available via `require`" . runLuaTest $ do + s <- Lua.dostring + "package.path = ''; package.cpath = ''; require 'lpeg'" + Lua.liftIO $ Lua.OK @=? s + + , testCase "module 're' is available via `require`" . runLuaTest $ do + s <- Lua.dostring + "package.path = ''; package.cpath = ''; require 're'" + Lua.liftIO $ Lua.OK @=? s + ] , testCase "informative error messages" . runLuaTest $ do Lua.pushboolean True diff --git a/test/Tests/Readers/RST.hs b/test/Tests/Readers/RST.hs --- a/test/Tests/Readers/RST.hs +++ b/test/Tests/Readers/RST.hs @@ -179,6 +179,15 @@ , "custom code role with language field" =: ".. role:: lhs(code)\n :language: haskell\n\n:lhs:`a`" =?> para (codeWith ("", ["lhs", "haskell"], []) "a") + , "custom role with class field" + =: ".. role:: classy\n :class: myclass\n\n:classy:`a`" + =?> para (spanWith ("", ["myclass"], []) "a") + , "custom role with class field containing multiple whitespace-separated classes" + =: ".. role:: classy\n :class: myclass1 myclass2\n myclass3\n\n:classy:`a`" + =?> para (spanWith ("", ["myclass1", "myclass2", "myclass3"], []) "a") + , "custom role with inherited class field" + =: ".. role:: classy\n :class: myclass1\n.. role:: classier(classy)\n :class: myclass2\n\n:classier:`a`" + =?> para (spanWith ("", ["myclass2", "myclass1"], []) "a") , "custom role with unspecified parent role" =: ".. role:: classy\n\n:classy:`text`" =?> para (spanWith ("", ["classy"], []) "text") diff --git a/test/Tests/Shared.hs b/test/Tests/Shared.hs --- a/test/Tests/Shared.hs +++ b/test/Tests/Shared.hs @@ -56,6 +56,7 @@ testLegacyTable = [ testCase "decomposes a table with head" $ gen1 @?= expect1 , testCase "decomposes a table without head" $ gen2 @?= expect2 + , testCase "decomposes the table from issue 7683" $ gen3 @?= expect3 ] where pln = toList . plain . str @@ -110,3 +111,18 @@ ,[pln "j", mempty, mempty]] ) gen2 = toLegacyTable emptyCaption spec1 (th []) [body1] (tf footRows1) + + spec3 = replicate 4 (AlignDefault, ColWidthDefault) + body3 = tb 0 + [] + [[cl "a" 2 1, cl "b" 1 2, cl "c" 2 1] + ,[cl "d" 1 1, cl "e" 1 1] + ] + expect3 = ( [] + , replicate 4 AlignDefault + , replicate 4 0 + , [] + , [[pln "a", pln "b", mempty, pln "c"] + ,[mempty, pln "d", pln "e", mempty]] + ) + gen3 = toLegacyTable emptyCaption spec3 (th []) [body3] (tf []) diff --git a/test/Tests/Writers/JATS.hs b/test/Tests/Writers/JATS.hs --- a/test/Tests/Writers/JATS.hs +++ b/test/Tests/Writers/JATS.hs @@ -61,21 +61,39 @@ , "

" ]) ] - , "bullet list" =: bulletList [ plain $ text "first" - , plain $ text "second" - , plain $ text "third" - ] - =?> "\n\ - \ \n\ - \

first

\n\ - \ \n\ - \ \n\ - \

second

\n\ - \ \n\ - \ \n\ - \

third

\n\ - \ \n\ - \" + , testGroup "bullet list" + [ "plain items" =: bulletList [ plain $ text "first" + , plain $ text "second" + , plain $ text "third" + ] + =?> "\n\ + \ \n\ + \

first

\n\ + \ \n\ + \ \n\ + \

second

\n\ + \ \n\ + \ \n\ + \

third

\n\ + \ \n\ + \" + + , "item with implicit figure" =: + bulletList [ simpleFigure (text "caption") "a.png" "" ] =?> + T.unlines + [ "" + , " " + , "

" + , " " + , "

caption

" + , " + " xlink:href=\"a.png\" xlink:title=\"\" />" + , " " + , "

" + , " " + , "" + ] + ] , testGroup "definition lists" [ "with internal link" =: definitionList [(link "#go" "" (str "testing"), [plain (text "hi there")])] =?> diff --git a/test/command/3681.md b/test/command/3681.md --- a/test/command/3681.md +++ b/test/command/3681.md @@ -45,7 +45,7 @@ , Note [ Para [ Link - ( "" , [] , [] ) + ( "" , [ "uri" ] , [] ) [ Str "https://en.wikipedia.org/wiki/CI/CD" ] ( "https://en.wikipedia.org/wiki/CI/CD" , "" ) ] diff --git a/test/command/4832.md b/test/command/4832.md --- a/test/command/4832.md +++ b/test/command/4832.md @@ -4,7 +4,7 @@ ^D [ Para [ Link - ( "" , [] , [] ) + ( "" , [ "uri" ] , [] ) [ Str "http://example.com/foo%20bar.htm" ] ( "http://example.com/foo%20bar.htm" , "" ) ] @@ -17,7 +17,7 @@ ^D [ Para [ Link - ( "" , [] , [] ) + ( "" , [ "uri" ] , [] ) [ Str "http://example.com/foo{bar}.htm" ] ( "http://example.com/foo{bar}.htm" , "" ) ] diff --git a/test/command/7546.md b/test/command/7546.md new file mode 100644 --- /dev/null +++ b/test/command/7546.md @@ -0,0 +1,6 @@ +``` +% pandoc -t html -f native +Span ("", [], [("","")]) [] +^D + +``` diff --git a/test/command/7668.md b/test/command/7668.md new file mode 100644 --- /dev/null +++ b/test/command/7668.md @@ -0,0 +1,36 @@ +``` +% pandoc -f bibtex -t csljson +%@Book{JW82, + author = {Richard A. Johnson and Dean W. Wichern}, + title = {Applied Multivariate Statistical Analysis}, + publisher= {Prentice-Hall}, + year = {1983} +} +@Book{JW83, + author = {Richard %A. + B. Johnson}, +% title = {Multivariate Analysis}, + year = "% + 1983" +} +^D +[ + { + "author": [ + { + "family": "Johnson", + "given": "Richard B." + } + ], + "id": "JW83", + "issued": { + "date-parts": [ + [ + 1983 + ] + ] + }, + "type": "book" + } +] +``` diff --git a/test/command/7692.md b/test/command/7692.md new file mode 100644 --- /dev/null +++ b/test/command/7692.md @@ -0,0 +1,13 @@ +``` +% pandoc -t markdown +[https://example.com](https://example.com){.clz} +^D +[https://example.com](https://example.com){.clz} +``` + +``` +% pandoc -f markdown -t html | pandoc -f html -t markdown + +^D + +``` diff --git a/test/command/7697.md b/test/command/7697.md new file mode 100644 --- /dev/null +++ b/test/command/7697.md @@ -0,0 +1,27 @@ +``` +% pandoc -f rst -t mediawiki +.. _refsubpage1: + +heading +------- + +ref to top of this section: `refsubpage1`_. +^D + += heading = + +ref to top of this section: [[#refsubpage1|refsubpage1]]. +``` +``` +% pandoc -f markdown -t mediawiki +# Heading {#foo} +^D + += Heading = +``` +``` +% pandoc -f markdown -t mediawiki +# My Heading {#My_Heading} +^D += My Heading = +``` diff --git a/test/command/jats-figure-alt-text.md b/test/command/jats-figure-alt-text.md new file mode 100644 --- /dev/null +++ b/test/command/jats-figure-alt-text.md @@ -0,0 +1,18 @@ +``` +% pandoc -f jats -t native + + +

bar

+ + alternative-decription + + +^D +[ Para + [ Image + ( "fig-1" , [] , [ ( "alt" , "alternative-decription" ) ] ) + [ Str "bar" ] + ( "foo.png" , "fig:" ) + ] +] +``` diff --git a/test/command/refs.md b/test/command/refs.md --- a/test/command/refs.md +++ b/test/command/refs.md @@ -60,6 +60,24 @@ ``` % pandoc -f latex -t native +\autoref{fig:flowchart} +^D +[ Para + [ Link + ( "" + , [] + , [ ( "reference-type" , "autoref" ) + , ( "reference" , "fig:flowchart" ) + ] + ) + [ Str "[fig:flowchart]" ] + ( "#fig:flowchart" , "" ) + ] +] +``` + +``` +% pandoc -f latex -t native Accuracy~\eqref{eq:Accuracy} is the proportion, measuring true results among all results. \begin{equation} diff --git a/test/docbook-reader.docbook b/test/docbook-reader.docbook --- a/test/docbook-reader.docbook +++ b/test/docbook-reader.docbook @@ -1603,4 +1603,20 @@ + + Index terms + + In the simplest case, index termsindex term consists of just a <primary> element, but index termmulti-level they can also consist of a <primary> and <secondary> element, and index termmulti-level3-level can even include a <tertiary> term. + + + Index terms can also refer to other index terms: index cross referencingindex termcross referencesindex cross referencingexclusively, using the <see> tag; or index cross referencingcross referencing as a reference to related terms, using the <seealso> tag. + + + foodbig baguette supremeNested content in index term elements is flattened. + + + + Abbreviated title + Abbr. title + diff --git a/test/docbook-reader.native b/test/docbook-reader.native --- a/test/docbook-reader.native +++ b/test/docbook-reader.native @@ -2930,4 +2930,198 @@ [ Str "A" , Space , Str "Final" , Space , Str "Step" ] ] ] + , Header + 1 + ( "indexterms" , [] , [] ) + [ Str "Index" , Space , Str "terms" ] + , Para + [ Str "In" + , Space + , Str "the" + , Space + , Str "simplest" + , Space + , Str "case," + , Space + , Str "index" + , Space + , Str "terms" + , Span + ( "" , [ "indexterm" ] , [ ( "primary" , "index term" ) ] ) + [] + , Space + , Str "consists" + , Space + , Str "of" + , Space + , Str "just" + , Space + , Str "a" + , Space + , Code ( "" , [] , [] ) "" + , Space + , Str "element," + , Space + , Str "but" + , Space + , Span + ( "" + , [ "indexterm" ] + , [ ( "primary" , "index term" ) + , ( "secondary" , "multi-level" ) + ] + ) + [] + , Space + , Str "they" + , Space + , Str "can" + , Space + , Str "also" + , Space + , Str "consist" + , Space + , Str "of" + , Space + , Str "a" + , Space + , Code ( "" , [] , [] ) "" + , Space + , Str "and" + , Space + , Code ( "" , [] , [] ) "" + , Space + , Str "element," + , Space + , Str "and" + , Space + , Span + ( "" + , [ "indexterm" ] + , [ ( "primary" , "index term" ) + , ( "secondary" , "multi-level" ) + , ( "tertiary" , "3-level" ) + ] + ) + [] + , Space + , Str "can" + , Space + , Str "even" + , Space + , Str "include" + , Space + , Str "a" + , Space + , Code ( "" , [] , [] ) "" + , Space + , Str "term." + ] + , Para + [ Str "Index" + , Space + , Str "terms" + , Space + , Str "can" + , Space + , Str "also" + , Space + , Str "refer" + , Space + , Str "to" + , Space + , Str "other" + , Space + , Str "index" + , Space + , Str "terms:" + , Space + , Span + ( "" + , [ "indexterm" ] + , [ ( "primary" , "index cross referencing" ) ] + ) + [] + , Span + ( "" + , [ "indexterm" ] + , [ ( "primary" , "index term" ) + , ( "secondary" , "cross references" ) + , ( "see" , "index cross referencing" ) + ] + ) + [] + , Str "exclusively," + , Space + , Str "using" + , Space + , Str "the" + , Space + , Code ( "" , [] , [] ) "" + , Space + , Str "tag;" + , Space + , Str "or" + , Space + , Span + ( "" + , [ "indexterm" ] + , [ ( "primary" , "index cross referencing" ) + , ( "seealso" , "cross referencing" ) + ] + ) + [] + , Space + , Str "as" + , Space + , Str "a" + , Space + , Str "reference" + , Space + , Str "to" + , Space + , Str "related" + , Space + , Str "terms," + , Space + , Str "using" + , Space + , Str "the" + , Space + , Code ( "" , [] , [] ) "" + , Space + , Str "tag." + ] + , Para + [ Span + ( "" + , [ "indexterm" ] + , [ ( "primary" , "food" ) + , ( "secondary" , "big baguette supreme" ) + ] + ) + [] + , Str "Nested" + , Space + , Str "content" + , Space + , Str "in" + , Space + , Str "index" + , Space + , Str "term" + , Space + , Str "elements" + , Space + , Str "is" + , Space + , Str "flattened." + ] + , Header + 1 + ( "titleabbrev" + , [] + , [ ( "titleabbrev" , "Abbr. title" ) ] + ) + [ Str "Abbreviated" , Space , Str "title" ] ] diff --git a/test/docbook-xref.native b/test/docbook-xref.native --- a/test/docbook-xref.native +++ b/test/docbook-xref.native @@ -156,7 +156,7 @@ [ Str "Some" , Space , Str "content" , Space , Str "here" ] , Header 1 - ( "ch04" , [] , [] ) + ( "ch04" , [] , [ ( "titleabbrev" , "Chapter 4" ) ] ) [ Str "The" , Space , Str "Fourth" , Space , Str "Chapter" ] , Para [ Str "Some" , Space , Str "content" , Space , Str "here" ] diff --git a/test/latex-reader.native b/test/latex-reader.native --- a/test/latex-reader.native +++ b/test/latex-reader.native @@ -1868,7 +1868,7 @@ , Str "ampersand:" , Space , Link - ( "" , [] , [] ) + ( "" , [ "uri" ] , [] ) [ Str "http://example.com/?foo=1&bar=2" ] ( "http://example.com/?foo=1&bar=2" , "" ) ] @@ -1878,7 +1878,7 @@ ] , [ Para [ Link - ( "" , [] , [] ) + ( "" , [ "uri" ] , [] ) [ Str "http://example.com/" ] ( "http://example.com/" , "" ) ] @@ -1902,7 +1902,7 @@ [ Str "Blockquoted:" , Space , Link - ( "" , [] , [] ) + ( "" , [ "uri" ] , [] ) [ Str "http://example.com/" ] ( "http://example.com/" , "" ) ] diff --git a/test/lua/module/pandoc.lua b/test/lua/module/pandoc.lua --- a/test/lua/module/pandoc.lua +++ b/test/lua/module/pandoc.lua @@ -151,17 +151,17 @@ group "Inline elements" { group 'Cite' { test('has property `content`', function () - local cite = pandoc.Cite({}, {pandoc.Emph 'important'}) + local cite = pandoc.Cite({pandoc.Emph 'important'}, {}) assert.are_same(cite.content, {pandoc.Emph {pandoc.Str 'important'}}) cite.content = 'boring' - assert.are_equal(cite, pandoc.Cite({}, {pandoc.Str 'boring'})) + assert.are_equal(cite, pandoc.Cite({pandoc.Str 'boring'}, {})) end), test('has list of citations in property `cite`', function () local citations = { pandoc.Citation('einstein1905', 'NormalCitation') } - local cite = pandoc.Cite(citations, 'relativity') + local cite = pandoc.Cite('relativity', citations) assert.are_same(cite.citations, citations) local new_citations = { @@ -169,7 +169,7 @@ pandoc.Citation('Poincaré1905', 'NormalCitation') } cite.citations = new_citations - assert.are_equal(cite, pandoc.Cite(new_citations, {'relativity'})) + assert.are_equal(cite, pandoc.Cite({'relativity'}, new_citations)) end), }, group 'Code' { @@ -809,7 +809,25 @@ ) assert.are_same(expected_table, new_table) end) - } + }, + group 'ReaderOptions' { + test('returns a userdata value', function () + local opts = pandoc.ReaderOptions {} + assert.are_equal(type(opts), 'userdata') + end), + test('can construct from table', function () + local opts = pandoc.ReaderOptions {columns = 66} + assert.are_equal(opts.columns, 66) + end), + test('can construct from other ReaderOptions value', function () + local orig = pandoc.ReaderOptions{columns = 65} + local copy = pandoc.ReaderOptions(orig) + for k, v in pairs(orig) do + assert.are_same(copy[k], v) + end + assert.are_equal(copy.columns, 65) + end), + }, }, group 'clone' { @@ -894,6 +912,16 @@ assert.error_matches( function () pandoc.read('foo', 'gfm+empty_paragraphs') end, 'Extension empty_paragraphs not supported for gfm' + ) + end), + test('read with other indented code classes', function() + local indented_code = ' return true' + local expected = pandoc.Pandoc({ + pandoc.CodeBlock('return true', {class='foo'}) + }) + assert.are_same( + expected, + pandoc.read(indented_code, 'markdown', {indented_code_classes={'foo'}}) ) end), test('failing read', function () diff --git a/test/pptx/layouts/deleted.pptx b/test/pptx/layouts/deleted.pptx new file mode 100644 Binary files /dev/null and b/test/pptx/layouts/deleted.pptx differ diff --git a/test/pptx/layouts/input.native b/test/pptx/layouts/input.native new file mode 100644 --- /dev/null +++ b/test/pptx/layouts/input.native @@ -0,0 +1,23 @@ +Pandoc (Meta {unMeta = fromList [("title",MetaInlines [Str "Testing",Space,Str "Layouts"])]}) +[Header 2 ("slide-1",[],[]) [Str "Slide",Space,Str "1"] +,Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "title",Space,Str "and",Space,Str "content",Space,Str "slide"] +,Header 2 ("slide-2",[],[]) [Str "Slide",Space,Str "2"] +,Div ("",["columns"],[]) + [Div ("",["column"],[]) + [Para [Str "This"]] + ,Div ("",["column"],[]) + [Para [Str "\8230is",Space,Str "a",Space,Str "two-column",Space,Str "slide"]]] +,Header 2 ("slide-3",[],[]) [Str "Slide",Space,Str "3"] +,Para [Str "This",Space,Str "slide",Space,Str "is",Space,Str "a",Space,Str "Content",Space,Str "with",Space,Str "Caption",Space,Str "slide"] +,Para [Image ("",[],[]) [Str "Content"] ("lalune.jpg","fig:")] +,Header 2 ("slide-4",[],[]) [Str "Slide",Space,Str "4"] +,Div ("",["columns"],[]) + [Div ("",["column"],[]) + [Para [Str "This",Space,Str "slide",Space,Str "is",Space,Str "a",Space,Str "Comparison",Space,Str "slide:"] + ,Para [Image ("",[],[]) [Str "Content"] ("lalune.jpg","fig:")]] + ,Div ("",["column"],[]) + [Para [Str "Here",Space,Str "is",Space,Str "some",Space,Str "other",Space,Str "text"]]] +,Header 1 ("section-header",[],[]) [Str "Section",Space,Str "header"] +,Header 2 ("section",[],[]) [] +,Div ("",["notes"],[]) + [Para [Str "This",Space,Str "is",Space,Str "a",Space,Str "blank",Space,Str "slide"]]] diff --git a/test/pptx/layouts/moved.pptx b/test/pptx/layouts/moved.pptx new file mode 100644 Binary files /dev/null and b/test/pptx/layouts/moved.pptx differ diff --git a/test/pptx/pauses/without-incremental/output.pptx b/test/pptx/pauses/without-incremental/output.pptx new file mode 100644 Binary files /dev/null and b/test/pptx/pauses/without-incremental/output.pptx differ diff --git a/test/pptx/pauses/without-incremental/templated.pptx b/test/pptx/pauses/without-incremental/templated.pptx new file mode 100644 Binary files /dev/null and b/test/pptx/pauses/without-incremental/templated.pptx differ diff --git a/test/writer.mediawiki b/test/writer.mediawiki --- a/test/writer.mediawiki +++ b/test/writer.mediawiki @@ -3,24 +3,33 @@ ----- + = Headers = + == Level 2 with an [[url|embedded link]] == + === Level 3 with ''emphasis'' === + ==== Level 4 ==== + ===== Level 5 ===== + = Level 1 = + == Level 2 with ''emphasis'' == + === Level 3 === with no blank line + == Level 2 == with no blank line @@ -28,6 +37,7 @@ ----- + = Paragraphs = Here’s a regular paragraph. @@ -42,6 +52,7 @@ ----- + = Block Quotes = E-mail style: @@ -71,6 +82,7 @@ ----- + = Code Blocks = Code: @@ -90,8 +102,10 @@ ----- + = Lists = + == Unordered == Asterisks tight: @@ -130,6 +144,7 @@ * Minus 2 * Minus 3 + == Ordered == Tight: @@ -164,6 +179,7 @@

  • Item 2.

  • Item 3.

    • + == Nested == * Tab @@ -188,6 +204,7 @@ #* Foe # Third + == Tabs and spaces == * this is a list item indented with tabs @@ -195,6 +212,7 @@ ** this is an example list item indented with tabs ** this is an example list item indented with spaces + == Fancy list markers ==
      @@ -239,6 +257,7 @@ ----- + = Definition Lists = Tight using spaces: @@ -307,6 +326,7 @@ ;# sublist ;# sublist + = HTML Blocks = Simple block on one line: @@ -416,6 +436,7 @@ ----- + = Inline Markup = This is ''emphasized'', and so ''is this''. @@ -445,6 +466,7 @@ ----- + = Smart quotes, ellipses, dashes = “Hello,” said the spider. “‘Shelob’ is my name.” @@ -466,6 +488,7 @@ ----- + = LaTeX = * @@ -490,6 +513,7 @@ ----- + = Special Characters = Here is some unicode: @@ -545,8 +569,10 @@ ----- + = Links = + == Explicit == Just a [[url/|URL]]. @@ -567,6 +593,7 @@ [[|Empty]]. + == Reference == Foo [[url/|bar]]. @@ -588,6 +615,7 @@ Foo [[url/|biz]]. + == With ampersands == Here’s a [http://example.com/?foo=1&bar=2 link with an ampersand in the URL]. @@ -598,6 +626,7 @@ Here’s an [[script?foo=1&bar=2|inline link in pointy braces]]. + == Autolinks == With an ampersand: http://example.com/?foo=1&bar=2 @@ -616,6 +645,7 @@ ----- + = Images = From “Voyage dans la Lune” by Georges Melies (1902): @@ -627,6 +657,7 @@ ----- + = Footnotes = Here is a footnote reference,Here is the footnote. It can go anywhere after the footnote reference. It need not be placed at the end of the document. and another.Here’s the long note. This one contains multiple blocks.