packages feed

pandoc 2.17.1.1 → 2.18

raw patch · 178 files changed

+10493/−3282 lines, 178 filesdep +hslua-module-doclayoutdep +xml-typesdep −hslua-marshallingdep ~citeprocdep ~commonmarkdep ~commonmark-extensionsbinary-added

Dependencies added: hslua-module-doclayout, xml-types

Dependencies removed: hslua-marshalling

Dependency ranges changed: citeproc, commonmark, commonmark-extensions, containers, doclayout, hslua, hslua-aeson, pandoc-lua-marshal, pandoc-types, skylighting, skylighting-core, texmath

Files

AUTHORS.md view
@@ -76,6 +76,7 @@ - Derek Chen-Becker - Diego Balseiro - Dimitri Sabadie+- Dimitris Apostolou - Dmitry Pogodin - Dmitry Volodin - Douglas Calvert@@ -113,6 +114,7 @@ - Greg Maslov - Greg Rundlett - Grégory Bataille+- Guriy Samarin - Gwern Branwen - Hamish Mackenzie - Hans-Peter Deifel@@ -126,7 +128,9 @@ - Igor Pashev - Ilya V. Portnov - Ivan Trubach+- Ivar de Bruin - Ivo Clarysse+- Jaap de Jong - J. B. Rainsberger - J. Lewis Muir - Jaime Marquínez Ferrándiz@@ -159,6 +163,7 @@ - Jonas Scholl - Jonas Smedegaard - Jonathan Daugherty+- Jonathan Dönszelmann - Jose Luis Duran - José de Mattos Neto - Josef Svenningsson@@ -184,8 +189,10 @@ - Lorenzo - Lucas Escot - Lucas Viana+- Lucas V. R - Luke Plant - Marc Schreiber+- Mario Lang - Mark Szepieniec - Mark Wright - Martin Linn@@ -215,6 +222,7 @@ - MinRK - Morton Fox - Nathan Gass+- Naveen - Neil Mayhew - Nick Bart - Nick Berendsen@@ -226,6 +234,7 @@ - Nixon Enraght-Moony - Nokome Bentley - OCzarnecki+- Ola Wolska - Ole Martin Ruud - Oliver Matthews - Ophir Lifshitz@@ -321,6 +330,7 @@ - cholonam - csforste - d-dorazio+- damon-sava-stanley - dbecher-ito - ebiim - favonia@@ -331,6 +341,7 @@ - lux-lth - lwolfsonkin - mbrackeantidot+- mjfs - mt_caret - niszet - nkalvi
INSTALL.md view
@@ -33,7 +33,12 @@ [MiKTeX] (to typeset PDFs with [LaTeX]):      choco install rsvg-convert python miktex+    +Or, you can install pandoc using+[winget](https://github.com/microsoft/winget-pkgs): +    winget install pandoc+ Using multiple installation methods can result in two separate installations of pandoc; it is recommended to properly uninstall pandoc before switching to an alternative installation method.@@ -305,8 +310,7 @@     - `embed_data_files`: embed all data files into the binary (default no).       This is helpful if you want to create a relocatable binary. -    - `https`:  enable support for downloading resources over https-      (using the `http-client` and `http-client-tls` libraries).+    - `lua53`: embed support for Lua 5.3 instead of 5.4.  3.  Build:
MANUAL.txt view
@@ -1,7 +1,7 @@ --- title: Pandoc User's Guide author: John MacFarlane-date: January 30, 2022+date: April 4, 2022 ---  # Synopsis@@ -235,6 +235,7 @@     - `docbook` ([DocBook])     - `docx` ([Word docx])     - `dokuwiki` ([DokuWiki markup])+    - `endnotexml` ([EndNote XML bibliography])     - `epub` ([EPUB])     - `fb2` ([FictionBook2] e-book)     - `gfm` ([GitHub-Flavored Markdown]),@@ -259,6 +260,7 @@     - `odt` ([ODT])     - `opml` ([OPML])     - `org` ([Emacs Org mode])+    - `ris` ([RIS] bibliography)     - `rtf` ([Rich Text Format])     - `rst` ([reStructuredText])     - `t2t` ([txt2tags])@@ -486,6 +488,7 @@ [roff ms]: https://man.cx/groff_ms(7) [Haskell]: https://www.haskell.org [GNU Texinfo]: https://www.gnu.org/software/texinfo/+[RIS]: https://en.wikipedia.org/wiki/RIS_(file_format) [Emacs Org mode]: https://orgmode.org [AsciiDoc]: https://www.methods.co.nz/asciidoc/ [AsciiDoctor]: https://asciidoctor.org/@@ -504,6 +507,7 @@ [BibTeX]: https://ctan.org/pkg/bibtex [BibLaTeX]: https://ctan.org/pkg/biblatex [Markua]: https://leanpub.com/markua/read+[EndNote XML bibliography]: https://support.clarivate.com/Endnote/s/article/EndNote-XML-Document-Type-Definition  ## Reader options {.options} @@ -637,10 +641,12 @@  :   Read metadata from the supplied YAML (or JSON) file. This     option can be used with every input format, but string scalars-    in the YAML file will always be parsed as Markdown. Generally,-    the input will be handled the same as in [YAML metadata-    blocks][Extension: `yaml_metadata_block`]. This option can be-    used repeatedly to include multiple metadata files; values in+    in the YAML file will always be parsed as Markdown. (If the+    input format is Markdown or a Markdown variant, then the+    same variant will be used to parse the metadata file;+    if it is a non-Markdown format, pandoc's default Markdown+    extensions will be used.) This option can be used+    repeatedly to include multiple metadata files; values in     files specified later on the command line will be preferred     over those specified in earlier files. Metadata values     specified inside the document, or by using `-M`, overwrite@@ -950,8 +956,7 @@     `data-external="1"` will be left alone; the documents they     link to will not be incorporated in the document.     Limitation: resources that are loaded dynamically through-    JavaScript cannot be incorporated; as a result,-    `--self-contained` does not work with `--mathjax`, and some+    JavaScript cannot be incorporated; as a result, some     advanced features (e.g.  zoom or speaker notes) may not work     in an offline "self-contained" `reveal.js` slide show. @@ -3162,7 +3167,8 @@ This extension can be enabled/disabled for the following formats:  input formats-:  `markdown`, `commonmark`, `latex`, `mediawiki`, `org`, `rst`, `twiki`+:  `markdown`, `commonmark`, `latex`, `mediawiki`, `org`, `rst`, `twiki`,+   `html`  output formats :  `markdown`, `latex`, `context`, `rst`@@ -3465,9 +3471,18 @@  #### Extension: `citations` {#org-citations} -Some aspects of [Pandoc's Markdown citation syntax](#citations)-are also accepted in `org` input.+When the `citations` extension is enabled in `org`,+org-cite and org-ref style citations will be parsed as+native pandoc citations. +  [org-cite]: https://orgmode.org/manual/Citations.html+  [org-ref]:  https://github.com/jkitchin/org-ref++When `citations` is enabled in `docx`, citations inserted+by Zotero or Mendeley or EndNote plugins will be parsed as native+pandoc citations.  (Otherwise, the formatted citations generated+by the bibliographic software will be parsed as regular text.)+ #### Extension: `fancy_lists` {#org-fancy-lists}  Some aspects of [Pandoc's Markdown fancy lists](#extension-fancy_lists) are also@@ -4784,7 +4799,7 @@  For display math, use `$$` delimiters.  (In this case, the delimiters may be separated from the formula by whitespace.  However, there can be-no blank lines betwen the opening and closing `$$` delimiters.)+no blank lines between the opening and closing `$$` delimiters.)  TeX math will be printed in all output formats. How it is rendered depends on the output format:@@ -5717,6 +5732,7 @@   BibTeX            .bibtex   CSL JSON          .json   CSL YAML          .yaml+  RIS               .ris  Note that `.bib` can be used with both BibTeX and BibLaTeX files; use the extension `.bibtex` to force interpretation as BibTeX.@@ -5959,6 +5975,19 @@ the `unnumbered` class will be added to this heading, so that the section will not be numbered. +If you want to put the bibliography into a variable in your+template, one way to do that is to put the div with id `refs`+into a metadata field, e.g.++    ---+    refs: |+       ::: {#refs}+       :::+    ...++You can then put the variable `$refs$` into your template where+you want the bibliography to be placed.+ ## Including uncited items in the bibliography  If you want to include items in the bibliography without actually@@ -6241,6 +6270,15 @@ Both methods allow incremental and nonincremental lists to be mixed in a single document. +If you want to include a block-quoted list, you can work around+this behavior by putting the list inside a fenced div, so that+it is not the direct child of the block quote:++    > ::: wrapper+    > - a+    > - list in a quote+    > :::+ ## Inserting pauses  You can add "pauses" within a slide by including a paragraph containing@@ -6974,9 +7012,6 @@  # A note on security -If you use pandoc to convert user-contributed content in a web-application, here are some things to keep in mind:- 1. Although pandoc itself will not create or modify any files other    than those you explicitly ask it create (with the exception    of temporary files used in producing PDFs), a filter or custom@@ -7002,7 +7037,8 @@    that fully isolates pandoc from your file system, by running the    pandoc operations in the `PandocPure` monad. See the document    [Using the pandoc API](https://pandoc.org/using-the-pandoc-api.html)-   for more details.+   for more details. (This corresponds to the use of the `--sandbox`+   option on the command line.)   5. Pandoc's parsers can exhibit pathological performance on some    corner cases.  It is wise to put any pandoc operations under@@ -7018,8 +7054,8 @@    If `raw_html` is enabled for the Markdown input, users can    inject arbitrary HTML.  Even if `raw_html` is disabled,    users can include dangerous content in URLs and attributes.-   To be safe, you should run all the generated HTML through-   an HTML sanitizer.+   To be safe, you should run all HTML generated from untrusted+   user input through an HTML sanitizer.  # Authors
README.md view
@@ -39,6 +39,8 @@ -   `docbook` ([DocBook](https://docbook.org)) -   `docx` ([Word docx](https://en.wikipedia.org/wiki/Office_Open_XML)) -   `dokuwiki` ([DokuWiki markup](https://www.dokuwiki.org/dokuwiki))+-   `endnotexml` ([EndNote XML+    bibliography](https://support.clarivate.com/Endnote/s/article/EndNote-XML-Document-Type-Definition)) -   `epub` ([EPUB](http://idpf.org/epub)) -   `fb2`     ([FictionBook2](http://www.fictionbook.org/index.php/Eng:XML_Schema_Fictionbook_2.1)@@ -76,6 +78,8 @@ -   `odt` ([ODT](https://en.wikipedia.org/wiki/OpenDocument)) -   `opml` ([OPML](http://dev.opml.org/spec2.html)) -   `org` ([Emacs Org mode](https://orgmode.org))+-   `ris` ([RIS](https://en.wikipedia.org/wiki/RIS_(file_format))+    bibliography) -   `rtf` ([Rich Text     Format](https://en.wikipedia.org/wiki/Rich_Text_Format)) -   `rst`
cabal.project view
@@ -3,8 +3,8 @@ flags: +embed_data_files constraints: aeson >= 2.0.1.0 + -- source-repository-package --   type: git---   location: https://github.com/jgm/ipynb.git---   tag: 00246af10885c2ad4413ace4f69a7e6c88297a08-+--   location: https://github.com/jgm/doclayout.git+--   tag: b274a5658f3655f31c0bc6175d5b27a9467c1456
changelog.md view
@@ -1,5 +1,361 @@ # Revision history for pandoc +## pandoc 2.18 (2022-04-22)++  * New input formats: `endnotexml` (EndNote XML bibliography),+    `ris` (RIS bibliography).++  * A RIS bibliography file may now be used with `--citeproc`.++  * Citeproc: Allow a formatted bibliography to be placed in metadata fields+    via a Div with class `refs` (#7969, #526).  Thus, one can include a+    metadata field, say `refs`, whose content is an+    empty div with id `refs`, and the formatted bibliography will be put into+    this metadata field.  It may then be interpolated into a template using the+    variable `refs`.++  * Ensure that you don't get PDF output to terminal.  `-t pdf` now behaves+    like `-t docx` and gives an error unless the output is redirected.++  * `--version` now prints hslua version (#7929) and+    Lua version (#7997, Albert Krewinkel).++  * Change `--metadata-file` parsing so that, when the input format is not+    markdown or a markdown variant, pandoc's markdown is used (#6832, #7926).+    When the input format is a markdown variant, the same format is used.+    Reason for the change: it doesn't make sense to run the markdown parser+    with a set of extensions designed for a non-markdown format, and+    this dramatically limits what people can do in metadata files.++  * Trim whitespace from math in `--webtex` (#7892).+    This fixes problems with --webtex and markdown output,+    when display math starts or ends with a newline.++  * New exported module Text.Pandoc.Readers.EndNote, exporting+    `readEndNoteXML`, `readEndNoteXMLCitation`,+    and `readEndNoteXMLReferences`. [API change]++  * `--self-contained`: issue warning rather than failing with an error+    if a resource can't be found (#7904).++  * New exported module, Text.Pandoc.Readers.RIS, exporting `readRIS` (#7894).++  * LaTeX reader:++    + Handle subequations as inline math environment (#7883).+    + Rudimentary support for `vbox` (#7939).+    + Support `\today` (#7905).+    + Handle `\label` and `\ref` for footnotes (#7930).+    + Allow inline groups starting with `\bgroup` (#7953).+    + Use custom TokStream that keeps track of whether macros are expanded.+      This allows us to improve performance a bit by avoiding unnecessary+      runs of the macro expansion code (e.g. from 24 ms to 20 ms on+      our standard benchmark).+    + Further optimizations for inline parsing.+    + Better handling of `\usepackage`.  If the package is local but+      causes parse errors, parse everything up to the error and skip the+      rest.  Issue a `CouldNotParseIncludeFile` warning indicating that+      parsing failed at that point.+    + Text.Pandoc.Readers.LaTeX.Parsing: Monoid and Semigroup instances for+      TokStream.++  * HTML reader:++    + Give warnings and emit empty note when parsing+      `<a epub:type="noteref">` and the identifier+      doesn't correspond to anything in the note table (#7884).+      Previously we just silently skipped these cases.+    + Fix parsing of epub footnotes (#7884).++  * DocBook reader:++    + Handle complete set of entities as specified at+      <https://www.w3.org/2003/entities/2007doc/byalpha.html> (#7938).+    + Handle abstract in info section (#7747).+    + Improve info parsing.+    + Simplify metadata parsing code (#7747).+      Handle abstract as block-level content.+      Report skipped info elements with `--verbose`.+    + Handle address and coyright in metadata (#7747).++  * DokuWiki reader:++    + Add DokuWiki table alignment (#5202, damon-sava-stanley).++  * RST reader:++    + Fix treatment of headerless simple tables (#7902).+    + Wrap math in Span to preserve attributes (#7998, Albert Krewinkel).+      Math elements with a name, classes, or other fields are wrapped in a+      `Span` with these attributes.++  * JATS reader:++    + Improve handling of fn-group elements (#6348, Albert Krewinkel).+      Footnotes in `<fn-group>` elements are collected and re-inserted into+      the document as proper footnotes in the place where they are referenced.+    + Handle `pub-date` (#8000).+    + Support PMID, DOI, issue in citations (#7995).+    + Improve refs parsing.  Handle `issn` and `isbn`; use simpler form+      for issued date.+    + Strip 'ref-' from ref id in constructing CSL id.  This allows better+      round-tripping, because the JATS writer adds the `ref-` prefix to+      the citation id to get the ref element's id.++  * Org reader:++    + Allow ":" in property drawer keys (Lucas V. R).+      Any non-space character is allowed as property drawer key, including ":"+      itself (so it is not really a delimiter). The real delimiter is a space+      character, so in a drawer like+      ```+      :PROPERTIES:+      ::k:ey:: value+      :END:+      ```+      ":k:ey:" is a key with value "value".++    + Allow comments above property drawer.+    + More flexible LaTeX environments (Lucas V. R).+    + Handle `#+bibliography:` as metadata so that it can work with+      `--citeproc`.+    + Parse `#+print_bibliography:` as Div with id `refs`.+    + Allow multiple `#+bibliography:`.++  * Markdown reader:++    + Allow one-column pipe tables with pipe on right (#7919).+    + Remove restriction on identifiers, so they no longer need to+      begin with a letter (#7920).++  * Docx reader:++    + Enable `citations` extension for docx reader (#7840).+      When enabled, Zotero, Mendeley, and EndNote citations embedded in+      a docx are parsed as native pandoc citations.+      (When disabled, the generated citation+      text and bibliography are passed through as regular text.)+      The bibliography generated by the plugin is suppressed.+      Instead, bibliographic data embedded in citation items is added+      to the `references` metadata field so that it can be used+      with `--citeproc`.++  * Docbook writer:++    + Interpret links without contents as cross-references (#7360, Jan Tojnar).+      Links without text contents are converted to `<xref>` elements. DocBook+      processors will generate appropriate cross-reference text when presented+      with an xref element.++  * Docx writer:++    + Single numbering ID for examples (#7895, mjfs).+      This change ensures that example list items all belong to a single+      number sequence, so that if items are added or deleted in a word+      processor, the other items will renumber automatically.+    + Add bookmark with table id to table (#7989, Nikolai+      Korobeinikov, #7285).  This allows tables with ids to be linked to.++  * Ipynb writer:++    + Handle metadata better (#7928).  Previously we used the markdown+      writer to render metadata.  This had some undesirable consequences+      (e.g. en dash expanded to `--` when `smart` enabled), so now we use+      the plain writer.++  * LaTeX writer:++    + Avoid extra space before `\CSLRightInline` (#7932).+    + Add `scrreport` to `chaptersClasses` (#6168, ivardb).+    + Support `page`,`trim`,`clip` attributes on images (#7181).+    + Add `()` after booktabs rules (#8001).  These commands take optional+      arguments with () and [], which can lead to problems if the content+      of the table cell begins with these characters.++  * RST writer:++    + Support all standard metadata ("bibliographic") fields.++  * HTML writer: performance improvements.++  * Org writer:++    + Stop indenting property drawers, quote blocks+      (#3245, Albert Krewinkel).  This follows the current default org-mode+      behavior.++  * Markdown writer:++    + Move table-related code into submodule (Albert Krewinkel).+    + Don't produce redundant header identifier when the+      `gfm_auto_identifiers` extension is set (#7941).+    + Update escaping rules for `\`.  We now escape `\` only if `raw_tex`+      is enabled or it is followed by a non-alphanumeric.++  * JATS writer:++    + Encode author "others" as `<etal/>` (Albert Krewinkel).+      Citeproc adopted the BibTeX convention to use the author name "others"+      when there are additional authors that are not named. JATS uses the+      `<etal>` element for this.+    + Avoid doubled ref-list element (#7990).  Previously when generating JATS+      with the `element_citations` extension enabled, the references were+      put in a doubly-nested ref-list element (`<ref-list><ref-list>...`).+    + Keep edition info in element citations (#7993, Albert Krewinkel).+    + Fix handling of CSL variable 'page' (not 'pages' as we had before).+      It should go to 'lpage' and 'rpage', not 'page-range'.++  * EPUB writer: refactor for clarity (#7991, Jonathan Dönszelmann,+    Ola Wolska, Ivar de Bruin, Jaap de Jong).++  * Custom writer (Albert Krewinkel):++    + Support new-style Writer function (Albert Krewinkel).+      See the documentation for custom writers for details.+    + Produce stacktrace if Writer function fails++  * Text.Pandoc.Logging: add `CouldNotParseIncludeFile` constructor+    for `LogMessage` [API change].++  * Text.Pandoc.Shared:++    + Put id attributes on TOC entries (#7907, damon-sava-stanley).+      Naming scheme of id is "toc-" + id of linked to header/section.+      Effects HTML, Markdown, Powerpoint, and RTF.+    + Define `ordNub` as alias for `nubOrd` from containers package (#7963,+      Albert Krewinkel).+    + Export `ensureValidXmlIdentifiers`.  This function changes identifiers+      that don't start with letters, and internal links to these identifiers,+      making them compatible with XML standards.  The change+      is simple: we add `id_` to the front.  There is potential+      for duplication if there are already `id_...` identifiers+      defined, but this seems rare enough not to worry too much+      about.++  * Ensure that valid XML identifiers are used in Docbook, EPUB, FB2,+    HTML4, S5, Slidy, Slideous, ICML, ODT, TEI writers.+    Thus, if you convert `[anchor]{#1} and [link to](#1)`,+    `id_1` will be used instead of `1` for the identifier.++  * Lua (Albert Krewinkel).++    + Add module `pandoc.layout` to format and layout text.+    + Move custom writer code into Lua hierarchy.+    + Use pandoc-lua-marshal 0.1.5.+    +  Allow any type of callable object as argument to List+       functions `filter`, `map`, and `find_if`. These previously+       required the argument to be of type `function`, which was too+       restrictive.+    + Inline: the type of Image captions is now `Inlines` instead of `List`.+    + Allow passing `Sources` to `pandoc.read` (#8002).++  * Text.Pandoc.PDF:++    + Restore `wkhtmltopdf` as default pdf engine for HTML (Albert Krewinkel).+    + Allow custom writer as format if engine is explicitly specified (#7898,+      Albert Krewinkel).  Note that it may be necessary to explicitly+      specify a template on the command line.++  * Text.Pandoc.MediaBag: improve detection of absolute paths (#7881).++  * Text.Pandoc.Extensions:++    + Remove `raw_tex` extension from list of commonmark extensions, and+      from the `commonmark_x` defaults.  commonmark doesn't parse raw TeX,+      and it doesn't make sense to write it if we don't parse it.+    + Remove `native_divs` from allowed gfm extensions (#7965).  This allows+      `<div>` to be suppressed using `-raw_html`.  Previously `native_divs`+      was enabled but could not be suppressed, because it was not in the+      list of available extensions for commonmark-based formats.++  * Text.Pandoc.Parsing:++    + Partition module into (internal) submodules (#7962, Albert Krewinkel).+    + Unify grid table parsing (#7971, Albert Krewinkel).+      Grid table parsing in Markdown and RST are updated use the same+      functions. Functions are generalized to meet requirements for both+      formats.  This change also lays the ground for further generalizations+      in table parsers, including support for advanced table features.+      [API change] in Text.Pandoc.Parsing:++      - Parse results of functions `tableWith'` and `gridTableWith'` are now a+        `mf TableComponents` instead of a quadruple of alignments, column+        widths, header rows and body rows.+      - Additional exports from Text.Pandoc.Parsing:+        `tableWith'`, `TableComponents`, `TableNormalization`,+        `toTableComponents`, `toTableComponents'`.++  * Text.Pandoc.XML.Light: add versions of the parsers that allow+    specifying a custom entity map.  Exports new functions:+    `parseXMLElementWithEntities`, `parseXMLContentsWithEntities` [API change].++  * Text.Pandoc.Writers.GridTable: improve module documentation (Albert+    Krewinkel).++  * Text.Pandoc.Parsing.GridTable: simplify column handling code (Albert+    Krewinkel).++  * Text.Pandoc.MIME: Add mime type for mkv extension (#7181).++  * Text.Pandoc.Asciify: Fix regression with `ascii_identifiers` and Turkish+    undotted i (#8003).++  * Relax upper bound for hslua, allow hslua-2.2. (#7929)+    Lua 5.4 is used by default after this is merged. Packagers may still+    include Lua 5.3 instead by building pandoc with `--constraint='hslua <2.2'`.+    Differences between 5.3 and 5.4 should not generally affect pandoc Lua+    filters.  See list of incompatible changes here:+    <https://www.lua.org/manual/5.4/manual.html#8.1>++  * JATS template: allow multiple licenses (Albert Krewinkel).++  * LaTeX template:++    + Skip `\babelprovide` if `babel-lang` is empty (#7945) to avoid an error.+    + Move hyperref near end of preamble (#5811).  It now comes after+      header-includes and right before title, author, date, abstract.+      Note:  Users who presuppose hyperref in their header-includes+      will now have to add `\usepackage{hyperref}` to their header-includes+      to make it available there.  (The redundant `\usepackage` will+      do no harm in this case.)+    + Remove special redefinition of `\sout`.  This used to be necessary to+      avoid problems with hyperref, when headings contain strikeout text,+      but it does not seem to be necessary any more (tested).++  * Tests: improve location reporting of failing tests (Albert Krewinkel).++  * Add tests for idempotency of `makeSections` (#7950).++  * Add armhf support in linux build script (#7944, Tony).++  * Use latest pandoc-types, so that toJSONFilter will work with+    Meta and MetaValue.++  * Use latest doclayout, texmath, commonmark, citeproc.++  * INSTALL.md: add reference to install via winget (#7951, Guriy Samarin).++  * MANUAL.txt:++    - Document way to get list in block quote in slide shows (#7916).+    - Minor changes to security section.+    - Note that `smart` works for html input.+    - Fix typos (#7934, Dimitris Apostolou).+    - Fix documentation for citations and org mode.+    - Remove claim that `--self-contained` does not work with `--mathjax`+      (#682).++  * doc/org.md: remove obsolete citations section.+    This mostly described citation formats we no longer support.++  * doc/lua-filters.md: Fix typo (#7981, Mario Lang).++  * Makefile: Use cabal for default build++  * Add `lua53` cabal flag.  It is false by default.  If set to true, compile+    with hslua 2.1 and Lua 5.3, otherwise hslua 2.2 and Lua 5.4.++ ## pandoc 2.17.1.1 (2022-01-31)    * Fix regression in 2.17.1 which caused problems finding
+ data/docbook-entities.txt view
@@ -0,0 +1,2237 @@+aacgr 03AC+Aacgr 0386+aacute 00E1+Aacute 00C1+abreve 0103+Abreve 0102+ac 223E+acd 223F+acE 223E  0333+acirc 00E2+Acirc 00C2+acute 00B4+acy 0430+Acy 0410+aelig 00E6+AElig 00C6+af 2061+afr 1D51E+Afr 1D504+agr 03B1+Agr 0391+agrave 00E0+Agrave 00C0+alefsym 2135+aleph 2135+alpha 03B1+Alpha 0391+amacr 0101+Amacr 0100+amalg 2A3F+amp 0026+AMP 0026+and 2227+And 2A53+andand 2A55+andd 2A5C+andslope 2A58+andv 2A5A+ang 2220+ange 29A4+angle 2220+angmsd 2221+angmsdaa 29A8+angmsdab 29A9+angmsdac 29AA+angmsdad 29AB+angmsdae 29AC+angmsdaf 29AD+angmsdag 29AE+angmsdah 29AF+angrt 221F+angrtvb 22BE+angrtvbd 299D+angsph 2222+angst 00C5+angzarr 237C+aogon 0105+Aogon 0104+aopf 1D552+Aopf 1D538+ap 2248+apacir 2A6F+ape 224A+apE 2A70+apid 224B+apos 0027+ApplyFunction 2061+approx 2248+approxeq 224A+aring 00E5+Aring 00C5+ascr 1D4B6+Ascr 1D49C+Assign 2254+ast 002A+asymp 2248+asympeq 224D+atilde 00E3+Atilde 00C3+auml 00E4+Auml 00C4+awconint 2233+awint 2A11+b.alpha 1D6C2+b.beta 1D6C3+b.chi 1D6D8+b.delta 1D6C5+b.Delta 1D6AB+b.epsi 1D6C6+b.epsiv 1D6DC+b.eta 1D6C8+b.gamma 1D6C4+b.Gamma 1D6AA+b.gammad 1D7CB+b.Gammad 1D7CA+b.iota 1D6CA+b.kappa 1D6CB+b.kappav 1D6DE+b.lambda 1D6CC+b.Lambda 1D6B2+b.mu 1D6CD+b.nu 1D6CE+b.omega 1D6DA+b.Omega 1D6C0+b.phi 1D6D7+b.Phi 1D6BD+b.phiv 1D6DF+b.pi 1D6D1+b.Pi 1D6B7+b.piv 1D6E1+b.psi 1D6D9+b.Psi 1D6BF+b.rho 1D6D2+b.rhov 1D6E0+b.sigma 1D6D4+b.Sigma 1D6BA+b.sigmav 1D6D3+b.tau 1D6D5+b.Theta 1D6AF+b.thetas 1D6C9+b.thetav 1D6DD+b.upsi 1D6D6+b.UpsiUpsilon+b.xi 1D6CF+b.Xi 1D6B5+b.zeta 1D6C7+backcong 224C+backepsilon 03F6+backprime 2035+backsim 223D+backsimeq 22CD+Backslash 2216+Barv 2AE7+barvee 22BD+barwed 2305+Barwed 2306+barwedge 2305+bbrk 23B5+bbrktbrk 23B6+bcong 224C+bcy 0431+Bcy 0411+bdquo 201E+becaus 2235+because 2235+Because 2235+bemptyv 29B0+bepsi 03F6+bernou 212C+Bernoullis 212C+beta 03B2+Beta 0392+beth 2136+between 226C+bfr 1D51F+Bfr 1D505+bgr 03B2+Bgr 0392+bigcap 22C2+bigcirc 25EF+bigcup 22C3+bigodot 2A00+bigoplus 2A01+bigotimes 2A02+bigsqcup 2A06+bigstarUB starf+bigtriangledown 25BD+bigtriangleup 25B3+biguplus 2A04+bigvee 22C1+bigwedge 22C0+bkarow 290D+blacklozengeUB lozf+blacksquare 25AA+blacktriangleUB utrif+blacktriangledownUB dtrif+blacktriangleleftUB ltrif+blacktrianglerightUB rtrif+blank 2423+blk12 2592+blk14 2591+blk34 2593+block 2588+bne 003D  20E5+bnequiv 2261  20E5+bnot 2310+bNot 2AED+bopf 1D553+Bopf 1D539+bot 22A5+bottom 22A5+bowtie 22C8+boxbox 29C9+boxdl 2510+boxdL 2555+boxDl 2556+boxDL 2557+boxdr 250C+boxdR 2552+boxDr 2553+boxDR 2554+boxh 2500+boxH 2550+boxhd 252C+boxhD 2565+boxHd 2564+boxHD 2566+boxhu 2534+boxhU 2568+boxHu 2567+boxHU 2569+boxminus 229F+boxplus 229E+boxtimes 22A0+boxul 2518+boxuL 255B+boxUl 255C+boxUL 255D+boxur 2514+boxuR 2558+boxUr 2559+boxUR 255A+boxv 2502+boxV 2551+boxvh 253C+boxvH 256A+boxVh 256B+boxVH 256C+boxvl 2524+boxvL 2561+boxVl 2562+boxVL 2563+boxvr 251C+boxvR 255E+boxVr 255F+boxVR 2560+bprime 2035+breve 02D8+Breve 02D8+brvbar 00A6+bscr 1D4B7+Bscr 212C+bsemi 204F+bsim 223D+bsime 22CD+bsol 005C+bsolb 29C5+bsolhsub 27C8+bull 2022+bulletUB bull+bump 224E+bumpe 224F+bumpE 2AAE+bumpeq 224F+Bumpeq 224E+cacute 0107+Cacute 0106+cap 2229+Cap 22D2+capand 2A44+capbrcup 2A49+capcap 2A4B+capcup 2A47+capdot 2A40+CapitalDifferentialD 2145+caps 2229  FE00+caret 2041+caron 02C7+Cayleys 212D+ccaps 2A4D+ccaron 010D+Ccaron 010C+ccedil 00E7+Ccedil 00C7+ccirc 0109+Ccirc 0108+Cconint 2230+ccups 2A4C+ccupssm 2A50+cdot 010B+Cdot 010A+cedil 00B8+Cedilla 00B8+cemptyv 29B2+cent 00A2+centerdotUM middot+CenterDotUM middot+cfr 1D520+Cfr 212D+chcy 0447+CHcy 0427+check 2713+checkmarkUB check+chi 03C7+Chi 03A7+cir 25CB+circ 02C6+circeq 2257+circlearrowleft 21BA+circlearrowright 21BB+circledast 229B+circledcirc 229A+circleddash 229D+CircleDot 2299+circledRUM reg+circledS 24C8+CircleMinus 2296+CirclePlus 2295+CircleTimes 2297+cire 2257+cirE 29C3+cirfnint 2A10+cirmid 2AEF+cirscir 29C2+ClockwiseContourIntegral 2232+CloseCurlyDoubleQuoteUM rdquo+CloseCurlyQuoteUM rsquo+clubs 2663+clubsuitUB clubs+colon 003A+Colon 2237+colone 2254+Colone 2A74+coloneq 2254+comma 002C+commat 0040+comp 2201+compfn 2218+complement 2201+complexes 2102+cong 2245+congdot 2A6D+Congruent 2261+conint 222E+Conint 222F+ContourIntegral 222E+copf 1D554+Copf 2102+coprod 2210+Coproduct 2210+copy 00A9+COPY 00A9+copysr 2117+CounterClockwiseContourIntegral 2233+crarr 21B5+cross 2717+Cross 2A2F+cscr 1D4B8+Cscr 1D49E+csub 2ACF+csube 2AD1+csup 2AD0+csupe 2AD2+ctdot 22EF+cudarrl 2938+cudarrr 2935+cuepr 22DE+cuesc 22DF+cularr 21B6+cularrp 293D+cup 222A+Cup 22D3+cupbrcap 2A48+cupcap 2A46+CupCap 224D+cupcup 2A4A+cupdot 228D+cupor 2A45+cups 222A  FE00+curarr 21B7+curarrm 293C+curlyeqprec 22DE+curlyeqsucc 22DF+curlyvee 22CE+curlywedge 22CF+curren 00A4+curvearrowleft 21B6+curvearrowright 21B7+cuvee 22CE+cuwed 22CF+cwconint 2232+cwint 2231+cylcty 232D+dagger 2020+Dagger 2021+daleth 2138+darr 2193+dArr 21D3+Darr 21A1+dash 2010+dashv 22A3+Dashv 2AE4+dbkarow 290F+dblac 02DD+dcaron 010F+Dcaron 010E+dcy 0434+Dcy 0414+dd 2146+DD 2145+ddaggerUB Dagger+ddarr 21CA+DDotrahd 2911+ddotseq 2A77+deg 00B0+Del 2207+delta 03B4+Delta 0394+demptyv 29B1+dfisht 297F+dfr 1D521+Dfr 1D507+dgr 03B4+Dgr 0394+dHar 2965+dharl 21C3+dharr 21C2+DiacriticalAcute 00B4+DiacriticalDot 02D9+DiacriticalDoubleAcute 02DD+DiacriticalGrave 0060+DiacriticalTilde 02DC+diam 22C4+diamond 22C4+Diamond 22C4+diamondsuitUB diams+diams 2666+die 00A8+DifferentialD 2146+digamma 03DD+disin 22F2+divUM divide+divide 00F7+divideontimes 22C7+divonx 22C7+djcy 0452+DJcy 0402+dlcorn 231E+dlcrop 230D+dollar 0024+dopf 1D555+Dopf 1D53B+dot 02D9+Dot 00A8+DotDot 20DC+doteq 2250+doteqdot 2251+DotEqual 2250+dotminus 2238+dotplus 2214+dotsquare 22A1+doublebarwedge 2306+DoubleContourIntegral 222F+DoubleDot 00A8+DoubleDownArrow 21D3+DoubleLeftArrow 21D0+DoubleLeftRightArrow 21D4+DoubleLeftTee 2AE4+DoubleLongLeftArrow 27F8+DoubleLongLeftRightArrow 27FA+DoubleLongRightArrow 27F9+DoubleRightArrow 21D2+DoubleRightTee 22A8+DoubleUpArrow 21D1+DoubleUpDownArrow 21D5+DoubleVerticalBar 2225+downarrowUM darr+Downarrow 21D3+DownArrowUM darr+DownArrowBar 2913+DownArrowUpArrow 21F5+DownBreve 0311+downdownarrows 21CA+downharpoonleft 21C3+downharpoonright 21C2+DownLeftRightVector 2950+DownLeftTeeVector 295E+DownLeftVector 21BD+DownLeftVectorBar 2956+DownRightTeeVector 295F+DownRightVector 21C1+DownRightVectorBar 2957+DownTee 22A4+DownTeeArrow 21A7+drbkarow 2910+drcorn 231F+drcrop 230C+dscr 1D4B9+Dscr 1D49F+dscy 0455+DScy 0405+dsol 29F6+dstrok 0111+Dstrok 0110+dtdot 22F1+dtri 25BF+dtrif 25BE+duarr 21F5+duhar 296F+dwangle 29A6+dzcy 045F+DZcy 040F+dzigrarr 27FF+eacgr 03AD+Eacgr 0388+eacute 00E9+Eacute 00C9+easter 2A6E+ecaron 011B+Ecaron 011A+ecir 2256+ecirc 00EA+Ecirc 00CA+ecolon 2255+ecy 044D+Ecy 042D+eDDot 2A77+edot 0117+eDot 2251+Edot 0116+ee 2147+eeacgr 03AE+EEacgr 0389+eegr 03B7+EEgr 0397+efDot 2252+efr 1D522+Efr 1D508+eg 2A9A+egr 03B5+Egr 0395+egrave 00E8+Egrave 00C8+egs 2A96+egsdot 2A98+el 2A99+Element 2208+elinters 23E7+ell 2113+els 2A95+elsdot 2A97+emacr 0113+Emacr 0112+empty 2205+emptyset 2205+EmptySmallSquare 25FB+emptyv 2205+EmptyVerySmallSquare 25AB+emsp 2003+emsp13 2004+emsp14 2005+eng 014B+ENG 014A+ensp 2002+eogon 0119+Eogon 0118+eopf 1D556+Eopf 1D53C+epar 22D5+eparsl 29E3+eplus 2A71+epsi 03B5+epsilon 03B5+Epsilon 0395+epsiv 03F5+eqcirc 2256+eqcolon 2255+eqsim 2242+eqslantgtr 2A96+eqslantless 2A95+Equal 2A75+equals 003D+EqualTilde 2242+equest 225F+Equilibrium 21CC+equiv 2261+equivDD 2A78+eqvparsl 29E5+erarr 2971+erDot 2253+escr 212F+Escr 2130+esdot 2250+esim 2242+Esim 2A73+eta 03B7+Eta 0397+eth 00F0+ETH 00D0+euml 00EB+Euml 00CB+euro 20AC+excl 0021+exist 2203+Exists 2203+expectation 2130+exponentiale 2147+ExponentialE 2147+fallingdotseq 2252+fcy 0444+Fcy 0424+female 2640+ffilig FB03+fflig FB00+ffllig FB04+ffr 1D523+Ffr 1D509+filig FB01+FilledSmallSquare 25FC+FilledVerySmallSquare 25AA+fjlig 0066  006A+flat 266D+fllig FB02+fltns 25B1+fnof 0192+fopf 1D557+Fopf 1D53D+forall 2200+ForAll 2200+fork 22D4+forkv 2AD9+Fouriertrf 2131+fpartint 2A0D+frac12 00BD+frac13 2153+frac14 00BC+frac15 2155+frac16 2159+frac18 215B+frac23 2154+frac25 2156+frac34 00BE+frac35 2157+frac38 215C+frac45 2158+frac56 215A+frac58 215D+frac78 215E+frasl 2044+frown 2322+fscr 1D4BB+Fscr 2131+gacute 01F5+gamma 03B3+Gamma 0393+gammad 03DD+Gammad 03DC+gap 2A86+gbreve 011F+Gbreve 011E+Gcedil 0122+gcirc 011D+Gcirc 011C+gcy 0433+Gcy 0413+gdot 0121+Gdot 0120+ge 2265+gE 2267+gel 22DB+gEl 2A8C+geq 2265+geqq 2267+geqslant 2A7E+ges 2A7E+gescc 2AA9+gesdot 2A80+gesdoto 2A82+gesdotol 2A84+gesl 22DB  FE00+gesles 2A94+gfr 1D524+Gfr 1D50A+gg 226B+Gg 22D9+ggg 22D9+ggr 03B3+Ggr 0393+gimel 2137+gjcy 0453+GJcy 0403+gl 2277+gla 2AA5+glE 2A92+glj 2AA4+gnap 2A8A+gnapprox 2A8A+gne 2A88+gnE 2269+gneq 2A88+gneqq 2269+gnsim 22E7+gopf 1D558+Gopf 1D53E+grave 0060+GreaterEqual 2265+GreaterEqualLess 22DB+GreaterFullEqual 2267+GreaterGreater 2AA2+GreaterLess 2277+GreaterSlantEqual 2A7E+GreaterTilde 2273+gscr 210A+Gscr 1D4A2+gsim 2273+gsime 2A8E+gsiml 2A90+gt 003E+Gt 226B+GT 003E+gtcc 2AA7+gtcir 2A7A+gtdot 22D7+gtlPar 2995+gtquest 2A7C+gtrapprox 2A86+gtrarr 2978+gtrdot 22D7+gtreqless 22DB+gtreqqless 2A8C+gtrless 2277+gtrsim 2273+gvertneqq 2269  FE00+gvnE 2269  FE00+Hacek 02C7+hairsp 200A+half 00BD+hamilt 210B+hardcy 044A+HARDcy 042A+harr 2194+hArr 21D4+harrcir 2948+harrw 21AD+Hat 005E+hbar 210F+hcirc 0125+Hcirc 0124+hearts 2665+heartsuitUB hearts+hellip 2026+hercon 22B9+hfr 1D525+Hfr 210C+HilbertSpace 210B+hksearow 2925+hkswarow 2926+hoarr 21FF+homtht 223B+hookleftarrow 21A9+hookrightarrow 21AA+hopf 1D559+Hopf 210D+horbar 2015+HorizontalLine 2500+hscr 1D4BD+Hscr 210B+hslash 210F+hstrok 0127+Hstrok 0126+HumpDownHump 224E+HumpEqual 224F+hybull 2043+hyphen 2010+iacgr 03AF+Iacgr 038A+iacute 00ED+Iacute 00CD+ic 2063+icirc 00EE+Icirc 00CE+icy 0438+Icy 0418+idiagr 0390+idigr 03CA+Idigr 03AA+Idot 0130+iecy 0435+IEcy 0415+iexcl 00A1+iff 21D4+ifr 1D526+Ifr 2111+igr 03B9+Igr 0399+igrave 00EC+Igrave 00CC+ii 2148+iiiint 2A0C+iiint 222D+iinfin 29DC+iiota 2129+ijlig 0133+IJlig 0132+Im 2111+imacr 012B+Imacr 012A+image 2111+ImaginaryI 2148+imagline 2110+imagpart 2111+imath 0131+imof 22B7+imped 01B5+Implies 21D2+in 2208+incare 2105+infin 221E+infintie 29DD+inodot 0131+int 222B+Int 222C+intcal 22BA+integers 2124+Integral 222B+intercal 22BA+Intersection 22C2+intlarhk 2A17+intprod 2A3C+InvisibleComma 2063+InvisibleTimes 2062+iocy 0451+IOcy 0401+iogon 012F+Iogon 012E+iopf 1D55A+Iopf 1D540+iota 03B9+Iota 0399+iprod 2A3C+iquest 00BF+iscr 1D4BE+Iscr 2110+isin 2208+isindot 22F5+isinE 22F9+isins 22F4+isinsv 22F3+isinv 2208+it 2062+itilde 0129+Itilde 0128+iukcyUkrainian+IukcyUkrainian+iuml 00EF+Iuml 00CF+jcirc 0135+Jcirc 0134+jcy 0439+Jcy 0419+jfr 1D527+Jfr 1D50D+jmath 0237+jopf 1D55B+Jopf 1D541+jscr 1D4BF+Jscr 1D4A5+jsercy 0458+Jsercy 0408+jukcyUkrainian+JukcyUkrainian+kappa 03BA+Kappa 039A+kappav 03F0+kcedil 0137+Kcedil 0136+kcy 043A+Kcy 041A+kfr 1D528+Kfr 1D50E+kgr 03BA+Kgr 039A+kgreen 0138+khcy 0445+KHcy 0425+khgr 03C7+KHgr 03A7+kjcy 045C+KJcy 040C+kopf 1D55C+Kopf 1D542+kscr 1D4C0+Kscr 1D4A6+lAarr 21DA+lacute 013A+Lacute 0139+laemptyv 29B4+lagran 2112+lambda 03BB+Lambda 039B+lang 27E8+Lang 27EA+langd 2991+langle 27E8+lap 2A85+Laplacetrf 2112+laquo 00AB+larr 2190+lArr 21D0+Larr 219E+larrb 21E4+larrbfs 291F+larrfs 291D+larrhk 21A9+larrlp 21AB+larrpl 2939+larrsim 2973+larrtl 21A2+lat 2AAB+latail 2919+lAtail 291B+late 2AAD+lates 2AAD  FE00+lbarr 290C+lBarr 290E+lbbrk 2772+lbraceUM lcub+lbrackUM lsqb+lbrke 298B+lbrksld 298F+lbrkslu 298D+lcaron 013E+Lcaron 013D+lcedil 013C+Lcedil 013B+lceil 2308+lcub 007B+lcy 043B+Lcy 041B+ldca 2936+ldquo 201C+ldquor 201E+ldrdhar 2967+ldrushar 294B+ldsh 21B2+le 2264+lE 2266+LeftAngleBracket 27E8+leftarrowUM larr+Leftarrow 21D0+LeftArrowUM larr+LeftArrowBar 21E4+LeftArrowRightArrow 21C6+leftarrowtail 21A2+LeftCeiling 2308+LeftDoubleBracket 27E6+LeftDownTeeVector 2961+LeftDownVector 21C3+LeftDownVectorBar 2959+LeftFloor 230A+leftharpoondown 21BD+leftharpoonup 21BC+leftleftarrows 21C7+leftrightarrow 2194+Leftrightarrow 21D4+LeftRightArrow 2194+leftrightarrows 21C6+leftrightharpoons 21CB+leftrightsquigarrow 21AD+LeftRightVector 294E+LeftTee 22A3+LeftTeeArrow 21A4+LeftTeeVector 295A+leftthreetimes 22CB+LeftTriangle 22B2+LeftTriangleBar 29CF+LeftTriangleEqual 22B4+LeftUpDownVector 2951+LeftUpTeeVector 2960+LeftUpVector 21BF+LeftUpVectorBar 2958+LeftVector 21BC+LeftVectorBar 2952+leg 22DA+lEg 2A8B+leq 2264+leqq 2266+leqslant 2A7D+les 2A7D+lescc 2AA8+lesdot 2A7F+lesdoto 2A81+lesdotor 2A83+lesg 22DA  FE00+lesges 2A93+lessapprox 2A85+lessdot 22D6+lesseqgtr 22DA+lesseqqgtr 2A8B+LessEqualGreater 22DA+LessFullEqual 2266+LessGreater 2276+lessgtr 2276+LessLess 2AA1+lesssim 2272+LessSlantEqual 2A7D+LessTilde 2272+lfisht 297C+lfloor 230A+lfr 1D529+Lfr 1D50F+lg 2276+lgE 2A91+lgr 03BB+Lgr 039B+lHar 2962+lhard 21BD+lharu 21BC+lharul 296A+lhblk 2584+ljcy 0459+LJcy 0409+ll 226A+Ll 22D8+llarr 21C7+llcorner 231E+Lleftarrow 21DA+llhard 296B+lltri 25FA+lmidot 0140+Lmidot 013F+lmoust 23B0+lmoustache 23B0+lnap 2A89+lnapprox 2A89+lne 2A87+lnE 2268+lneq 2A87+lneqq 2268+lnsim 22E6+loang 27EC+loarr 21FD+lobrk 27E6+longleftarrow 27F5+Longleftarrow 27F8+LongLeftArrow 27F5+longleftrightarrow 27F7+Longleftrightarrow 27FA+LongLeftRightArrow 27F7+longmapsto 27FC+longrightarrow 27F6+Longrightarrow 27F9+LongRightArrow 27F6+looparrowleft 21AB+looparrowright 21AC+lopar 2985+lopf 1D55D+Lopf 1D543+loplus 2A2D+lotimes 2A34+lowast 2217+lowbar 005F+LowerLeftArrow 2199+LowerRightArrow 2198+loz 25CA+lozengeUB loz+lozf 29EB+lpar 0028+lparlt 2993+lrarr 21C6+lrcorner 231F+lrhar 21CB+lrhard 296D+lrm 200E+lrtri 22BF+lsaquo 2039+lscr 1D4C1+Lscr 2112+lsh 21B0+Lsh 21B0+lsim 2272+lsime 2A8D+lsimg 2A8F+lsqb 005B+lsquo 2018+lsquor 201A+lstrok 0142+Lstrok 0141+lt 003C+Lt 226A+LT 003C+ltcc 2AA6+ltcir 2A79+ltdot 22D6+lthree 22CB+ltimes 22C9+ltlarr 2976+ltquest 2A7B+ltri 25C3+ltrie 22B4+ltrif 25C2+ltrPar 2996+lurdshar 294A+luruhar 2966+lvertneqq 2268  FE00+lvnE 2268  FE00+macr 00AF+male 2642+malt 2720+malteseUB malt+map 21A6+Map 2905+mapsto 21A6+mapstodown 21A7+mapstoleft 21A4+mapstoup 21A5+marker 25AE+mcomma 2A29+mcy 043C+Mcy 041C+mdash 2014+mDDot 223A+measuredangle 2221+MediumSpace 205F+Mellintrf 2133+mfr 1D52A+Mfr 1D510+mgr 03BC+Mgr 039C+mho 2127+micro 00B5+mid 2223+midast 002A+midcir 2AF0+middot 00B7+minus 2212+minusb 229F+minusd 2238+minusdu 2A2A+MinusPlus 2213+mlcp 2ADB+mldr 2026+mnplus 2213+models 22A7+mopf 1D55E+Mopf 1D544+mp 2213+mscr 1D4C2+Mscr 2133+mstpos 223E+mu 03BC+Mu 039C+multimap 22B8+mumap 22B8+nabla 2207+nacute 0144+Nacute 0143+nang 2220  20D2+nap 2249+napE 2A70  0338+napid 224B  0338+napos 0149+napprox 2249+natur 266E+naturalUB natur+naturals 2115+nbsp 00A0+nbump 224E  0338+nbumpe 224F  0338+ncap 2A43+ncaron 0148+Ncaron 0147+ncedil 0146+Ncedil 0145+ncong 2247+ncongdot 2A6D  0338+ncup 2A42+ncy 043D+Ncy 041D+ndash 2013+ne 2260+nearhk 2924+nearr 2197+neArr 21D7+nearrow 2197+nedot 2250  0338+NegativeMediumSpace 200B+NegativeThickSpace 200B+NegativeThinSpace 200B+NegativeVeryThinSpace 200B+nequiv 2262+nesear 2928+nesim 2242  0338+NestedGreaterGreater 226B+NestedLessLess 226A+NewLine 000A+nexist 2204+nexists 2204+nfr 1D52B+Nfr 1D511+nge 2271+ngE 2267  0338+ngeq 2271+ngeqq 2267  0338+ngeqslant 2A7E  0338+nges 2A7E  0338+nGg 22D9  0338+ngr 03BD+Ngr 039D+ngsim 2275+ngt 226F+nGt 226B  20D2+ngtr 226F+nGtv 226B  0338+nharr 21AE+nhArr 21CE+nhpar 2AF2+ni 220B+nis 22FC+nisd 22FA+niv 220B+njcy 045A+NJcy 040A+nlarr 219A+nlArr 21CD+nldr 2025+nle 2270+nlE 2266  0338+nleftarrow 219A+nLeftarrow 21CD+nleftrightarrow 21AE+nLeftrightarrow 21CE+nleq 2270+nleqq 2266  0338+nleqslant 2A7D  0338+nles 2A7D  0338+nless 226E+nLl 22D8  0338+nlsim 2274+nlt 226E+nLt 226A  20D2+nltri 22EA+nltrie 22EC+nLtv 226A  0338+nmid 2224+NoBreak 2060+NonBreakingSpaceUM nbsp+nopf 1D55F+Nopf 2115+not 00AC+Not 2AEC+NotCongruent 2262+NotCupCap 226D+NotDoubleVerticalBar 2226+NotElement 2209+NotEqual 2260+NotEqualTilde 2242  0338+NotExists 2204+NotGreater 226F+NotGreaterEqual 2271+NotGreaterFullEqual 2267  0338+NotGreaterGreater 226B  0338+NotGreaterLess 2279+NotGreaterSlantEqual 2A7E  0338+NotGreaterTilde 2275+NotHumpDownHump 224E  0338+NotHumpEqual 224F  0338+notin 2209+notindot 22F5  0338+notinE 22F9  0338+notinva 2209+notinvb 22F7+notinvc 22F6+NotLeftTriangle 22EA+NotLeftTriangleBar 29CF  0338+NotLeftTriangleEqual 22EC+NotLess 226E+NotLessEqual 2270+NotLessGreater 2278+NotLessLess 226A  0338+NotLessSlantEqual 2A7D  0338+NotLessTilde 2274+NotNestedGreaterGreater 2AA2  0338+NotNestedLessLess 2AA1  0338+notni 220C+notniva 220C+notnivb 22FE+notnivc 22FD+NotPrecedes 2280+NotPrecedesEqual 2AAF  0338+NotPrecedesSlantEqual 22E0+NotReverseElement 220C+NotRightTriangle 22EB+NotRightTriangleBar 29D0  0338+NotRightTriangleEqual 22ED+NotSquareSubset 228F  0338+NotSquareSubsetEqual 22E2+NotSquareSuperset 2290  0338+NotSquareSupersetEqual 22E3+NotSubset 2282  20D2+NotSubsetEqual 2288+NotSucceeds 2281+NotSucceedsEqual 2AB0  0338+NotSucceedsSlantEqual 22E1+NotSucceedsTilde 227F  0338+NotSuperset 2283  20D2+NotSupersetEqual 2289+NotTilde 2241+NotTildeEqual 2244+NotTildeFullEqual 2247+NotTildeTilde 2249+NotVerticalBar 2224+npar 2226+nparallel 2226+nparsl 2AFD  20E5+npart 2202  0338+npolint 2A14+npr 2280+nprcue 22E0+npre 2AAF  0338+nprec 2280+npreceq 2AAF  0338+nrarr 219B+nrArr 21CF+nrarrc 2933  0338+nrarrw 219D  0338+nrightarrow 219B+nRightarrow 21CF+nrtri 22EB+nrtrie 22ED+nsc 2281+nsccue 22E1+nsce 2AB0  0338+nscr 1D4C3+Nscr 1D4A9+nshortmid 2224+nshortparallel 2226+nsim 2241+nsime 2244+nsimeq 2244+nsmid 2224+nspar 2226+nsqsube 22E2+nsqsupe 22E3+nsub 2284+nsube 2288+nsubE 2AC5  0338+nsubset 2282  20D2+nsubseteq 2288+nsubseteqq 2AC5  0338+nsucc 2281+nsucceq 2AB0  0338+nsup 2285+nsupe 2289+nsupE 2AC6  0338+nsupset 2283  20D2+nsupseteq 2289+nsupseteqq 2AC6  0338+ntgl 2279+ntilde 00F1+Ntilde 00D1+ntlg 2278+ntriangleleft 22EA+ntrianglelefteq 22EC+ntriangleright 22EB+ntrianglerighteq 22ED+nu 03BD+Nu 039D+num 0023+numero 2116+numsp 2007+nvap 224D  20D2+nvdash 22AC+nvDash 22AD+nVdash 22AE+nVDash 22AF+nvge 2265  20D2+nvgt 003E  20D2+nvHarr 2904+nvinfin 29DE+nvlArr 2902+nvle 2264  20D2+nvlt 003C  20D2+nvltrie 22B4  20D2+nvrArr 2903+nvrtrie 22B5  20D2+nvsim 223C  20D2+nwarhk 2923+nwarr 2196+nwArr 21D6+nwarrow 2196+nwnear 2927+oacgr 03CC+Oacgr 038C+oacute 00F3+Oacute 00D3+oast 229B+ocir 229A+ocirc 00F4+Ocirc 00D4+ocy 043E+Ocy 041E+odash 229D+odblac 0151+Odblac 0150+odiv 2A38+odot 2299+odsold 29BC+oelig 0153+OElig 0152+ofcir 29BF+ofr 1D52C+Ofr 1D512+ogon 02DB+ogr 03BF+Ogr 039F+ograve 00F2+Ograve 00D2+ogt 29C1+ohacgr 03CE+OHacgr 038F+ohbar 29B5+ohgr 03C9+OHgr 03A9+ohm 03A9+oint 222E+olarr 21BA+olcir 29BE+olcross 29BB+oline 203E+olt 29C0+omacr 014D+Omacr 014C+omega 03C9+Omega 03A9+omicron 03BF+Omicron 039F+omid 29B6+ominus 2296+oopf 1D560+Oopf 1D546+opar 29B7+OpenCurlyDoubleQuoteUM ldquo+OpenCurlyQuoteUM lsquo+operp 29B9+oplus 2295+or 2228+Or 2A54+orarr 21BB+ord 2A5D+order 2134+orderof 2134+ordf 00AA+ordm 00BA+origof 22B6+oror 2A56+orslope 2A57+orv 2A5B+oS 24C8+oscr 2134+Oscr 1D4AA+oslash 00F8+Oslash 00D8+osol 2298+otilde 00F5+Otilde 00D5+otimes 2297+Otimes 2A37+otimesas 2A36+ouml 00F6+Ouml 00D6+ovbar 233D+OverBar 203E+OverBrace 23DE+OverBracket 23B4+OverParenthesis 23DC+par 2225+para 00B6+parallel 2225+parsim 2AF3+parsl 2AFD+part 2202+PartialD 2202+pcy 043F+Pcy 041F+percnt 0025+period 002E+permil 2030+perp 22A5+pertenk 2031+pfr 1D52D+Pfr 1D513+pgr 03C0+Pgr 03A0+phgr 03C6+PHgr 03A6+phi 03C6+Phi 03A6+phiv 03D5+phmmat 2133+phone 260E+pi 03C0+Pi 03A0+pitchfork 22D4+piv 03D6+planck 210F+planckh 210E+plankv 210F+plus 002B+plusacir 2A23+plusb 229E+pluscir 2A22+plusdo 2214+plusdu 2A25+pluse 2A72+PlusMinusUM plusmn+plusmn 00B1+plussim 2A26+plustwo 2A27+pmUM plusmn+Poincareplane 210C+pointint 2A15+popf 1D561+Popf 2119+pound 00A3+pr 227A+Pr 2ABB+prap 2AB7+prcue 227C+pre 2AAF+prE 2AB3+prec 227A+precapprox 2AB7+preccurlyeq 227C+Precedes 227A+PrecedesEqual 2AAF+PrecedesSlantEqual 227C+PrecedesTilde 227E+preceq 2AAF+precnapprox 2AB9+precneqq 2AB5+precnsim 22E8+precsim 227E+prime 2032+Prime 2033+primes 2119+prnap 2AB9+prnE 2AB5+prnsim 22E8+prod 220F+Product 220F+profalar 232E+profline 2312+profsurf 2313+prop 221D+Proportion 2237+Proportional 221D+propto 221D+prsim 227E+prurel 22B0+pscr 1D4C5+Pscr 1D4AB+psgr 03C8+PSgr 03A8+psi 03C8+Psi 03A8+puncsp 2008+qfr 1D52E+Qfr 1D514+qint 2A0C+qopf 1D562+Qopf 211A+qprime 2057+qscr 1D4C6+Qscr 1D4AC+quaternions 210D+quatint 2A16+quest 003F+questeq 225F+quot 0022+QUOT 0022+rAarr 21DB+race 223D  0331+racute 0155+Racute 0154+radic 221A+raemptyv 29B3+rang 27E9+Rang 27EB+rangd 2992+range 29A5+rangle 27E9+raquo 00BB+rarr 2192+rArr 21D2+Rarr 21A0+rarrap 2975+rarrb 21E5+rarrbfs 2920+rarrc 2933+rarrfs 291E+rarrhk 21AA+rarrlp 21AC+rarrpl 2945+rarrsim 2974+rarrtl 21A3+Rarrtl 2916+rarrw 219D+ratail 291A+rAtail 291C+ratio 2236+rationals 211A+rbarr 290D+rBarr 290F+RBarr 2910+rbbrk 2773+rbraceUM rcub+rbrackUM rsqb+rbrke 298C+rbrksld 298E+rbrkslu 2990+rcaron 0159+Rcaron 0158+rcedil 0157+Rcedil 0156+rceil 2309+rcub 007D+rcy 0440+Rcy 0420+rdca 2937+rdldhar 2969+rdquo 201D+rdquor 201D+rdsh 21B3+Re 211C+real 211C+realine 211B+realpart 211C+reals 211D+rect 25AD+reg 00AE+REG 00AE+ReverseElement 220B+ReverseEquilibrium 21CB+ReverseUpEquilibrium 296F+rfisht 297D+rfloor 230B+rfr 1D52F+Rfr 211C+rgr 03C1+Rgr 03A1+rHar 2964+rhard 21C1+rharu 21C0+rharul 296C+rho 03C1+Rho 03A1+rhov 03F1+RightAngleBracket 27E9+rightarrowUM rarr+Rightarrow 21D2+RightArrowUM rarr+RightArrowBar 21E5+RightArrowLeftArrow 21C4+rightarrowtail 21A3+RightCeiling 2309+RightDoubleBracket 27E7+RightDownTeeVector 295D+RightDownVector 21C2+RightDownVectorBar 2955+RightFloor 230B+rightharpoondown 21C1+rightharpoonup 21C0+rightleftarrows 21C4+rightleftharpoons 21CC+rightrightarrows 21C9+rightsquigarrow 219D+RightTee 22A2+RightTeeArrow 21A6+RightTeeVector 295B+rightthreetimes 22CC+RightTriangle 22B3+RightTriangleBar 29D0+RightTriangleEqual 22B5+RightUpDownVector 294F+RightUpTeeVector 295C+RightUpVector 21BE+RightUpVectorBar 2954+RightVector 21C0+RightVectorBar 2953+ring 02DA+risingdotseq 2253+rlarr 21C4+rlhar 21CC+rlm 200F+rmoust 23B1+rmoustache 23B1+rnmid 2AEE+roang 27ED+roarr 21FE+robrk 27E7+ropar 2986+ropf 1D563+Ropf 211D+roplus 2A2E+rotimes 2A35+RoundImplies 2970+rpar 0029+rpargt 2994+rppolint 2A12+rrarr 21C9+Rrightarrow 21DB+rsaquo 203A+rscr 1D4C7+Rscr 211B+rsh 21B1+Rsh 21B1+rsqb 005D+rsquo 2019+rsquor 2019+rthree 22CC+rtimes 22CA+rtri 25B9+rtrie 22B5+rtrif 25B8+rtriltri 29CE+RuleDelayed 29F4+ruluhar 2968+rx 211E+sacute 015B+Sacute 015A+sbquo 201A+sc 227B+Sc 2ABC+scap 2AB8+scaron 0161+Scaron 0160+sccue 227D+sce 2AB0+scE 2AB4+scedil 015F+Scedil 015E+scirc 015D+Scirc 015C+scnap 2ABA+scnE 2AB6+scnsim 22E9+scpolint 2A13+scsim 227F+scy 0441+Scy 0421+sdot 22C5+sdotb 22A1+sdote 2A66+searhk 2925+searr 2198+seArr 21D8+searrow 2198+sect 00A7+semi 003B+seswar 2929+setminus 2216+setmn 2216+sext 2736+sfgr 03C2+sfr 1D530+Sfr 1D516+sfrown 2322+sgr 03C3+Sgr 03A3+sharp 266F+shchcy 0449+SHCHcy 0429+shcy 0448+SHcy 0428+ShortDownArrow 2193+ShortLeftArrow 2190+shortmid 2223+shortparallel 2225+ShortRightArrow 2192+ShortUpArrow 2191+shy 00AD+sigma 03C3+Sigma 03A3+sigmaf 03C2+sigmav 03C2+sim 223C+simdot 2A6A+sime 2243+simeq 2243+simg 2A9E+simgE 2AA0+siml 2A9D+simlE 2A9F+simne 2246+simplus 2A24+simrarr 2972+slarr 2190+SmallCircle 2218+smallsetminus 2216+smashp 2A33+smeparsl 29E4+smid 2223+smile 2323+smt 2AAA+smte 2AAC+smtes 2AAC  FE00+softcy 044C+SOFTcy 042C+sol 002F+solb 29C4+solbar 233F+sopf 1D564+Sopf 1D54A+spades 2660+spadesuitUB spades+spar 2225+sqcap 2293+sqcaps 2293  FE00+sqcup 2294+sqcups 2294  FE00+Sqrt 221A+sqsub 228F+sqsube 2291+sqsubset 228F+sqsubseteq 2291+sqsup 2290+sqsupe 2292+sqsupset 2290+sqsupseteq 2292+squ 25A1+square 25A1+Square 25A1+SquareIntersection 2293+SquareSubset 228F+SquareSubsetEqual 2291+SquareSuperset 2290+SquareSupersetEqual 2292+SquareUnion 2294+squarf 25AA+squf 25AA+srarr 2192+sscr 1D4C8+Sscr 1D4AE+ssetmn 2216+ssmile 2323+sstarf 22C6+star 2606+Star 22C6+starf 2605+straightepsilon 03F5+straightphi 03D5+strns 00AF+sub 2282+Sub 22D0+subdot 2ABD+sube 2286+subE 2AC5+subedot 2AC3+submult 2AC1+subne 228A+subnE 2ACB+subplus 2ABF+subrarr 2979+subset 2282+Subset 22D0+subseteq 2286+subseteqq 2AC5+SubsetEqual 2286+subsetneq 228A+subsetneqq 2ACB+subsim 2AC7+subsub 2AD5+subsup 2AD3+succ 227B+succapprox 2AB8+succcurlyeq 227D+Succeeds 227B+SucceedsEqual 2AB0+SucceedsSlantEqual 227D+SucceedsTilde 227F+succeq 2AB0+succnapprox 2ABA+succneqq 2AB6+succnsim 22E9+succsim 227F+SuchThat 220B+sum 2211+Sum 2211+sung 266A+sup 2283+Sup 22D1+sup1 00B9+sup2 00B2+sup3 00B3+supdot 2ABE+supdsub 2AD8+supe 2287+supE 2AC6+supedot 2AC4+Superset 2283+SupersetEqual 2287+suphsol 27C9+suphsub 2AD7+suplarr 297B+supmult 2AC2+supne 228B+supnE 2ACC+supplus 2AC0+supset 2283+Supset 22D1+supseteq 2287+supseteqq 2AC6+supsetneq 228B+supsetneqq 2ACC+supsim 2AC8+supsub 2AD4+supsup 2AD6+swarhk 2926+swarr 2199+swArr 21D9+swarrow 2199+swnwar 292A+szlig 00DF+Tab 0009+target 2316+tau 03C4+Tau 03A4+tbrk 23B4+tcaron 0165+Tcaron 0164+tcedil 0163+Tcedil 0162+tcy 0442+Tcy 0422+tdot 20DB+telrec 2315+tfr 1D531+Tfr 1D517+tgr 03C4+Tgr 03A4+there4 2234+therefore 2234+Therefore 2234+theta 03B8+Theta 0398+thetasym 03D1+thetav 03D1+thgr 03B8+THgr 0398+thickapprox 2248+thicksim 223C+ThickSpace 205F  200A+thinsp 2009+ThinSpaceUB thinsp+thkap 2248+thksim 223C+thorn 00FE+THORN 00DE+tilde 02DC+Tilde 223C+TildeEqual 2243+TildeFullEqual 2245+TildeTilde 2248+times 00D7+timesb 22A0+timesbar 2A31+timesd 2A30+tint 222D+toea 2928+top 22A4+topbot 2336+topcir 2AF1+topf 1D565+Topf 1D54B+topfork 2ADA+tosa 2929+tprime 2034+trade 2122+TRADE 2122+triangleUB utri+triangledownUB dtri+triangleleftUB ltri+trianglelefteq 22B4+triangleq 225C+trianglerightUB rtri+trianglerighteq 22B5+tridot 25EC+trie 225C+triminus 2A3A+TripleDot 20DB+triplus 2A39+trisb 29CD+tritime 2A3B+trpezium 23E2+tscr 1D4C9+Tscr 1D4AF+tscy 0446+TScy 0426+tshcy 045B+TSHcy 040B+tstrok 0167+Tstrok 0166+twixt 226C+twoheadleftarrow 219E+twoheadrightarrow 21A0+uacgr 03CD+UacgrUpsilon+uacute 00FA+UacuteU with acute+uarr 2191+uArr 21D1+Uarr 219F+Uarrocir 2949+ubrcy 045E+UbrcyU+ubreve 016D+UbreveU+ucirc 00FB+UcircU with circumflex+ucy 0443+UcyU+udarr 21C5+udblac 0171+UdblacU+udhar 296E+udiagr 03B0+udigr 03CB+UdigrUpsilon+ufisht 297E+ufr 1D532+UfrU+ugr 03C5+UgrUpsilon+ugrave 00F9+UgraveU with grave+uHar 2963+uharl 21BF+uharr 21BE+uhblk 2580+ulcorn 231C+ulcorner 231C+ulcrop 230F+ultri 25F8+umacr 016B+UmacrU+uml 00A8+UnderBar 005F+UnderBrace 23DF+UnderBracket 23B5+UnderParenthesis 23DD+Union 22C3+UnionPlus 228E+uogon 0173+UogonU+uopf 1D566+UopfU+uparrowUM uarr+Uparrow 21D1+UpArrowUM uarr+UpArrowBar 2912+UpArrowDownArrow 21C5+updownarrow 2195+Updownarrow 21D5+UpDownArrow 2195+UpEquilibrium 296E+upharpoonleft 21BF+upharpoonright 21BE+uplus 228E+UpperLeftArrow 2196+UpperRightArrow 2197+upsi 03C5+UpsiUpsilon capital Upsilon+upsih 03D2+upsilon 03C5+UpsilonUgr+UpTee 22A5+UpTeeArrow 21A5+upuparrows 21C8+urcorn 231D+urcorner 231D+urcrop 230E+uring 016F+UringU+urtri 25F9+uscr 1D4CA+UscrU+utdot 22F0+utilde 0169+UtildeU+utri 25B5+utrif 25B4+uuarr 21C8+uuml 00FC+UumlU with diaeresis+uwangle 29A7+vangrt 299C+varepsilon 03F5+varkappa 03F0+varnothing 2205+varphi 03D5+varpi 03D6+varpropto 221D+varr 2195+vArrUpdownarrow A: up&down dbl arrow+varrho 03F1+varsigma 03C2+varsubsetneq 228A  FE00+varsubsetneqq 2ACB  FE00+varsupsetneq 228B  FE00+varsupsetneqq 2ACC  FE00+vartheta 03D1+vartriangleleft 22B2+vartriangleright 22B3+vBar 2AE8+Vbar 2AEB+vBarv 2AE9+vcy 0432+Vcy 0412+vdash 22A2+vDash 22A8+Vdash 22A9+VDash 22AB+Vdashl 2AE6+vee 2228+Vee 22C1+veebar 22BB+veeeq 225A+vellip 22EE+verbar 007C+Verbar 2016+vertUM verbar+Vert 2016+VerticalBar 2223+VerticalLineUM verbar+VerticalSeparator 2758+VerticalTilde 2240+VeryThinSpaceUB hairsp+vfr 1D533+Vfr 1D519+vltri 22B2+vnsub 2282  20D2+vnsup 2283  20D2+vopf 1D567+Vopf 1D54D+vprop 221D+vrtri 22B3+vscr 1D4CB+Vscr 1D4B1+vsubne 228A  FE00+vsubnE 2ACB  FE00+vsupne 228B  FE00+vsupnE 2ACC  FE00+Vvdash 22AA+vzigzag 299A+wcirc 0175+Wcirc 0174+wedbar 2A5F+wedge 2227+Wedge 22C0+wedgeq 2259+weierp 2118+wfr 1D534+Wfr 1D51A+wopf 1D568+Wopf 1D54E+wp 2118+wr 2240+wreath 2240+wscr 1D4CC+Wscr 1D4B2+xcap 22C2+xcirc 25EF+xcup 22C3+xdtri 25BD+xfr 1D535+Xfr 1D51B+xgr 03BE+Xgr 039E+xharr 27F7+xhArr 27FA+xi 03BE+Xi 039E+xlarr 27F5+xlArr 27F8+xmap 27FC+xnis 22FB+xodot 2A00+xopf 1D569+Xopf 1D54F+xoplus 2A01+xotime 2A02+xrarr 27F6+xrArr 27F9+xscr 1D4CD+Xscr 1D4B3+xsqcup 2A06+xuplus 2A04+xutri 25B3+xvee 22C1+xwedge 22C0+yacute 00FD+Yacute 00DD+yacy 044F+YAcy 042F+ycirc 0177+Ycirc 0176+ycy 044B+YcyU+yen 00A5+yfr 1D536+Yfr 1D51C+yicyUkrainian+YIcyUkrainian+yopf 1D56A+Yopf 1D550+yscr 1D4CE+Yscr 1D4B4+yucy 044E+YUcyU+yuml 00FF+Yuml 0178+zacute 017A+Zacute 0179+zcaron 017E+Zcaron 017D+zcy 0437+Zcy 0417+zdot 017C+Zdot 017B+zeetrf 2128+ZeroWidthSpace 200B+zeta 03B6+Zeta 0396+zfr 1D537+Zfr 2128+zgr 03B6+Zgr 0396+zhcy 0436+ZHcy 0416+zigrarr 21DD+zopf 1D56B+Zopf 2124+zscr 1D4CF+Zscr 1D4B5+zwj 200D+zwnj 200C
data/templates/article.jats_publishing view
@@ -174,20 +174,25 @@ $endif$ $if(copyright)$ <permissions>-$if(copyright.statement)$+$for(copyright.statement)$ <copyright-statement>$copyright.statement$</copyright-statement>-$endif$-$if(copyright.year)$+$endfor$+$for(copyright.year)$ <copyright-year>$copyright.year$</copyright-year>-$endif$-$if(copyright.holder)$+$endfor$+$for(copyright.holder)$ <copyright-holder>$copyright.holder$</copyright-holder>-$endif$+$endfor$ $if(copyright.text)$ <license license-type="$copyright.type$" xlink:href="$copyright.link$"> <license-p>$copyright.text$</license-p> </license> $endif$+$for(license)$+<license$if(it.type)$ license-type="${it.type}"$endif$$if(it.link)$ xlink:href="${it.link}"$endif$>+<license-p>$if(it.text)$${it.text}$else$${it}$endif$</license-p>+</license>+$endfor$ </permissions> $endif$ $if(abstract)$
data/templates/default.jats_articleauthoring view
@@ -51,20 +51,25 @@ $endif$ $if(copyright)$ <permissions>-$if(copyright.statement)$+$for(copyright.statement)$ <copyright-statement>$copyright.statement$</copyright-statement>-$endif$-$if(copyright.year)$+$endfor$+$for(copyright.year)$ <copyright-year>$copyright.year$</copyright-year>-$endif$-$if(copyright.holder)$+$endfor$+$for(copyright.holder)$ <copyright-holder>$copyright.holder$</copyright-holder>-$endif$+$endfor$ $if(copyright.text)$ <license license-type="$copyright.type$" xlink:href="$copyright.link$"> <license-p>$copyright.text$</license-p> </license> $endif$+$for(license)$+<license$if(it.type)$ license-type="${it.type}"$endif$$if(it.link)$ xlink:href="${it.link}"$endif$>+<license-p>$if(it.text)$${it.text}$else$${it}$endif$</license-p>+</license>+$endfor$ </permissions> $endif$ <abstract>
data/templates/default.latex view
@@ -210,38 +210,6 @@ \usepackage{fancyvrb} $endif$ \usepackage{xcolor}-\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available-\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}-\hypersetup{-$if(title-meta)$-  pdftitle={$title-meta$},-$endif$-$if(author-meta)$-  pdfauthor={$author-meta$},-$endif$-$if(lang)$-  pdflang={$lang$},-$endif$-$if(subject)$-  pdfsubject={$subject$},-$endif$-$if(keywords)$-  pdfkeywords={$for(keywords)$$keywords$$sep$, $endfor$},-$endif$-$if(colorlinks)$-  colorlinks=true,-  linkcolor={$if(linkcolor)$$linkcolor$$else$Maroon$endif$},-  filecolor={$if(filecolor)$$filecolor$$else$Maroon$endif$},-  citecolor={$if(citecolor)$$citecolor$$else$Blue$endif$},-  urlcolor={$if(urlcolor)$$urlcolor$$else$Blue$endif$},-$else$-  hidelinks,-$endif$-  pdfcreator={LaTeX via pandoc}}-\urlstyle{same} % disable monospaced font for URLs-$if(verbatim-in-note)$-\VerbatimFootnotes % allow verbatim text in footnotes-$endif$ $if(geometry)$ $if(beamer)$ \geometry{$for(geometry)$$geometry$$sep$,$endfor$}@@ -309,8 +277,6 @@ $if(strikeout)$ $-- also used for underline \usepackage[normalem]{ulem}-% Avoid problems with \sout in headers with hyperref-\pdfstringdefDisableCommands{\renewcommand{\sout}{}} $endif$ \setlength{\emergencystretch}{3em} % prevent overfull lines \providecommand{\tightlist}{%@@ -368,7 +334,9 @@ \else \usepackage[bidi=default]{babel} \fi+$if(babel-lang)$ \babelprovide[main,import]{$babel-lang$}+$endif$ $for(babel-otherlangs)$ \babelprovide[import]{$babel-otherlangs$} $endfor$@@ -407,6 +375,38 @@ $if(csquotes)$ \usepackage{csquotes} $endif$+\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}+\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available+\urlstyle{same} % disable monospaced font for URLs+$if(verbatim-in-note)$+\VerbatimFootnotes % allow verbatim text in footnotes+$endif$+\hypersetup{+$if(title-meta)$+  pdftitle={$title-meta$},+$endif$+$if(author-meta)$+  pdfauthor={$author-meta$},+$endif$+$if(lang)$+  pdflang={$lang$},+$endif$+$if(subject)$+  pdfsubject={$subject$},+$endif$+$if(keywords)$+  pdfkeywords={$for(keywords)$$keywords$$sep$, $endfor$},+$endif$+$if(colorlinks)$+  colorlinks=true,+  linkcolor={$if(linkcolor)$$linkcolor$$else$Maroon$endif$},+  filecolor={$if(filecolor)$$filecolor$$else$Maroon$endif$},+  citecolor={$if(citecolor)$$citecolor$$else$Blue$endif$},+  urlcolor={$if(urlcolor)$$urlcolor$$else$Blue$endif$},+$else$+  hidelinks,+$endif$+  pdfcreator={LaTeX via pandoc}}  $if(title)$ \title{$title$$if(thanks)$\thanks{$thanks$}$endif$}
data/templates/default.rst view
@@ -1,19 +1,45 @@ $if(titleblock)$ $titleblock$ -$endif$ $for(author)$-:Author: $author$+:Author: $^$$author$ $endfor$-$if(date)$-:Date:   $date$+$if(authors)$+:Authors:+   $author$ $endif$-$if(author)$--$else$ $if(date)$-+:Date: $^$$date$ $endif$+$if(address)$+:Addresss: $^$$address$+$endif$+$if(contact)$+:Contact: $^$$contact$+$endif$+$if(copyright)$+:Copyright: $^$$copyright$+$endif$+$if(dedication)$+:Dedication: $^$$dedication$+$endif$+$if(organization)$+:Organization: $^$$organization$+$endif$+$if(revision)$+:Revision: $^$$revision$+$endif$+$if(status)$+:Status: $^$$status$+$endif$+$if(version)$+:Version: $^$$version$+$endif$+$if(abstract)$+:Abstract:+   $abstract$+$endif$+ $endif$ $if(rawtex)$ .. role:: raw-latex(raw)
man/pandoc.1 view
@@ -1,5 +1,5 @@ '\" t-.\" Automatically generated by Pandoc 2.17.1+.\" Automatically generated by Pandoc 2.17.1.1 .\" .\" Define V font for inline verbatim, using C font in formats .\" that render this, and otherwise B font.@@ -15,7 +15,7 @@ . ftr VB CB . ftr VBI CBI .\}-.TH "Pandoc User\[cq]s Guide" "" "January 30, 2022" "pandoc 2.17.1.1" ""+.TH "Pandoc User\[cq]s Guide" "" "April 4, 2022" "pandoc 2.18" "" .hy .SH NAME pandoc - general markup converter@@ -260,6 +260,8 @@ .IP \[bu] 2 \f[V]dokuwiki\f[R] (DokuWiki markup) .IP \[bu] 2+\f[V]endnotexml\f[R] (EndNote XML bibliography)+.IP \[bu] 2 \f[V]epub\f[R] (EPUB) .IP \[bu] 2 \f[V]fb2\f[R] (FictionBook2 e-book)@@ -304,6 +306,8 @@ .IP \[bu] 2 \f[V]org\f[R] (Emacs Org mode) .IP \[bu] 2+\f[V]ris\f[R] (RIS bibliography)+.IP \[bu] 2 \f[V]rtf\f[R] (Rich Text Format) .IP \[bu] 2 \f[V]rst\f[R] (reStructuredText)@@ -693,8 +697,9 @@ Read metadata from the supplied YAML (or JSON) file. This option can be used with every input format, but string scalars in the YAML file will always be parsed as Markdown.-Generally, the input will be handled the same as in YAML metadata-blocks.+(If the input format is Markdown or a Markdown variant, then the same+variant will be used to parse the metadata file; if it is a non-Markdown+format, pandoc\[cq]s default Markdown extensions will be used.) This option can be used repeatedly to include multiple metadata files; values in files specified later on the command line will be preferred over those specified in earlier files.@@ -998,9 +1003,8 @@ left alone; the documents they link to will not be incorporated in the document. Limitation: resources that are loaded dynamically through JavaScript-cannot be incorporated; as a result, \f[V]--self-contained\f[R] does not-work with \f[V]--mathjax\f[R], and some advanced features (e.g.\ zoom or-speaker notes) may not work in an offline \[lq]self-contained\[rq]+cannot be incorporated; as a result, some advanced features (e.g.\ zoom+or speaker notes) may not work in an offline \[lq]self-contained\[rq] \f[V]reveal.js\f[R] slide show. .TP \f[V]--html-q-tags\f[R]@@ -4852,7 +4856,8 @@ .TP input formats \f[V]markdown\f[R], \f[V]commonmark\f[R], \f[V]latex\f[R],-\f[V]mediawiki\f[R], \f[V]org\f[R], \f[V]rst\f[R], \f[V]twiki\f[R]+\f[V]mediawiki\f[R], \f[V]org\f[R], \f[V]rst\f[R], \f[V]twiki\f[R],+\f[V]html\f[R] .TP output formats \f[V]markdown\f[R], \f[V]latex\f[R], \f[V]context\f[R], \f[V]rst\f[R]@@ -5208,8 +5213,15 @@ output format. .SS Extension: \f[V]citations\f[R] .PP-Some aspects of Pandoc\[cq]s Markdown citation syntax are also accepted-in \f[V]org\f[R] input.+When the \f[V]citations\f[R] extension is enabled in \f[V]org\f[R],+org-cite and org-ref style citations will be parsed as native pandoc+citations.+.PP+When \f[V]citations\f[R] is enabled in \f[V]docx\f[R], citations+inserted by Zotero or Mendeley or EndNote plugins will be parsed as+native pandoc citations.+(Otherwise, the formatted citations generated by the bibliographic+software will be parsed as regular text.) .SS Extension: \f[V]fancy_lists\f[R] .PP Some aspects of Pandoc\[cq]s Markdown fancy lists are also accepted in@@ -6886,7 +6898,7 @@ For display math, use \f[V]$$\f[R] delimiters. (In this case, the delimiters may be separated from the formula by whitespace.-However, there can be no blank lines betwen the opening and closing+However, there can be no blank lines between the opening and closing \f[V]$$\f[R] delimiters.) .PP TeX math will be printed in all output formats.@@ -8023,6 +8035,11 @@ T}@T{ \&.yaml T}+T{+RIS+T}@T{+\&.ris+T} .TE .PP Note that \f[V].bib\f[R] can be used with both BibTeX and BibLaTeX@@ -8324,6 +8341,23 @@ The bibliography will be inserted after this heading. Note that the \f[V]unnumbered\f[R] class will be added to this heading, so that the section will not be numbered.+.PP+If you want to put the bibliography into a variable in your template,+one way to do that is to put the div with id \f[V]refs\f[R] into a+metadata field, e.g.+.IP+.nf+\f[C]+---+refs: |+   ::: {#refs}+   :::+\&...+\f[R]+.fi+.PP+You can then put the variable \f[V]$refs$\f[R] into your template where+you want the bibliography to be placed. .SS Including uncited items in the bibliography .PP If you want to include items in the bibliography without actually citing@@ -8653,6 +8687,19 @@ .PP Both methods allow incremental and nonincremental lists to be mixed in a single document.+.PP+If you want to include a block-quoted list, you can work around this+behavior by putting the list inside a fenced div, so that it is not the+direct child of the block quote:+.IP+.nf+\f[C]+> ::: wrapper+> - a+> - list in a quote+> :::+\f[R]+.fi .SS Inserting pauses .PP You can add \[lq]pauses\[rq] within a slide by including a paragraph@@ -9598,9 +9645,6 @@ For EPUB, this can be set explicitly by setting the \f[V]identifier\f[R] metadata field (see EPUB Metadata, above). .SH A NOTE ON SECURITY-.PP-If you use pandoc to convert user-contributed content in a web-application, here are some things to keep in mind: .IP "1." 3 Although pandoc itself will not create or modify any files other than those you explicitly ask it create (with the exception of temporary@@ -9629,6 +9673,8 @@ fully isolates pandoc from your file system, by running the pandoc operations in the \f[V]PandocPure\f[R] monad. See the document Using the pandoc API for more details.+(This corresponds to the use of the \f[V]--sandbox\f[R] option on the+command line.) .IP "5." 3 Pandoc\[cq]s parsers can exhibit pathological performance on some corner cases.@@ -9647,8 +9693,8 @@ inject arbitrary HTML. Even if \f[V]raw_html\f[R] is disabled, users can include dangerous content in URLs and attributes.-To be safe, you should run all the generated HTML through an HTML-sanitizer.+To be safe, you should run all HTML generated from untrusted user input+through an HTML sanitizer. .SH AUTHORS .PP Copyright 2006\[en]2022 John MacFarlane (jgm\[at]berkeley.edu).
pandoc.cabal view
@@ -1,6 +1,6 @@ cabal-version:   2.4 name:            pandoc-version:         2.17.1.1+version:         2.18 build-type:      Simple license:         GPL-2.0-or-later license-file:    COPYING.md@@ -11,7 +11,8 @@ stability:       alpha homepage:        https://pandoc.org category:        Text-tested-with:     GHC == 8.6.5, GHC == 8.8.4, GHC == 8.10.7, GHC == 9.0.1+tested-with:     GHC == 8.6.5, GHC == 8.8.4, GHC == 8.10.7, GHC == 9.0.1,+                 GHC == 9.2.2 synopsis:        Conversion between markup formats description:     Pandoc is a Haskell library for converting from one markup                  format to another, and a command-line tool that uses@@ -93,6 +94,8 @@                  data/templates/default.markua                  -- translations                  data/translations/*.yaml+                 -- entities+                 data/docbook-entities.txt                  -- source files for reference.docx                  data/docx/[Content_Types].xml                  data/docx/_rels/.rels@@ -259,6 +262,7 @@                  test/docbook-chapter.docbook                  test/docbook-reader.docbook                  test/docbook-xref.docbook+                 test/endnotexml-reader.xml                  test/html-reader.html                  test/opml-reader.opml                  test/org-select-tags.org@@ -416,6 +420,10 @@   Description:   Embed data files in binary for relocatable executable.   Default:       False +flag lua53+  Description:   Embed Lua 5.3 instead of 5.4.+  Default:       False+ flag trypandoc   Description:   Build trypandoc cgi executable.   Default:       False@@ -464,25 +472,23 @@                  blaze-markup          >= 0.8      && < 0.9,                  bytestring            >= 0.9      && < 0.12,                  case-insensitive      >= 1.2      && < 1.3,-                 citeproc              >= 0.6.0.1  && < 0.7,-                 commonmark            >= 0.2.1.1  && < 0.3,-                 commonmark-extensions >= 0.2.3    && < 0.3,+                 citeproc              >= 0.7      && < 0.8,+                 commonmark            >= 0.2.2    && < 0.3,+                 commonmark-extensions >= 0.2.3.1  && < 0.3,                  commonmark-pandoc     >= 0.2.1.2  && < 0.3,                  connection            >= 0.3.1,-                 containers            >= 0.4.2.1  && < 0.7,+                 containers            >= 0.6.0.1  && < 0.7,                  data-default          >= 0.4      && < 0.8,                  deepseq               >= 1.3      && < 1.5,                  directory             >= 1.2.3    && < 1.4,-                 doclayout             >= 0.3.1.1  && < 0.4,+                 doclayout             >= 0.4      && < 0.5,                  doctemplates          >= 0.10     && < 0.11,                  emojis                >= 0.1      && < 0.2,                  exceptions            >= 0.8      && < 0.11,                  file-embed            >= 0.0      && < 0.1,                  filepath              >= 1.1      && < 1.5,                  haddock-library       >= 1.10     && < 1.11,-                 hslua                 >= 2.1      && < 2.2,-                 hslua-aeson           >= 2.1      && < 2.2,-                 hslua-marshalling     >= 2.1      && < 2.2,+                 hslua-module-doclayout>= 1.0.4    && < 1.1,                  hslua-module-path     >= 1.0      && < 1.1,                  hslua-module-system   >= 1.0      && < 1.1,                  hslua-module-text     >= 1.0      && < 1.1,@@ -496,8 +502,8 @@                  mtl                   >= 2.2      && < 2.3,                  network               >= 2.6,                  network-uri           >= 2.6      && < 2.8,-                 pandoc-lua-marshal    >= 0.1.3.1  && < 0.2,-                 pandoc-types          >= 1.22.1   && < 1.23,+                 pandoc-lua-marshal    >= 0.1.5    && < 0.2,+                 pandoc-types          >= 1.22.2   && < 1.23,                  parsec                >= 3.1      && < 3.2,                  pretty                >= 1.1      && < 1.2,                  pretty-show           >= 1.10     && < 1.11,@@ -505,13 +511,13 @@                  random                >= 1        && < 1.3,                  safe                  >= 0.3.18   && < 0.4,                  scientific            >= 0.3      && < 0.4,-                 skylighting           >= 0.12.2   && < 0.13,-                 skylighting-core      >= 0.12.2   && < 0.13,+                 skylighting           >= 0.12.3   && < 0.13,+                 skylighting-core      >= 0.12.3   && < 0.13,                  split                 >= 0.2      && < 0.3,                  syb                   >= 0.1      && < 0.8,                  tagsoup               >= 0.14.6   && < 0.15,                  temporary             >= 1.1      && < 1.4,-                 texmath               >= 0.12.4   && < 0.12.5,+                 texmath               >= 0.12.5   && < 0.12.6,                  text                  >= 1.1.1.0  && < 2.1,                  text-conversions      >= 0.3      && < 0.4,                  time                  >= 1.5      && < 1.14,@@ -519,11 +525,17 @@                  unicode-transforms    >= 0.3      && < 0.5,                  xml                   >= 1.3.12   && < 1.4,                  xml-conduit           >= 1.9.1.1  && < 1.10,+                 xml-types             >= 0.3      && < 0.4,                  yaml                  >= 0.11     && < 0.12,                  zip-archive           >= 0.2.3.4  && < 0.5,                  zlib                  >= 0.5      && < 0.7   if !os(windows)     build-depends:  unix >= 2.4 && < 2.8+  if flag(lua53)+    build-depends:  hslua >= 2.1 && < 2.2,+                    hslua-aeson >= 2.1 && < 2.3+  else+    build-depends:  hslua >= 2.2 && < 2.3   if flag(embed_data_files)      cpp-options:   -DEMBED_DATA_FILES      other-modules: Text.Pandoc.Data@@ -545,6 +557,8 @@                    Text.Pandoc.Readers.CommonMark,                    Text.Pandoc.Readers.Creole,                    Text.Pandoc.Readers.BibTeX,+                   Text.Pandoc.Readers.EndNote,+                   Text.Pandoc.Readers.RIS,                    Text.Pandoc.Readers.CslJson,                    Text.Pandoc.Readers.MediaWiki,                    Text.Pandoc.Readers.Vimwiki,@@ -642,6 +656,15 @@                    Text.Pandoc.Filter.JSON,                    Text.Pandoc.Filter.Lua,                    Text.Pandoc.Filter.Path,+                   Text.Pandoc.Parsing.Capabilities,+                   Text.Pandoc.Parsing.Citations,+                   Text.Pandoc.Parsing.General,+                   Text.Pandoc.Parsing.GridTable,+                   Text.Pandoc.Parsing.Lists,+                   Text.Pandoc.Parsing.Math,+                   Text.Pandoc.Parsing.Smart,+                   Text.Pandoc.Parsing.State,+                   Text.Pandoc.Parsing.Types,                    Text.Pandoc.Readers.Docx.Lists,                    Text.Pandoc.Readers.Docx.Combine,                    Text.Pandoc.Readers.Docx.Parse,@@ -697,6 +720,7 @@                    Text.Pandoc.Writers.LaTeX.Types,                    Text.Pandoc.Writers.LaTeX.Citation,                    Text.Pandoc.Writers.LaTeX.Util,+                   Text.Pandoc.Writers.Markdown.Table,                    Text.Pandoc.Writers.Markdown.Types,                    Text.Pandoc.Writers.Markdown.Inline,                    Text.Pandoc.Writers.Roff,@@ -724,6 +748,7 @@                    Text.Pandoc.Lua.Orphans,                    Text.Pandoc.Lua.Packages,                    Text.Pandoc.Lua.PandocLua,+                   Text.Pandoc.Lua.Writer.Classic,                    Text.Pandoc.XML.Light,                    Text.Pandoc.XML.Light.Types,                    Text.Pandoc.XML.Light.Proc,@@ -781,9 +806,9 @@                   doctemplates      >= 0.10    && < 0.11,                   exceptions        >= 0.8     && < 0.11,                   filepath          >= 1.1     && < 1.5,-                  hslua             >= 2.1     && < 2.2,+                  hslua             >= 2.1     && < 2.3,                   mtl               >= 2.2     && < 2.3,-                  pandoc-types      >= 1.22.1  && < 1.23,+                  pandoc-types      >= 1.22.2  && < 1.23,                   process           >= 1.2.3   && < 1.7,                   tasty             >= 0.11    && < 1.5,                   tasty-golden      >= 2.3     && < 2.4,
src/Text/Pandoc/App.hs view
@@ -32,7 +32,6 @@ import qualified Data.ByteString as BS import qualified Data.ByteString.Char8 as B8 import qualified Data.ByteString.Lazy as BL-import Data.Char (toLower) import Data.Maybe (fromMaybe, isJust, isNothing) import qualified Data.Set as Set import Data.Text (Text)@@ -129,17 +128,7 @@                              return "markdown"      let readerNameBase = T.takeWhile (\c -> c /= '+' && c /= '-') readerName-    let pdfOutput = map toLower (takeExtension outputFile) == ".pdf" -    when (pdfOutput && readerNameBase == "latex") $-      case optInputFiles opts of-        Just (inputFile:_) -> report $ UnusualConversion $ T.pack $-          "to convert a .tex file to PDF, you get better results by using pdflatex "-            <> "(or lualatex or xelatex) directly, try `pdflatex " <> inputFile-            <> "` instead of `pandoc " <> inputFile <> " -o " <> outputFile <> "`."-        _ -> return ()--     let makeSandboxed pureReader =           let files = maybe id (:) (optReferenceDoc opts) .                       maybe id (:) (optEpubMetadata opts) .@@ -169,6 +158,8 @@     let writerNameBase = T.takeWhile (\c -> c /= '+' && c /= '-') writerName     let writerOptions = outputWriterOptions outputSettings +    let pdfOutput = isJust $ outputPdfProgram outputSettings+     let bibOutput = writerNameBase == "bibtex" ||                     writerNameBase == "biblatex" ||                     writerNameBase == "csljson"@@ -178,6 +169,14 @@                      pdfOutput ||                      bibOutput +    when (pdfOutput && readerNameBase == "latex") $+      case optInputFiles opts of+        Just (inputFile:_) -> report $ UnusualConversion $ T.pack $+          "to convert a .tex file to PDF, you get better results by using pdflatex "+            <> "(or lualatex or xelatex) directly, try `pdflatex " <> inputFile+            <> "` instead of `pandoc " <> inputFile <> " -o " <> outputFile <> "`."+        _ -> return ()+     -- We don't want to send output to the terminal if the user     -- does 'pandoc -t docx input.txt'; though we allow them to     -- force this with '-o -'.  On posix systems, we detect@@ -186,7 +185,8 @@     when ((pdfOutput || not (isTextFormat format)) &&              istty && isNothing ( optOutputFile opts)) $       throwError $ PandocAppError $-              "Cannot write " <> format <> " output to terminal.\n" <>+              "Cannot write " <> (if pdfOutput then "pdf" else format) <>+              " output to terminal.\n" <>               "Specify an output file using the -o option, or " <>               "use '-o -' to force output to stdout." @@ -218,9 +218,16 @@     metadataFromFile <-       case optMetadataFiles opts of         []    -> return mempty-        paths -> mconcat <$>-           mapM (\path -> do raw <- readMetadataFile path-                             yamlToMeta readerOpts (Just path) raw) paths+        paths -> do+          -- If format is markdown or commonmark, use the enabled extensions,+          -- otherwise treat metadata as pandoc markdown (see #7926, #6832)+          let readerOptsMeta =+                if readerNameBase == "markdown" || readerNameBase == "commonmark"+                   then readerOpts+                   else readerOpts{ readerExtensions = pandocExtensions }+          mconcat <$> mapM+                (\path -> do raw <- readMetadataFile path+                             yamlToMeta readerOptsMeta (Just path) raw) paths      let transforms = (case optShiftHeadingLevelBy opts of                           0             -> id
src/Text/Pandoc/App/CommandLineOptions.hs view
@@ -4,6 +4,7 @@ {-# LANGUAGE TupleSections       #-} {-# LANGUAGE OverloadedStrings   #-} {-# LANGUAGE FlexibleContexts    #-}+{-# LANGUAGE TypeApplications    #-} {- |    Module      : Text.Pandoc.App.CommandLineOptions    Copyright   : Copyright (C) 2006-2022 John MacFarlane@@ -37,6 +38,7 @@ #endif import Data.Maybe (fromMaybe, isJust) import Data.Text (Text)+import HsLua (Exception, getglobal, openlibs, peek, run, top) import Safe (tailDef) import Skylighting (Style, Syntax (..), defaultSyntaxMap, parseTheme) import System.Console.GetOpt@@ -105,11 +107,15 @@                      not (null (optIncludeBeforeBody opts)) ||                      not (null (optIncludeAfterBody opts)) } +-- | Supported LaTeX engines; the first item is used as default engine+-- when going through LaTeX. latexEngines :: [String] latexEngines  = ["pdflatex", "lualatex", "xelatex", "latexmk", "tectonic"] +-- | Supported HTML PDF engines; the first item is used as default+-- engine when going through HTML. htmlEngines :: [String]-htmlEngines  = ["pagedjs-cli", "wkhtmltopdf", "weasyprint", "prince"]+htmlEngines  = ["wkhtmltopdf", "weasyprint", "pagedjs-cli", "prince"]  engines :: [(Text, String)] engines = map ("html",) htmlEngines ++@@ -942,10 +948,14 @@                   (\_ -> do                      prg <- getProgName                      defaultDatadir <- defaultUserDataDir+                     luaVersion <- HsLua.run @HsLua.Exception $ do+                       openlibs+                       getglobal "_VERSION"+                       peek top                      UTF8.hPutStrLn stdout                       $ T.pack                       $ prg ++ " " ++ T.unpack pandocVersion ++-                        compileInfo +++                        compileInfo ++ "\nScripting engine: " ++ luaVersion ++                         "\nUser data directory: " ++ defaultDatadir ++                         ('\n':copyrightMessage)                      exitSuccess ))@@ -985,7 +995,7 @@   "\nCompiled with pandoc-types " ++ VERSION_pandoc_types ++   ", texmath " ++ VERSION_texmath ++ ", skylighting " ++   VERSION_skylighting ++ ",\nciteproc " ++ VERSION_citeproc ++-  ", ipynb " ++ VERSION_ipynb+  ", ipynb " ++ VERSION_ipynb ++ ", hslua " ++ VERSION_hslua  handleUnrecognizedOption :: String -> [String] -> [String] handleUnrecognizedOption "--smart" =
src/Text/Pandoc/App/FormatHeuristics.hs view
@@ -69,6 +69,7 @@     ".org"      -> Just "org"     ".pdf"      -> Just "pdf"  -- so we get an "unknown reader" error     ".pptx"     -> Just "pptx"+    ".ris"      -> Just "ris"     ".roff"     -> Just "ms"     ".rst"      -> Just "rst"     ".rtf"      -> Just "rtf"
src/Text/Pandoc/App/OutputSettings.hs view
@@ -254,6 +254,10 @@       go Nothing Nothing       = Right ("latex", "pdflatex")       go (Just writer) Nothing = (writer,) <$> engineForWriter writer       go Nothing (Just engine) = (,engine) <$> writerForEngine (takeBaseName engine)+      go (Just writer) (Just engine) | isCustomWriter writer =+           -- custom writers can produce any format, so assume the user knows+           -- what they are doing.+           Right (writer, engine)       go (Just writer) (Just engine) =            case find (== (baseWriterName writer, takeBaseName engine)) engines of                 Just _  -> Right (writer, engine)@@ -266,10 +270,12 @@                                    "pdf-engine " <> T.pack eng <> " not known"        engineForWriter "pdf" = Left "pdf writer"-      engineForWriter w = case [e |  (f,e) <- engines, f == baseWriterName w] of+      engineForWriter w = case [e | (f,e) <- engines, f == baseWriterName w] of                                 eng : _ -> Right eng                                 []      -> Left $                                    "cannot produce pdf output from " <> w++      isCustomWriter w = ".lua" `T.isSuffixOf` w  isTextFormat :: T.Text -> Bool isTextFormat s =
src/Text/Pandoc/Asciify.hs view
@@ -18,11 +18,15 @@ import qualified Data.Text as T  toAsciiText :: Text -> Text-toAsciiText = T.filter isAscii . TN.normalize (TN.NFD)+toAsciiText = T.filter isAscii . T.map specialCase . TN.normalize (TN.NFD)+ where+  specialCase '\x131' = 'i' -- Turkish undotted i+  specialCase c = c  toAsciiChar :: Char -> Maybe Char toAsciiChar c = case T.unpack (TN.normalize TN.NFD (T.singleton c)) of                   (x:xs) | isAscii x                          , all isMark xs                          -> Just x+                  ['\x131'] -> Just 'i'  -- Turkish undotted i                   _      -> Nothing
src/Text/Pandoc/Citeproc.hs view
@@ -16,6 +16,7 @@                                      LocatorInfo(..)) import Text.Pandoc.Citeproc.CslJson (cslJsonToReferences) import Text.Pandoc.Citeproc.BibTeX (readBibtexString, Variant(..))+import Text.Pandoc.Readers.RIS (readRIS) import Text.Pandoc.Citeproc.MetaValue (metaValueToReference, metaValueToText) import Text.Pandoc.Readers.Markdown (yamlToRefs) import Text.Pandoc.Builder (Inlines, Many(..), deleteMeta, setMeta)@@ -107,8 +108,8 @@          evalState (walkM insertResolvedCitations $ Pandoc meta' bs)          $ cits   return $ walk removeQuoteSpan-         $ Pandoc meta''-         $ insertRefs refkvs classes meta'' (B.toList bibs) bs'+         $ insertRefs refkvs classes (B.toList bibs)+         $ Pandoc meta'' bs'  removeQuoteSpan :: Inline -> Inline removeQuoteSpan (Span ("",["csl-quoted"],[]) xs) = Span nullAttr xs@@ -267,6 +268,11 @@               (T.unpack <$> mbfp)               raw       return $ mapMaybe metaValueToReference rs+    Format_ris -> do+      Pandoc meta _ <- readRIS def (UTF8.toText raw)+      case lookupMeta "references" meta of+        Just (MetaList rs) -> return $ mapMaybe metaValueToReference rs+        _ -> return []  -- assumes we walk in same order as query insertResolvedCitations :: Inline -> State [Inlines] Inline@@ -321,7 +327,7 @@                , citationItemSuffix = case suffix of                                         [] -> Nothing                                         ils -> Just $ B.fromList ils-               }+               , citationItemData = Nothing }      in if Pandoc.citationId c == "*"            then []            else@@ -343,6 +349,7 @@   | Format_bibtex   | Format_json   | Format_yaml+  | Format_ris   deriving (Show, Eq, Ord)  formatFromExtension :: FilePath -> Maybe BibFormat@@ -353,6 +360,7 @@                            "json"     -> Just Format_json                            "yaml"     -> Just Format_yaml                            "yml"      -> Just Format_yaml+                           "ris"      -> Just Format_ris                            _          -> Nothing  @@ -453,23 +461,24 @@ -- if document contains a Div with id="refs", insert -- references as its contents.  Otherwise, insert references -- at the end of the document in a Div with id="refs"-insertRefs :: [(Text,Text)] -> [Text] -> Meta -> [Block] -> [Block] -> [Block]-insertRefs _ _ _  []   bs = bs-insertRefs refkvs refclasses meta refs bs =+insertRefs :: [(Text,Text)] -> [Text] -> [Block] -> Pandoc -> Pandoc+insertRefs _ _ [] d = d+insertRefs refkvs refclasses refs (Pandoc meta bs) =   if isRefRemove meta-     then bs-     else case runState (walkM go bs) False of-               (bs', True) -> bs'-               (_, False)-                 -> case refTitle meta of+     then Pandoc meta bs+     else case runState (walkM go (Pandoc meta bs)) False of+               (d', True) -> d'+               (Pandoc meta' bs', False)+                 -> Pandoc meta' $+                    case refTitle meta of                       Nothing ->-                        case reverse bs of+                        case reverse bs' of                           Header lev (id',classes,kvs) ys : xs ->                             reverse xs ++                             [Header lev (id',addUnNumbered classes,kvs) ys,                              Div ("refs",refclasses,refkvs) refs]-                          _ -> bs ++ [refDiv]-                      Just ils -> bs +++                          _ -> bs' ++ [refDiv]+                      Just ils -> bs' ++                         [Header 1 ("bibliography", ["unnumbered"], []) ils,                          refDiv]   where
src/Text/Pandoc/Citeproc/BibTeX.hs view
@@ -19,6 +19,7 @@     ( Variant(..)     , readBibtexString     , writeBibtexString+    , toName     )     where @@ -1173,7 +1174,7 @@           , nameStaticOrdering      = False           } -toName :: Options -> [Inline] -> Bib Name+toName :: MonadPlus m => Options -> [Inline] -> m Name toName _ [Str "others"] =   return emptyName{ nameLiteral = Just "others" } toName _ [Span ("",[],[]) ils] = -- corporate author
src/Text/Pandoc/Class/IO.hs view
@@ -53,6 +53,7 @@ import System.Directory (createDirectoryIfMissing) import System.Environment (getEnv) import System.FilePath ((</>), takeDirectory, normalise)+import qualified System.FilePath.Posix as Posix import System.IO (stderr) import System.IO.Error import System.Random (StdGen)@@ -234,6 +235,6 @@   case lookupMedia (T.unpack src) mediabag of     Nothing -> Image attr lab (src, tit)     Just item ->-      let fullpath = dir <> "/" <> mediaPath item+      let fullpath = dir Posix.</> mediaPath item       in  Image attr lab (T.pack fullpath, tit) adjustImagePath _ _ x = x
src/Text/Pandoc/Extensions.hs view
@@ -265,7 +265,6 @@ githubMarkdownExtensions = extensionsFromList   [ Ext_pipe_tables   , Ext_raw_html-  , Ext_native_divs   , Ext_auto_identifiers   , Ext_gfm_auto_identifiers   , Ext_autolink_bare_uris@@ -356,7 +355,6 @@ getDefaultExtensions "gfm"             = extensionsFromList   [ Ext_pipe_tables   , Ext_raw_html-  , Ext_native_divs   , Ext_auto_identifiers   , Ext_gfm_auto_identifiers   , Ext_autolink_bare_uris@@ -377,7 +375,6 @@   , Ext_emoji   , Ext_pipe_tables   , Ext_raw_html-  , Ext_raw_tex            -- only supported in writer (for math)   , Ext_smart   , Ext_tex_math_dollars   , Ext_superscript@@ -488,6 +485,7 @@     [ Ext_empty_paragraphs     , Ext_native_numbering     , Ext_styles+    , Ext_citations     ]   getAll "opendocument"    = extensionsFromList     [ Ext_empty_paragraphs@@ -512,7 +510,6 @@     , Ext_task_lists     , Ext_emoji     , Ext_raw_html-    , Ext_raw_tex            -- only supported in writer (for math)     , Ext_implicit_figures     , Ext_hard_line_breaks     , Ext_smart
src/Text/Pandoc/Logging.hs view
@@ -63,6 +63,7 @@   | UndefinedToggle Text SourcePos   | ParsingUnescaped Text SourcePos   | CouldNotLoadIncludeFile Text SourcePos+  | CouldNotParseIncludeFile Text SourcePos   | MacroAlreadyDefined Text SourcePos   | InlineNotRendered Inline   | BlockNotRendered Block@@ -156,6 +157,11 @@             "source" .= sourceName pos,             "line" .= toJSON (sourceLine pos),             "column" .= toJSON (sourceColumn pos)]+      CouldNotParseIncludeFile fp pos ->+           ["path" .= fp,+            "source" .= sourceName pos,+            "line" .= toJSON (sourceLine pos),+            "column" .= toJSON (sourceColumn pos)]       MacroAlreadyDefined name pos ->            ["name" .= name,             "source" .= sourceName pos,@@ -280,6 +286,8 @@          "Parsing unescaped '" <> s <> "' at " <> showPos pos        CouldNotLoadIncludeFile fp pos ->          "Could not load include file " <> fp <> " at " <> showPos pos+       CouldNotParseIncludeFile fp pos ->+         "Parsing include file " <> fp <> " failed at " <> showPos pos        MacroAlreadyDefined name pos ->          "Macro '" <> name <> "' already defined, ignoring at " <> showPos pos        InlineNotRendered il ->@@ -383,6 +391,7 @@        CouldNotLoadIncludeFile f _         | ".sty" `Text.isSuffixOf` f -> INFO         | otherwise                  -> WARNING+       CouldNotParseIncludeFile{}    -> WARNING        MacroAlreadyDefined{}         -> WARNING        ParsingUnescaped{}            -> INFO        InlineNotRendered{}           -> INFO
src/Text/Pandoc/Lua/ErrorConversion.hs view
@@ -14,8 +14,7 @@   ( addContextToException   ) where -import HsLua (LuaError, LuaE, top)-import HsLua.Marshalling (resultToEither, runPeek)+import HsLua (LuaError, LuaE, resultToEither, runPeek, top) import Text.Pandoc.Error (PandocError (PandocLuaError)) import Text.Pandoc.Lua.Marshal.PandocError (pushPandocError, peekPandocError)
src/Text/Pandoc/Lua/Init.hs view
@@ -45,6 +45,7 @@ loadedModules :: [(Name, Name)] loadedModules =   [ ("pandoc.List", "List")+  , ("pandoc.layout", "layout")   , ("pandoc.mediabag", "mediabag")   , ("pandoc.path", "path")   , ("pandoc.system", "system")
src/Text/Pandoc/Lua/Marshal/CommonState.hs view
@@ -15,9 +15,7 @@   , pushCommonState   ) where -import HsLua.Core-import HsLua.Marshalling-import HsLua.Packaging+import HsLua import Text.Pandoc.Class (CommonState (..)) import Text.Pandoc.Logging (LogMessage, showLogMessage) import Text.Pandoc.Lua.Marshal.List (pushPandocList)
src/Text/Pandoc/Lua/Marshal/PandocError.hs view
@@ -19,8 +19,7 @@   )   where -import HsLua.Core (LuaError)-import HsLua.Marshalling (Peeker, Pusher, pushString, liftLua)+import HsLua (LuaError, Peeker, Pusher, liftLua, pushString) import HsLua.Packaging import Text.Pandoc.Error (PandocError (PandocLuaError))
src/Text/Pandoc/Lua/Marshal/ReaderOptions.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE CPP                  #-} {-# LANGUAGE LambdaCase           #-} {-# LANGUAGE OverloadedStrings    #-} {-# LANGUAGE ScopedTypeVariables  #-}@@ -21,7 +22,9 @@  import Data.Default (def) import HsLua as Lua+#if !MIN_VERSION_hslua(2,2,0) import HsLua.Aeson (peekViaJSON, pushViaJSON)+#endif import Text.Pandoc.Lua.Marshal.List (pushPandocList) import Text.Pandoc.Options (ReaderOptions (..))
src/Text/Pandoc/Lua/Marshal/Sources.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE LambdaCase           #-} {-# LANGUAGE OverloadedStrings    #-} {-# OPTIONS_GHC -fno-warn-orphans #-} {- |@@ -9,13 +10,15 @@ Marshal 'Sources'. -} module Text.Pandoc.Lua.Marshal.Sources-  ( pushSources+  ( peekSources+  , pushSources   ) where +import Control.Monad ((<$!>)) import Data.Text (Text) import HsLua as Lua import Text.Pandoc.Lua.Marshal.List (newListMetatable)-import Text.Pandoc.Sources (Sources (..))+import Text.Pandoc.Sources (Sources (..), toSources) import Text.Parsec (SourcePos, sourceName)  -- | Pushes the 'Sources' as a list of lazy Lua objects.@@ -30,6 +33,13 @@       return 1     rawset (nth 3)   setmetatable (nth 2)++-- | Retrieves sources from the stack.+peekSources :: LuaError e => Peeker e Sources+peekSources idx = liftLua (ltype idx) >>= \case+  TypeString -> toSources <$!> peekText idx+  TypeTable  -> Sources <$!> peekList (peekUD typeSource) idx+  _          -> Sources . (:[]) <$!> peekUD typeSource idx  -- | Source object type. typeSource :: LuaError e => DocumentedType e (SourcePos, Text)
src/Text/Pandoc/Lua/Marshal/WriterOptions.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE CPP                  #-} {-# LANGUAGE LambdaCase           #-} {-# LANGUAGE OverloadedStrings    #-} {-# LANGUAGE ScopedTypeVariables  #-}@@ -20,7 +21,9 @@ import Control.Applicative (optional) import Data.Default (def) import HsLua as Lua+#if !MIN_VERSION_hslua(2,2,0) import HsLua.Aeson (peekViaJSON, pushViaJSON)+#endif import Text.Pandoc.Lua.Marshal.List (pushPandocList) import Text.Pandoc.Lua.Marshal.Template (peekTemplate, pushTemplate) import Text.Pandoc.Options (WriterOptions (..))
src/Text/Pandoc/Lua/Module/Pandoc.hs view
@@ -19,6 +19,7 @@   ) where  import Prelude hiding (read)+import Control.Applicative ((<|>)) import Control.Monad (forM_, when) import Control.Monad.Catch (catch, throwM) import Data.Data (Data, dataTypeConstrs, dataTypeOf, showConstr)@@ -34,6 +35,7 @@ import Text.Pandoc.Lua.Marshal.Filter (peekFilter) import Text.Pandoc.Lua.Marshal.ReaderOptions ( peekReaderOptions                                              , pushReaderOptions)+import Text.Pandoc.Lua.Marshal.Sources (peekSources) import Text.Pandoc.Lua.Marshal.WriterOptions ( peekWriterOptions                                              , pushWriterOptions) import Text.Pandoc.Lua.Module.Utils (sha1)@@ -42,6 +44,7 @@                            , WriterOptions (writerExtensions) ) import Text.Pandoc.Process (pipeProcess) import Text.Pandoc.Readers (Reader (..), getReader)+import Text.Pandoc.Sources (toSources) import Text.Pandoc.Writers (Writer (..), getWriter)  import qualified HsLua as Lua@@ -180,10 +183,17 @@             let formatSpec = fromMaybe "markdown" mformatspec                 readerOpts = fromMaybe def mreaderOptions                 readAction = getReader formatSpec >>= \case-                  (TextReader r, es)      ->-                    r readerOpts{readerExtensions = es} (UTF8.toText content)+                  (TextReader r, es)       ->+                    r readerOpts{readerExtensions = es}+                      (case content of+                         Left bs       -> toSources $ UTF8.toText bs+                         Right sources -> sources)                   (ByteStringReader r, es) ->-                    r readerOpts{readerExtensions = es} (BSL.fromStrict content)+                    case content of+                      Left bs -> r readerOpts{readerExtensions = es}+                                   (BSL.fromStrict bs)+                      Right _ -> liftPandocLua $ Lua.failLua+                                 "Cannot use bytestring reader with Sources"             try (unPandocLua readAction) >>= \case               Right pd ->                 -- success, got a Pandoc document@@ -195,7 +205,9 @@                 "Extension " <> e <> " not supported for " <> f               Left e ->                 throwM e)-    <#> parameter peekByteString "string" "content" "text to parse"+    <#> parameter (\idx -> (Left  <$> peekByteString idx)+                       <|> (Right <$> peekSources idx))+          "string|Sources" "content" "text to parse"     <#> opt (textParam "formatspec" "format and extensions")     <#> opt (parameter peekReaderOptions "ReaderOptions" "reader_options"              "reader options")
src/Text/Pandoc/Lua/Packages.hs view
@@ -16,11 +16,13 @@   ) where  import Control.Monad (forM_)+import Data.String (fromString) import Text.Pandoc.Error (PandocError) import Text.Pandoc.Lua.Marshal.List (pushListModule) import Text.Pandoc.Lua.PandocLua (PandocLua, liftPandocLua)  import qualified HsLua as Lua+import qualified HsLua.Module.DocLayout as DocLayout import qualified HsLua.Module.Path as Path import qualified HsLua.Module.Text as Text import qualified Text.Pandoc.Lua.Module.Pandoc as Pandoc@@ -48,6 +50,7 @@ pandocPackageSearcher pkgName =   case pkgName of     "pandoc"          -> pushModuleLoader Pandoc.documentedModule+    "pandoc.layout"   -> pushModuleLoader DocLayout.documentedModule     "pandoc.mediabag" -> pushModuleLoader MediaBag.documentedModule     "pandoc.path"     -> pushModuleLoader Path.documentedModule     "pandoc.system"   -> pushModuleLoader System.documentedModule@@ -60,8 +63,9 @@     _                 -> reportPandocSearcherFailure  where   pushModuleLoader mdl = liftPandocLua $ do+    let mdl' = mdl { Lua.moduleName = fromString pkgName }     Lua.pushHaskellFunction $-      Lua.NumResults 1 <$ Lua.pushModule @PandocError mdl+      Lua.NumResults 1 <$ Lua.pushModule mdl'     return (Lua.NumResults 1)   pushWrappedHsFun f = liftPandocLua $ do     Lua.pushHaskellFunction f
+ src/Text/Pandoc/Lua/Writer/Classic.hs view
@@ -0,0 +1,250 @@+{-# LANGUAGE CPP                 #-}+{-# LANGUAGE FlexibleContexts    #-}+{-# LANGUAGE FlexibleInstances   #-}+{-# LANGUAGE OverloadedStrings   #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeApplications    #-}+{- |+   Module      : Text.Pandoc.Lua.Writer.Classic+   Copyright   : Copyright (C) 2012-2022 John MacFarlane+   License     : GNU GPL, version 2 or above++   Maintainer  : John MacFarlane <jgm@berkeley.edu>+   Stability   : alpha+   Portability : portable++Conversion of Pandoc documents using a \"classic\" custom Lua writer.+-}+module Text.Pandoc.Lua.Writer.Classic+  ( runCustom+  ) where+import Control.Applicative (optional)+import Control.Arrow ((***))+import Data.List (intersperse)+import Data.Maybe (fromMaybe)+import qualified Data.Text as T+import Data.Text (Text, pack)+import HsLua as Lua hiding (Operation (Div))+#if !MIN_VERSION_hslua(2,2,0)+import HsLua.Aeson (peekViaJSON)+#endif+import Text.DocLayout (literal, render)+import Text.DocTemplates (Context)+import Text.Pandoc.Definition+import Text.Pandoc.Lua.Marshal.Attr (pushAttributeList)+import Text.Pandoc.Lua.Orphans ()+import Text.Pandoc.Options+import Text.Pandoc.Templates (renderTemplate)+import Text.Pandoc.Writers.Shared++-- | List of key-value pairs that is pushed to Lua as AttributeList+-- userdata.+newtype AttributeList = AttributeList [(Text, Text)]+instance Pushable AttributeList where+  push (AttributeList kvs) = pushAttributeList kvs++attrToMap :: Attr -> AttributeList+attrToMap (id',classes,keyvals) = AttributeList+    $ ("id", id')+    : ("class", T.unwords classes)+    : keyvals++newtype Stringify a = Stringify a++instance Pushable (Stringify Format) where+  push (Stringify (Format f)) = Lua.push (T.toLower f)++instance Pushable (Stringify [Inline]) where+  push (Stringify ils) = Lua.push =<< inlineListToCustom ils++instance Pushable (Stringify [Block]) where+  push (Stringify blks) = Lua.push =<< blockListToCustom blks++instance Pushable (Stringify MetaValue) where+  push (Stringify (MetaMap m))       = Lua.push (fmap Stringify m)+  push (Stringify (MetaList xs))     = Lua.push (map Stringify xs)+  push (Stringify (MetaBool x))      = Lua.push x+  push (Stringify (MetaString s))    = Lua.push s+  push (Stringify (MetaInlines ils)) = Lua.push (Stringify ils)+  push (Stringify (MetaBlocks bs))   = Lua.push (Stringify bs)++instance Pushable (Stringify Citation) where+  push (Stringify cit) = flip pushAsTable cit+    [ ("citationId", push . citationId)+    , ("citationPrefix",  push . Stringify . citationPrefix)+    , ("citationSuffix",  push . Stringify . citationSuffix)+    , ("citationMode",    push . citationMode)+    , ("citationNoteNum", push . citationNoteNum)+    , ("citationHash",    push . citationHash)+    ]++-- | Key-value pair, pushed as a table with @a@ as the only key and @v@ as the+-- associated value.+newtype KeyValue a b = KeyValue (a, b)++instance (Pushable a, Pushable b) => Pushable (KeyValue a b) where+  push (KeyValue (k, v)) = do+    Lua.newtable+    Lua.push k+    Lua.push v+    Lua.rawset (Lua.nth 3)++-- | Convert Pandoc to custom markup using a classic Lua writer.+runCustom :: LuaError e+          => WriterOptions+          -> Pandoc+          -> LuaE e Text+runCustom opts doc@(Pandoc meta _) = do+  (body, context) <- docToCustom opts doc+  -- convert metavalues to a template context (variables)+  metaContext <- metaToContext opts+                   (fmap (literal . pack) . blockListToCustom)+                   (fmap (literal . pack) . inlineListToCustom)+                   meta+  -- merge contexts from metadata and variables+  let renderContext = context <> metaContext+  return $ case writerTemplate opts of+    Nothing  -> body+    Just tpl -> render Nothing $+                renderTemplate tpl $ setField "body" body renderContext++-- | Converts a Pandoc value to custom markup using a classic Lua writer.+docToCustom :: forall e. LuaError e+            => WriterOptions -> Pandoc -> LuaE e (Text, Context Text)+docToCustom opts (Pandoc (Meta metamap) blocks) = do+  body <- blockListToCustom blocks+  -- invoke doesn't work with multiple return values, so we have to call+  -- `Doc` manually.+  Lua.getglobal "Doc"                 -- function+  push body                           -- argument 1+  push (fmap Stringify  metamap)      -- argument 2+  push (writerVariables opts)         -- argument 3+  call 3 2+  rendered  <- peek (nth 2)       -- first return value+  context <- forcePeek . optional $ peekViaJSON top  -- snd return value+  return (rendered, fromMaybe mempty context)+++-- | Convert Pandoc block element to Custom.+blockToCustom :: forall e. LuaError e+              => Block         -- ^ Block element+              -> LuaE e String++blockToCustom Null = return ""++blockToCustom (Plain inlines) = invoke "Plain" (Stringify inlines)++blockToCustom (Para [Image attr txt (src,tit)]) =+  invoke "CaptionedImage" src tit (Stringify txt) (attrToMap attr)++blockToCustom (Para inlines) = invoke "Para" (Stringify inlines)++blockToCustom (LineBlock linesList) =+  invoke "LineBlock" (map (Stringify) linesList)++blockToCustom (RawBlock format str) =+  invoke "RawBlock" (Stringify format) str++blockToCustom HorizontalRule = invoke "HorizontalRule"++blockToCustom (Header level attr inlines) =+  invoke "Header" level (Stringify inlines) (attrToMap attr)++blockToCustom (CodeBlock attr str) =+  invoke "CodeBlock" str (attrToMap attr)++blockToCustom (BlockQuote blocks) =+  invoke "BlockQuote" (Stringify blocks)++blockToCustom (Table _ blkCapt specs thead tbody tfoot) =+  let (capt, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot+      aligns' = map show aligns+      capt' = Stringify capt+      headers' = map (Stringify) headers+      rows' = map (map (Stringify)) rows+  in invoke "Table" capt' aligns' widths headers' rows'++blockToCustom (BulletList items) =+  invoke "BulletList" (map (Stringify) items)++blockToCustom (OrderedList (num,sty,delim) items) =+  invoke "OrderedList" (map (Stringify) items) num (show sty) (show delim)++blockToCustom (DefinitionList items) =+  invoke "DefinitionList"+               (map (KeyValue . (Stringify *** map (Stringify))) items)++blockToCustom (Div attr items) =+  invoke "Div" (Stringify items) (attrToMap attr)++-- | Convert list of Pandoc block elements to Custom.+blockListToCustom :: forall e. LuaError e+                  => [Block]       -- ^ List of block elements+                  -> LuaE e String+blockListToCustom xs = do+  blocksep <- invoke "Blocksep"+  bs <- mapM blockToCustom xs+  return $ mconcat $ intersperse blocksep bs++-- | Convert list of Pandoc inline elements to Custom.+inlineListToCustom :: forall e. LuaError e => [Inline] -> LuaE e String+inlineListToCustom lst = do+  xs <- mapM (inlineToCustom @e) lst+  return $ mconcat xs++-- | Convert Pandoc inline element to Custom.+inlineToCustom :: forall e. LuaError e => Inline -> LuaE e String++inlineToCustom (Str str) = invoke "Str" str++inlineToCustom Space = invoke "Space"++inlineToCustom SoftBreak = invoke "SoftBreak"++inlineToCustom (Emph lst) = invoke "Emph" (Stringify lst)++inlineToCustom (Underline lst) = invoke "Underline" (Stringify lst)++inlineToCustom (Strong lst) = invoke "Strong" (Stringify lst)++inlineToCustom (Strikeout lst) = invoke "Strikeout" (Stringify lst)++inlineToCustom (Superscript lst) = invoke "Superscript" (Stringify lst)++inlineToCustom (Subscript lst) = invoke "Subscript" (Stringify lst)++inlineToCustom (SmallCaps lst) = invoke "SmallCaps" (Stringify lst)++inlineToCustom (Quoted SingleQuote lst) =+  invoke "SingleQuoted" (Stringify lst)++inlineToCustom (Quoted DoubleQuote lst) =+  invoke "DoubleQuoted" (Stringify lst)++inlineToCustom (Cite cs lst) =+  invoke "Cite" (Stringify lst) (map (Stringify) cs)++inlineToCustom (Code attr str) =+  invoke "Code" str (attrToMap attr)++inlineToCustom (Math DisplayMath str) =+  invoke "DisplayMath" str++inlineToCustom (Math InlineMath str) =+  invoke "InlineMath" str++inlineToCustom (RawInline format str) =+  invoke "RawInline" (Stringify format) str++inlineToCustom LineBreak = invoke "LineBreak"++inlineToCustom (Link attr txt (src,tit)) =+  invoke "Link" (Stringify txt) src tit (attrToMap attr)++inlineToCustom (Image attr alt (src,tit)) =+  invoke "Image" (Stringify alt) src tit (attrToMap attr)++inlineToCustom (Note contents) = invoke "Note" (Stringify contents)++inlineToCustom (Span attr items) =+  invoke "Span" (Stringify items) (attrToMap attr)
src/Text/Pandoc/MIME.hs view
@@ -296,6 +296,7 @@            ,("mid","audio/midi")            ,("midi","audio/midi")            ,("mif","application/x-mif")+           ,("mkv","video/x-matroska")            ,("mm","application/x-freemind")            ,("mmd","chemical/x-macromodel-input")            ,("mmf","application/vnd.smaf")
src/Text/Pandoc/MediaBag.hs view
@@ -29,6 +29,8 @@ import Data.Maybe (fromMaybe, isNothing) import Data.Typeable (Typeable) import System.FilePath+import qualified System.FilePath.Posix as Posix+import qualified System.FilePath.Windows as Windows import Text.Pandoc.MIME (MimeType, getMimeTypeDef, extensionFromMimeType) import Data.Text (Text) import qualified Data.Text as T@@ -78,7 +80,8 @@                              , mediaMimeType = mt }         fp' = canonicalize fp         uri = parseURI fp-        newpath = if isRelative fp+        newpath = if Posix.isRelative fp+                       && Windows.isRelative fp                        && isNothing uri                        && ".." `notElem` splitDirectories fp                      then T.unpack fp'
src/Text/Pandoc/Parsing.hs view
@@ -1,1692 +1,333 @@-{-# LANGUAGE CPP                        #-}-{-# LANGUAGE ExplicitForAll             #-}-{-# LANGUAGE FlexibleContexts           #-}-{-# LANGUAGE FlexibleInstances          #-}-{-# LANGUAGE GeneralizedNewtypeDeriving #-}-{-# LANGUAGE IncoherentInstances        #-}-{-# LANGUAGE MultiParamTypeClasses      #-}-{-# LANGUAGE OverloadedStrings          #-}-{- |-   Module      : Text.Pandoc.Parsing-   Copyright   : Copyright (C) 2006-2022 John MacFarlane-   License     : GNU GPL, version 2 or above--   Maintainer  : John MacFarlane <jgm@berkeley.edu>-   Stability   : alpha-   Portability : portable--A utility library with parsers used in pandoc readers.--}--module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,-                             countChar,-                             textStr,-                             anyLine,-                             anyLineNewline,-                             indentWith,-                             manyChar,-                             many1Char,-                             manyTillChar,-                             many1TillChar,-                             many1Till,-                             manyUntil,-                             manyUntilChar,-                             sepBy1',-                             notFollowedBy',-                             oneOfStrings,-                             oneOfStringsCI,-                             spaceChar,-                             nonspaceChar,-                             skipSpaces,-                             blankline,-                             blanklines,-                             gobbleSpaces,-                             gobbleAtMostSpaces,-                             enclosed,-                             stringAnyCase,-                             parseFromString,-                             parseFromString',-                             lineClump,-                             charsInBalanced,-                             romanNumeral,-                             emailAddress,-                             uri,-                             mathInline,-                             mathDisplay,-                             withHorizDisplacement,-                             withRaw,-                             escaped,-                             characterReference,-                             upperRoman,-                             lowerRoman,-                             decimal,-                             lowerAlpha,-                             upperAlpha,-                             anyOrderedListMarker,-                             orderedListMarker,-                             charRef,-                             lineBlockLines,-                             tableWith,-                             widthsFromIndices,-                             gridTableWith,-                             gridTableWith',-                             readWith,-                             readWithM,-                             testStringWith,-                             guardEnabled,-                             guardDisabled,-                             updateLastStrPos,-                             notAfterString,-                             logMessage,-                             reportLogMessages,-                             ParserState (..),-                             HasReaderOptions (..),-                             HasIdentifierList (..),-                             HasMacros (..),-                             HasLogMessages (..),-                             HasLastStrPosition (..),-                             HasIncludeFiles (..),-                             defaultParserState,-                             HeaderType (..),-                             ParserContext (..),-                             QuoteContext (..),-                             HasQuoteContext (..),-                             NoteTable,-                             NoteTable',-                             KeyTable,-                             SubstTable,-                             Key (..),-                             toKey,-                             registerHeader,-                             smartPunctuation,-                             singleQuoteStart,-                             singleQuoteEnd,-                             doubleQuoteStart,-                             doubleQuoteEnd,-                             apostrophe,-                             doubleCloseQuote,-                             ellipses,-                             dash,-                             nested,-                             citeKey,-                             Parser,-                             ParserT,-                             Future(..),-                             runF,-                             askF,-                             asksF,-                             returnF,-                             trimInlinesF,-                             token,-                             (<+?>),-                             extractIdClass,-                             insertIncludedFile,-                             -- * Re-exports from Text.Parsec-                             Stream,-                             runParser,-                             runParserT,-                             parse,-                             tokenPrim,-                             anyToken,-                             getInput,-                             setInput,-                             unexpected,-                             skipMany,-                             skipMany1,-                             count,-                             eof,-                             lookAhead,-                             notFollowedBy,-                             many,-                             many1,-                             manyTill,-                             (<|>),-                             (<?>),-                             choice,-                             try,-                             sepBy,-                             sepBy1,-                             sepEndBy,-                             sepEndBy1,-                             endBy,-                             endBy1,-                             option,-                             optional,-                             optionMaybe,-                             getState,-                             setState,-                             updateState,-                             SourcePos,-                             getPosition,-                             setPosition,-                             sourceName,-                             setSourceName,-                             sourceColumn,-                             sourceLine,-                             setSourceColumn,-                             setSourceLine,-                             incSourceColumn,-                             incSourceLine,-                             newPos,-                             initialPos,-                             Line,-                             Column,-                             ParseError-                             )-where--import Control.Monad.Identity-    ( guard,-      join,-      unless,-      when,-      void,-      liftM2,-      liftM,-      Identity(..),-      MonadPlus(mzero) )-import Control.Monad.Reader-    ( asks, runReader, MonadReader(ask), Reader, ReaderT(ReaderT) )-import Data.Char (chr, isAlphaNum, isAscii, isAsciiUpper, isAsciiLower,-                  isSpace, ord, toLower, toUpper)-import Data.Default ( Default(..) )-import Data.Functor (($>))-import Data.List (intercalate, transpose)-import qualified Data.Map as M-import Data.Maybe (fromMaybe)-import qualified Data.Set as Set-import Data.Text (Text)-import qualified Data.Text as T-import Text.HTML.TagSoup.Entity (lookupEntity)-import Text.Pandoc.Asciify (toAsciiText)-import Text.Pandoc.Builder (Blocks, HasMeta (..), Inlines, trimInlines)-import qualified Text.Pandoc.Builder as B-import Text.Pandoc.Class.PandocMonad (PandocMonad, readFileFromDirs, report)-import Text.Pandoc.Definition-    ( Target,-      nullMeta,-      nullAttr,-      Meta,-      ColWidth(ColWidthDefault, ColWidth),-      TableFoot(TableFoot),-      TableBody(TableBody),-      Attr,-      TableHead(TableHead),-      Row(..),-      Alignment(..),-      Inline(Str),-      ListNumberDelim(..),-      ListAttributes,-      ListNumberStyle(..) )-import Text.Pandoc.Logging-    ( LogMessage(CouldNotLoadIncludeFile, DuplicateIdentifier) )-import Text.Pandoc.Options-    ( extensionEnabled,-      Extension(Ext_old_dashes, Ext_tex_math_dollars,-                Ext_tex_math_single_backslash, Ext_tex_math_double_backslash,-                Ext_auto_identifiers, Ext_ascii_identifiers, Ext_smart),-      ReaderOptions(readerTabStop, readerColumns, readerExtensions) )-import Text.Pandoc.Readers.LaTeX.Types (Macro)-import Text.Pandoc.Shared-    ( uniqueIdent,-      tshow,-      mapLeft,-      compactify,-      trim,-      trimr,-      splitTextByIndices,-      safeRead,-      trimMath,-      schemes,-      escapeURI )-import Text.Pandoc.Sources-import qualified Text.Pandoc.UTF8 as UTF8 (putStrLn)-import Text.Pandoc.XML (fromEntities)-import Text.Parsec-    ( between,-      setSourceName,-      Parsec,-      Column,-      Line,-      incSourceLine,-      incSourceColumn,-      setSourceLine,-      setSourceColumn,-      sourceLine,-      sourceColumn,-      sourceName,-      setSourceName,-      setPosition,-      getPosition,-      updateState,-      setState,-      getState,-      optionMaybe,-      optional,-      option,-      endBy1,-      endBy,-      sepEndBy1,-      sepEndBy,-      sepBy1,-      sepBy,-      try,-      choice,-      (<?>),-      (<|>),-      manyTill,-      many1,-      many,-      notFollowedBy,-      lookAhead,-      eof,-      count,-      skipMany1,-      skipMany,-      unexpected,-      setInput,-      getInput,-      anyToken,-      tokenPrim,-      parse,-      runParserT,-      runParser,-      ParseError,-      ParsecT,-      SourcePos,-      Stream(..) )-import Text.Parsec.Pos (initialPos, newPos)-import Control.Monad.Except ( MonadError(throwError) )-import Text.Pandoc.Error-    ( PandocError(PandocParseError, PandocParsecError) )--type Parser t s = Parsec t s--type ParserT = ParsecT----- | Reader monad wrapping the parser state. This is used to possibly delay--- evaluation until all relevant information has been parsed and made available--- in the parser state.-newtype Future s a = Future { runDelayed :: Reader s a }-  deriving (Monad, Applicative, Functor)--runF :: Future s a -> s -> a-runF = runReader . runDelayed--askF :: Future s s-askF = Future ask--asksF :: (s -> a) -> Future s a-asksF f = Future $ asks f--returnF :: Monad m => a -> m (Future s a)-returnF = return . return--trimInlinesF :: Future s Inlines -> Future s Inlines-trimInlinesF = liftM trimInlines--instance Semigroup a => Semigroup (Future s a) where-  (<>) = liftM2 (<>)-instance (Semigroup a, Monoid a) => Monoid (Future s a) where-  mempty = return mempty-  mappend = (<>)---- | Like @count@, but packs its result-countChar :: (Stream s m Char, UpdateSourcePos s Char, Monad m)-          => Int-          -> ParsecT s st m Char-          -> ParsecT s st m Text-countChar n = fmap T.pack . count n---- | Like @string@, but uses @Text@.-textStr :: (Stream s m Char, UpdateSourcePos s Char)-        => Text -> ParsecT s u m Text-textStr t = string (T.unpack t) $> t----- | Parse any line of text, returning the contents without the--- final newline.-anyLine :: Monad m => ParserT Sources st m Text-anyLine = do-  -- This is much faster than:-  -- manyTill anyChar newline-  inp <- getInput-  case inp of-    Sources [] -> mzero-    Sources ((fp,t):inps) ->-      -- we assume that lines don't span different input files-      case T.break (=='\n') t of-           (this, rest)-             | T.null rest-             , not (null inps) ->-                -- line may span different input files, so do it-                 -- character by character-                 T.pack <$> manyTill anyChar newline-             | otherwise -> do --  either end of inputs or newline in rest-                 setInput $ Sources ((fp, rest):inps)-                 char '\n' -- needed so parsec knows we won't match empty string-                           -- and so source pos is updated-                 return this---- | Parse any line, include the final newline in the output-anyLineNewline :: Monad m => ParserT Sources st m Text-anyLineNewline = (<> "\n") <$> anyLine---- | Parse indent by specified number of spaces (or equiv. tabs)-indentWith :: (Stream s m Char, UpdateSourcePos s Char)-           => HasReaderOptions st-           => Int -> ParserT s st m Text-indentWith num = do-  tabStop <- getOption readerTabStop-  if num < tabStop-     then countChar num (char ' ')-     else choice [ try (countChar num (char ' '))-                 , try (char '\t' >> indentWith (num - tabStop)) ]---- | Like @many@, but packs its result.-manyChar :: Stream s m t-         => ParserT s st m Char-         -> ParserT s st m Text-manyChar = fmap T.pack . many---- | Like @many1@, but packs its result.-many1Char :: Stream s m t-          => ParserT s st m Char-          -> ParserT s st m Text-many1Char = fmap T.pack . many1---- | Like @manyTill@, but packs its result.-manyTillChar :: Stream s m t-             => ParserT s st m Char-             -> ParserT s st m a-             -> ParserT s st m Text-manyTillChar p = fmap T.pack . manyTill p---- | Like @manyTill@, but reads at least one item.-many1Till :: (Show end, Stream s m t)-          => ParserT s st m a-          -> ParserT s st m end-          -> ParserT s st m [a]-many1Till p end = do-         notFollowedBy' end-         first <- p-         rest <- manyTill p end-         return (first:rest)---- | Like @many1Till@, but packs its result-many1TillChar :: (Show end, Stream s m t)-              => ParserT s st m Char-              -> ParserT s st m end-              -> ParserT s st m Text-many1TillChar p = fmap T.pack . many1Till p---- | Like @manyTill@, but also returns the result of end parser.-manyUntil :: ParserT s u m a-          -> ParserT s u m b-          -> ParserT s u m ([a], b)-manyUntil p end = scan-  where scan =-          (do e <- end-              return ([], e)-          ) <|>-          (do x <- p-              (xs, e) <- scan-              return (x:xs, e))---- | Like @manyUntil@, but also packs its result.-manyUntilChar :: ParserT s u m Char-              -> ParserT s u m b-              -> ParserT s u m (Text, b)-manyUntilChar p = fmap go . manyUntil p-  where-    go (x, y) = (T.pack x, y)---- | Like @sepBy1@ from Parsec,--- but does not fail if it @sep@ succeeds and @p@ fails.-sepBy1' :: ParsecT s u m a-        -> ParsecT s u m sep-        -> ParsecT s u m [a]-sepBy1' p sep = (:) <$> p <*> many (try $ sep >> p)---- | A more general form of @notFollowedBy@.  This one allows any--- type of parser to be specified, and succeeds only if that parser fails.--- It does not consume any input.-notFollowedBy' :: (Show b, Stream s m a) => ParserT s st m b -> ParserT s st m ()-notFollowedBy' p  = try $ join $  do  a <- try p-                                      return (unexpected (show a))-                                  <|>-                                  return (return ())--- (This version due to Andrew Pimlott on the Haskell mailing list.)--oneOfStrings' :: (Stream s m Char, UpdateSourcePos s Char)-              => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text-oneOfStrings' f = fmap T.pack . oneOfStrings'' f . fmap T.unpack---- TODO: This should be re-implemented in a Text-aware way-oneOfStrings'' :: (Stream s m Char, UpdateSourcePos s Char)-               => (Char -> Char -> Bool) -> [String] -> ParserT s st m String-oneOfStrings'' _ []   = Prelude.fail "no strings"-oneOfStrings'' matches strs = try $ do-  c <- anyChar-  let strs' = [xs | (x:xs) <- strs, x `matches` c]-  case strs' of-       []  -> Prelude.fail "not found"-       _   -> (c:) <$> oneOfStrings'' matches strs'-               <|> if "" `elem` strs'-                      then return [c]-                      else Prelude.fail "not found"---- | Parses one of a list of strings.  If the list contains--- two strings one of which is a prefix of the other, the longer--- string will be matched if possible.-oneOfStrings :: (Stream s m Char, UpdateSourcePos s Char)-             => [Text] -> ParserT s st m Text-oneOfStrings = oneOfStrings' (==)---- | Parses one of a list of strings (tried in order), case insensitive.---- TODO: This will not be accurate with general Unicode (neither--- Text.toLower nor Text.toCaseFold can be implemented with a map)-oneOfStringsCI :: (Stream s m Char, UpdateSourcePos s Char)-               => [Text] -> ParserT s st m Text-oneOfStringsCI = oneOfStrings' ciMatch-  where ciMatch x y = toLower' x == toLower' y-        -- this optimizes toLower by checking common ASCII case-        -- first, before calling the expensive unicode-aware-        -- function:-        toLower' c | isAsciiUpper c = chr (ord c + 32)-                   | isAscii c = c-                   | otherwise = toLower c---- | Parses a space or tab.-spaceChar :: (Stream s m Char, UpdateSourcePos s Char)-          => ParserT s st m Char-spaceChar = satisfy $ \c -> c == ' ' || c == '\t'---- | Parses a nonspace, nonnewline character.-nonspaceChar :: (Stream s m Char, UpdateSourcePos s Char)-             => ParserT s st m Char-nonspaceChar = satisfy (not . isSpaceChar)--isSpaceChar :: Char -> Bool-isSpaceChar ' '  = True-isSpaceChar '\t' = True-isSpaceChar '\n' = True-isSpaceChar '\r' = True-isSpaceChar _    = False---- | Skips zero or more spaces or tabs.-skipSpaces :: (Stream s m Char, UpdateSourcePos s Char)-           => ParserT s st m ()-skipSpaces = skipMany spaceChar---- | Skips zero or more spaces or tabs, then reads a newline.-blankline :: (Stream s m Char, UpdateSourcePos s Char)-          => ParserT s st m Char-blankline = try $ skipSpaces >> newline---- | Parses one or more blank lines and returns a string of newlines.-blanklines :: (Stream s m Char, UpdateSourcePos s Char)-           => ParserT s st m Text-blanklines = T.pack <$> many1 blankline---- | Gobble n spaces; if tabs are encountered, expand them--- and gobble some or all of their spaces, leaving the rest.-gobbleSpaces :: (HasReaderOptions st, Monad m)-             => Int -> ParserT Sources st m ()-gobbleSpaces 0 = return ()-gobbleSpaces n-  | n < 0     = error "gobbleSpaces called with negative number"-  | otherwise = try $ do-      char ' ' <|> eatOneSpaceOfTab-      gobbleSpaces (n - 1)--eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Sources st m Char-eatOneSpaceOfTab = do-  lookAhead (char '\t')-  pos <- getPosition-  tabstop <- getOption readerTabStop-  -- replace the tab on the input stream with spaces-  let numSpaces = tabstop - ((sourceColumn pos - 1) `mod` tabstop)-  inp <- getInput-  setInput $-    case inp of-      Sources [] -> error "eatOneSpaceOfTab - empty Sources list"-      Sources ((fp,t):rest) ->-        -- drop the tab and add spaces-        Sources ((fp, T.replicate numSpaces " " <> T.drop 1 t):rest)-  char ' '---- | Gobble up to n spaces; if tabs are encountered, expand them--- and gobble some or all of their spaces, leaving the rest.-gobbleAtMostSpaces :: (HasReaderOptions st, Monad m)-                   => Int -> ParserT Sources st m Int-gobbleAtMostSpaces 0 = return 0-gobbleAtMostSpaces n-  | n < 0     = error "gobbleAtMostSpaces called with negative number"-  | otherwise = option 0 $ do-      char ' ' <|> eatOneSpaceOfTab-      (+ 1) <$> gobbleAtMostSpaces (n - 1)---- | Parses material enclosed between start and end parsers.-enclosed :: (Show end, Stream s m Char, UpdateSourcePos s Char)-         => ParserT s st m t   -- ^ start parser-         -> ParserT s st m end  -- ^ end parser-         -> ParserT s st m a    -- ^ content parser (to be used repeatedly)-         -> ParserT s st m [a]-enclosed start end parser = try $-  start >> notFollowedBy space >> many1Till parser end---- | Parse string, case insensitive.-stringAnyCase :: (Stream s m Char, UpdateSourcePos s Char)-              => Text -> ParserT s st m Text-stringAnyCase = fmap T.pack . stringAnyCase' . T.unpack--stringAnyCase' :: (Stream s m Char, UpdateSourcePos s Char)-               => String -> ParserT s st m String-stringAnyCase' [] = string ""-stringAnyCase' (x:xs) = do-  firstChar <- char (toUpper x) <|> char (toLower x)-  rest <- stringAnyCase' xs-  return (firstChar:rest)---- TODO rewrite by just adding to Sources stream?--- | Parse contents of 'str' using 'parser' and return result.-parseFromString :: Monad m-                => ParserT Sources st m r-                -> Text-                -> ParserT Sources st m r-parseFromString parser str = do-  oldPos <- getPosition-  oldInput <- getInput-  setInput $ toSources str-  setPosition $ initialPos $ sourceName oldPos <> "_chunk"-  result <- parser-  spaces-  setInput oldInput-  setPosition oldPos-  return result---- | Like 'parseFromString' but specialized for 'ParserState'.--- This resets 'stateLastStrPos', which is almost always what we want.-parseFromString' :: (Monad m, HasLastStrPosition u)-                 => ParserT Sources u m a-                 -> Text-                 -> ParserT Sources u m a-parseFromString' parser str = do-  oldLastStrPos <- getLastStrPos <$> getState-  updateState $ setLastStrPos Nothing-  res <- parseFromString parser str-  updateState $ setLastStrPos oldLastStrPos-  return res---- | Parse raw line block up to and including blank lines.-lineClump :: Monad m => ParserT Sources st m Text-lineClump = blanklines-          <|> (T.unlines <$> many1 (notFollowedBy blankline >> anyLine))---- | Parse a string of characters between an open character--- and a close character, including text between balanced--- pairs of open and close, which must be different. For example,--- @charsInBalanced '(' ')' anyChar@ will parse "(hello (there))"--- and return "hello (there)".-charsInBalanced :: (Stream s m Char, UpdateSourcePos s Char) => Char -> Char -> ParserT s st m Char-                -> ParserT s st m Text-charsInBalanced open close parser = try $ do-  char open-  let isDelim c = c == open || c == close-  raw <- many $  T.pack <$> many1 (notFollowedBy (satisfy isDelim) >> parser)-             <|> (do res <- charsInBalanced open close parser-                     return $ T.singleton open <> res <> T.singleton close)-  char close-  return $ T.concat raw---- old charsInBalanced would be:--- charsInBalanced open close (noneOf "\n" <|> char '\n' >> notFollowedBy blankline)--- old charsInBalanced' would be:--- charsInBalanced open close anyChar---- Auxiliary functions for romanNumeral:---- | Parses a roman numeral (uppercase or lowercase), returns number.-romanNumeral :: (Stream s m Char, UpdateSourcePos s Char) => Bool                  -- ^ Uppercase if true-             -> ParserT s st m Int-romanNumeral upperCase = do-    let rchar uc = char $ if upperCase then uc else toLower uc-    let one         = rchar 'I'-    let five        = rchar 'V'-    let ten         = rchar 'X'-    let fifty       = rchar 'L'-    let hundred     = rchar 'C'-    let fivehundred = rchar 'D'-    let thousand    = rchar 'M'-    lookAhead $ choice [one, five, ten, fifty, hundred, fivehundred, thousand]-    thousands <- (1000 *) . length <$> many thousand-    ninehundreds <- option 0 $ try $ hundred >> thousand >> return 900-    fivehundreds <- option 0 $ 500 <$ fivehundred-    fourhundreds <- option 0 $ try $ hundred >> fivehundred >> return 400-    hundreds <- (100 *) . length <$> many hundred-    nineties <- option 0 $ try $ ten >> hundred >> return 90-    fifties <- option 0 (50 <$ fifty)-    forties <- option 0 $ try $ ten >> fifty >> return 40-    tens <- (10 *) . length <$> many ten-    nines <- option 0 $ try $ one >> ten >> return 9-    fives <- option 0 (5 <$ five)-    fours <- option 0 $ try $ one >> five >> return 4-    ones <- length <$> many one-    let total = thousands + ninehundreds + fivehundreds + fourhundreds +-                hundreds + nineties + fifties + forties + tens + nines +-                fives + fours + ones-    if total == 0-       then Prelude.fail "not a roman numeral"-       else return total---- Parsers for email addresses and URIs---- | Parses an email address; returns original and corresponding--- escaped mailto: URI.-emailAddress :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)-emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)- where toResult mbox dom = let full = fromEntities $ T.pack $ mbox ++ '@':dom-                           in  (full, escapeURI $ "mailto:" <> full)-       mailbox           = intercalate "." <$> (emailWord `sepBy1'` dot)-       domain            = intercalate "." <$> (subdomain `sepBy1'` dot)-       dot               = char '.'-       subdomain         = many1 $ alphaNum <|> innerPunct (=='-')-       -- this excludes some valid email addresses, since an-       -- email could contain e.g. '__', but gives better results-       -- for our purposes, when combined with markdown parsing:-       innerPunct f      = try (satisfy f-                                 <* notFollowedBy (satisfy (not . isAlphaNum)))-       -- technically an email address could begin with a symbol,-       -- but allowing this creates too many problems.-       -- See e.g. https://github.com/jgm/pandoc/issues/2940-       emailWord         = do x <- satisfy isAlphaNum-                              xs <- many (satisfy isEmailChar)-                              return (x:xs)-       isEmailChar c     = isAlphaNum c || isEmailPunct c-       isEmailPunct c    = T.any (== c) "!\"#$%&'*+-/=?^_{|}~;"---uriScheme :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Text-uriScheme = oneOfStringsCI (Set.toList schemes)---- | Parses a URI. Returns pair of original and URI-escaped version.-uri :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)-uri = try $ do-  scheme <- uriScheme-  char ':'-  -- Avoid parsing e.g. "**Notes:**" as a raw URI:-  notFollowedBy $ satisfy (\c -> c == '*' || c == '_' || c == ']')-  -- We allow sentence punctuation except at the end, since-  -- we don't want the trailing '.' in 'http://google.com.' We want to allow-  -- http://en.wikipedia.org/wiki/State_of_emergency_(disambiguation)-  -- as a URL, while NOT picking up the closing paren in-  -- (http://wikipedia.org). So we include balanced parens in the URL.-  str <- T.concat <$> many1 (uriChunkBetween '(' ')'-                        <|> uriChunkBetween '{' '}'-                        <|> uriChunkBetween '[' ']'-                        <|> T.pack <$> uriChunk)-  str' <- option str $ char '/' >> return (str <> "/")-  let uri' = scheme <> ":" <> fromEntities str'-  return (uri', escapeURI uri')-  where-    isWordChar '#' = True-    isWordChar '$' = True-    isWordChar '%' = True-    isWordChar '+' = True-    isWordChar '/' = True-    isWordChar '@' = True-    isWordChar '\\' = True-    isWordChar '_' = True-    isWordChar '-' = True-    isWordChar '&' = True-    isWordChar '=' = True-    isWordChar c   = isAlphaNum c--    wordChar = satisfy isWordChar-    percentEscaped = try $ (:) <$> char '%' <*> many1 hexDigit-    entity = try $ pure <$> characterReference-    punct = try $ many1 (char ',') <|> fmap pure (satisfy (\c -> not (isSpace c) && c /= '<' && c /= '>'))-    uriChunk = many1 wordChar-           <|> percentEscaped-           <|> entity-           <|> try (punct <* lookAhead (void wordChar <|> void percentEscaped))-    uriChunkBetween l r = try $ do chunk <- between (char l) (char r) uriChunk-                                   return (T.pack $ [l] ++ chunk ++ [r])--mathInlineWith :: (Stream s m Char, UpdateSourcePos s Char)  => Text -> Text -> ParserT s st m Text-mathInlineWith op cl = try $ do-  textStr op-  when (op == "$") $ notFollowedBy space-  words' <- many1Till (-                       (T.singleton <$>-                          satisfy (\c -> not (isSpaceChar c || c == '\\')))-                   <|> (char '\\' >>-                           -- This next clause is needed because \text{..} can-                           -- contain $, \(\), etc.-                           (try (string "text" >>-                                 (("\\text" <>) <$> inBalancedBraces 0 ""))-                            <|>  (\c -> T.pack ['\\',c]) <$> anyChar))-                   <|> do (blankline <* notFollowedBy' blankline) <|>-                             (spaceChar <* skipMany spaceChar)-                          notFollowedBy (char '$')-                          return " "-                    ) (try $ textStr cl)-  notFollowedBy digit  -- to prevent capture of $5-  return $ trimMath $ T.concat words'- where-  inBalancedBraces :: (Stream s m Char, UpdateSourcePos s Char) => Int -> Text -> ParserT s st m Text-  inBalancedBraces n = fmap T.pack . inBalancedBraces' n . T.unpack--  inBalancedBraces' :: (Stream s m Char, UpdateSourcePos s Char) => Int -> String -> ParserT s st m String-  inBalancedBraces' 0 "" = do-    c <- anyChar-    if c == '{'-       then inBalancedBraces' 1 "{"-       else mzero-  inBalancedBraces' 0 s = return $ reverse s-  inBalancedBraces' numOpen ('\\':xs) = do-    c <- anyChar-    inBalancedBraces' numOpen (c:'\\':xs)-  inBalancedBraces' numOpen xs = do-    c <- anyChar-    case c of-         '}' -> inBalancedBraces' (numOpen - 1) (c:xs)-         '{' -> inBalancedBraces' (numOpen + 1) (c:xs)-         _   -> inBalancedBraces' numOpen (c:xs)--mathDisplayWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text-mathDisplayWith op cl = try $ fmap T.pack $ do-  textStr op-  many1Till (satisfy (/= '\n') <|> (newline <* notFollowedBy' blankline))-            (try $ textStr cl)--mathDisplay :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)-            => ParserT s st m Text-mathDisplay =-      (guardEnabled Ext_tex_math_dollars >> mathDisplayWith "$$" "$$")-  <|> (guardEnabled Ext_tex_math_single_backslash >>-       mathDisplayWith "\\[" "\\]")-  <|> (guardEnabled Ext_tex_math_double_backslash >>-       mathDisplayWith "\\\\[" "\\\\]")--mathInline :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)-           => ParserT s st m Text-mathInline =-      (guardEnabled Ext_tex_math_dollars >> mathInlineWith "$" "$")-  <|> (guardEnabled Ext_tex_math_single_backslash >>-       mathInlineWith "\\(" "\\)")-  <|> (guardEnabled Ext_tex_math_double_backslash >>-       mathInlineWith "\\\\(" "\\\\)")---- | Applies a parser, returns tuple of its results and its horizontal--- displacement (the difference between the source column at the end--- and the source column at the beginning). Vertical displacement--- (source row) is ignored.-withHorizDisplacement :: (Stream s m Char, UpdateSourcePos s Char)-                      => ParserT s st m a  -- ^ Parser to apply-                      -> ParserT s st m (a, Int) -- ^ (result, displacement)-withHorizDisplacement parser = do-  pos1 <- getPosition-  result <- parser-  pos2 <- getPosition-  return (result, sourceColumn pos2 - sourceColumn pos1)---- | Applies a parser and returns the raw string that was parsed,--- along with the value produced by the parser.-withRaw :: Monad m-        => ParsecT Sources st m a-        -> ParsecT Sources st m (a, Text)-withRaw parser = do-  inps1 <- getInput-  result <- parser-  inps2 <- getInput-  -- 'raw' is the difference between inps1 and inps2-  return (result, sourcesDifference inps1 inps2)--sourcesDifference :: Sources -> Sources -> Text-sourcesDifference (Sources is1) (Sources is2) = go is1 is2- where-   go inps1 inps2 =-    case (inps1, inps2) of-      ([], _) -> mempty-      (_, []) -> mconcat $ map snd inps1-      ((p1,t1):rest1, (p2, t2):rest2)-        | p1 == p2-        , t1 == t2  -> go rest1 rest2-        | p1 == p2-        , t1 /= t2  -> fromMaybe mempty $ T.stripSuffix t2 t1-        | otherwise -> t1 <> go rest1 inps2---- | Parses backslash, then applies character parser.-escaped :: (Stream s m Char, UpdateSourcePos s Char)-        => ParserT s st m Char  -- ^ Parser for character to escape-        -> ParserT s st m Char-escaped parser = try $ char '\\' >> parser---- | Parse character entity.-characterReference :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char-characterReference = try $ do-  char '&'-  ent <- many1Till nonspaceChar (char ';')-  let ent' = case ent of-                  '#':'X':xs -> '#':'x':xs  -- workaround tagsoup bug-                  '#':_      -> ent-                  _          -> ent ++ ";"-  case lookupEntity ent' of-       Just (c : _) -> return c-       _            -> Prelude.fail "entity not found"---- | Parses an uppercase roman numeral and returns (UpperRoman, number).-upperRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-upperRoman = do-  num <- romanNumeral True-  return (UpperRoman, num)---- | Parses a lowercase roman numeral and returns (LowerRoman, number).-lowerRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-lowerRoman = do-  num <- romanNumeral False-  return (LowerRoman, num)---- | Parses a decimal numeral and returns (Decimal, number).-decimal :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-decimal = do-  num <- many1 digit-  return (Decimal, fromMaybe 1 $ safeRead $ T.pack num)---- | Parses a '@' and optional label and--- returns (DefaultStyle, [next example number]).  The next--- example number is incremented in parser state, and the label--- (if present) is added to the label table.-exampleNum :: (Stream s m Char, UpdateSourcePos s Char)-           => ParserT s ParserState m (ListNumberStyle, Int)-exampleNum = do-  char '@'-  lab <- mconcat . map T.pack <$>-                    many (many1 alphaNum <|>-                          try (do c <- char '_' <|> char '-'-                                  cs <- many1 alphaNum-                                  return (c:cs)))-  st <- getState-  let num = stateNextExample st-  let newlabels = if T.null lab-                     then stateExamples st-                     else M.insert lab num $ stateExamples st-  updateState $ \s -> s{ stateNextExample = num + 1-                       , stateExamples    = newlabels }-  return (Example, num)---- | Parses a '#' returns (DefaultStyle, 1).-defaultNum :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-defaultNum = do-  char '#'-  return (DefaultStyle, 1)---- | Parses a lowercase letter and returns (LowerAlpha, number).-lowerAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-lowerAlpha = do-  ch <- satisfy isAsciiLower-  return (LowerAlpha, ord ch - ord 'a' + 1)---- | Parses an uppercase letter and returns (UpperAlpha, number).-upperAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-upperAlpha = do-  ch <- satisfy isAsciiUpper-  return (UpperAlpha, ord ch - ord 'A' + 1)---- | Parses a roman numeral i or I-romanOne :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)-romanOne = (char 'i' >> return (LowerRoman, 1)) <|>-           (char 'I' >> return (UpperRoman, 1))---- | Parses an ordered list marker and returns list attributes.-anyOrderedListMarker :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s ParserState m ListAttributes-anyOrderedListMarker = choice-  [delimParser numParser | delimParser <- [inPeriod, inOneParen, inTwoParens],-                           numParser <- [decimal, exampleNum, defaultNum, romanOne,-                           lowerAlpha, lowerRoman, upperAlpha, upperRoman]]---- | Parses a list number (num) followed by a period, returns list attributes.-inPeriod :: (Stream s m Char, UpdateSourcePos s Char)-         => ParserT s st m (ListNumberStyle, Int)-         -> ParserT s st m ListAttributes-inPeriod num = try $ do-  (style, start) <- num-  char '.'-  let delim = if style == DefaultStyle-                 then DefaultDelim-                 else Period-  return (start, style, delim)---- | Parses a list number (num) followed by a paren, returns list attributes.-inOneParen :: (Stream s m Char, UpdateSourcePos s Char)-           => ParserT s st m (ListNumberStyle, Int)-           -> ParserT s st m ListAttributes-inOneParen num = try $ do-  (style, start) <- num-  char ')'-  return (start, style, OneParen)---- | Parses a list number (num) enclosed in parens, returns list attributes.-inTwoParens :: (Stream s m Char, UpdateSourcePos s Char)-            => ParserT s st m (ListNumberStyle, Int)-            -> ParserT s st m ListAttributes-inTwoParens num = try $ do-  char '('-  (style, start) <- num-  char ')'-  return (start, style, TwoParens)---- | Parses an ordered list marker with a given style and delimiter,--- returns number.-orderedListMarker :: (Stream s m Char, UpdateSourcePos s Char)-                  => ListNumberStyle-                  -> ListNumberDelim-                  -> ParserT s ParserState m Int-orderedListMarker style delim = do-  let num = defaultNum <|>  -- # can continue any kind of list-            case style of-               DefaultStyle -> decimal-               Example      -> exampleNum-               Decimal      -> decimal-               UpperRoman   -> upperRoman-               LowerRoman   -> lowerRoman-               UpperAlpha   -> upperAlpha-               LowerAlpha   -> lowerAlpha-  let context = case delim of-               DefaultDelim -> inPeriod-               Period       -> inPeriod-               OneParen     -> inOneParen-               TwoParens    -> inTwoParens-  (start, _, _) <- context num-  return start---- | Parses a character reference and returns a Str element.-charRef :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inline-charRef = Str . T.singleton <$> characterReference--lineBlockLine :: Monad m => ParserT Sources st m Text-lineBlockLine = try $ do-  char '|'-  char ' '-  white <- T.pack <$> many (spaceChar >> return '\160')-  notFollowedBy newline-  line <- anyLine-  continuations <- many (try $ char ' ' >> anyLine)-  return $ white <> T.unwords (line : continuations)--blankLineBlockLine :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char-blankLineBlockLine = try (char '|' >> blankline)---- | Parses an RST-style line block and returns a list of strings.-lineBlockLines :: Monad m => ParserT Sources st m [Text]-lineBlockLines = try $ do-  lines' <- many1 (lineBlockLine <|> (T.singleton <$> blankLineBlockLine))-  skipMany blankline-  return lines'---- | Parse a table using 'headerParser', 'rowParser',--- 'lineParser', and 'footerParser'.-tableWith :: (Stream s m Char, UpdateSourcePos s Char,-              HasReaderOptions st, Monad mf)-          => ParserT s st m (mf [Blocks], [Alignment], [Int])-          -> ([Int] -> ParserT s st m (mf [Blocks]))-          -> ParserT s st m sep-          -> ParserT s st m end-          -> ParserT s st m (mf Blocks)-tableWith headerParser rowParser lineParser footerParser = try $ do-  (aligns, widths, heads, rows) <- tableWith' headerParser rowParser-                                                lineParser footerParser-  let th = TableHead nullAttr <$> heads-      tb = (:[]) . TableBody nullAttr 0 [] <$> rows-      tf = pure $ TableFoot nullAttr []-  return $ B.table B.emptyCaption (zip aligns (map fromWidth widths)) <$> th <*> tb <*> tf-  where-    fromWidth n-      | n > 0     = ColWidth n-      | otherwise = ColWidthDefault--type TableComponents mf = ([Alignment], [Double], mf [Row], mf [Row])--tableWith' :: (Stream s m Char, UpdateSourcePos s Char,-               HasReaderOptions st, Monad mf)-           => ParserT s st m (mf [Blocks], [Alignment], [Int])-           -> ([Int] -> ParserT s st m (mf [Blocks]))-           -> ParserT s st m sep-           -> ParserT s st m end-           -> ParserT s st m (TableComponents mf)-tableWith' headerParser rowParser lineParser footerParser = try $ do-    (heads, aligns, indices) <- headerParser-    lines' <- sequence <$> rowParser indices `sepEndBy1` lineParser-    footerParser-    numColumns <- getOption readerColumns-    let widths = if null indices-                    then replicate (length aligns) 0.0-                    else widthsFromIndices numColumns indices-    let toRow =  Row nullAttr . map B.simpleCell-        toHeaderRow l = [toRow l | not (null l)]-    return (aligns, widths, toHeaderRow <$> heads, map toRow <$> lines')---- Calculate relative widths of table columns, based on indices-widthsFromIndices :: Int      -- Number of columns on terminal-                  -> [Int]    -- Indices-                  -> [Double] -- Fractional relative sizes of columns-widthsFromIndices _ [] = []-widthsFromIndices numColumns' indices =-  let numColumns = max numColumns' (if null indices then 0 else last indices)-      lengths' = zipWith (-) indices (0:indices)-      lengths  = reverse $-                 case reverse lengths' of-                      []       -> []-                      [x]      -> [x]-                      -- compensate for the fact that intercolumn-                      -- spaces are counted in widths of all columns-                      -- but the last...-                      (x:y:zs) -> if x < y && y - x <= 2-                                     then y:y:zs-                                     else x:y:zs-      totLength = sum lengths-      quotient = if totLength > numColumns-                   then fromIntegral totLength-                   else fromIntegral numColumns-      fracs = map (\l -> fromIntegral l / quotient) lengths in-  tail fracs--------- Parse a grid table:  starts with row of '-' on top, then header--- (which may be grid), then the rows,--- which may be grid, separated by blank lines, and--- ending with a footer (dashed line followed by blank line).-gridTableWith :: (Monad m, HasReaderOptions st, HasLastStrPosition st, Monad mf)-              => ParserT Sources st m (mf Blocks)  -- ^ Block list parser-              -> Bool                        -- ^ Headerless table-              -> ParserT Sources st m (mf Blocks)-gridTableWith blocks headless =-  tableWith (gridTableHeader headless blocks) (gridTableRow blocks)-            (gridTableSep '-') gridTableFooter--gridTableWith' :: (Monad m, HasReaderOptions st, HasLastStrPosition st,-                   Monad mf)-               => ParserT Sources st m (mf Blocks)  -- ^ Block list parser-               -> Bool                        -- ^ Headerless table-               -> ParserT Sources st m (TableComponents mf)-gridTableWith' blocks headless =-  tableWith' (gridTableHeader headless blocks) (gridTableRow blocks)-             (gridTableSep '-') gridTableFooter--gridTableSplitLine :: [Int] -> Text -> [Text]-gridTableSplitLine indices line = map removeFinalBar $ tail $-  splitTextByIndices (init indices) $ trimr line--gridPart :: Monad m => Char -> ParserT Sources st m ((Int, Int), Alignment)-gridPart ch = do-  leftColon <- option False (True <$ char ':')-  dashes <- many1 (char ch)-  rightColon <- option False (True <$ char ':')-  char '+'-  let lengthDashes = length dashes + (if leftColon then 1 else 0) +-                       (if rightColon then 1 else 0)-  let alignment = case (leftColon, rightColon) of-                       (True, True)   -> AlignCenter-                       (True, False)  -> AlignLeft-                       (False, True)  -> AlignRight-                       (False, False) -> AlignDefault-  return ((lengthDashes, lengthDashes + 1), alignment)--gridDashedLines :: Monad m => Char -> ParserT Sources st m [((Int, Int), Alignment)]-gridDashedLines ch = try $ char '+' >> many1 (gridPart ch) <* blankline--removeFinalBar :: Text -> Text-removeFinalBar = T.dropWhileEnd go . T.dropWhileEnd (=='|')-  where-    go c = T.any (== c) " \t"---- | Separator between rows of grid table.-gridTableSep :: Monad m => Char -> ParserT Sources st m Char-gridTableSep ch = try $ gridDashedLines ch >> return '\n'---- | Parse header for a grid table.-gridTableHeader :: (Monad m, Monad mf, HasLastStrPosition st)-                => Bool -- ^ Headerless table-                -> ParserT Sources st m (mf Blocks)-                -> ParserT Sources st m (mf [Blocks], [Alignment], [Int])-gridTableHeader True _ = do-  optional blanklines-  dashes <- gridDashedLines '-'-  let aligns = map snd dashes-  let lines'   = map (snd . fst) dashes-  let indices  = scanl (+) 0 lines'-  return (return [], aligns, indices)-gridTableHeader False blocks = try $ do-  optional blanklines-  dashes <- gridDashedLines '-'-  rawContent  <- many1 (notFollowedBy (gridTableSep '=') >> char '|' >>-                           T.pack <$> many1Till anyChar newline)-  underDashes <- gridDashedLines '='-  guard $ length dashes == length underDashes-  let lines'   = map (snd . fst) underDashes-  let indices  = scanl (+) 0 lines'-  let aligns   = map snd underDashes-  let rawHeads = map (T.unlines . map trim) $ transpose-                       $ map (gridTableSplitLine indices) rawContent-  heads <- sequence <$> mapM (parseFromString' blocks . trim) rawHeads-  return (heads, aligns, indices)--gridTableRawLine :: (Stream s m Char, UpdateSourcePos s Char) => [Int] -> ParserT s st m [Text]-gridTableRawLine indices = do-  char '|'-  line <- many1Till anyChar newline-  return (gridTableSplitLine indices $ T.pack line)---- | Parse row of grid table.-gridTableRow :: (Monad m, Monad mf, HasLastStrPosition st)-             => ParserT Sources st m (mf Blocks)-             -> [Int]-             -> ParserT Sources st m (mf [Blocks])-gridTableRow blocks indices = do-  colLines <- many1 (gridTableRawLine indices)-  let cols = map ((<> "\n") . T.unlines . removeOneLeadingSpace) $-               transpose colLines-      compactifyCell bs = case compactify [bs] of-                            []  -> mempty-                            x:_ -> x-  cells <- sequence <$> mapM (parseFromString' blocks) cols-  return $ fmap (map compactifyCell) cells--removeOneLeadingSpace :: [Text] -> [Text]-removeOneLeadingSpace xs =-  if all startsWithSpace xs-     then map (T.drop 1) xs-     else xs-   where startsWithSpace t = case T.uncons t of-           Nothing     -> True-           Just (c, _) -> c == ' '---- | Parse footer for a grid table.-gridTableFooter :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m ()-gridTableFooter = optional blanklines--------- | Removes the ParsecT layer from the monad transformer stack-readWithM :: (Monad m, ToSources t)-          => ParserT Sources st m a  -- ^ parser-          -> st                      -- ^ initial state-          -> t                       -- ^ input-          -> m (Either PandocError a)-readWithM parser state input =-    mapLeft (PandocParsecError sources)-      <$> runParserT parser state (initialSourceName sources) sources- where-   sources = toSources input---- | Parse a string with a given parser and state-readWith :: ToSources t-         => Parser Sources st a-         -> st-         -> t-         -> Either PandocError a-readWith p t inp = runIdentity $ readWithM p t inp---- | Parse a string with @parser@ (for testing).-testStringWith :: Show a-               => ParserT Sources ParserState Identity a-               -> Text-               -> IO ()-testStringWith parser str = UTF8.putStrLn $ tshow $-                            readWith parser defaultParserState (toSources str)---- | Parsing options.-data ParserState = ParserState-    { stateOptions           :: ReaderOptions, -- ^ User options-      stateParserContext     :: ParserContext, -- ^ Inside list?-      stateQuoteContext      :: QuoteContext,  -- ^ Inside quoted environment?-      stateAllowLinks        :: Bool,          -- ^ Allow parsing of links-      stateAllowLineBreaks   :: Bool,          -- ^ Allow parsing of line breaks-      stateMaxNestingLevel   :: Int,           -- ^ Max # of nested Strong/Emph-      stateLastStrPos        :: Maybe SourcePos, -- ^ Position after last str parsed-      stateKeys              :: KeyTable,      -- ^ List of reference keys-      stateHeaderKeys        :: KeyTable,      -- ^ List of implicit header ref keys-      stateSubstitutions     :: SubstTable,    -- ^ List of substitution references-      stateNotes             :: NoteTable,     -- ^ List of notes (raw bodies)-      stateNotes'            :: NoteTable',    -- ^ List of notes (parsed bodies)-      stateNoteRefs          :: Set.Set Text, -- ^ List of note references used-      stateInNote            :: Bool,          -- ^ True if parsing note contents-      stateNoteNumber        :: Int,           -- ^ Last note number for citations-      stateMeta              :: Meta,          -- ^ Document metadata-      stateMeta'             :: Future ParserState Meta, -- ^ Document metadata-      stateCitations         :: M.Map Text Text, -- ^ RST-style citations-      stateHeaderTable       :: [HeaderType],  -- ^ Ordered list of header types used-      stateIdentifiers       :: Set.Set Text, -- ^ Header identifiers used-      stateNextExample       :: Int,           -- ^ Number of next example-      stateExamples          :: M.Map Text Int, -- ^ Map from example labels to numbers-      stateMacros            :: M.Map Text Macro, -- ^ Table of macros defined so far-      stateRstDefaultRole    :: Text,        -- ^ Current rST default interpreted text role-      stateRstHighlight      :: Maybe Text,  -- ^ Current rST literal block language-      stateRstCustomRoles    :: M.Map Text (Text, Maybe Text, Attr), -- ^ Current rST custom text roles-      -- Triple represents: 1) Base role, 2) Optional format (only for :raw:-      -- roles), 3) Additional classes (rest of Attr is unused)).-      stateCaption           :: Maybe Inlines, -- ^ Caption in current environment-      stateInHtmlBlock       :: Maybe Text,  -- ^ Tag type of HTML block being parsed-      stateFencedDivLevel    :: Int,           -- ^ Depth of fenced div-      stateContainers        :: [Text],      -- ^ parent include files-      stateLogMessages       :: [LogMessage],  -- ^ log messages-      stateMarkdownAttribute :: Bool         -- ^ True if in markdown=1 context-    }--instance Default ParserState where-  def = defaultParserState--instance HasMeta ParserState where-  setMeta field val st =-    st{ stateMeta = setMeta field val $ stateMeta st }-  deleteMeta field st =-    st{ stateMeta = deleteMeta field $ stateMeta st }--class HasReaderOptions st where-  extractReaderOptions :: st -> ReaderOptions-  getOption            :: (Stream s m t) => (ReaderOptions -> b) -> ParserT s st m b-  -- default-  getOption  f         = f . extractReaderOptions <$> getState--instance HasReaderOptions ParserState where-  extractReaderOptions = stateOptions--class HasQuoteContext st m where-  getQuoteContext :: (Stream s m t) => ParsecT s st m QuoteContext-  withQuoteContext :: QuoteContext -> ParsecT s st m a -> ParsecT s st m a--instance Monad m => HasQuoteContext ParserState m where-  getQuoteContext = stateQuoteContext <$> getState-  withQuoteContext context parser = do-    oldState <- getState-    let oldQuoteContext = stateQuoteContext oldState-    setState oldState { stateQuoteContext = context }-    result <- parser-    newState <- getState-    setState newState { stateQuoteContext = oldQuoteContext }-    return result--class HasIdentifierList st where-  extractIdentifierList  :: st -> Set.Set Text-  updateIdentifierList   :: (Set.Set Text -> Set.Set Text) -> st -> st--instance HasIdentifierList ParserState where-  extractIdentifierList     = stateIdentifiers-  updateIdentifierList f st = st{ stateIdentifiers = f $ stateIdentifiers st }--class HasMacros st where-  extractMacros         :: st -> M.Map Text Macro-  updateMacros          :: (M.Map Text Macro -> M.Map Text Macro) -> st -> st--instance HasMacros ParserState where-  extractMacros        = stateMacros-  updateMacros f st    = st{ stateMacros = f $ stateMacros st }--class HasLastStrPosition st where-  setLastStrPos  :: Maybe SourcePos -> st -> st-  getLastStrPos  :: st -> Maybe SourcePos--instance HasLastStrPosition ParserState where-  setLastStrPos pos st = st{ stateLastStrPos = pos }-  getLastStrPos st     = stateLastStrPos st--class HasLogMessages st where-  addLogMessage :: LogMessage -> st -> st-  getLogMessages :: st -> [LogMessage]--instance HasLogMessages ParserState where-  addLogMessage msg st = st{ stateLogMessages = msg : stateLogMessages st }-  getLogMessages st = reverse $ stateLogMessages st--class HasIncludeFiles st where-  getIncludeFiles :: st -> [Text]-  addIncludeFile :: Text -> st -> st-  dropLatestIncludeFile :: st -> st--instance HasIncludeFiles ParserState where-  getIncludeFiles = stateContainers-  addIncludeFile f s = s{ stateContainers = f : stateContainers s }-  dropLatestIncludeFile s = s { stateContainers = drop 1 $ stateContainers s }--defaultParserState :: ParserState-defaultParserState =-    ParserState { stateOptions         = def,-                  stateParserContext   = NullState,-                  stateQuoteContext    = NoQuote,-                  stateAllowLinks      = True,-                  stateAllowLineBreaks = True,-                  stateMaxNestingLevel = 6,-                  stateLastStrPos      = Nothing,-                  stateKeys            = M.empty,-                  stateHeaderKeys      = M.empty,-                  stateSubstitutions   = M.empty,-                  stateNotes           = [],-                  stateNotes'          = M.empty,-                  stateNoteRefs        = Set.empty,-                  stateInNote          = False,-                  stateNoteNumber      = 0,-                  stateMeta            = nullMeta,-                  stateMeta'           = return nullMeta,-                  stateCitations       = M.empty,-                  stateHeaderTable     = [],-                  stateIdentifiers     = Set.empty,-                  stateNextExample     = 1,-                  stateExamples        = M.empty,-                  stateMacros          = M.empty,-                  stateRstDefaultRole  = "title-reference",-                  stateRstHighlight    = Nothing,-                  stateRstCustomRoles  = M.empty,-                  stateCaption         = Nothing,-                  stateInHtmlBlock     = Nothing,-                  stateFencedDivLevel  = 0,-                  stateContainers      = [],-                  stateLogMessages     = [],-                  stateMarkdownAttribute = False-                  }---- | Add a log message.-logMessage :: (Stream s m a, HasLogMessages st)-           => LogMessage -> ParserT s st m ()-logMessage msg = updateState (addLogMessage msg)---- | Report all the accumulated log messages, according to verbosity level.-reportLogMessages :: (PandocMonad m, HasLogMessages st) => ParserT s st m ()-reportLogMessages = do-  msgs <- getLogMessages <$> getState-  mapM_ report msgs---- | Succeed only if the extension is enabled.-guardEnabled :: (Stream s m a,  HasReaderOptions st) => Extension -> ParserT s st m ()-guardEnabled ext = getOption readerExtensions >>= guard . extensionEnabled ext---- | Succeed only if the extension is disabled.-guardDisabled :: (Stream s m a, HasReaderOptions st) => Extension -> ParserT s st m ()-guardDisabled ext = getOption readerExtensions >>= guard . not . extensionEnabled ext---- | Update the position on which the last string ended.-updateLastStrPos :: (Stream s m a, HasLastStrPosition st) => ParserT s st m ()-updateLastStrPos = getPosition >>= updateState . setLastStrPos . Just---- | Whether we are right after the end of a string.-notAfterString :: (Stream s m a, HasLastStrPosition st) => ParserT s st m Bool-notAfterString = do-  pos <- getPosition-  st  <- getState-  return $ getLastStrPos st /= Just pos--data HeaderType-    = SingleHeader Char  -- ^ Single line of characters underneath-    | DoubleHeader Char  -- ^ Lines of characters above and below-    deriving (Eq, Show)--data ParserContext-    = ListItemState   -- ^ Used when running parser on list item contents-    | NullState       -- ^ Default state-    deriving (Eq, Show)--data QuoteContext-    = InSingleQuote   -- ^ Used when parsing inside single quotes-    | InDoubleQuote   -- ^ Used when parsing inside double quotes-    | NoQuote         -- ^ Used when not parsing inside quotes-    deriving (Eq, Show)--type NoteTable = [(Text, Text)]--type NoteTable' = M.Map Text (SourcePos, Future ParserState Blocks)--- used in markdown reader--newtype Key = Key Text deriving (Show, Read, Eq, Ord)--toKey :: Text -> Key-toKey = Key . T.toLower . T.unwords . T.words . unbracket-  where unbracket t-          | Just ('[', t') <- T.uncons t-          , Just (t'', ']') <- T.unsnoc t'-          = t''-          | otherwise-          = t--type KeyTable = M.Map Key (Target, Attr)--type SubstTable = M.Map Key Inlines----  | Add header to the list of headers in state, together---  with its associated identifier.  If the identifier is null---  and the auto_identifiers extension is set, generate a new---  unique identifier, and update the list of identifiers---  in state.  Issue a warning if an explicit identifier---  is encountered that duplicates an earlier identifier---  (explicit or automatically generated).-registerHeader :: (Stream s m a, HasReaderOptions st,-                   HasLogMessages st, HasIdentifierList st)-               => Attr -> Inlines -> ParserT s st m Attr-registerHeader (ident,classes,kvs) header' = do-  ids <- extractIdentifierList <$> getState-  exts <- getOption readerExtensions-  if T.null ident && Ext_auto_identifiers `extensionEnabled` exts-     then do-       let id' = uniqueIdent exts (B.toList header') ids-       let id'' = if Ext_ascii_identifiers `extensionEnabled` exts-                     then toAsciiText id'-                     else id'-       updateState $ updateIdentifierList $ Set.insert id'-       updateState $ updateIdentifierList $ Set.insert id''-       return (id'',classes,kvs)-     else do-        unless (T.null ident) $ do-          when (ident `Set.member` ids) $ do-            pos <- getPosition-            logMessage $ DuplicateIdentifier ident pos-          updateState $ updateIdentifierList $ Set.insert ident-        return (ident,classes,kvs)--smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st,-                     HasQuoteContext st m,-                     Stream s m Char, UpdateSourcePos s Char)-                 => ParserT s st m Inlines-                 -> ParserT s st m Inlines-smartPunctuation inlineParser = do-  guardEnabled Ext_smart-  choice [ quoted inlineParser, apostrophe, doubleCloseQuote, dash, ellipses ]--quoted :: (HasLastStrPosition st, HasQuoteContext st m,-           Stream s m Char, UpdateSourcePos s Char)-       => ParserT s st m Inlines-       -> ParserT s st m Inlines-quoted inlineParser = doubleQuoted inlineParser <|> singleQuoted inlineParser--singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m,-                 Stream s m Char, UpdateSourcePos s Char)-             => ParserT s st m Inlines-             -> ParserT s st m Inlines-singleQuoted inlineParser = do-  singleQuoteStart-  (B.singleQuoted . mconcat <$>-    try-     (withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)))-   <|> pure "\8217"--doubleQuoted :: (HasQuoteContext st m, HasLastStrPosition st,-                 Stream s m Char, UpdateSourcePos s Char)-             => ParserT s st m Inlines-             -> ParserT s st m Inlines-doubleQuoted inlineParser = do-  doubleQuoteStart-  (B.doubleQuoted . mconcat <$>-    try-     (withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)))-   <|> pure (B.str "\8220")--failIfInQuoteContext :: (HasQuoteContext st m, Stream s m t)-                     => QuoteContext-                     -> ParserT s st m ()-failIfInQuoteContext context = do-  context' <- getQuoteContext-  when (context' == context) $ Prelude.fail "already inside quotes"--charOrRef :: (Stream s m Char, UpdateSourcePos s Char) => [Char] -> ParserT s st m Char-charOrRef cs =-  oneOf cs <|> try (do c <- characterReference-                       guard (c `elem` cs)-                       return c)--singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m,-                     Stream s m Char, UpdateSourcePos s Char)-                 => ParserT s st m ()-singleQuoteStart = do-  failIfInQuoteContext InSingleQuote-  -- single quote start can't be right after str-  guard =<< notAfterString-  try $ do-    charOrRef "'\8216\145"-    void $ lookAhead (satisfy (not . isSpaceChar))--singleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)-               => ParserT s st m ()-singleQuoteEnd = try $ do-  charOrRef "'\8217\146"-  notFollowedBy alphaNum--doubleQuoteStart :: (HasLastStrPosition st,-                     HasQuoteContext st m,-                     Stream s m Char, UpdateSourcePos s Char)-                 => ParserT s st m ()-doubleQuoteStart = do-  failIfInQuoteContext InDoubleQuote-  guard =<< notAfterString-  try $ do charOrRef "\"\8220\147"-           void $ lookAhead (satisfy (not . isSpaceChar))--doubleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)-               => ParserT s st m ()-doubleQuoteEnd = void (charOrRef "\"\8221\148")--apostrophe :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines-apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\8217")--doubleCloseQuote :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inlines-doubleCloseQuote = B.str "\8221" <$ char '"'--ellipses :: (Stream s m Char, UpdateSourcePos s Char)-         => ParserT s st m Inlines-ellipses = try (string "..." >> return (B.str "\8230"))--dash :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)-     => ParserT s st m Inlines-dash = try $ do-  oldDashes <- extensionEnabled Ext_old_dashes <$> getOption readerExtensions-  if oldDashes-     then do-       char '-'-       (char '-' >> return (B.str "\8212"))-         <|> (lookAhead digit >> return (B.str "\8211"))-     else do-       string "--"-       (char '-' >> return (B.str "\8212"))-         <|> return (B.str "\8211")---- This is used to prevent exponential blowups for things like:--- a**a*a**a*a**a*a**a*a**a*a**a*a**-nested :: Stream s m a-       => ParserT s ParserState m a-       -> ParserT s ParserState m a-nested p = do-  nestlevel <- stateMaxNestingLevel <$>  getState-  guard $ nestlevel > 0-  updateState $ \st -> st{ stateMaxNestingLevel = stateMaxNestingLevel st - 1 }-  res <- p-  updateState $ \st -> st{ stateMaxNestingLevel = nestlevel }-  return res--citeKey :: (Stream s m Char, UpdateSourcePos s Char, HasLastStrPosition st)-        => Bool -- ^ If True, allow expanded @{..} syntax.-        -> ParserT s st m (Bool, Text)-citeKey allowBraced = try $ do-  guard =<< notAfterString-  suppress_author <- option False (True <$ char '-')-  char '@'-  key <- simpleCiteIdentifier-        <|> if allowBraced-               then charsInBalanced '{' '}' (satisfy (not . isSpace))-               else mzero-  return (suppress_author, key)--simpleCiteIdentifier :: (Stream s m Char, UpdateSourcePos s Char)-                      => ParserT s st m Text-simpleCiteIdentifier = do-  firstChar <- alphaNum <|> char '_' <|> char '*' -- @* for wildcard in nocite-  let regchar = satisfy (\c -> isAlphaNum c || c == '_')-  let internal p = try $ p <* lookAhead regchar-  rest <- many $ regchar <|> internal (oneOf ":.#$%&-+?<>~/") <|>-                 try (oneOf ":/" <* lookAhead (char '/'))-  return $ T.pack $ firstChar:rest--token :: (Stream s m t)-      => (t -> Text)-      -> (t -> SourcePos)-      -> (t -> Maybe a)-      -> ParsecT s st m a-token pp pos match = tokenPrim (T.unpack . pp) (\_ t _ -> pos t) match--infixr 5 <+?>-(<+?>) :: (Monoid a) => ParserT s st m a -> ParserT s st m a -> ParserT s st m a-a <+?> b = a >>= flip fmap (try b <|> return mempty) . mappend--extractIdClass :: Attr -> Attr-extractIdClass (ident, cls, kvs) = (ident', cls', kvs')-  where-    ident' = fromMaybe ident (lookup "id" kvs)-    cls'   = maybe cls T.words $ lookup "class" kvs-    kvs'   = filter (\(k,_) -> k /= "id" || k /= "class") kvs--insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)-                   => ParserT a st m b -- ^ parser to apply-                   -> (Text -> a) -- ^ convert Text to stream type-                   -> [FilePath]  -- ^ search path (directories)-                   -> FilePath    -- ^ path of file to include-                   -> Maybe Int   -- ^ start line (negative counts from end)-                   -> Maybe Int   -- ^ end line (negative counts from end)-                   -> ParserT a st m b-insertIncludedFile parser toStream dirs f mbstartline mbendline = do-  oldPos <- getPosition-  oldInput <- getInput-  containers <- getIncludeFiles <$> getState-  when (T.pack f `elem` containers) $-    throwError $ PandocParseError $ T.pack $ "Include file loop at " ++ show oldPos-  updateState $ addIncludeFile $ T.pack f-  mbcontents <- readFileFromDirs dirs f-  contents <- case mbcontents of-                   Just s -> return $ exciseLines mbstartline mbendline s-                   Nothing -> do-                     report $ CouldNotLoadIncludeFile (T.pack f) oldPos-                     return ""-  setInput $ toStream contents-  setPosition $ newPos f (fromMaybe 1 mbstartline) 1-  result <- parser-  setInput oldInput-  setPosition oldPos-  updateState dropLatestIncludeFile-  return result--exciseLines :: Maybe Int -> Maybe Int -> Text -> Text-exciseLines Nothing Nothing t = t-exciseLines mbstartline mbendline t =-  T.unlines $ take (endline' - (startline' - 1))-            $ drop (startline' - 1) contentLines- where-  contentLines = T.lines t-  numLines = length contentLines-  startline' = case mbstartline of-                 Nothing -> 1-                 Just x | x >= 0 -> x-                        | otherwise -> numLines + x -- negative from end-  endline' = case mbendline of-                 Nothing -> numLines-                 Just x | x >= 0 -> x-                        | otherwise -> numLines + x -- negative from end-+{- |+   Module      : Text.Pandoc.Parsing+   Copyright   : Copyright (C) 2006-2022 John MacFarlane+   License     : GNU GPL, version 2 or above++   Maintainer  : John MacFarlane <jgm@berkeley.edu>+   Stability   : alpha+   Portability : portable++A utility library with parsers used in pandoc readers.+-}++module Text.Pandoc.Parsing ( module Text.Pandoc.Sources,+                             countChar,+                             textStr,+                             anyLine,+                             anyLineNewline,+                             indentWith,+                             manyChar,+                             many1Char,+                             manyTillChar,+                             many1TillChar,+                             many1Till,+                             manyUntil,+                             manyUntilChar,+                             sepBy1',+                             notFollowedBy',+                             oneOfStrings,+                             oneOfStringsCI,+                             spaceChar,+                             nonspaceChar,+                             skipSpaces,+                             blankline,+                             blanklines,+                             gobbleSpaces,+                             gobbleAtMostSpaces,+                             enclosed,+                             stringAnyCase,+                             parseFromString,+                             parseFromString',+                             lineClump,+                             charsInBalanced,+                             romanNumeral,+                             emailAddress,+                             uri,+                             mathInline,+                             mathDisplay,+                             withHorizDisplacement,+                             withRaw,+                             escaped,+                             characterReference,+                             upperRoman,+                             lowerRoman,+                             decimal,+                             lowerAlpha,+                             upperAlpha,+                             anyOrderedListMarker,+                             orderedListMarker,+                             charRef,+                             lineBlockLines,+                             tableWith,+                             tableWith',+                             widthsFromIndices,+                             gridTableWith,+                             gridTableWith',+                             TableComponents (..),+                             TableNormalization (..),+                             toTableComponents,+                             toTableComponents',+                             readWith,+                             readWithM,+                             testStringWith,+                             guardEnabled,+                             guardDisabled,+                             updateLastStrPos,+                             notAfterString,+                             logMessage,+                             reportLogMessages,+                             ParserState (..),+                             HasReaderOptions (..),+                             HasIdentifierList (..),+                             HasMacros (..),+                             HasLogMessages (..),+                             HasLastStrPosition (..),+                             HasIncludeFiles (..),+                             defaultParserState,+                             HeaderType (..),+                             ParserContext (..),+                             QuoteContext (..),+                             HasQuoteContext (..),+                             NoteTable,+                             NoteTable',+                             KeyTable,+                             SubstTable,+                             Key (..),+                             toKey,+                             registerHeader,+                             smartPunctuation,+                             singleQuoteStart,+                             singleQuoteEnd,+                             doubleQuoteStart,+                             doubleQuoteEnd,+                             apostrophe,+                             doubleCloseQuote,+                             ellipses,+                             dash,+                             nested,+                             citeKey,+                             Parser,+                             ParserT,+                             Future(..),+                             runF,+                             askF,+                             asksF,+                             returnF,+                             trimInlinesF,+                             token,+                             (<+?>),+                             extractIdClass,+                             insertIncludedFile,+                             -- * Re-exports from Text.Parsec+                             Stream,+                             runParser,+                             runParserT,+                             parse,+                             tokenPrim,+                             anyToken,+                             getInput,+                             setInput,+                             unexpected,+                             skipMany,+                             skipMany1,+                             count,+                             eof,+                             lookAhead,+                             notFollowedBy,+                             many,+                             many1,+                             manyTill,+                             (<|>),+                             (<?>),+                             choice,+                             try,+                             sepBy,+                             sepBy1,+                             sepEndBy,+                             sepEndBy1,+                             endBy,+                             endBy1,+                             option,+                             optional,+                             optionMaybe,+                             getState,+                             setState,+                             updateState,+                             SourcePos,+                             getPosition,+                             setPosition,+                             sourceName,+                             setSourceName,+                             sourceColumn,+                             sourceLine,+                             setSourceColumn,+                             setSourceLine,+                             incSourceColumn,+                             incSourceLine,+                             newPos,+                             initialPos,+                             Line,+                             Column,+                             ParseError+                             )+where++import Text.Pandoc.Sources+import Text.Parsec+    ( setSourceName,+      Column,+      Line,+      incSourceLine,+      incSourceColumn,+      setSourceLine,+      setSourceColumn,+      sourceLine,+      sourceColumn,+      sourceName,+      setSourceName,+      setPosition,+      getPosition,+      updateState,+      setState,+      getState,+      optionMaybe,+      optional,+      option,+      endBy1,+      endBy,+      sepEndBy1,+      sepEndBy,+      sepBy1,+      sepBy,+      try,+      choice,+      (<?>),+      (<|>),+      manyTill,+      many1,+      many,+      notFollowedBy,+      lookAhead,+      eof,+      count,+      skipMany1,+      skipMany,+      unexpected,+      setInput,+      getInput,+      anyToken,+      tokenPrim,+      parse,+      runParserT,+      runParser,+      ParseError,+      SourcePos,+      Stream(..) )+import Text.Parsec.Pos (initialPos, newPos)+import Text.Pandoc.Parsing.Capabilities+    ( guardDisabled,+      guardEnabled,+      logMessage,+      notAfterString,+      reportLogMessages,+      updateLastStrPos,+      HasIdentifierList(..),+      HasIncludeFiles(..),+      HasLastStrPosition(..),+      HasLogMessages(..),+      HasMacros(..),+      HasQuoteContext(..),+      HasReaderOptions(..),+      QuoteContext(..) )+import Text.Pandoc.Parsing.Citations ( citeKey )+import Text.Pandoc.Parsing.General+    ( readWithM,+      (<+?>),+      anyLine,+      anyLineNewline,+      blankline,+      blanklines,+      charRef,+      characterReference,+      charsInBalanced,+      countChar,+      emailAddress,+      enclosed,+      escaped,+      extractIdClass,+      gobbleAtMostSpaces,+      gobbleSpaces,+      indentWith,+      insertIncludedFile,+      lineBlockLines,+      lineClump,+      many1Char,+      many1Till,+      many1TillChar,+      manyChar,+      manyTillChar,+      manyUntil,+      manyUntilChar,+      nested,+      nonspaceChar,+      notFollowedBy',+      oneOfStrings,+      oneOfStringsCI,+      parseFromString,+      parseFromString',+      readWith,+      registerHeader,+      sepBy1',+      skipSpaces,+      spaceChar,+      stringAnyCase,+      testStringWith,+      textStr,+      token,+      trimInlinesF,+      uri,+      withHorizDisplacement,+      withRaw )+import Text.Pandoc.Parsing.GridTable+    ( gridTableWith,+      gridTableWith',+      tableWith,+      tableWith',+      toTableComponents,+      toTableComponents',+      widthsFromIndices,+      TableComponents(..),+      TableNormalization(..) )+import Text.Pandoc.Parsing.Lists+    ( anyOrderedListMarker,+      decimal,+      lowerAlpha,+      lowerRoman,+      orderedListMarker,+      romanNumeral,+      upperAlpha,+      upperRoman )+import Text.Pandoc.Parsing.Math ( mathDisplay, mathInline )+import Text.Pandoc.Parsing.Smart+    ( apostrophe,+      dash,+      doubleCloseQuote,+      doubleQuoteEnd,+      doubleQuoteStart,+      ellipses,+      singleQuoteEnd,+      singleQuoteStart,+      smartPunctuation )+import Text.Pandoc.Parsing.State+    ( defaultParserState,+      toKey,+      HeaderType(..),+      Key(..),+      KeyTable,+      NoteTable,+      NoteTable',+      ParserContext(..),+      ParserState(..),+      SubstTable )+import Text.Pandoc.Parsing.Types+    ( ParserT, askF, asksF, returnF, runF, Future(..), Parser )
+ src/Text/Pandoc/Parsing/Capabilities.hs view
@@ -0,0 +1,137 @@+{-# LANGUAGE MultiParamTypeClasses      #-}+{- |+   Module      : Text.Pandoc.Parsing+   Copyright   : © 2006-2022 John MacFarlane+   License     : GPL-2.0-or-later+   Maintainer  : John MacFarlane <jgm@berkeley.edu>++Parser state capabilities.+-}+module Text.Pandoc.Parsing.Capabilities+  ( -- * Capabilities++    -- ** Element identifiers+    HasIdentifierList (..)++    -- ** Include files+  , HasIncludeFiles (..)++    -- ** String/Word boundaries+  , HasLastStrPosition (..)+  , updateLastStrPos+  , notAfterString++    -- ** Logging+  , HasLogMessages (..)+  , logMessage+  , reportLogMessages++    -- ** Macros+  , HasMacros (..)++    -- ** Quote context+  , QuoteContext (..)+  , HasQuoteContext (..)+  , failIfInQuoteContext++    -- ** Reader options+  , HasReaderOptions (..)+  , guardEnabled+  , guardDisabled+  )+where++import Control.Monad (guard, when)+import Data.Text (Text)+import Text.Parsec (ParsecT, SourcePos, Stream, getPosition, getState, updateState)+import Text.Pandoc.Class.PandocMonad (PandocMonad, report)+import Text.Pandoc.Logging (LogMessage)+import Text.Pandoc.Options+  ( Extension+  , ReaderOptions(readerExtensions)+  , extensionEnabled+  )+import Text.Pandoc.Parsing.Types+import Text.Pandoc.Readers.LaTeX.Types (Macro)++import qualified Data.Map as M+import qualified Data.Set as Set++class HasReaderOptions st where+  extractReaderOptions :: st -> ReaderOptions+  getOption            :: (Stream s m t) => (ReaderOptions -> b) -> ParserT s st m b+  -- default+  getOption  f         = f . extractReaderOptions <$> getState++class HasQuoteContext st m where+  getQuoteContext :: (Stream s m t) => ParsecT s st m QuoteContext+  withQuoteContext :: QuoteContext -> ParsecT s st m a -> ParsecT s st m a++failIfInQuoteContext :: (HasQuoteContext st m, Stream s m t)+                     => QuoteContext+                     -> ParserT s st m ()+failIfInQuoteContext context = do+  context' <- getQuoteContext+  when (context' == context) $ Prelude.fail "already inside quotes"++class HasIdentifierList st where+  extractIdentifierList  :: st -> Set.Set Text+  updateIdentifierList   :: (Set.Set Text -> Set.Set Text) -> st -> st++class HasMacros st where+  extractMacros          :: st -> M.Map Text Macro+  updateMacros           :: (M.Map Text Macro -> M.Map Text Macro) -> st -> st++class HasLastStrPosition st where+  setLastStrPos  :: Maybe SourcePos -> st -> st+  getLastStrPos  :: st -> Maybe SourcePos++class HasLogMessages st where+  addLogMessage :: LogMessage -> st -> st+  getLogMessages :: st -> [LogMessage]++class HasIncludeFiles st where+  getIncludeFiles :: st -> [Text]+  addIncludeFile :: Text -> st -> st+  dropLatestIncludeFile :: st -> st++-- | Add a log message.+logMessage :: (Stream s m a, HasLogMessages st)+           => LogMessage -> ParserT s st m ()+logMessage msg = updateState (addLogMessage msg)++-- | Report all the accumulated log messages, according to verbosity level.+reportLogMessages :: (PandocMonad m, HasLogMessages st) => ParserT s st m ()+reportLogMessages = do+  msgs <- getLogMessages <$> getState+  mapM_ report msgs++-- | Succeed only if the extension is enabled.+guardEnabled :: (Stream s m a,  HasReaderOptions st)+             => Extension -> ParserT s st m ()+guardEnabled ext =+  getOption readerExtensions >>= guard . extensionEnabled ext++-- | Succeed only if the extension is disabled.+guardDisabled :: (Stream s m a, HasReaderOptions st)+              => Extension -> ParserT s st m ()+guardDisabled ext =+  getOption readerExtensions >>= guard . not . extensionEnabled ext++-- | Update the position on which the last string ended.+updateLastStrPos :: (Stream s m a, HasLastStrPosition st)+                 => ParserT s st m ()+updateLastStrPos = getPosition >>= updateState . setLastStrPos . Just++-- | Whether we are right after the end of a string.+notAfterString :: (Stream s m a, HasLastStrPosition st) => ParserT s st m Bool+notAfterString = do+  pos <- getPosition+  st  <- getState+  return $ getLastStrPos st /= Just pos++data QuoteContext+    = InSingleQuote   -- ^ Used when parsing inside single quotes+    | InDoubleQuote   -- ^ Used when parsing inside double quotes+    | NoQuote         -- ^ Used when not parsing inside quotes+    deriving (Eq, Show)
+ src/Text/Pandoc/Parsing/Citations.hs view
@@ -0,0 +1,57 @@+{-# LANGUAGE FlexibleContexts           #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+Module      : Text.Pandoc.Parsing.Citations+Copyright   : © 2006-2022 John MacFarlane+License     : GPL-2.0-or-later+Maintainer  : John MacFarlane <jgm@berkeley.edu>++Citation parsing.+-}++module Text.Pandoc.Parsing.Citations+  ( citeKey+  )+where++import Control.Monad (guard, MonadPlus(mzero))+import Data.Char (isAlphaNum , isSpace)+import Data.Text (Text)+import Text.Pandoc.Sources+import Text.Parsec+  ( (<|>)+  , Stream(..)+  , lookAhead+  , many+  , option+  , try+  )+import Text.Pandoc.Parsing.Capabilities (HasLastStrPosition, notAfterString)+import Text.Pandoc.Parsing.General+import Text.Pandoc.Parsing.Types (ParserT)++import qualified Data.Text as T++citeKey :: (Stream s m Char, UpdateSourcePos s Char, HasLastStrPosition st)+        => Bool -- ^ If True, allow expanded @{..} syntax.+        -> ParserT s st m (Bool, Text)+citeKey allowBraced = try $ do+  guard =<< notAfterString+  suppress_author <- option False (True <$ char '-')+  char '@'+  key <- simpleCiteIdentifier+        <|> if allowBraced+               then charsInBalanced '{' '}' (satisfy (not . isSpace))+               else mzero+  return (suppress_author, key)++simpleCiteIdentifier :: (Stream s m Char, UpdateSourcePos s Char)+                      => ParserT s st m Text+simpleCiteIdentifier = do+  firstChar <- alphaNum <|> char '_' <|> char '*' -- @* for wildcard in nocite+  let regchar = satisfy (\c -> isAlphaNum c || c == '_')+  let internal p = try $ p <* lookAhead regchar+  rest <- many $ regchar <|> internal (oneOf ":.#$%&-+?<>~/") <|>+                 try (oneOf ":/" <* lookAhead (char '/'))+  return $ T.pack $ firstChar:rest+
+ src/Text/Pandoc/Parsing/General.hs view
@@ -0,0 +1,751 @@+{-# LANGUAGE FlexibleContexts           #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+Module      : Text.Pandoc.Parsing.General+Copyright   : © 2006-2022 John MacFarlane+License     : GPL-2.0-or-later+Maintainer  : John MacFarlane <jgm@berkeley.edu>++Parser combinators for pandoc format readers.+-}++module Text.Pandoc.Parsing.General+  ( (<+?>)+  , anyLine+  , anyLineNewline+  , blankline+  , blanklines+  , charRef+  , characterReference+  , charsInBalanced+  , countChar+  , emailAddress+  , enclosed+  , escaped+  , extractIdClass+  , gobbleAtMostSpaces+  , gobbleSpaces+  , indentWith+  , insertIncludedFile+  , isSpaceChar          -- not re-exported from T.P.Parsing+  , lineBlockLines+  , lineClump+  , many1Char+  , many1Till+  , many1TillChar+  , manyChar+  , manyTillChar+  , manyUntil+  , manyUntilChar+  , nested+  , nonspaceChar+  , notFollowedBy'+  , oneOfStrings+  , oneOfStringsCI+  , parseFromString+  , parseFromString'+  , readWith+  , readWithM+  , registerHeader+  , sepBy1'+  , skipSpaces+  , spaceChar+  , stringAnyCase+  , testStringWith+  , textStr+  , token+  , trimInlinesF+  , uri+  , withHorizDisplacement+  , withRaw+  )+where++import Control.Monad+  ( guard+  , join+  , liftM+  , unless+  , void+  , when+  )+import Control.Monad.Except ( MonadError(throwError) )+import Control.Monad.Identity ( Identity(..), MonadPlus(mzero) )+import Data.Char+  ( chr+  , isAlphaNum+  , isAscii+  , isAsciiUpper+  , isSpace+  , ord+  , toLower+  , toUpper+  )+import Data.Functor (($>))+import Data.List (intercalate)+import Data.Maybe (fromMaybe)+import Data.Text (Text)+import Text.HTML.TagSoup.Entity (lookupEntity)+import Text.Pandoc.Asciify (toAsciiText)+import Text.Pandoc.Builder (Attr, Inline(Str), Inlines, trimInlines)+import Text.Pandoc.Class.PandocMonad (PandocMonad, readFileFromDirs, report)+import Text.Pandoc.Logging+  ( LogMessage(CouldNotLoadIncludeFile, DuplicateIdentifier) )+import Text.Pandoc.Options+  ( extensionEnabled+  , Extension(Ext_auto_identifiers, Ext_ascii_identifiers)+  , ReaderOptions(readerTabStop, readerExtensions) )+import Text.Pandoc.Shared (escapeURI, mapLeft, schemes, tshow, uniqueIdent)+import Text.Pandoc.Sources+import Text.Pandoc.XML (fromEntities)+import Text.Parsec+  ( (<|>)+  , ParsecT+  , SourcePos+  , Stream(..)+  , between+  , choice+  , count+  , getInput+  , getPosition+  , getState+  , lookAhead+  , many+  , many1+  , manyTill+  , notFollowedBy+  , option+  , runParserT+  , setInput+  , setPosition+  , skipMany+  , sourceColumn+  , sourceName+  , tokenPrim+  , try+  , unexpected+  , updateState+  )+import Text.Parsec.Pos (initialPos, newPos)+import Text.Pandoc.Error+  ( PandocError(PandocParseError, PandocParsecError) )+import Text.Pandoc.Parsing.Capabilities+import Text.Pandoc.Parsing.State+import Text.Pandoc.Parsing.Types ( Parser, ParserT, Future (..))++import qualified Data.Set as Set+import qualified Data.Text as T+import qualified Text.Pandoc.Builder as B+import qualified Text.Pandoc.UTF8 as UTF8 (putStrLn)++-- | Remove whitespace from start and end; just like @'trimInlines'@,+-- but lifted into the 'Future' type.+trimInlinesF :: Future s Inlines -> Future s Inlines+trimInlinesF = liftM trimInlines++-- | Like @count@, but packs its result+countChar :: (Stream s m Char, UpdateSourcePos s Char, Monad m)+          => Int+          -> ParsecT s st m Char+          -> ParsecT s st m Text+countChar n = fmap T.pack . count n++-- | Like @string@, but uses @Text@.+textStr :: (Stream s m Char, UpdateSourcePos s Char)+        => Text -> ParsecT s u m Text+textStr t = string (T.unpack t) $> t+++-- | Parse any line of text, returning the contents without the+-- final newline.+anyLine :: Monad m => ParserT Sources st m Text+anyLine = do+  -- This is much faster than:+  -- manyTill anyChar newline+  inp <- getInput+  case inp of+    Sources [] -> mzero+    Sources ((fp,t):inps) ->+      -- we assume that lines don't span different input files+      case T.break (=='\n') t of+           (this, rest)+             | T.null rest+             , not (null inps) ->+                -- line may span different input files, so do it+                 -- character by character+                 T.pack <$> manyTill anyChar newline+             | otherwise -> do --  either end of inputs or newline in rest+                 setInput $ Sources ((fp, rest):inps)+                 char '\n' -- needed so parsec knows we won't match empty string+                           -- and so source pos is updated+                 return this++-- | Parse any line, include the final newline in the output+anyLineNewline :: Monad m => ParserT Sources st m Text+anyLineNewline = (<> "\n") <$> anyLine++-- | Parse indent by specified number of spaces (or equiv. tabs)+indentWith :: (Stream s m Char, UpdateSourcePos s Char)+           => HasReaderOptions st+           => Int -> ParserT s st m Text+indentWith num = do+  tabStop <- getOption readerTabStop+  if num < tabStop+     then countChar num (char ' ')+     else choice [ try (countChar num (char ' '))+                 , try (char '\t' >> indentWith (num - tabStop)) ]++-- | Like @many@, but packs its result.+manyChar :: Stream s m t+         => ParserT s st m Char+         -> ParserT s st m Text+manyChar = fmap T.pack . many++-- | Like @many1@, but packs its result.+many1Char :: Stream s m t+          => ParserT s st m Char+          -> ParserT s st m Text+many1Char = fmap T.pack . many1++-- | Like @manyTill@, but packs its result.+manyTillChar :: Stream s m t+             => ParserT s st m Char+             -> ParserT s st m a+             -> ParserT s st m Text+manyTillChar p = fmap T.pack . manyTill p++-- | Like @manyTill@, but reads at least one item.+many1Till :: (Show end, Stream s m t)+          => ParserT s st m a+          -> ParserT s st m end+          -> ParserT s st m [a]+many1Till p end = do+         notFollowedBy' end+         first <- p+         rest <- manyTill p end+         return (first:rest)++-- | Like @many1Till@, but packs its result+many1TillChar :: (Show end, Stream s m t)+              => ParserT s st m Char+              -> ParserT s st m end+              -> ParserT s st m Text+many1TillChar p = fmap T.pack . many1Till p++-- | Like @manyTill@, but also returns the result of end parser.+manyUntil :: ParserT s u m a+          -> ParserT s u m b+          -> ParserT s u m ([a], b)+manyUntil p end = scan+  where scan =+          (do e <- end+              return ([], e)+          ) <|>+          (do x <- p+              (xs, e) <- scan+              return (x:xs, e))++-- | Like @manyUntil@, but also packs its result.+manyUntilChar :: ParserT s u m Char+              -> ParserT s u m b+              -> ParserT s u m (Text, b)+manyUntilChar p = fmap go . manyUntil p+  where+    go (x, y) = (T.pack x, y)++-- | Like @sepBy1@ from Parsec,+-- but does not fail if it @sep@ succeeds and @p@ fails.+sepBy1' :: ParsecT s u m a+        -> ParsecT s u m sep+        -> ParsecT s u m [a]+sepBy1' p sep = (:) <$> p <*> many (try $ sep >> p)++-- | A more general form of @notFollowedBy@.  This one allows any+-- type of parser to be specified, and succeeds only if that parser fails.+-- It does not consume any input.+notFollowedBy' :: (Show b, Stream s m a) => ParserT s st m b -> ParserT s st m ()+notFollowedBy' p  = try $ join $  do  a <- try p+                                      return (unexpected (show a))+                                  <|>+                                  return (return ())+-- (This version due to Andrew Pimlott on the Haskell mailing list.)++oneOfStrings' :: (Stream s m Char, UpdateSourcePos s Char)+              => (Char -> Char -> Bool) -> [Text] -> ParserT s st m Text+oneOfStrings' f = fmap T.pack . oneOfStrings'' f . fmap T.unpack++-- TODO: This should be re-implemented in a Text-aware way+oneOfStrings'' :: (Stream s m Char, UpdateSourcePos s Char)+               => (Char -> Char -> Bool) -> [String] -> ParserT s st m String+oneOfStrings'' _ []   = Prelude.fail "no strings"+oneOfStrings'' matches strs = try $ do+  c <- anyChar+  let strs' = [xs | (x:xs) <- strs, x `matches` c]+  case strs' of+       []  -> Prelude.fail "not found"+       _   -> (c:) <$> oneOfStrings'' matches strs'+               <|> if "" `elem` strs'+                      then return [c]+                      else Prelude.fail "not found"++-- | Parses one of a list of strings.  If the list contains+-- two strings one of which is a prefix of the other, the longer+-- string will be matched if possible.+oneOfStrings :: (Stream s m Char, UpdateSourcePos s Char)+             => [Text] -> ParserT s st m Text+oneOfStrings = oneOfStrings' (==)++-- | Parses one of a list of strings (tried in order), case insensitive.++-- TODO: This will not be accurate with general Unicode (neither+-- Text.toLower nor Text.toCaseFold can be implemented with a map)+oneOfStringsCI :: (Stream s m Char, UpdateSourcePos s Char)+               => [Text] -> ParserT s st m Text+oneOfStringsCI = oneOfStrings' ciMatch+  where ciMatch x y = toLower' x == toLower' y+        -- this optimizes toLower by checking common ASCII case+        -- first, before calling the expensive unicode-aware+        -- function:+        toLower' c | isAsciiUpper c = chr (ord c + 32)+                   | isAscii c = c+                   | otherwise = toLower c++-- | Parses a space or tab.+spaceChar :: (Stream s m Char, UpdateSourcePos s Char)+          => ParserT s st m Char+spaceChar = satisfy $ \c -> c == ' ' || c == '\t'++-- | Parses a nonspace, nonnewline character.+nonspaceChar :: (Stream s m Char, UpdateSourcePos s Char)+             => ParserT s st m Char+nonspaceChar = satisfy (not . isSpaceChar)++isSpaceChar :: Char -> Bool+isSpaceChar ' '  = True+isSpaceChar '\t' = True+isSpaceChar '\n' = True+isSpaceChar '\r' = True+isSpaceChar _    = False++-- | Skips zero or more spaces or tabs.+skipSpaces :: (Stream s m Char, UpdateSourcePos s Char)+           => ParserT s st m ()+skipSpaces = skipMany spaceChar++-- | Skips zero or more spaces or tabs, then reads a newline.+blankline :: (Stream s m Char, UpdateSourcePos s Char)+          => ParserT s st m Char+blankline = try $ skipSpaces >> newline++-- | Parses one or more blank lines and returns a string of newlines.+blanklines :: (Stream s m Char, UpdateSourcePos s Char)+           => ParserT s st m Text+blanklines = T.pack <$> many1 blankline++-- | Gobble n spaces; if tabs are encountered, expand them+-- and gobble some or all of their spaces, leaving the rest.+gobbleSpaces :: (HasReaderOptions st, Monad m)+             => Int -> ParserT Sources st m ()+gobbleSpaces 0 = return ()+gobbleSpaces n+  | n < 0     = error "gobbleSpaces called with negative number"+  | otherwise = try $ do+      char ' ' <|> eatOneSpaceOfTab+      gobbleSpaces (n - 1)++eatOneSpaceOfTab :: (HasReaderOptions st, Monad m) => ParserT Sources st m Char+eatOneSpaceOfTab = do+  lookAhead (char '\t')+  pos <- getPosition+  tabstop <- getOption readerTabStop+  -- replace the tab on the input stream with spaces+  let numSpaces = tabstop - ((sourceColumn pos - 1) `mod` tabstop)+  inp <- getInput+  setInput $+    case inp of+      Sources [] -> error "eatOneSpaceOfTab - empty Sources list"+      Sources ((fp,t):rest) ->+        -- drop the tab and add spaces+        Sources ((fp, T.replicate numSpaces " " <> T.drop 1 t):rest)+  char ' '++-- | Gobble up to n spaces; if tabs are encountered, expand them+-- and gobble some or all of their spaces, leaving the rest.+gobbleAtMostSpaces :: (HasReaderOptions st, Monad m)+                   => Int -> ParserT Sources st m Int+gobbleAtMostSpaces 0 = return 0+gobbleAtMostSpaces n+  | n < 0     = error "gobbleAtMostSpaces called with negative number"+  | otherwise = option 0 $ do+      char ' ' <|> eatOneSpaceOfTab+      (+ 1) <$> gobbleAtMostSpaces (n - 1)++-- | Parses material enclosed between start and end parsers.+enclosed :: (Show end, Stream s m Char, UpdateSourcePos s Char)+         => ParserT s st m t   -- ^ start parser+         -> ParserT s st m end  -- ^ end parser+         -> ParserT s st m a    -- ^ content parser (to be used repeatedly)+         -> ParserT s st m [a]+enclosed start end parser = try $+  start >> notFollowedBy space >> many1Till parser end++-- | Parse string, case insensitive.+stringAnyCase :: (Stream s m Char, UpdateSourcePos s Char)+              => Text -> ParserT s st m Text+stringAnyCase = fmap T.pack . stringAnyCase' . T.unpack++stringAnyCase' :: (Stream s m Char, UpdateSourcePos s Char)+               => String -> ParserT s st m String+stringAnyCase' [] = string ""+stringAnyCase' (x:xs) = do+  firstChar <- char (toUpper x) <|> char (toLower x)+  rest <- stringAnyCase' xs+  return (firstChar:rest)++-- TODO rewrite by just adding to Sources stream?+-- | Parse contents of 'str' using 'parser' and return result.+parseFromString :: Monad m+                => ParserT Sources st m r+                -> Text+                -> ParserT Sources st m r+parseFromString parser str = do+  oldPos <- getPosition+  oldInput <- getInput+  setInput $ toSources str+  setPosition $ initialPos $ sourceName oldPos <> "_chunk"+  result <- parser+  spaces+  setInput oldInput+  setPosition oldPos+  return result++-- | Like 'parseFromString' but specialized for 'ParserState'.+-- This resets 'stateLastStrPos', which is almost always what we want.+parseFromString' :: (Monad m, HasLastStrPosition u)+                 => ParserT Sources u m a+                 -> Text+                 -> ParserT Sources u m a+parseFromString' parser str = do+  oldLastStrPos <- getLastStrPos <$> getState+  updateState $ setLastStrPos Nothing+  res <- parseFromString parser str+  updateState $ setLastStrPos oldLastStrPos+  return res++-- | Parse raw line block up to and including blank lines.+lineClump :: Monad m => ParserT Sources st m Text+lineClump = blanklines+          <|> (T.unlines <$> many1 (notFollowedBy blankline >> anyLine))++-- | Parse a string of characters between an open character+-- and a close character, including text between balanced+-- pairs of open and close, which must be different. For example,+-- @charsInBalanced '(' ')' anyChar@ will parse "(hello (there))"+-- and return "hello (there)".+charsInBalanced :: (Stream s m Char, UpdateSourcePos s Char) => Char -> Char -> ParserT s st m Char+                -> ParserT s st m Text+charsInBalanced open close parser = try $ do+  char open+  let isDelim c = c == open || c == close+  raw <- many $  T.pack <$> many1 (notFollowedBy (satisfy isDelim) >> parser)+             <|> (do res <- charsInBalanced open close parser+                     return $ T.singleton open <> res <> T.singleton close)+  char close+  return $ T.concat raw++-- old charsInBalanced would be:+-- charsInBalanced open close (noneOf "\n" <|> char '\n' >> notFollowedBy blankline)+-- old charsInBalanced' would be:+-- charsInBalanced open close anyChar++-- Parsers for email addresses and URIs++-- | Parses an email address; returns original and corresponding+-- escaped mailto: URI.+emailAddress :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)+emailAddress = try $ toResult <$> mailbox <*> (char '@' *> domain)+ where toResult mbox dom = let full = fromEntities $ T.pack $ mbox ++ '@':dom+                           in  (full, escapeURI $ "mailto:" <> full)+       mailbox           = intercalate "." <$> (emailWord `sepBy1'` dot)+       domain            = intercalate "." <$> (subdomain `sepBy1'` dot)+       dot               = char '.'+       subdomain         = many1 $ alphaNum <|> innerPunct (=='-')+       -- this excludes some valid email addresses, since an+       -- email could contain e.g. '__', but gives better results+       -- for our purposes, when combined with markdown parsing:+       innerPunct f      = try (satisfy f+                                 <* notFollowedBy (satisfy (not . isAlphaNum)))+       -- technically an email address could begin with a symbol,+       -- but allowing this creates too many problems.+       -- See e.g. https://github.com/jgm/pandoc/issues/2940+       emailWord         = do x <- satisfy isAlphaNum+                              xs <- many (satisfy isEmailChar)+                              return (x:xs)+       isEmailChar c     = isAlphaNum c || isEmailPunct c+       isEmailPunct c    = T.any (== c) "!\"#$%&'*+-/=?^_{|}~;"+++uriScheme :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Text+uriScheme = oneOfStringsCI (Set.toList schemes)++-- | Parses a URI. Returns pair of original and URI-escaped version.+uri :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (Text, Text)+uri = try $ do+  scheme <- uriScheme+  char ':'+  -- Avoid parsing e.g. "**Notes:**" as a raw URI:+  notFollowedBy $ satisfy (\c -> c == '*' || c == '_' || c == ']')+  -- We allow sentence punctuation except at the end, since+  -- we don't want the trailing '.' in 'http://google.com.' We want to allow+  -- http://en.wikipedia.org/wiki/State_of_emergency_(disambiguation)+  -- as a URL, while NOT picking up the closing paren in+  -- (http://wikipedia.org). So we include balanced parens in the URL.+  str <- T.concat <$> many1 (uriChunkBetween '(' ')'+                        <|> uriChunkBetween '{' '}'+                        <|> uriChunkBetween '[' ']'+                        <|> T.pack <$> uriChunk)+  str' <- option str $ char '/' >> return (str <> "/")+  let uri' = scheme <> ":" <> fromEntities str'+  return (uri', escapeURI uri')+  where+    isWordChar '#' = True+    isWordChar '$' = True+    isWordChar '%' = True+    isWordChar '+' = True+    isWordChar '/' = True+    isWordChar '@' = True+    isWordChar '\\' = True+    isWordChar '_' = True+    isWordChar '-' = True+    isWordChar '&' = True+    isWordChar '=' = True+    isWordChar c   = isAlphaNum c++    wordChar = satisfy isWordChar+    percentEscaped = try $ (:) <$> char '%' <*> many1 hexDigit+    entity = try $ pure <$> characterReference+    punct = try $ many1 (char ',') <|> fmap pure (satisfy (\c -> not (isSpace c) && c /= '<' && c /= '>'))+    uriChunk = many1 wordChar+           <|> percentEscaped+           <|> entity+           <|> try (punct <* lookAhead (void wordChar <|> void percentEscaped))+    uriChunkBetween l r = try $ do chunk <- between (char l) (char r) uriChunk+                                   return (T.pack $ [l] ++ chunk ++ [r])++-- | Applies a parser, returns tuple of its results and its horizontal+-- displacement (the difference between the source column at the end+-- and the source column at the beginning). Vertical displacement+-- (source row) is ignored.+withHorizDisplacement :: (Stream s m Char, UpdateSourcePos s Char)+                      => ParserT s st m a  -- ^ Parser to apply+                      -> ParserT s st m (a, Int) -- ^ (result, displacement)+withHorizDisplacement parser = do+  pos1 <- getPosition+  result <- parser+  pos2 <- getPosition+  return (result, sourceColumn pos2 - sourceColumn pos1)++-- | Applies a parser and returns the raw string that was parsed,+-- along with the value produced by the parser.+withRaw :: Monad m+        => ParsecT Sources st m a+        -> ParsecT Sources st m (a, Text)+withRaw parser = do+  inps1 <- getInput+  result <- parser+  inps2 <- getInput+  -- 'raw' is the difference between inps1 and inps2+  return (result, sourcesDifference inps1 inps2)++sourcesDifference :: Sources -> Sources -> Text+sourcesDifference (Sources is1) (Sources is2) = go is1 is2+ where+   go inps1 inps2 =+    case (inps1, inps2) of+      ([], _) -> mempty+      (_, []) -> mconcat $ map snd inps1+      ((p1,t1):rest1, (p2, t2):rest2)+        | p1 == p2+        , t1 == t2  -> go rest1 rest2+        | p1 == p2+        , t1 /= t2  -> fromMaybe mempty $ T.stripSuffix t2 t1+        | otherwise -> t1 <> go rest1 inps2++-- | Parses backslash, then applies character parser.+escaped :: (Stream s m Char, UpdateSourcePos s Char)+        => ParserT s st m Char  -- ^ Parser for character to escape+        -> ParserT s st m Char+escaped parser = try $ char '\\' >> parser++-- | Parse character entity.+characterReference :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char+characterReference = try $ do+  char '&'+  ent <- many1Till nonspaceChar (char ';')+  let ent' = case ent of+                  '#':'X':xs -> '#':'x':xs  -- workaround tagsoup bug+                  '#':_      -> ent+                  _          -> ent ++ ";"+  case lookupEntity ent' of+       Just (c : _) -> return c+       _            -> Prelude.fail "entity not found"++-- | Parses a character reference and returns a Str element.+charRef :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Inline+charRef = Str . T.singleton <$> characterReference++lineBlockLine :: Monad m => ParserT Sources st m Text+lineBlockLine = try $ do+  char '|'+  char ' '+  white <- T.pack <$> many (spaceChar >> return '\160')+  notFollowedBy newline+  line <- anyLine+  continuations <- many (try $ char ' ' >> anyLine)+  return $ white <> T.unwords (line : continuations)++blankLineBlockLine :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m Char+blankLineBlockLine = try (char '|' >> blankline)++-- | Parses an RST-style line block and returns a list of strings.+lineBlockLines :: Monad m => ParserT Sources st m [Text]+lineBlockLines = try $ do+  lines' <- many1 (lineBlockLine <|> (T.singleton <$> blankLineBlockLine))+  skipMany blankline+  return lines'+++-- | Removes the ParsecT layer from the monad transformer stack+readWithM :: (Monad m, ToSources t)+          => ParserT Sources st m a  -- ^ parser+          -> st                      -- ^ initial state+          -> t                       -- ^ input+          -> m (Either PandocError a)+readWithM parser state input =+    mapLeft (PandocParsecError sources)+      <$> runParserT parser state (initialSourceName sources) sources+ where+   sources = toSources input++-- | Parse a string with a given parser and state+readWith :: ToSources t+         => Parser Sources st a+         -> st+         -> t+         -> Either PandocError a+readWith p t inp = runIdentity $ readWithM p t inp++-- | Parse a string with @parser@ (for testing).+testStringWith :: Show a+               => ParserT Sources ParserState Identity a+               -> Text+               -> IO ()+testStringWith parser str = UTF8.putStrLn $ tshow $+                            readWith parser defaultParserState (toSources str)++-- | Add header to the list of headers in state, together+--  with its associated identifier.  If the identifier is null+--  and the auto_identifiers extension is set, generate a new+--  unique identifier, and update the list of identifiers+--  in state.  Issue a warning if an explicit identifier+--  is encountered that duplicates an earlier identifier+--  (explicit or automatically generated).+registerHeader :: (Stream s m a, HasReaderOptions st,+                   HasLogMessages st, HasIdentifierList st)+               => Attr -> Inlines -> ParserT s st m Attr+registerHeader (ident,classes,kvs) header' = do+  ids <- extractIdentifierList <$> getState+  exts <- getOption readerExtensions+  if T.null ident && Ext_auto_identifiers `extensionEnabled` exts+     then do+       let id' = uniqueIdent exts (B.toList header') ids+       let id'' = if Ext_ascii_identifiers `extensionEnabled` exts+                     then toAsciiText id'+                     else id'+       updateState $ updateIdentifierList $ Set.insert id'+       updateState $ updateIdentifierList $ Set.insert id''+       return (id'',classes,kvs)+     else do+        unless (T.null ident) $ do+          when (ident `Set.member` ids) $ do+            pos <- getPosition+            logMessage $ DuplicateIdentifier ident pos+          updateState $ updateIdentifierList $ Set.insert ident+        return (ident,classes,kvs)++-- This is used to prevent exponential blowups for things like:+-- a**a*a**a*a**a*a**a*a**a*a**a*a**+nested :: Stream s m a+       => ParserT s ParserState m a+       -> ParserT s ParserState m a+nested p = do+  nestlevel <- stateMaxNestingLevel <$>  getState+  guard $ nestlevel > 0+  updateState $ \st -> st{ stateMaxNestingLevel = stateMaxNestingLevel st - 1 }+  res <- p+  updateState $ \st -> st{ stateMaxNestingLevel = nestlevel }+  return res++token :: (Stream s m t)+      => (t -> Text)+      -> (t -> SourcePos)+      -> (t -> Maybe a)+      -> ParsecT s st m a+token pp pos match = tokenPrim (T.unpack . pp) (\_ t _ -> pos t) match++infixr 5 <+?>+(<+?>) :: (Monoid a) => ParserT s st m a -> ParserT s st m a -> ParserT s st m a+a <+?> b = a >>= flip fmap (try b <|> return mempty) . mappend++extractIdClass :: Attr -> Attr+extractIdClass (ident, cls, kvs) = (ident', cls', kvs')+  where+    ident' = fromMaybe ident (lookup "id" kvs)+    cls'   = maybe cls T.words $ lookup "class" kvs+    kvs'   = filter (\(k,_) -> k /= "id" || k /= "class") kvs++insertIncludedFile :: (PandocMonad m, HasIncludeFiles st)+                   => ParserT a st m b -- ^ parser to apply+                   -> (Text -> a) -- ^ convert Text to stream type+                   -> [FilePath]  -- ^ search path (directories)+                   -> FilePath    -- ^ path of file to include+                   -> Maybe Int   -- ^ start line (negative counts from end)+                   -> Maybe Int   -- ^ end line (negative counts from end)+                   -> ParserT a st m b+insertIncludedFile parser toStream dirs f mbstartline mbendline = do+  oldPos <- getPosition+  oldInput <- getInput+  containers <- getIncludeFiles <$> getState+  when (T.pack f `elem` containers) $+    throwError $ PandocParseError $ T.pack $ "Include file loop at " ++ show oldPos+  updateState $ addIncludeFile $ T.pack f+  mbcontents <- readFileFromDirs dirs f+  contents <- case mbcontents of+                   Just s -> return $ exciseLines mbstartline mbendline s+                   Nothing -> do+                     report $ CouldNotLoadIncludeFile (T.pack f) oldPos+                     return ""+  setInput $ toStream contents+  setPosition $ newPos f (fromMaybe 1 mbstartline) 1+  result <- parser+  setInput oldInput+  setPosition oldPos+  updateState dropLatestIncludeFile+  return result++exciseLines :: Maybe Int -> Maybe Int -> Text -> Text+exciseLines Nothing Nothing t = t+exciseLines mbstartline mbendline t =+  T.unlines $ take (endline' - (startline' - 1))+            $ drop (startline' - 1) contentLines+ where+  contentLines = T.lines t+  numLines = length contentLines+  startline' = case mbstartline of+                 Nothing -> 1+                 Just x | x >= 0 -> x+                        | otherwise -> numLines + x -- negative from end+  endline' = case mbendline of+                 Nothing -> numLines+                 Just x | x >= 0 -> x+                        | otherwise -> numLines + x -- negative from end
+ src/Text/Pandoc/Parsing/GridTable.hs view
@@ -0,0 +1,289 @@+{-# LANGUAGE FlexibleContexts           #-}+{-# LANGUAGE LambdaCase                 #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+   Module      : Text.Pandoc.Parsing.GridTable+   Copyright   : Copyright (C) 2006-2022 John MacFarlane+   License     : GPL-2.0-or-later+   Maintainer  : John MacFarlane <jgm@berkeley.edu>++Shared parsers for plaintext tables.+-}+module Text.Pandoc.Parsing.GridTable+  ( gridTableWith+  , gridTableWith'+  , tableWith+  , tableWith'+  , widthsFromIndices+    -- * Components of a plain-text table+  , TableComponents (..)+  , TableNormalization (..)+  , toTableComponents+  , toTableComponents'+  )+where++import Control.Monad (guard)+import Data.List (transpose)+import Data.Text (Text)+import Text.Pandoc.Options (ReaderOptions (readerColumns))+import Text.Pandoc.Builder (Blocks)+import Text.Pandoc.Definition+import Text.Pandoc.Parsing.Capabilities+import Text.Pandoc.Parsing.General+import Text.Pandoc.Parsing.Types+import Text.Pandoc.Shared (compactify, splitTextByIndices, trim, trimr)+import Text.Pandoc.Sources+import Text.Parsec+  ( Stream (..), many1, notFollowedBy, option, optional, sepEndBy1, try )++import qualified Data.Text as T+import qualified Text.Pandoc.Builder as B++-- | Collection of components making up a Table block.+data TableComponents = TableComponents+  { tableAttr     :: Attr+  , tableCaption  :: Caption+  , tableColSpecs :: [ColSpec]+  , tableHead     :: TableHead+  , tableBodies   :: [TableBody]+  , tableFoot     :: TableFoot+  }++-- | Creates a table block from the collection of table parts.+tableFromComponents :: TableComponents -> Blocks+tableFromComponents (TableComponents attr capt colspecs th tb tf) =+  B.tableWith attr capt colspecs th tb tf++-- | Bundles basic table components into a single value.+toTableComponents :: [Alignment] -> [Double] -> [Blocks] -> [[Blocks]]+                  -> TableComponents+toTableComponents = toTableComponents' NoNormalization++-- | Bundles basic table components into a single value, performing+-- normalizations as necessary.+toTableComponents' :: TableNormalization+                   -> [Alignment] -> [Double] -> [Blocks] -> [[Blocks]]+                   -> TableComponents+toTableComponents' normalization aligns widths heads rows =+  let th = TableHead nullAttr (toHeaderRow normalization heads)+      tb = TableBody nullAttr 0 [] (map toRow rows)+      tf = TableFoot nullAttr []+      colspecs = toColSpecs aligns widths+  in TableComponents nullAttr B.emptyCaption colspecs th [tb] tf++-- | Combine a list of column alignments and column widths into a list+-- of column specifiers. Both input lists should have the same length.+toColSpecs :: [Alignment]   -- ^ column alignments+           -> [Double]      -- ^ column widths+           -> [ColSpec]+toColSpecs aligns widths = zip aligns (map fromWidth widths')+  where+    fromWidth n+      | n > 0     = ColWidth n+      | otherwise = ColWidthDefault++    -- renormalize widths if greater than 100%:+    totalWidth = sum widths+    widths' = if totalWidth < 1+              then widths+              else map (/ totalWidth) widths++-- | Whether the table header should be normalized, i.e., whether an header row+-- with only empty cells should be omitted.+data TableNormalization+  = NoNormalization+  | NormalizeHeader++--+-- Grid Tables+--++-- | Parse a grid table: starts with row of '-' on top, then header+-- (which may be grid), then the rows, which may be grid, separated by+-- blank lines, and ending with a footer (dashed line followed by blank+-- line).+gridTableWith :: (Monad m, Monad mf, HasLastStrPosition st, HasReaderOptions st)+              => ParserT Sources st m (mf Blocks)  -- ^ Block list parser+              -> Bool                              -- ^ Headerless table+              -> ParserT Sources st m (mf Blocks)+gridTableWith blocks headless =+  tableWith (gridTableHeader headless blocks) (gridTableRow blocks)+            (gridTableSep '-') gridTableFooter++-- | Like @'gridTableWith'@, but returns 'TableComponents' instead of a+-- Table.+gridTableWith' :: (Monad m, Monad mf,+                   HasReaderOptions st, HasLastStrPosition st)+               => TableNormalization+               -> ParserT Sources st m (mf Blocks) -- ^ Block list parser+               -> Bool                             -- ^ Headerless table+               -> ParserT Sources st m (mf TableComponents)+gridTableWith' normalization blocks headless =+  tableWith' normalization+             (gridTableHeader headless blocks) (gridTableRow blocks)+             (gridTableSep '-') gridTableFooter++gridTableSplitLine :: [Int] -> Text -> [Text]+gridTableSplitLine indices line = map removeFinalBar $ tail $+  splitTextByIndices (init indices) $ trimr line++-- | Parses a grid segment, where the grid line is made up from the+-- given char and terminated with a plus (@+@). The grid line may begin+-- and/or end with a colon, signaling column alignment. Returns the size+-- of the grid part and column alignment+gridPart :: Monad m => Char -> ParserT Sources st m (Int, Alignment)+gridPart ch = do+  leftColon <- option False (True <$ char ':')+  dashes <- many1 (char ch)+  rightColon <- option False (True <$ char ':')+  char '+'+  let lengthDashes = length dashes + (if leftColon then 1 else 0) ++                       (if rightColon then 1 else 0)+  let alignment = case (leftColon, rightColon) of+                       (True, True)   -> AlignCenter+                       (True, False)  -> AlignLeft+                       (False, True)  -> AlignRight+                       (False, False) -> AlignDefault+  return (lengthDashes + 1, alignment)++gridDashedLines :: Monad m+                => Char -> ParserT Sources st m [(Int, Alignment)]+gridDashedLines ch = try $ char '+' >> many1 (gridPart ch) <* blankline++removeFinalBar :: Text -> Text+removeFinalBar = T.dropWhileEnd go . T.dropWhileEnd (=='|')+  where+    go c = T.any (== c) " \t"++-- | Separator between rows of grid table.+gridTableSep :: Monad m => Char -> ParserT Sources st m Char+gridTableSep ch = try $ gridDashedLines ch >> return '\n'++-- | Parse header for a grid table.+gridTableHeader :: (Monad m, Monad mf, HasLastStrPosition st)+                => Bool -- ^ Headerless table+                -> ParserT Sources st m (mf Blocks)+                -> ParserT Sources st m (mf [Blocks], [Alignment], [Int])+gridTableHeader True _ = do+  optional blanklines+  dashes <- gridDashedLines '-'+  let aligns = map snd dashes+  let lines'   = map fst dashes+  let indices  = scanl (+) 0 lines'+  return (return [], aligns, indices)+gridTableHeader False blocks = try $ do+  optional blanklines+  dashes <- gridDashedLines '-'+  rawContent  <- many1 (notFollowedBy (gridTableSep '=') >> char '|' >>+                           T.pack <$> many1Till anyChar newline)+  underDashes <- gridDashedLines '='+  guard $ length dashes == length underDashes+  let lines'   = map fst underDashes+  let indices  = scanl (+) 0 lines'+  let aligns   = map snd underDashes+  let rawHeads = map (T.unlines . map trim) $ transpose+                       $ map (gridTableSplitLine indices) rawContent+  heads <- sequence <$> mapM (parseFromString' blocks . trim) rawHeads+  return (heads, aligns, indices)++gridTableRawLine :: (Stream s m Char, UpdateSourcePos s Char)+                 => [Int] -> ParserT s st m [Text]+gridTableRawLine indices = do+  char '|'+  line <- many1Till anyChar newline+  return (gridTableSplitLine indices $ T.pack line)++-- | Parse row of grid table.+gridTableRow :: (Monad m, Monad mf, HasLastStrPosition st)+             => ParserT Sources st m (mf Blocks)+             -> [Int]+             -> ParserT Sources st m (mf [Blocks])+gridTableRow blocks indices = do+  colLines <- many1 (gridTableRawLine indices)+  let cols = map ((<> "\n") . T.unlines . removeOneLeadingSpace) $+               transpose colLines+      compactifyCell bs = case compactify [bs] of+                            []  -> mempty+                            x:_ -> x+  cells <- sequence <$> mapM (parseFromString' blocks) cols+  return $ fmap (map compactifyCell) cells++removeOneLeadingSpace :: [Text] -> [Text]+removeOneLeadingSpace xs =+  if all startsWithSpace xs+     then map (T.drop 1) xs+     else xs+   where startsWithSpace t = case T.uncons t of+           Nothing     -> True+           Just (c, _) -> c == ' '++-- | Parse footer for a grid table.+gridTableFooter :: (Stream s m Char, UpdateSourcePos s Char)+                => ParserT s st m ()+gridTableFooter = optional blanklines++---++-- | Parse a table using 'headerParser', 'rowParser',+-- 'lineParser', and 'footerParser'.+tableWith :: (Stream s m Char, UpdateSourcePos s Char,+              HasReaderOptions st, Monad mf)+          => ParserT s st m (mf [Blocks], [Alignment], [Int]) -- ^ header parser+          -> ([Int] -> ParserT s st m (mf [Blocks]))  -- ^ row parser+          -> ParserT s st m sep                       -- ^ line parser+          -> ParserT s st m end                       -- ^ footer parser+          -> ParserT s st m (mf Blocks)+tableWith hp rp lp fp = fmap tableFromComponents <$>+  tableWith' NoNormalization hp rp lp fp++tableWith' :: (Stream s m Char, UpdateSourcePos s Char,+               HasReaderOptions st, Monad mf)+           => TableNormalization+           -> ParserT s st m (mf [Blocks], [Alignment], [Int]) -- ^ header parser+           -> ([Int] -> ParserT s st m (mf [Blocks]))  -- ^ row parser+           -> ParserT s st m sep                       -- ^ line parser+           -> ParserT s st m end                       -- ^ footer parser+           -> ParserT s st m (mf TableComponents)+tableWith' n11n headerParser rowParser lineParser footerParser = try $ do+  (heads, aligns, indices) <- headerParser+  lines' <- sequence <$> rowParser indices `sepEndBy1` lineParser+  footerParser+  numColumns <- getOption readerColumns+  let widths = if null indices+               then replicate (length aligns) 0.0+               else widthsFromIndices numColumns indices+  return $ toTableComponents' n11n aligns widths <$> heads <*> lines'++toRow :: [Blocks] -> Row+toRow =  Row nullAttr . map B.simpleCell++toHeaderRow :: TableNormalization -> [Blocks] -> [Row]+toHeaderRow = \case+  NoNormalization -> \l -> [toRow l | not (null l)]+  NormalizeHeader -> \l -> [toRow l | not (null l) && not (all null l)]++-- Calculate relative widths of table columns, based on indices+widthsFromIndices :: Int      -- Number of columns on terminal+                  -> [Int]    -- Indices+                  -> [Double] -- Fractional relative sizes of columns+widthsFromIndices _ [] = []+widthsFromIndices numColumns' indices =+  let numColumns = max numColumns' (if null indices then 0 else last indices)+      lengths' = zipWith (-) indices (0:indices)+      lengths  = reverse $+                 case reverse lengths' of+                      []       -> []+                      [x]      -> [x]+                      -- compensate for the fact that intercolumn+                      -- spaces are counted in widths of all columns+                      -- but the last...+                      (x:y:zs) -> if x < y && y - x <= 2+                                     then y:y:zs+                                     else x:y:zs+      totLength = sum lengths+      quotient = if totLength > numColumns+                   then fromIntegral totLength+                   else fromIntegral numColumns+      fracs = map (\l -> fromIntegral l / quotient) lengths in+  tail fracs
+ src/Text/Pandoc/Parsing/Lists.hs view
@@ -0,0 +1,213 @@+{-# LANGUAGE FlexibleContexts           #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+Module      : Text.Pandoc.Parsing.Lists+Copyright   : © 2006-2022 John MacFarlane+License     : GPL-2.0-or-later+Maintainer  : John MacFarlane <jgm@berkeley.edu>++Parsers for list markers.+-}++module Text.Pandoc.Parsing.Lists+  ( anyOrderedListMarker+  , decimal+  , lowerAlpha+  , lowerRoman+  , orderedListMarker+  , romanNumeral+  , upperAlpha+  , upperRoman+  )+where++import Data.Char+  ( isAsciiUpper+  , isAsciiLower+  , ord+  , toLower+  )+import Data.Maybe (fromMaybe)+import Text.Pandoc.Definition+  ( ListNumberDelim(..)+  , ListAttributes+  , ListNumberStyle(..)+  )+import Text.Pandoc.Shared (safeRead)+import Text.Pandoc.Sources+import Text.Parsec+  ( (<|>)+  , Stream(..)+  , choice+  , getState+  , lookAhead+  , many+  , many1+  , option+  , try+  , updateState+  )+import Text.Pandoc.Parsing.State+import Text.Pandoc.Parsing.Types (ParserT)++import qualified Data.Map as M+import qualified Data.Text as T++-- | Parses a roman numeral (uppercase or lowercase), returns number.+romanNumeral :: (Stream s m Char, UpdateSourcePos s Char)+             => Bool                  -- ^ Uppercase if true+             -> ParserT s st m Int+romanNumeral upperCase = do+    let rchar uc = char $ if upperCase then uc else toLower uc+    let one         = rchar 'I'+    let five        = rchar 'V'+    let ten         = rchar 'X'+    let fifty       = rchar 'L'+    let hundred     = rchar 'C'+    let fivehundred = rchar 'D'+    let thousand    = rchar 'M'+    lookAhead $ choice [one, five, ten, fifty, hundred, fivehundred, thousand]+    thousands <- (1000 *) . length <$> many thousand+    ninehundreds <- option 0 $ try $ hundred >> thousand >> return 900+    fivehundreds <- option 0 $ 500 <$ fivehundred+    fourhundreds <- option 0 $ try $ hundred >> fivehundred >> return 400+    hundreds <- (100 *) . length <$> many hundred+    nineties <- option 0 $ try $ ten >> hundred >> return 90+    fifties <- option 0 (50 <$ fifty)+    forties <- option 0 $ try $ ten >> fifty >> return 40+    tens <- (10 *) . length <$> many ten+    nines <- option 0 $ try $ one >> ten >> return 9+    fives <- option 0 (5 <$ five)+    fours <- option 0 $ try $ one >> five >> return 4+    ones <- length <$> many one+    let total = thousands + ninehundreds + fivehundreds + fourhundreds ++                hundreds + nineties + fifties + forties + tens + nines ++                fives + fours + ones+    if total == 0+       then Prelude.fail "not a roman numeral"+       else return total++-- | Parses an uppercase roman numeral and returns (UpperRoman, number).+upperRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+upperRoman = do+  num <- romanNumeral True+  return (UpperRoman, num)++-- | Parses a lowercase roman numeral and returns (LowerRoman, number).+lowerRoman :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+lowerRoman = do+  num <- romanNumeral False+  return (LowerRoman, num)++-- | Parses a decimal numeral and returns (Decimal, number).+decimal :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+decimal = do+  num <- many1 digit+  return (Decimal, fromMaybe 1 $ safeRead $ T.pack num)++-- | Parses a '@' and optional label and+-- returns (DefaultStyle, [next example number]).  The next+-- example number is incremented in parser state, and the label+-- (if present) is added to the label table.+exampleNum :: (Stream s m Char, UpdateSourcePos s Char)+           => ParserT s ParserState m (ListNumberStyle, Int)+exampleNum = do+  char '@'+  lab <- mconcat . map T.pack <$>+                    many (many1 alphaNum <|>+                          try (do c <- char '_' <|> char '-'+                                  cs <- many1 alphaNum+                                  return (c:cs)))+  st <- getState+  let num = stateNextExample st+  let newlabels = if T.null lab+                     then stateExamples st+                     else M.insert lab num $ stateExamples st+  updateState $ \s -> s{ stateNextExample = num + 1+                       , stateExamples    = newlabels }+  return (Example, num)++-- | Parses a '#' returns (DefaultStyle, 1).+defaultNum :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+defaultNum = do+  char '#'+  return (DefaultStyle, 1)++-- | Parses a lowercase letter and returns (LowerAlpha, number).+lowerAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+lowerAlpha = do+  ch <- satisfy isAsciiLower+  return (LowerAlpha, ord ch - ord 'a' + 1)++-- | Parses an uppercase letter and returns (UpperAlpha, number).+upperAlpha :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+upperAlpha = do+  ch <- satisfy isAsciiUpper+  return (UpperAlpha, ord ch - ord 'A' + 1)++-- | Parses a roman numeral i or I+romanOne :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s st m (ListNumberStyle, Int)+romanOne = (char 'i' >> return (LowerRoman, 1)) <|>+           (char 'I' >> return (UpperRoman, 1))++-- | Parses an ordered list marker and returns list attributes.+anyOrderedListMarker :: (Stream s m Char, UpdateSourcePos s Char) => ParserT s ParserState m ListAttributes+anyOrderedListMarker = choice+  [delimParser numParser | delimParser <- [inPeriod, inOneParen, inTwoParens],+                           numParser <- [decimal, exampleNum, defaultNum, romanOne,+                           lowerAlpha, lowerRoman, upperAlpha, upperRoman]]++-- | Parses a list number (num) followed by a period, returns list attributes.+inPeriod :: (Stream s m Char, UpdateSourcePos s Char)+         => ParserT s st m (ListNumberStyle, Int)+         -> ParserT s st m ListAttributes+inPeriod num = try $ do+  (style, start) <- num+  char '.'+  let delim = if style == DefaultStyle+                 then DefaultDelim+                 else Period+  return (start, style, delim)++-- | Parses a list number (num) followed by a paren, returns list attributes.+inOneParen :: (Stream s m Char, UpdateSourcePos s Char)+           => ParserT s st m (ListNumberStyle, Int)+           -> ParserT s st m ListAttributes+inOneParen num = try $ do+  (style, start) <- num+  char ')'+  return (start, style, OneParen)++-- | Parses a list number (num) enclosed in parens, returns list attributes.+inTwoParens :: (Stream s m Char, UpdateSourcePos s Char)+            => ParserT s st m (ListNumberStyle, Int)+            -> ParserT s st m ListAttributes+inTwoParens num = try $ do+  char '('+  (style, start) <- num+  char ')'+  return (start, style, TwoParens)++-- | Parses an ordered list marker with a given style and delimiter,+-- returns number.+orderedListMarker :: (Stream s m Char, UpdateSourcePos s Char)+                  => ListNumberStyle+                  -> ListNumberDelim+                  -> ParserT s ParserState m Int+orderedListMarker style delim = do+  let num = defaultNum <|>  -- # can continue any kind of list+            case style of+               DefaultStyle -> decimal+               Example      -> exampleNum+               Decimal      -> decimal+               UpperRoman   -> upperRoman+               LowerRoman   -> lowerRoman+               UpperAlpha   -> upperAlpha+               LowerAlpha   -> lowerAlpha+  let context = case delim of+               DefaultDelim -> inPeriod+               Period       -> inPeriod+               OneParen     -> inOneParen+               TwoParens    -> inTwoParens+  (start, _, _) <- context num+  return start
+ src/Text/Pandoc/Parsing/Math.hs view
@@ -0,0 +1,96 @@+{-# LANGUAGE FlexibleContexts           #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+Module      : Text.Pandoc.Parsing.Math+Copyright   : © 2006-2022 John MacFarlane+License     : GPL-2.0-or-later+Maintainer  : John MacFarlane <jgm@berkeley.edu>++Parsing of LaTeX math.+-}++module Text.Pandoc.Parsing.Math+  ( mathDisplay+  , mathInline+  )+where++import Control.Monad (mzero, when)+import Data.Text (Text)+import Text.Parsec ((<|>), Stream(..), notFollowedBy, skipMany, try)+import Text.Pandoc.Options+  ( Extension(Ext_tex_math_dollars, Ext_tex_math_single_backslash,+              Ext_tex_math_double_backslash) )+import Text.Pandoc.Parsing.Capabilities (HasReaderOptions, guardEnabled)+import Text.Pandoc.Parsing.General+import Text.Pandoc.Parsing.Types (ParserT)+import Text.Pandoc.Shared (trimMath)+import Text.Pandoc.Sources+  (UpdateSourcePos, anyChar, char, digit, newline, satisfy, space, string)++import qualified Data.Text as T++mathInlineWith :: (Stream s m Char, UpdateSourcePos s Char)  => Text -> Text -> ParserT s st m Text+mathInlineWith op cl = try $ do+  textStr op+  when (op == "$") $ notFollowedBy space+  words' <- many1Till (+                       (T.singleton <$>+                          satisfy (\c -> not (isSpaceChar c || c == '\\')))+                   <|> (char '\\' >>+                           -- This next clause is needed because \text{..} can+                           -- contain $, \(\), etc.+                           (try (string "text" >>+                                 (("\\text" <>) <$> inBalancedBraces 0 ""))+                            <|>  (\c -> T.pack ['\\',c]) <$> anyChar))+                   <|> do (blankline <* notFollowedBy' blankline) <|>+                             (spaceChar <* skipMany spaceChar)+                          notFollowedBy (char '$')+                          return " "+                    ) (try $ textStr cl)+  notFollowedBy digit  -- to prevent capture of $5+  return $ trimMath $ T.concat words'+ where+  inBalancedBraces :: (Stream s m Char, UpdateSourcePos s Char) => Int -> Text -> ParserT s st m Text+  inBalancedBraces n = fmap T.pack . inBalancedBraces' n . T.unpack++  inBalancedBraces' :: (Stream s m Char, UpdateSourcePos s Char) => Int -> String -> ParserT s st m String+  inBalancedBraces' 0 "" = do+    c <- anyChar+    if c == '{'+       then inBalancedBraces' 1 "{"+       else mzero+  inBalancedBraces' 0 s = return $ reverse s+  inBalancedBraces' numOpen ('\\':xs) = do+    c <- anyChar+    inBalancedBraces' numOpen (c:'\\':xs)+  inBalancedBraces' numOpen xs = do+    c <- anyChar+    case c of+         '}' -> inBalancedBraces' (numOpen - 1) (c:xs)+         '{' -> inBalancedBraces' (numOpen + 1) (c:xs)+         _   -> inBalancedBraces' numOpen (c:xs)++mathDisplayWith :: (Stream s m Char, UpdateSourcePos s Char) => Text -> Text -> ParserT s st m Text+mathDisplayWith op cl = try $ fmap T.pack $ do+  textStr op+  many1Till (satisfy (/= '\n') <|> (newline <* notFollowedBy' blankline))+            (try $ textStr cl)++mathDisplay :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)+            => ParserT s st m Text+mathDisplay =+      (guardEnabled Ext_tex_math_dollars >> mathDisplayWith "$$" "$$")+  <|> (guardEnabled Ext_tex_math_single_backslash >>+       mathDisplayWith "\\[" "\\]")+  <|> (guardEnabled Ext_tex_math_double_backslash >>+       mathDisplayWith "\\\\[" "\\\\]")++mathInline :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)+           => ParserT s st m Text+mathInline =+      (guardEnabled Ext_tex_math_dollars >> mathInlineWith "$" "$")+  <|> (guardEnabled Ext_tex_math_single_backslash >>+       mathInlineWith "\\(" "\\)")+  <|> (guardEnabled Ext_tex_math_double_backslash >>+       mathInlineWith "\\\\(" "\\\\)")
+ src/Text/Pandoc/Parsing/Smart.hs view
@@ -0,0 +1,186 @@+{-# LANGUAGE FlexibleContexts           #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+Module      : Text.Pandoc.Parsing.Smart+Copyright   : © 2006-2022 John MacFarlane+License     : GPL-2.0-or-later+Maintainer  : John MacFarlane <jgm@berkeley.edu>++Smart parsing of quotes, dashes, and other character combinations.+-}++module Text.Pandoc.Parsing.Smart+  ( apostrophe+  , dash+  , doubleCloseQuote+  , doubleQuoteEnd+  , doubleQuoteStart+  , doubleQuoted+  , ellipses+  , singleQuoteEnd+  , singleQuoteStart+  , singleQuoted+  , smartPunctuation+  )+where++import Control.Monad (guard , void)+import Text.Pandoc.Builder (Inlines)+import Text.Pandoc.Options+  ( extensionEnabled+  , Extension(Ext_old_dashes, Ext_smart)+  , ReaderOptions(readerExtensions) )+import Text.Pandoc.Sources+import Text.Pandoc.Parsing.Capabilities+import Text.Pandoc.Parsing.General+import Text.Pandoc.Parsing.Types (ParserT)+import Text.Parsec+  ( (<|>)+  , Stream(..)+  , choice+  , lookAhead+  , manyTill+  , notFollowedBy+  , try+  )++import qualified Text.Pandoc.Builder as B++-- | Parses various ASCII punctuation, quotes, and apostrophe in a smart+-- way, inferring their semantic meaning.+--+-- Fails unless the 'Ext_smart' extension has been enabled.+smartPunctuation :: (HasReaderOptions st, HasLastStrPosition st,+                     HasQuoteContext st m,+                     Stream s m Char, UpdateSourcePos s Char)+                 => ParserT s st m Inlines+                 -> ParserT s st m Inlines+smartPunctuation inlineParser = do+  guardEnabled Ext_smart+  choice [ quoted inlineParser, apostrophe, doubleCloseQuote, dash, ellipses ]++-- | Parses inline text in single or double quotes, assumes English+-- quoting conventions.+quoted :: (HasLastStrPosition st, HasQuoteContext st m,+           Stream s m Char, UpdateSourcePos s Char)+       => ParserT s st m Inlines+       -> ParserT s st m Inlines+quoted inlineParser = doubleQuoted inlineParser <|> singleQuoted inlineParser++-- | Parses inline text in single quotes, assumes English quoting+-- conventions.+singleQuoted :: (HasLastStrPosition st, HasQuoteContext st m,+                 Stream s m Char, UpdateSourcePos s Char)+             => ParserT s st m Inlines+             -> ParserT s st m Inlines+singleQuoted inlineParser = do+  singleQuoteStart+  (B.singleQuoted . mconcat <$>+    try+     (withQuoteContext InSingleQuote (many1Till inlineParser singleQuoteEnd)))+   <|> pure "\8217"++-- | Parses inline text in double quotes; assumes English quoting+-- conventions.+doubleQuoted :: (HasQuoteContext st m, HasLastStrPosition st,+                 Stream s m Char, UpdateSourcePos s Char)+             => ParserT s st m Inlines+             -> ParserT s st m Inlines+doubleQuoted inlineParser = do+  doubleQuoteStart+  (B.doubleQuoted . mconcat <$>+    try+     (withQuoteContext InDoubleQuote (manyTill inlineParser doubleQuoteEnd)))+   <|> pure (B.str "\8220")++charOrRef :: (Stream s m Char, UpdateSourcePos s Char) => [Char] -> ParserT s st m Char+charOrRef cs =+  oneOf cs <|> try (do c <- characterReference+                       guard (c `elem` cs)+                       return c)++-- | Succeeds if the parser is+--+-- * not within single quoted text;+-- * not directly after a word; and+-- * looking at an opening single quote char that's not followed by a+--   space.+--+-- Gobbles the quote character on success.+singleQuoteStart :: (HasLastStrPosition st, HasQuoteContext st m,+                     Stream s m Char, UpdateSourcePos s Char)+                 => ParserT s st m ()+singleQuoteStart = do+  failIfInQuoteContext InSingleQuote+  -- single quote start can't be right after str+  guard =<< notAfterString+  try $ do+    charOrRef "'\8216\145"+    void $ lookAhead (satisfy (not . isSpaceChar))++singleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)+               => ParserT s st m ()+singleQuoteEnd = try $ do+  charOrRef "'\8217\146"+  notFollowedBy alphaNum++-- | Succeeds if the parser is+--+-- * not within a double quoted text;+--+-- * not directly after a word; and+--+-- * looking at an opening double quote char that's not followed by a+--   space.+--+-- Gobbles the quote character on success.+doubleQuoteStart :: (HasLastStrPosition st,+                     HasQuoteContext st m,+                     Stream s m Char, UpdateSourcePos s Char)+                 => ParserT s st m ()+doubleQuoteStart = do+  failIfInQuoteContext InDoubleQuote+  guard =<< notAfterString+  try $ do charOrRef "\"\8220\147"+           void $ lookAhead (satisfy (not . isSpaceChar))++-- | Parses a closing quote character.+doubleQuoteEnd :: (Stream s m Char, UpdateSourcePos s Char)+               => ParserT s st m ()+doubleQuoteEnd = void (charOrRef "\"\8221\148")++-- | Parses an ASCII apostrophe (@'@) or right single quotation mark and+-- returns a RIGHT SINGLE QUOtatiON MARK character.+apostrophe :: (Stream s m Char, UpdateSourcePos s Char)+           => ParserT s st m Inlines+apostrophe = (char '\'' <|> char '\8217') >> return (B.str "\8217")++-- | Parses an ASCII quotation mark character and returns a RIGHT DOUBLE+-- QUOTATION MARK.+doubleCloseQuote :: (Stream s m Char, UpdateSourcePos s Char)+                 => ParserT s st m Inlines+doubleCloseQuote = B.str "\8221" <$ char '"'++-- | Parses three dots as HORIZONTAL ELLIPSIS.+ellipses :: (Stream s m Char, UpdateSourcePos s Char)+         => ParserT s st m Inlines+ellipses = try (string "..." >> return (B.str "\8230"))++-- | Parses two hyphens as EN DASH and three as EM DASH.+--+-- If the extension @'Ext_old_dashes'@ is enabled, then two hyphens are+-- parsed as EM DASH, and one hyphen is parsed as EN DASH if it is+-- followed by a digit.+dash :: (HasReaderOptions st, Stream s m Char, UpdateSourcePos s Char)+     => ParserT s st m Inlines+dash = try $ do+  oldDashes <- extensionEnabled Ext_old_dashes <$> getOption readerExtensions+  if oldDashes+     then do+       char '-'+       (char '-' >> return (B.str "\8212"))+         <|> (lookAhead digit >> return (B.str "\8211"))+     else do+       string "--"+       (char '-' >> return (B.str "\8212"))+         <|> return (B.str "\8211")
+ src/Text/Pandoc/Parsing/State.hs view
@@ -0,0 +1,191 @@+{-# LANGUAGE FlexibleInstances          #-}+{-# LANGUAGE MultiParamTypeClasses      #-}+{-# LANGUAGE OverloadedStrings          #-}+{- |+   Module      : Text.Pandoc.Parsing+   Copyright   : Copyright (C) 2006-2022 John MacFarlane+   License     : GPL-2.0-or-later+   Maintainer  : John MacFarlane <jgm@berkeley.edu>++A default parser state with commonly used properties.+-}++module Text.Pandoc.Parsing.State+  ( ParserState (..)+  , ParserContext (..)+  , HeaderType (..)+  , NoteTable+  , NoteTable'+  , Key (..)+  , KeyTable+  , SubstTable+  , defaultParserState+  , toKey+  )+where++import Data.Default (Default (def))+import Data.Text (Text)+import Text.Parsec (SourcePos, getState, setState)+import Text.Pandoc.Builder (Blocks, HasMeta (..), Inlines)+import Text.Pandoc.Definition (Attr, Meta, Target, nullMeta)+import Text.Pandoc.Logging (LogMessage)+import Text.Pandoc.Options (ReaderOptions)+import Text.Pandoc.Parsing.Capabilities+import Text.Pandoc.Parsing.Types+import Text.Pandoc.Readers.LaTeX.Types (Macro)++import qualified Data.Map as M+import qualified Data.Set as Set+import qualified Data.Text as T++-- | Parsing options.+data ParserState = ParserState+  { stateOptions         :: ReaderOptions -- ^ User options+  , stateParserContext   :: ParserContext -- ^ Inside list?+  , stateQuoteContext    :: QuoteContext  -- ^ Inside quoted environment?+  , stateAllowLinks      :: Bool          -- ^ Allow parsing of links+  , stateAllowLineBreaks :: Bool          -- ^ Allow parsing of line breaks+  , stateMaxNestingLevel :: Int           -- ^ Max # of nested Strong/Emph+  , stateLastStrPos      :: Maybe SourcePos -- ^ Position after last str parsed+  , stateKeys            :: KeyTable      -- ^ List of reference keys+  , stateHeaderKeys      :: KeyTable      -- ^ List of implicit header ref keys+  , stateSubstitutions   :: SubstTable    -- ^ List of substitution references+  , stateNotes           :: NoteTable     -- ^ List of notes (raw bodies)+  , stateNotes'          :: NoteTable'    -- ^ List of notes (parsed bodies)+  , stateNoteRefs        :: Set.Set Text  -- ^ List of note references used+  , stateInNote          :: Bool          -- ^ True if parsing note contents+  , stateNoteNumber      :: Int           -- ^ Last note number for citations+  , stateMeta            :: Meta          -- ^ Document metadata+  , stateMeta'           :: Future ParserState Meta -- ^ Document metadata+  , stateCitations       :: M.Map Text Text -- ^ RST-style citations+  , stateHeaderTable     :: [HeaderType]  -- ^ Ordered list of header types used+  , stateIdentifiers     :: Set.Set Text  -- ^ Header identifiers used+  , stateNextExample     :: Int           -- ^ Number of next example+  , stateExamples        :: M.Map Text Int -- ^ Map from example labels to numbers+  , stateMacros          :: M.Map Text Macro -- ^ Table of macros defined so far+  , stateRstDefaultRole  :: Text          -- ^ Current rST default+                                           -- interpreted text role+  , stateRstHighlight    :: Maybe Text    -- ^ Current rST literal block+                                           -- language+  , stateRstCustomRoles  :: M.Map Text (Text, Maybe Text, Attr)+    -- ^ Current rST cust text roles;+    -- Triple represents:) Base role 2) Optional format (only for :raw:+    -- roles) 3) Addition classes (rest of Attr is unused)).+  , stateCaption         :: Maybe Inlines -- ^ Caption in current environment+  , stateInHtmlBlock     :: Maybe Text    -- ^ Tag type of HTML block being parsed+  , stateFencedDivLevel  :: Int           -- ^ Depth of fenced div+  , stateContainers      :: [Text]        -- ^ parent include files+  , stateLogMessages     :: [LogMessage]  -- ^ log messages+  , stateMarkdownAttribute :: Bool        -- ^ True if in markdown=1 context+  }++instance Default ParserState where+  def = defaultParserState++instance HasMeta ParserState where+  setMeta field val st =+    st{ stateMeta = setMeta field val $ stateMeta st }+  deleteMeta field st =+    st{ stateMeta = deleteMeta field $ stateMeta st }++instance HasReaderOptions ParserState where+  extractReaderOptions = stateOptions++instance Monad m => HasQuoteContext ParserState m where+  getQuoteContext = stateQuoteContext <$> getState+  withQuoteContext context parser = do+    oldState <- getState+    let oldQuoteContext = stateQuoteContext oldState+    setState oldState { stateQuoteContext = context }+    result <- parser+    newState <- getState+    setState newState { stateQuoteContext = oldQuoteContext }+    return result++instance HasIdentifierList ParserState where+  extractIdentifierList     = stateIdentifiers+  updateIdentifierList f st = st{ stateIdentifiers = f $ stateIdentifiers st }++instance HasMacros ParserState where+  extractMacros        = stateMacros+  updateMacros f st    = st{ stateMacros = f $ stateMacros st }++instance HasLastStrPosition ParserState where+  setLastStrPos pos st = st{ stateLastStrPos = pos }+  getLastStrPos st     = stateLastStrPos st++instance HasLogMessages ParserState where+  addLogMessage msg st = st{ stateLogMessages = msg : stateLogMessages st }+  getLogMessages st = reverse $ stateLogMessages st++instance HasIncludeFiles ParserState where+  getIncludeFiles = stateContainers+  addIncludeFile f s = s{ stateContainers = f : stateContainers s }+  dropLatestIncludeFile s = s { stateContainers = drop 1 $ stateContainers s }++data ParserContext+    = ListItemState   -- ^ Used when running parser on list item contents+    | NullState       -- ^ Default state+    deriving (Eq, Show)++data HeaderType+    = SingleHeader Char  -- ^ Single line of characters underneath+    | DoubleHeader Char  -- ^ Lines of characters above and below+    deriving (Eq, Show)++defaultParserState :: ParserState+defaultParserState = ParserState+  { stateOptions         = def+  , stateParserContext   = NullState+  , stateQuoteContext    = NoQuote+  , stateAllowLinks      = True+  , stateAllowLineBreaks = True+  , stateMaxNestingLevel = 6+  , stateLastStrPos      = Nothing+  , stateKeys            = M.empty+  , stateHeaderKeys      = M.empty+  , stateSubstitutions   = M.empty+  , stateNotes           = []+  , stateNotes'          = M.empty+  , stateNoteRefs        = Set.empty+  , stateInNote          = False+  , stateNoteNumber      = 0+  , stateMeta            = nullMeta+  , stateMeta'           = return nullMeta+  , stateCitations       = M.empty+  , stateHeaderTable     = []+  , stateIdentifiers     = Set.empty+  , stateNextExample     = 1+  , stateExamples        = M.empty+  , stateMacros          = M.empty+  , stateRstDefaultRole  = "title-reference"+  , stateRstHighlight    = Nothing+  , stateRstCustomRoles  = M.empty+  , stateCaption         = Nothing+  , stateInHtmlBlock     = Nothing+  , stateFencedDivLevel  = 0+  , stateContainers      = []+  , stateLogMessages     = []+  , stateMarkdownAttribute = False+  }++type NoteTable = [(Text, Text)]++type NoteTable' = M.Map Text (SourcePos, Future ParserState Blocks)+-- used in markdown reader++newtype Key = Key Text deriving (Show, Read, Eq, Ord)++toKey :: Text -> Key+toKey = Key . T.toLower . T.unwords . T.words . unbracket+  where unbracket t+          | Just ('[', t') <- T.uncons t+          , Just (t'', ']') <- T.unsnoc t'+          = t''+          | otherwise+          = t++type KeyTable = M.Map Key (Target, Attr)++type SubstTable = M.Map Key Inlines
+ src/Text/Pandoc/Parsing/Types.hs view
@@ -0,0 +1,57 @@+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{- |+   Module      : Text.Pandoc.Parsing+   Copyright   : Copyright (C) 2006-2022 John MacFarlane+   License     : GPL-2.0-or-later+   Maintainer  : John MacFarlane <jgm@berkeley.edu>++Types and type-related functions for parsers.+-}++module Text.Pandoc.Parsing.Types+  ( Parser+  , ParserT+  , Future (..)+  , runF+  , askF+  , asksF+  , returnF+  )+where++import Control.Applicative (liftA2)+import Control.Monad.Reader+  ( asks, runReader, MonadReader(ask), Reader, ReaderT(ReaderT) )+import Text.Parsec ( Parsec , ParsecT )++-- | Generic parser type used by many pandoc readers.+type Parser t s = Parsec t s++-- | Generic parser transformer used by many pandoc readers.+type ParserT = ParsecT++-- | Reader monad wrapping the parser state. This is used to possibly+-- delay evaluation until all relevant information has been parsed and+-- made available in the parser state.+newtype Future s a = Future { runDelayed :: Reader s a }+  deriving (Monad, Applicative, Functor)++instance Semigroup a => Semigroup (Future s a) where+  (<>) = liftA2 (<>)++instance (Semigroup a, Monoid a) => Monoid (Future s a) where+  mempty = return mempty+  mappend = (<>)++-- | Run a delayed action with the given state.+runF :: Future s a -> s -> a+runF = runReader . runDelayed++askF :: Future s s+askF = Future ask++asksF :: (s -> a) -> Future s a+asksF f = Future $ asks f++returnF :: Monad m => a -> m (Future s a)+returnF = return . return
src/Text/Pandoc/Readers.hs view
@@ -55,6 +55,8 @@   , readCslJson   , readBibTeX   , readBibLaTeX+  , readEndNoteXML+  , readRIS   , readRTF   -- * Miscellaneous   , getReader@@ -103,6 +105,8 @@ import Text.Pandoc.Readers.CSV import Text.Pandoc.Readers.CslJson import Text.Pandoc.Readers.BibTeX+import Text.Pandoc.Readers.EndNote+import Text.Pandoc.Readers.RIS import Text.Pandoc.Readers.RTF import qualified Text.Pandoc.UTF8 as UTF8 import Text.Pandoc.Sources (ToSources(..), sourcesToText)@@ -151,6 +155,8 @@           ,("csljson"      , TextReader readCslJson)           ,("bibtex"       , TextReader readBibTeX)           ,("biblatex"     , TextReader readBibLaTeX)+          ,("endnotexml"   , TextReader readEndNoteXML)+          ,("ris"          , TextReader readRIS)           ,("rtf"          , TextReader readRTF)            ]
src/Text/Pandoc/Readers/DocBook.hs view
@@ -1,4 +1,6 @@ {-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE TemplateHaskell #-} {- |    Module      : Text.Pandoc.Readers.DocBook    Copyright   : Copyright (C) 2006-2022 John MacFarlane@@ -12,15 +14,20 @@ -} module Text.Pandoc.Readers.DocBook ( readDocBook ) where import Control.Monad.State.Strict-import Data.Char (isSpace, isLetter)+import Data.ByteString (ByteString)+import Data.FileEmbed+import Data.Char (isSpace, isLetter, chr) import Data.Default import Data.Either (rights) import Data.Foldable (asum) import Data.Generics import Data.List (intersperse,elemIndex)+import qualified Data.Set as Set import Data.List.NonEmpty (nonEmpty) import Data.Maybe (catMaybes,fromMaybe,mapMaybe,maybeToList) import Data.Text (Text)+import Data.Text.Read as TR+import Data.Text.Encoding (decodeUtf8) import qualified Data.Text as T import qualified Data.Text.Lazy as TL import Control.Monad.Except (throwError)@@ -33,6 +40,7 @@ import Text.Pandoc.Shared (safeRead, extractSpaces) import Text.Pandoc.Sources (ToSources(..), sourcesToText) import Text.TeXMath (readMathML, writeTeX)+import qualified Data.Map as M import Text.Pandoc.XML.Light  {-@@ -548,7 +556,8 @@ readDocBook _ inp = do   let sources = toSources inp   tree <- either (throwError . PandocXMLError "") return $-            parseXMLContents+            parseXMLContentsWithEntities+            docbookEntityMap               (TL.fromStrict . handleInstructions . sourcesToText $ sources)   (bs, st') <- flip runStateT (def{ dbContent = tree }) $ mapM parseBlock tree   return $ Pandoc (dbMeta st') (toList . mconcat $ bs)@@ -591,35 +600,84 @@ --  addMetadataFromElement :: PandocMonad m => Element -> DB m Blocks-addMetadataFromElement e = do-    case filterChild (named "title") e of-         Nothing -> return ()-         Just z  -> do-           getInlines z >>= addMeta "title"-           addMetaField "subtitle" z-    case filterChild (named "authorgroup") e of-         Nothing -> return ()-         Just z  -> addMetaField "author" z-    addMetaField "subtitle" e-    addAuthor e-    addMetaField "date" e-    addMetaField "release" e-    addMetaField "releaseinfo" e-    return mempty-  where-   addAuthor elt =-     case filterChildren (named "author") elt of-       [] -> return ()-       [z] -> fromAuthor z >>= addMeta "author"-       zs  -> mapM fromAuthor zs >>= addMeta "author"-   fromAuthor elt =-     mconcat . intersperse space <$> mapM getInlines (elChildren elt)-   addMetaField fieldname elt =-     case filterChildren (named fieldname) elt of-       []  -> return ()-       [z] -> getInlines z >>= addMeta fieldname-       zs  -> mapM getInlines zs >>= addMeta fieldname+addMetadataFromElement e =+  mempty <$ mapM_ handleMetadataElement+                  (filterChildren ((isMetadataField . qName . elName)) e)+ where+  handleMetadataElement elt =+    case qName (elName elt) of+      "title" -> addContentsToMetadata "title" elt+      "subtitle" -> addContentsToMetadata "subtitle" elt+      "abstract" -> addContentsToMetadata "abstract" elt+      "date" -> addContentsToMetadata "date" elt+      "release" -> addContentsToMetadata "release" elt+      "releaseinfo" -> addContentsToMetadata "releaseinfo" elt+      "address" -> addContentsToMetadata "address" elt+      "copyright" -> addContentsToMetadata "copyright" elt+      "author" -> fromAuthor elt >>= addMeta "author"+      "authorgroup" ->+        mapM fromAuthor (filterChildren (named "author") elt) >>= addMeta "author"+      _ -> report . IgnoredElement . qName . elName $ elt +  fromAuthor elt =+    mconcat . intersperse space . filter (not . null)+      <$> mapM getInlines (elChildren elt)++  addContentsToMetadata fieldname elt =+    if any ((`Set.member` blockTags) . qName . elName) (elChildren elt)+       then getBlocks elt >>= addMeta fieldname+       else getInlines elt >>= addMeta fieldname++  isMetadataField "abstract" = True+  isMetadataField "address" = True+  isMetadataField "annotation" = True+  isMetadataField "artpagenums" = True+  isMetadataField "author" = True+  isMetadataField "authorgroup" = True+  isMetadataField "authorinitials" = True+  isMetadataField "bibliocoverage" = True+  isMetadataField "biblioid" = True+  isMetadataField "bibliomisc" = True+  isMetadataField "bibliomset" = True+  isMetadataField "bibliorelation" = True+  isMetadataField "biblioset" = True+  isMetadataField "bibliosource" = True+  isMetadataField "collab" = True+  isMetadataField "confgroup" = True+  isMetadataField "contractnum" = True+  isMetadataField "contractsponsor" = True+  isMetadataField "copyright" = True+  isMetadataField "cover" = True+  isMetadataField "date" = True+  isMetadataField "edition" = True+  isMetadataField "editor" = True+  isMetadataField "extendedlink" = True+  isMetadataField "issuenum" = True+  isMetadataField "itermset" = True+  isMetadataField "keywordset" = True+  isMetadataField "legalnotice" = True+  isMetadataField "mediaobject" = True+  isMetadataField "org" = True+  isMetadataField "orgname" = True+  isMetadataField "othercredit" = True+  isMetadataField "pagenums" = True+  isMetadataField "printhistory" = True+  isMetadataField "productname" = True+  isMetadataField "productnumber" = True+  isMetadataField "pubdate" = True+  isMetadataField "publisher" = True+  isMetadataField "publishername" = True+  isMetadataField "releaseinfo" = True+  isMetadataField "revhistory" = True+  isMetadataField "seriesvolnums" = True+  isMetadataField "subjectset" = True+  isMetadataField "subtitle" = True+  isMetadataField "title" = True+  isMetadataField "titleabbrev" = True+  isMetadataField "volumenum" = True+  isMetadataField _ = False++ addMeta :: PandocMonad m => ToMetaValue a => Text -> a -> DB m () addMeta field val = modify (setMeta field val) @@ -628,11 +686,11 @@   deleteMeta field s = s {dbMeta = deleteMeta field (dbMeta s)}  isBlockElement :: Content -> Bool-isBlockElement (Elem e) = qName (elName e) `elem` blockTags+isBlockElement (Elem e) = qName (elName e) `Set.member` blockTags isBlockElement _ = False -blockTags :: [Text]-blockTags =+blockTags :: Set.Set Text+blockTags = Set.fromList $   [ "abstract"   , "ackno"   , "answer"@@ -894,7 +952,7 @@         "?xml"  -> return mempty         "title" -> return mempty     -- handled in parent element         "subtitle" -> return mempty  -- handled in parent element-        _       -> skip >> getBlocks e+        _ -> skip >> getBlocks e    where skip = do            let qn = qName $ elName e            let name = if "pi-" `T.isPrefixOf` qn@@ -1335,3 +1393,17 @@ paraToPlain (Para ils) = Plain ils paraToPlain x = x +docbookEntityMap :: M.Map Text Text+docbookEntityMap = M.fromList+  (map lineToPair (T.lines (decodeUtf8 docbookEntities)))+ where+   lineToPair l =+     case T.words l of+       (x:ys) -> (x, T.pack (mapMaybe readHex ys))+       [] -> ("","")+   readHex t = case TR.hexadecimal t of+                 Left _ -> Nothing+                 Right (n,_) -> Just (chr n)++docbookEntities :: ByteString+docbookEntities = $(embedFile "data/docbook-entities.txt")
src/Text/Pandoc/Readers/Docx.hs view
@@ -68,10 +68,12 @@ import Data.Char (isSpace) import qualified Data.Map as M import qualified Data.Text as T-import Data.Maybe (catMaybes, isJust, fromMaybe)+import Data.Maybe (catMaybes, isJust, fromMaybe, mapMaybe) import Data.Sequence (ViewL (..), viewl) import qualified Data.Sequence as Seq import qualified Data.Set as Set+import Citeproc (ItemId(..), Reference(..), CitationItem(..))+import qualified Citeproc import Text.Pandoc.Builder as Pandoc import Text.Pandoc.MediaBag (MediaBag) import Text.Pandoc.Options@@ -87,6 +89,12 @@ import Text.Pandoc.Error import Text.Pandoc.Logging import Data.List.NonEmpty (nonEmpty)+import Data.Aeson (eitherDecode)+import qualified Data.Text.Lazy as TL+import Text.Pandoc.UTF8 (fromTextLazy)+import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)+import Text.Pandoc.Readers.EndNote (readEndNoteXMLCitation)+import Text.Pandoc.Sources (toSources)  readDocx :: PandocMonad m          => ReaderOptions@@ -115,6 +123,7 @@                      , docxListState :: M.Map (T.Text, T.Text) Integer                      , docxPrevPara  :: Inlines                      , docxTableCaptions :: [Blocks]+                     , docxReferences :: M.Map ItemId (Reference Inlines)                      }  instance Default DState where@@ -126,6 +135,7 @@                , docxListState = M.empty                , docxPrevPara  = mempty                , docxTableCaptions = []+               , docxReferences = mempty                }  data DEnv = DEnv { docxOptions       :: ReaderOptions@@ -449,8 +459,70 @@   case info of     HyperlinkField url -> parPartToInlines' $ ExternalHyperLink url children     PagerefField fieldAnchor True -> parPartToInlines' $ InternalHyperLink fieldAnchor children+    EndNoteCite t -> do+      formattedCite <- smushInlines <$> mapM parPartToInlines' children+      opts <- asks docxOptions+      if isEnabled Ext_citations opts+         then do+           citation <- readEndNoteXMLCitation (toSources t)+           cs <- handleCitation citation+           return $ cite cs formattedCite+         else return formattedCite+    CslCitation t -> do+      formattedCite <- smushInlines <$> mapM parPartToInlines' children+      opts <- asks docxOptions+      if isEnabled Ext_citations opts+         then do+           let bs = fromTextLazy $ TL.fromStrict t+           case eitherDecode bs of+             Left _err -> return formattedCite+             Right citation -> do+               cs <- handleCitation citation+               return $ cite cs formattedCite+         else return formattedCite+    CslBibliography -> do+      opts <- asks docxOptions+      if isEnabled Ext_citations opts+         then return mempty -- omit Zotero-generated bibliography+         else smushInlines <$> mapM parPartToInlines' children+    EndNoteRefList -> do+      opts <- asks docxOptions+      if isEnabled Ext_citations opts+         then return mempty -- omit EndNote-generated bibliography+         else smushInlines <$> mapM parPartToInlines' children     _ -> smushInlines <$> mapM parPartToInlines' children +-- Turn a 'Citeproc.Citation' into a list of 'Text.Pandoc.Definition.Citation',+-- and store the embedded bibliographic data in state.+handleCitation :: PandocMonad m+               => Citeproc.Citation T.Text+               -> DocxContext m [Citation]+handleCitation citation = do+  let toPandocCitation item =+        Citation{ citationId = unItemId (Citeproc.citationItemId item)+                , citationPrefix = maybe [] (toList . text) $+                                     Citeproc.citationItemPrefix item+                , citationSuffix = (toList . text) $+                    maybe mempty (\x -> ", " <>+                       maybe "" (<>" ") (Citeproc.citationItemLabel item)+                         <> x <> " ")+                     (Citeproc.citationItemLocator item)+                    <> fromMaybe mempty (Citeproc.citationItemSuffix item)+                , citationMode = NormalCitation -- TODO for now+                , citationNoteNum = 0+                , citationHash = 0 }+  let items = Citeproc.citationItems citation+  let cs = map toPandocCitation items+  refs <- mapM (traverse (return . text)) $+            mapMaybe Citeproc.citationItemData items+  modify $ \st ->+    st{ docxReferences = foldr+          (\ref -> M.insert (referenceId ref) ref)+          (docxReferences st)+          refs }+  return cs++ isAnchorSpan :: Inline -> Bool isAnchorSpan (Span (_, ["anchor"], []) _) = True isAnchorSpan _ = False@@ -744,7 +816,11 @@   blks <- smushBlocks <$> mapM bodyPartToBlocks blkbps   blks' <- rewriteLinks $ blocksToDefinitions $ blocksToBullets $ toList blks   blks'' <- removeOrphanAnchors blks'-  return (meta, blks'')+  refs <- gets (map referenceToMetaValue . M.elems . docxReferences)+  let meta' = if null refs+                 then meta+                 else setMeta "references" refs meta+  return (meta', blks'')  docxToOutput :: PandocMonad m              => ReaderOptions
src/Text/Pandoc/Readers/Docx/Fields.hs view
@@ -26,8 +26,10 @@ data FieldInfo = HyperlinkField URL                 -- The boolean indicates whether the field is a hyperlink.                | PagerefField Anchor Bool-               | ZoteroItem T.Text-               | ZoteroBibliography+               | CslCitation T.Text+               | CslBibliography+               | EndNoteCite T.Text+               | EndNoteRefList                | UnknownField                deriving (Show) @@ -38,7 +40,7 @@ fieldInfo =   try (HyperlinkField <$> hyperlink)   <|>-  try ((uncurry PagerefField) <$> pageref) +  try ((uncurry PagerefField) <$> pageref)   <|>   try addIn   <|>@@ -49,20 +51,32 @@   spaces   string "ADDIN"   spaces-  try zoteroItem <|> zoteroBibliography+  try cslCitation <|> cslBibliography <|> endnoteCite <|> endnoteRefList -zoteroItem :: Parser FieldInfo-zoteroItem = do-  string "ZOTERO_ITEM"+cslCitation :: Parser FieldInfo+cslCitation = do+  optional (string "ZOTERO_ITEM")   spaces   string "CSL_CITATION"   spaces-  ZoteroItem <$> getInput+  CslCitation <$> getInput -zoteroBibliography :: Parser FieldInfo-zoteroBibliography = do-  string "ZOTERO_BIBL"-  return ZoteroBibliography+cslBibliography :: Parser FieldInfo+cslBibliography = do+  string "ZOTERO_BIBL" <|> string "Mendeley Bibliography CSL_BIBLIOGRAPHY"+  return CslBibliography++endnoteCite :: Parser FieldInfo+endnoteCite = try $ do+  string "EN.CITE"+  spaces+  EndNoteCite <$> getInput++endnoteRefList :: Parser FieldInfo+endnoteRefList = try $ do+  string "EN.REFLIST"+  return EndNoteRefList+  escapedQuote :: Parser T.Text escapedQuote = string "\\\"" $> "\\\""
src/Text/Pandoc/Readers/DokuWiki.hs view
@@ -19,8 +19,8 @@ import Control.Monad.Except (throwError) import Data.Char (isAlphaNum, isDigit) import qualified Data.Foldable as F-import Data.List (transpose) import Data.Maybe (fromMaybe, catMaybes)+import Data.Bifunctor (second) import Data.Text (Text) import qualified Data.Text as T import qualified Text.Pandoc.Builder as B@@ -30,6 +30,8 @@ import Text.Pandoc.Options import Text.Pandoc.Parsing hiding (enclosed, nested) import Text.Pandoc.Shared (trim, stringify, tshow)+import Data.List (isPrefixOf, isSuffixOf)+import qualified Safe  -- | Read DokuWiki from an input string and return a Pandoc document. readDokuWiki :: (PandocMonad m, ToSources a)@@ -96,7 +98,11 @@ -- | Parse any inline element but softbreak. inline' :: PandocMonad m => DWParser m B.Inlines inline' = whitespace-      <|> br+      <|> inline''++-- | Parse any inline element but whitespace.+inline'' :: PandocMonad m => DWParser m B.Inlines+inline'' = br       <|> bold       <|> italic       <|> underlined@@ -121,6 +127,10 @@       <|> symbol       <?> "inline" +-- | Parse any inline element but soft breaks and do not consolidate spaces.+inlineUnconsolidatedWhitespace :: PandocMonad m => DWParser m B.Inlines+inlineUnconsolidatedWhitespace = (B.space <$ spaceChar) <|> inline'+ -- | Parse any inline element, including soft break. inline :: PandocMonad m => DWParser m B.Inlines inline = endline <|> inline'@@ -468,22 +478,28 @@ table = do   firstSeparator <- lookAhead tableCellSeparator   rows <- tableRows+  let firstRow = fromMaybe [] . Safe.headMay $ rows   let (headerRow, body) = if firstSeparator == '^'-                            then (head rows, tail rows)+                            then (firstRow, tail rows)                             else ([], rows)-  let attrs = (AlignDefault, ColWidthDefault) <$ transpose rows+  -- Since Pandoc only has column level alignment, we have to make an arbitrary+  -- choice of how to reconcile potentially different alignments in the row.+  -- Here we end up assuming that the alignment of the header / first row is+  -- what the user wants to apply to the whole thing.+  let attrs =  map (\(a, _) -> (a, ColWidthDefault)) firstRow   let toRow = Row nullAttr . map B.simpleCell       toHeaderRow l = [toRow l | not (null l)]   pure $ B.table B.emptyCaption                  attrs-                 (TableHead nullAttr $ toHeaderRow headerRow)-                 [TableBody nullAttr 0 [] $ map toRow body]+                 (TableHead nullAttr $ toHeaderRow (map snd headerRow))+                 [TableBody nullAttr 0 [] $ map (toRow . (map snd)) body]                  (TableFoot nullAttr []) -tableRows :: PandocMonad m => DWParser m [[B.Blocks]]++tableRows :: PandocMonad m => DWParser m [[(Alignment, B.Blocks)]] tableRows = many1 tableRow -tableRow :: PandocMonad m => DWParser m [B.Blocks]+tableRow :: PandocMonad m => DWParser m [(Alignment, B.Blocks)] tableRow = many1Till tableCell tableRowEnd  tableRowEnd :: PandocMonad m => DWParser m Char@@ -492,11 +508,23 @@ tableCellSeparator :: PandocMonad m => DWParser m Char tableCellSeparator = char '|' <|> char '^' -tableCell :: PandocMonad m => DWParser m B.Blocks-tableCell = try $ B.plain . B.trimInlines . mconcat <$> (normalCell <|> headerCell)+tableCell :: PandocMonad m => DWParser m (Alignment, B.Blocks)+tableCell = try $ (second (B.plain . B.trimInlines . mconcat)) <$> cellContent   where-    normalCell = char '|' *> manyTill inline' (lookAhead tableCellSeparator)-    headerCell = char '^' *> manyTill inline' (lookAhead tableCellSeparator)+    cellContent = do+      -- https://www.dokuwiki.org/wiki:syntax#tables+      -- DokuWiki represents the alignment of cells with two spaces padding.+      tableCellSeparator+      cellInline <- manyTill inlineUnconsolidatedWhitespace (lookAhead tableCellSeparator)+      let left  = [B.space, B.space] `isPrefixOf` cellInline+      let right = [B.space, B.space] `isSuffixOf` cellInline+      let alignment = case (left, right) of+                           (True, True)   -> AlignCenter+                           (True, False)  -> AlignRight+                           (False, True)  -> AlignLeft+                           (False, False) -> AlignDefault+      return (alignment, cellInline)+  blockCode :: PandocMonad m => DWParser m B.Blocks blockCode = codeTag B.codeBlockWith "code"
+ src/Text/Pandoc/Readers/EndNote.hs view
@@ -0,0 +1,246 @@+{-# LANGUAGE TupleSections #-}+{-# LANGUAGE OverloadedStrings #-}+{- |+   Module      : Text.Pandoc.Readers.EndNote+   Copyright   : Copyright (C) 2022 John MacFarlane+   License     : GNU GPL, version 2 or above++   Maintainer  : John MacFarlane <jgm@berkeley.edu>+   Stability   : alpha+   Portability : portable++Parses EndNote XML bibliographies into a Pandoc document+with empty body and `references` and `nocite` fields+in the metadata.  A wildcard `nocite` is used so that+if the document is rendered in another format, the+entire bibliography will be printed.+-}+module Text.Pandoc.Readers.EndNote+  ( readEndNoteXML+  , readEndNoteXMLCitation+  )+where++import Text.Pandoc.Options+import Text.Pandoc.Definition+import Citeproc (Reference(..), ItemId(..), Val(..), Date(..), DateParts(..))+import qualified Citeproc+import Text.Pandoc.Builder as B+import Text.Pandoc.Error (PandocError(..))+import Text.Pandoc.Class (PandocMonad)+import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)+import Text.Pandoc.Sources (Sources(..), ToSources(..), sourcesToText)+import Text.Pandoc.Citeproc.BibTeX (toName)+import Control.Applicative ((<|>))+import Control.Monad.Except (throwError)+import Control.Monad (mzero, unless)+import Text.Pandoc.XML.Light+    ( filterElementName,+      strContent,+      QName(qName),+      Element(..),+      Content(..),+      CData(..),+      filterElementsName,+      filterChildName,+      filterChildrenName,+      findAttrBy,+      parseXMLElement )+import qualified Data.Text.Lazy as TL+import qualified Data.Text as T+import Data.Text (Text)+import qualified Data.Map as M+import Safe (readMay)++-- | Read EndNote XML from an input string and return a Pandoc document.+-- The document will have only metadata, with an empty body.+-- The metadata will contain a `references` field with the+-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.+readEndNoteXML :: (PandocMonad m, ToSources a)+               => ReaderOptions -> a -> m Pandoc+readEndNoteXML _opts inp = do+  let sources = toSources inp+  refs <- readEndNoteXMLReferences sources >>= mapM (traverse (return . text))+  return $+    setMeta "nocite" (cite [Citation {citationId = "*"+                                     , citationPrefix = []+                                     , citationSuffix = []+                                     , citationMode = NormalCitation+                                     , citationNoteNum = 0+                                     , citationHash = 0}] (str "[@*]")) $+    setMeta "references" (map referenceToMetaValue refs) $+    B.doc mempty++readEndNoteXMLCitation :: PandocMonad m+                    => Sources -> m (Citeproc.Citation Text)+readEndNoteXMLCitation sources = do+  tree <- either (throwError . PandocXMLError "") return $+              parseXMLElement (TL.fromStrict . sourcesToText $ sources)+  unless (qName (elName tree) == "EndNote") $+    throwError $ PandocXMLError "" "Expected EndNote element"+  let items = map toCitationItem $ filterElementsName (name "Cite") tree+  return $ Citeproc.Citation{+                     Citeproc.citationId = Nothing+                   , Citeproc.citationNoteNumber = Nothing+                   , Citeproc.citationItems = items+                   }++readEndNoteXMLReferences :: PandocMonad m+                         => Sources -> m [Reference Text]+readEndNoteXMLReferences sources = do+  tree <- either (throwError . PandocXMLError "") return $+              parseXMLElement (TL.fromStrict . sourcesToText $ sources)+  let records = filterElementsName (name "record") tree+  return $ map recordToReference records+++toCitationItem :: Element -> Citeproc.CitationItem Text+toCitationItem el =+    Citeproc.CitationItem{ Citeproc.citationItemId =+                              maybe mempty referenceId mbref+                         , Citeproc.citationItemLabel = Nothing+                         , Citeproc.citationItemLocator = mbpages+                         , Citeproc.citationItemType = Citeproc.NormalCite+                         , Citeproc.citationItemPrefix = mbprefix+                         , Citeproc.citationItemSuffix = mbsuffix+                         , Citeproc.citationItemData = mbref+                         }+ where+  mbref = recordToReference <$> filterChildName (name "record") el+  mbprefix = getText <$> filterChildName (name "Prefix") el+  mbsuffix = getText <$> filterChildName (name "Suffix") el+  mbpages  = getText <$> filterChildName (name "Pages") el++name :: Text -> (QName -> Bool)+name t = (== t) . qName++getText :: Element -> Text+getText el = getText' (Elem el)+ where+  getText' (Elem el') = mconcat $ map getText' $ elContent el'+  getText' (Text cd) = cdData cd+  getText' (CRef _) = mempty++recordToReference :: Element -> Reference Text+recordToReference e =+  Reference{ referenceId = ItemId refid,+             referenceType = reftype,+             referenceDisambiguation = Nothing,+             referenceVariables = refvars }++ where+   -- get strContent, recursing inside style elements:+   refid = maybe mempty (T.strip . strContent)+           (filterElementName (name "key") e+            <|> filterElementName (name "rec-number") e)+   reftype = maybe "document" toCslReferenceType+              (filterElementName (name "ref-type") e >>=+                findAttrBy (name "name"))+   authors =+     filterChildrenName (name "contributors") e >>=+     filterChildrenName (name "authors") >>=+     filterChildrenName (name "author") >>=+     toName [] . B.toList .  B.text . T.strip . getText+   titles = do+     x <- filterChildrenName (name "titles") e+     (key, name') <- [("title", "title"),+                      ("container-title", "secondary-title")]+     (key,) . FancyVal . T.strip . getText <$>+                    filterChildrenName (name name') x+   pages = ("pages",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "pages") e+   volume = ("volume",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "volume") e+   number = ("number",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "number") e+   isbn = ("isbn",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "isbn") e+   publisher = ("publisher",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "publisher") e+   originalPublisher =+     ("original-publisher",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "orig-pub") e+   publisherPlace =+     ("publisher-place",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "pub-location") e+   abstract = ("abstract",) . FancyVal . T.strip . getText <$>+                filterChildrenName (name "abstract") e+   dates = ("issued",) . toDate <$> filterChildrenName (name "dates") e+   toDate e' = DateVal $+    Date { dateParts = toDateParts e'+         , dateCirca = False+         , dateSeason = Nothing+         , dateLiteral = Nothing }+   toDateParts e' = do+    x <- filterChildrenName (name "year") e'+    case readMay . T.unpack . T.strip . getText $ x of+      Nothing -> mzero+      Just y  -> return $ DateParts [y]++   refvars = M.fromList $+     [ ("author", NamesVal authors) | not (null authors) ] +++     titles +++     pages +++     volume +++     number +++     isbn +++     dates +++     publisher +++     originalPublisher +++     publisherPlace +++     abstract++toCslReferenceType :: Text -> Text+toCslReferenceType t =+  case t of+    "Aggregated Database" -> "dataset"+    "Ancient Text" -> "classic"+    "Artwork" -> "graphic"+    "Audiovisual Material" -> "graphic"+    "Bill" -> "legislation"+    "Blog" -> "post-weblog"+    "Book" -> "book"+    "Book Section" -> "chapter"+    "Case" -> "legal_case"+    "Catalog" -> "document"+    "Chart or Table" -> "graphic"+    "Classical Work" -> "classic"+    "Computer program" -> "software"+    "Conference Paper" -> "article"+    "Conference Proceedings" -> "periodical"+    "Dataset" -> "dataset"+    "Dictionary" -> "book"+    "Edited Book" -> "book"+    "Electronic Article" -> "article"+    "Electronic Book" -> "book"+    "Electronic Book Section" -> "chapter"+    "Encyclopedia" -> "book"+    "Equation" -> "document"+    "Figure" -> "graphic"+    "Film or Broadcast" -> "motion_picture"+    "Government Document" -> "document"+    "Grant" -> "document"+    "Hearing" -> "hearing"+    "Interview" -> "interview"+    "Journal Article" -> "article-journal"+    "Legal Rule or Regulation" -> "regulation"+    "Magazine Article" -> "article-magazine"+    "Manuscript" -> "manuscript"+    "Map" -> "map"+    "Music" -> "musical_score"+    "Newspaper Article" -> "article-newspaper"+    "Online Database" -> "dataset"+    "Online Multimedia" -> "webpage"+    "Pamphlet" -> "pamphlet"+    "Patent" -> "patent"+    "Personal Communication" -> "personal_communication"+    "Podcast" -> "document"+    "Press Release" -> "report"+    "Report" -> "report"+    "Serial" -> "periodical"+    "Standard" -> "standard"+    "Statute" -> "legislation"+    "Thesis" -> "thesis"+    "Unpublished Work" -> "unpublished"+    "Web Page" -> "webpage"+    _ -> "document"
src/Text/Pandoc/Readers/HTML.hs view
@@ -89,7 +89,7 @@   result <- flip runReaderT def $        runParserT parseDoc        (HTMLState def{ stateOptions = opts }-         [] Nothing Set.empty [] M.empty opts)+         [] Nothing Set.empty [] M.empty opts False)        "source" tags   case result of     Right doc -> return doc@@ -106,13 +106,19 @@  replaceNotes :: PandocMonad m => [Block] -> TagParser m [Block] replaceNotes bs = do-  st <- getState-  return $ walk (replaceNotes' (noteTable st)) bs+  notes <- noteTable <$> getState+  walkM (replaceNotes' notes) bs -replaceNotes' :: [(Text, Blocks)] -> Inline -> Inline+replaceNotes' :: PandocMonad m+              => [(Text, Blocks)] -> Inline -> TagParser m Inline replaceNotes' noteTbl (RawInline (Format "noteref") ref) =-  maybe (Str "") (Note . B.toList) $ lookup ref noteTbl-replaceNotes' _ x = x+  maybe warnNotFound (pure . Note . B.toList) $ lookup ref noteTbl+ where+  warnNotFound = do+    pos <- getPosition+    logMessage $ ReferenceNotFound ref pos+    pure (Note [])+replaceNotes' _ x = pure x  setInChapter :: PandocMonad m => HTMLParser m s a -> HTMLParser m s a setInChapter = local (\s -> s {inChapter = True})@@ -173,6 +179,9 @@           , "chapter" `T.isInfixOf` type'           -> eSection         _ | epubExts+          , type' `elem` ["footnotes", "rearnotes"]+          -> eFootnotes+        _ | epubExts           , type' `elem` ["footnote", "rearnote"]           -> mempty <$ eFootnote         _ | epubExts@@ -250,19 +259,39 @@     Nothing -> Nothing <$ manyTill pAny (pSatisfy (matchTagClose "case"))  eFootnote :: PandocMonad m => TagParser m ()-eFootnote = try $ do-  let notes = ["footnote", "rearnote"]+eFootnote = do   guardEnabled Ext_epub_html_exts-  (TagOpen tag attr') <- lookAhead pAny+  TagOpen tag attr' <- lookAhead $ pSatisfy+    (\case+       TagOpen _ attr'+         -> case lookup "type" attr' <|> lookup "epub:type" attr' of+              Just "footnote" -> True+              Just "rearnote" -> True+              _ -> False+       _ -> False)   let attr = toStringAttr attr'-  guard $ maybe False (`elem` notes)-          (lookup "type" attr <|> lookup "epub:type" attr)   let ident = fromMaybe "" (lookup "id" attr)   content <- pInTags tag block-  addNote ident content+  updateState $ \s ->+    s {noteTable = (ident, content) : noteTable s} -addNote :: PandocMonad m => Text -> Blocks -> TagParser m ()-addNote uid cont = updateState (\s -> s {noteTable = (uid, cont) : noteTable s})+eFootnotes :: PandocMonad m => TagParser m Blocks+eFootnotes = try $ do+  let notes = ["footnotes", "rearnotes"]+  guardEnabled Ext_epub_html_exts+  (TagOpen tag attr') <- lookAhead pAny+  let attr = toStringAttr attr'+  guard $ maybe False (`elem` notes)+          (lookup "type" attr <|> lookup "epub:type" attr)+  updateState $ \s -> s{ inFootnotes = True }+  result <- pInTags tag block+  updateState $ \s -> s{ inFootnotes = False }+  if null result+     -- if it just contains notes, we don't need the container:+     then return result+     -- but there might be content other than notes, in which case+     -- we want a div:+     else return $ B.divWith (toAttr attr') result  eNoteref :: PandocMonad m => TagParser m Inlines eNoteref = try $ do@@ -331,6 +360,7 @@ pOrderedList :: PandocMonad m => TagParser m Blocks pOrderedList = try $ do   TagOpen _ attribs' <- pSatisfy (matchTagOpen "ol" [])+  isNoteList <- inFootnotes <$> getState   let attribs = toStringAttr attribs'   let start = fromMaybe 1 $ lookup "start" attribs >>= safeRead   let style = fromMaybe DefaultStyle@@ -346,8 +376,14 @@   -- note: if they have an <ol> or <ul> not in scope of a <li>,   -- treat it as a list item, though it's not valid xhtml...   skipMany nonItem-  items <- manyTill (pListItem nonItem) (pCloses "ol")-  return $ B.orderedListWith (start, style, DefaultDelim) $ map (fixPlains True) items+  if isNoteList+     then do+       _ <- manyTill (eFootnote <|> pBlank) (pCloses "ol")+       return mempty+     else do+       items <- manyTill (pListItem nonItem) (pCloses "ol")+       return $ B.orderedListWith (start, style, DefaultDelim) $+                map (fixPlains True) items  pDefinitionList :: PandocMonad m => TagParser m Blocks pDefinitionList = try $ do@@ -512,7 +548,10 @@ pHrule :: PandocMonad m => TagParser m Blocks pHrule = do   pSelfClosing (=="hr") (const True)-  return B.horizontalRule+  inNotes <- inFootnotes <$> getState+  return $ if inNotes+              then mempty+              else B.horizontalRule  pBlockQuote :: PandocMonad m => TagParser m Blocks pBlockQuote = do@@ -709,14 +748,18 @@   let title = fromAttrib "title" tag   let attr = toAttr $ filter (\(k,_) -> k /= "title" && k /= "href") attr'   lab <- mconcat <$> manyTill inline (pCloses "a")-  -- check for href; if href, then a link, otherwise a span-  case maybeFromAttrib "href" tag of-       Nothing   ->-         return $ extractSpaces (B.spanWith attr) lab-       Just url' -> do-         url <- canonicalizeUrl url'-         return $ extractSpaces-                   (B.linkWith attr (escapeURI url) title) lab+  st <- getState+  if inFootnotes st && maybeFromAttrib "role" tag == Just "doc-backlink"+     then return mempty+     else do+       -- check for href; if href, then a link, otherwise a span+       case maybeFromAttrib "href" tag of+            Nothing   ->+              return $ extractSpaces (B.spanWith attr) lab+            Just url' -> do+              url <- canonicalizeUrl url'+              return $ extractSpaces+                        (B.linkWith attr (escapeURI url) title) lab  pImage :: PandocMonad m => TagParser m Inlines pImage = do
src/Text/Pandoc/Readers/HTML/Types.hs view
@@ -52,6 +52,7 @@   , logMessages :: [LogMessage]   , macros      :: Map Text Macro   , readerOpts  :: ReaderOptions+  , inFootnotes :: Bool   }  -- | Local HTML parser state
src/Text/Pandoc/Readers/JATS.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE TupleSections #-} {-# LANGUAGE OverloadedStrings #-} {- |@@ -21,7 +22,7 @@ import Data.Generics import Data.List (foldl', intersperse) import qualified Data.Map as Map-import Data.Maybe (maybeToList, fromMaybe)+import Data.Maybe (maybeToList, fromMaybe, catMaybes) import Data.Text (Text) import qualified Data.Text as T import qualified Data.Text.Lazy as TL@@ -30,8 +31,9 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad) import Text.Pandoc.Options import Text.Pandoc.Shared (safeRead, extractSpaces)-import Text.TeXMath (readMathML, writeTeX)+import Text.Pandoc.Walk (walk) import Text.Pandoc.XML.Light+import Text.TeXMath (readMathML, writeTeX) import qualified Data.Set as S (fromList, member) import Data.Set ((\\)) import Text.Pandoc.Sources (ToSources(..), sourcesToText)@@ -43,6 +45,7 @@                           , jatsQuoteType    :: QuoteType                           , jatsMeta         :: Meta                           , jatsBook         :: Bool+                          , jatsFootnotes    :: Map.Map Text Blocks                           , jatsContent      :: [Content]                           } deriving Show @@ -51,6 +54,7 @@                  , jatsQuoteType = DoubleQuote                  , jatsMeta = mempty                  , jatsBook = False+                 , jatsFootnotes = mempty                  , jatsContent = [] }  @@ -63,7 +67,17 @@   tree <- either (throwError . PandocXMLError "") return $             parseXMLContents (TL.fromStrict . sourcesToText $ sources)   (bs, st') <- flip runStateT (def{ jatsContent = tree }) $ mapM parseBlock tree-  return $ Pandoc (jatsMeta st') (toList . mconcat $ bs)+  let footnotes = jatsFootnotes st'+  let blockList = toList $ mconcat bs+  let linkToFootnotes :: Inline -> Inline+      linkToFootnotes link'@(Link _attr _txt (href, _title)) =+        case T.uncons href of+          Just ('#', rid) -> case Map.lookup rid footnotes of+                               Just footnote -> Note (toList footnote)+                               Nothing       -> link'+          _               -> link'+      linkToFootnotes inline = inline+  return $ Pandoc (jatsMeta st') (walk linkToFootnotes blockList)  -- convenience function to get an attribute value, defaulting to "" attrValue :: Text -> Element -> Text@@ -175,6 +189,7 @@         "table-wrap" -> divWith (attrValue "id" e, ["table-wrap"], [])                           <$> getBlocks e         "caption" -> divWith (attrValue "id" e, ["caption"], []) <$> sect 6+        "fn-group" -> parseFootnoteGroup         "ref-list" -> parseRefList e         "?xml"  -> return mempty         _       -> getBlocks e@@ -239,6 +254,13 @@                           (attrValue "title" g)                    _   -> divWith (attrValue "id" e, ["fig"], []) <$> getBlocks e+         parseFootnoteGroup = do+           forM_ (filterChildren (named "fn") e) $ \fn -> do+             let id' = attrValue "id" fn+             contents <- getBlocks fn+             modify $ \st ->+               st { jatsFootnotes = Map.insert id' contents (jatsFootnotes st) }+           return mempty           parseTable = do                       let isCaption x = named "title" x || named "caption" x@@ -319,6 +341,7 @@   getAuthors e   getAffiliations e   getAbstract e+  getPubDate e   return mempty  getTitle :: PandocMonad m => Element -> JATS m ()@@ -358,6 +381,19 @@       addMeta "abstract" blks     Nothing -> pure () +getPubDate :: PandocMonad m => Element -> JATS m ()+getPubDate e =+  case filterElement (named "pub-date") e of+    Just d -> do+      case maybeAttrValue "iso-8601-date" d of+        Just isod -> addMeta "date" (text isod)+        Nothing -> do+          let yr = strContent <$> filterElement (named "year") d+          let mon = strContent <$> filterElement (named "month") d+          let day = strContent <$> filterElement (named "day") d+          addMeta "date" $ text $ T.intercalate "-" $ catMaybes [yr, mon, day]+    Nothing -> pure ()+ getContrib :: PandocMonad m => Element -> JATS m Inlines getContrib x = do   given <- maybe (return mempty) getInlines@@ -379,64 +415,69 @@ parseRef :: PandocMonad m          => Element -> JATS m (Map.Map Text MetaValue) parseRef e = do-  let refId = text $ attrValue "id" e-  let getInlineText n = maybe (return mempty) getInlines . filterChild (named n)-  case filterChild (named "element-citation") e of-       Just c  -> do-         let refType = text $-               case attrValue "publication-type" c of-                  "journal" -> "article-journal"-                  x -> x-         (refTitle, refContainerTitle) <- do-           t <- getInlineText "article-title" c-           ct <- getInlineText "source" c-           if t == mempty-              then return (ct, mempty)-              else return (t, ct)-         refLabel <- getInlineText "label" c-         refYear <- getInlineText "year" c-         refVolume <- getInlineText "volume" c-         refFirstPage <- getInlineText "fpage" c-         refLastPage <- getInlineText "lpage" c-         refPublisher <- getInlineText "publisher-name" c-         refPublisherPlace <- getInlineText "publisher-loc" c-         let refPages = refFirstPage <> (if refLastPage == mempty-                                            then mempty-                                            else text "\x2013" <> refLastPage)-         let personGroups' = filterChildren (named "person-group") c-         let getName nm = do-               given <- maybe (return mempty) getInlines-                         $ filterChild (named "given-names") nm-               family <- maybe (return mempty) getInlines-                         $ filterChild (named "surname") nm-               return $ toMetaValue $ Map.fromList [-                   ("given" :: Text, given)-                 , ("family", family)-                 ]-         personGroups <- mapM (\pg ->-                                do names <- mapM getName-                                            (filterChildren (named "name") pg)-                                   return (attrValue "person-group-type" pg,-                                           toMetaValue names))-                         personGroups'-         return $ Map.fromList $-           [ ("id" :: Text, toMetaValue refId)-           , ("type", toMetaValue refType)-           , ("title", toMetaValue refTitle)-           , ("container-title", toMetaValue refContainerTitle)-           , ("publisher", toMetaValue refPublisher)-           , ("publisher-place", toMetaValue refPublisherPlace)-           , ("title", toMetaValue refTitle)-           , ("issued", toMetaValue-                        $ Map.fromList [-                            ("year" :: Text, refYear)-                          ])-           , ("volume", toMetaValue refVolume)-           , ("page", toMetaValue refPages)-           , ("citation-label", toMetaValue refLabel)-           ] ++ personGroups-       Nothing -> return $ Map.insert "id" (toMetaValue refId) mempty-       -- TODO handle mixed-citation+  let combineWithDash x y = x <> text "-" <> y+  let getName nm = do+        given <- maybe (return mempty) getInlines+                  $ filterChild (named "given-names") nm+        family <- maybe (return mempty) getInlines+                  $ filterChild (named "surname") nm+        return $ toMetaValue $ Map.fromList [+            ("given" :: Text, given)+          , ("family", family)+          ]+  let refElement :: PandocMonad m+                 => Element -> Element -> JATS m (Maybe (Text, MetaValue))+      refElement c el =+        case qName (elName el) of+          "article-title" -> Just . ("title",) . toMetaValue <$> getInlines el+          "source" ->+            case filterChild (named "article-title") c of+              Just _ -> Just . ("container-title",) . toMetaValue <$> getInlines el+              Nothing -> Just . ("title",) . toMetaValue <$> getInlines el+          "label" -> Just . ("citation-label",) . toMetaValue <$> getInlines el+          "year" -> case filterChild (named "month") c of+                      Just m -> Just . ("issued",) . toMetaValue <$>+                                 (combineWithDash+                                    <$> getInlines el <*> getInlines m)+                      Nothing -> Just . ("issued",) . toMetaValue <$> getInlines el+          "volume" -> Just . ("volume",) . toMetaValue <$> getInlines el+          "issue" -> Just . ("issue",) . toMetaValue <$> getInlines el+          "isbn" -> Just . ("ISBN",) . toMetaValue <$> getInlines el+          "issn" -> Just . ("ISSN",) . toMetaValue <$> getInlines el+          "fpage" ->+            case filterChild (named "lpage") c of+              Just lp -> Just . ("page",) . toMetaValue <$>+                          (combineWithDash <$> getInlines el <*> getInlines lp)+              Nothing -> Just . ("page-first",) . toMetaValue <$> getInlines el+          "publisher-name" -> Just . ("publisher",) . toMetaValue <$> getInlines el+          "publisher-loc" -> Just . ("publisher-place",) . toMetaValue+                                <$> getInlines el+          "person-group" -> do names <- mapM getName+                                            (filterChildren (named "name") el)+                               pure $ Just (attrValue "person-group-type" el,+                                            toMetaValue names)+          "pub-id" -> case attrValue "pub-id-type" el of+                         "doi"  -> pure $ Just ("DOI",  toMetaValue $ strContent el)+                         "pmid" -> pure $ Just ("PMID", toMetaValue $ strContent el)+                         _      -> pure Nothing+          "object-id" -> case attrValue "pub-id-type" el of+                         "doi"  -> pure $ Just ("DOI",  toMetaValue $ strContent el)+                         "pmid" -> pure $ Just ("PMID", toMetaValue $ strContent el)+                         _      -> pure Nothing+++          _ -> pure Nothing+  refVariables <-+    case filterChild (named "element-citation") e of+      Just c -> (("type", toMetaValue $ case attrValue "publication-type" c of+                            "journal" -> "article-journal"+                            x -> x) :) .+                  catMaybes <$> mapM (refElement c) (elChildren c)+      Nothing -> pure []   -- TODO handle mixed-citation+  -- allows round-tripping, since JATS writer puts ref- in front of citation ids:+  let stripPref t = fromMaybe t $ T.stripPrefix "ref-" t+  return $ Map.fromList (("id", toMetaValue $ stripPref $ attrValue "id" e)+                        : refVariables)  textContent :: Element -> Text textContent = strContent
src/Text/Pandoc/Readers/LaTeX.hs view
@@ -36,10 +36,10 @@ import System.FilePath (addExtension, replaceExtension, takeExtension) import Text.Collate.Lang (renderLang) import Text.Pandoc.Builder as B-import Text.Pandoc.Class.PandocPure (PandocPure)-import Text.Pandoc.Class.PandocMonad (PandocMonad (..), getResourcePath,-                                      readFileFromDirs, report,-                                      setResourcePath)+import Text.Pandoc.Class (PandocPure, PandocMonad (..), getResourcePath,+                          readFileFromDirs, report,+                          setResourcePath, getZonedTime)+import Data.Time (ZonedTime(..), LocalTime(..), showGregorian) import Text.Pandoc.Error (PandocError (PandocParseError, PandocParsecError)) import Text.Pandoc.Highlighting (languagesByExtension) import Text.Pandoc.ImageSize (numUnit, showFl)@@ -84,7 +84,7 @@ readLaTeX opts ltx = do   let sources = toSources ltx   parsed <- runParserT parseLaTeX def{ sOptions = opts } "source"-               (tokenizeSources sources)+               (TokStream False (tokenizeSources sources))   case parsed of     Right result -> return result     Left e       -> throwError $ PandocParsecError sources e@@ -188,9 +188,6 @@  -- inline elements: -word :: PandocMonad m => LP m Inlines-word = str . untoken <$> satisfyTok isWordTok- inlineGroup :: PandocMonad m => LP m Inlines inlineGroup = do   ils <- grouped inline@@ -360,6 +357,7 @@     , ("underline", underline <$> tok)     , ("mbox", rawInlineOr "mbox" $ processHBox <$> tok)     , ("hbox", rawInlineOr "hbox" $ processHBox <$> tok)+    , ("vbox", rawInlineOr "vbox" tok)     , ("lettrine", rawInlineOr "lettrine" lettrine)     , ("(", mathInline . untokenize <$> manyTill anyTok (controlSeq ")"))     , ("[", mathDisplay . untokenize <$> manyTill anyTok (controlSeq "]"))@@ -383,7 +381,7 @@     , ("MakeTextLowercase", makeLowercase <$> tok)     , ("lowercase", makeLowercase <$> tok)     , ("thanks", skipopts >> note <$> grouped block)-    , ("footnote", skipopts >> note <$> grouped block)+    , ("footnote", skipopts >> footnote)     , ("passthrough", tok) -- \passthrough macro used by latex writer                            -- for listings     , ("includegraphics", do options <- option [] keyvals@@ -422,8 +420,31 @@     , ("uline", underline <$> tok)     -- plain tex stuff that should just be passed through as raw tex     , ("ifdim", ifdim)+    -- generally only used in \date+    , ("today", today)     ] +today :: PandocMonad m => LP m Inlines+today =+  text . T.pack . showGregorian . localDay . zonedTimeToLocalTime+    <$> getZonedTime++footnote :: PandocMonad m => LP m Inlines+footnote = do+  updateState $ \st -> st{ sLastNoteNum = sLastNoteNum st + 1 }+  contents <- grouped block >>= walkM resolveNoteLabel+  return $ note contents++resolveNoteLabel :: PandocMonad m => Inline -> LP m Inline+resolveNoteLabel (Span (_,cls,kvs) _)+  | Just lab <- lookup "label" kvs = do+      updateState $ \st -> st{+        sLabels = M.insert lab (toList . text . tshow $ sLastNoteNum st)+                      $ sLabels st }+      return $ Span (lab,cls,kvs) []+resolveNoteLabel il = return il++ lettrine :: PandocMonad m => LP m Inlines lettrine = do   optional rawopt@@ -492,11 +513,11 @@   spaces   no <- braced   toggles <- sToggles <$> getState-  inp <- getInput+  TokStream _ inp <- getInput   let name' = untokenize name   case M.lookup name' toggles of-                Just True  -> setInput (yes ++ inp)-                Just False -> setInput (no  ++ inp)+                Just True  -> setInput $ TokStream False (yes ++ inp)+                Just False -> setInput $ TokStream False (no  ++ inp)                 Nothing    -> do                   pos <- getPosition                   report $ UndefinedToggle name' pos@@ -508,9 +529,10 @@   str2 <- tok   ifequal <- braced   ifnotequal <- braced+  TokStream _ ts <- getInput   if str1 == str2-     then getInput >>= setInput . (ifequal ++)-     else getInput >>= setInput . (ifnotequal ++)+     then setInput $ TokStream False (ifequal ++ ts)+     else setInput $ TokStream False (ifnotequal ++ ts)   return mempty  coloredInline :: PandocMonad m => Text -> LP m Inlines@@ -578,34 +600,33 @@  inline :: PandocMonad m => LP m Inlines inline = do-  Tok pos toktype t <- lookAhead anyTok-  let symbolAsString = str . untoken <$> anySymbol+  Tok pos toktype t <- peekTok+  let eatOneToken = satisfyTok (const True)+  let symbolAsString = str t <$ eatOneToken   let unescapedSymbolAsString =-        do s <- untoken <$> anySymbol-           report $ ParsingUnescaped s pos-           return $ str s+        do eatOneToken+           report $ ParsingUnescaped t pos+           return $ str t   case toktype of-    Comment     -> mempty <$ comment-    Spaces      -> space <$ whitespace+    Comment     -> mempty <$ eatOneToken+    Spaces      -> space <$ eatOneToken     Newline     -> softbreak <$ endline-    Word        -> word-    Esc1        -> str . T.singleton <$> primEscape-    Esc2        -> str . T.singleton <$> primEscape+    Word        -> str t <$ eatOneToken     Symbol      ->       case t of-        "-"     -> symbol '-' *>+        "-"     -> eatOneToken *>                     option (str "-") (symbol '-' *>                       option (str "–") (str "—" <$ symbol '-'))-        "'"     -> symbol '\'' *>-                  option (str "’") (str  "”" <$ symbol '\'')-        "~"     -> str "\160" <$ symbol '~'+        "'"     -> eatOneToken *>+                    option (str "’") (str  "”" <$ symbol '\'')+        "~"     -> str "\160" <$ eatOneToken         "`"     -> doubleQuote <|> singleQuote <|> symbolAsString         "\""    -> doubleQuote <|> singleQuote <|> symbolAsString         "“"     -> doubleQuote <|> symbolAsString         "‘"     -> singleQuote <|> symbolAsString         "$"     -> dollarsMath <|> unescapedSymbolAsString         "|"     -> (guardEnabled Ext_literate_haskell *>-                    symbol '|' *> doLHSverb) <|> symbolAsString+                    eatOneToken *> doLHSverb) <|> symbolAsString         "{"     -> inlineGroup         "#"     -> unescapedSymbolAsString         "&"     -> unescapedSymbolAsString@@ -615,9 +636,11 @@         "}"     -> mzero         _       -> symbolAsString     CtrlSeq _   -> macroDef (rawInline "latex")+                  <|> inlineGroup                   <|> inlineCommand'                   <|> inlineEnvironment-                  <|> inlineGroup+    Esc1        -> str . T.singleton <$> primEscape+    Esc2        -> str . T.singleton <$> primEscape     _           -> mzero  inlines :: PandocMonad m => LP m Inlines@@ -628,7 +651,8 @@   toks <- try (sp *> bracketedToks <* sp)   -- now parse the toks as inlines   st <- getState-  parsed <- runParserT (mconcat <$> many inline) st "bracketed option" toks+  parsed <- runParserT (mconcat <$> many inline) st "bracketed option"+              (TokStream False toks)   case parsed of     Right result -> return result     Left e       -> throwError $ PandocParsecError (toSources toks) e@@ -676,7 +700,7 @@   skipMany opt   f <- T.unpack . removeDoubleQuotes . T.strip . untokenize <$> braced   oldToks <- getInput-  setInput []+  setInput $ TokStream False []   insertIncluded ".tex" f   bs <- blocks   eof@@ -684,15 +708,29 @@   return bs  include :: (PandocMonad m, Monoid a) => Text -> LP m a-include name = do+include _name = do   skipMany opt   fs <- map (T.unpack . removeDoubleQuotes . T.strip) . T.splitOn "," .          untokenize <$> braced-  let defaultExt | name == "usepackage" = ".sty"-                 | otherwise            = ".tex"+  let defaultExt = ".tex"   mapM_ (insertIncluded defaultExt) fs   return mempty +usepackage :: (PandocMonad m, Monoid a) => LP m a+usepackage = do+  skipMany opt+  fs <- map (T.unpack . removeDoubleQuotes . T.strip) . T.splitOn "," .+         untokenize <$> braced+  let parsePackage f = do+        TokStream _ ts <- getIncludedToks ".sty" f+        parseFromToks (do _ <- blocks+                          eof <|>+                            do pos <- getPosition+                               report $ CouldNotParseIncludeFile (T.pack f) pos)+                      ts+  mapM_ parsePackage fs+  return mempty+ readFileFromTexinputs :: PandocMonad m => FilePath -> LP m (Maybe Text) readFileFromTexinputs fp = do   fileContentsMap <- sFileContents <$> getState@@ -703,11 +741,11 @@                <$> lookupEnv "TEXINPUTS"       readFileFromDirs dirs fp -insertIncluded :: PandocMonad m-               => FilePath-               -> FilePath-               -> LP m ()-insertIncluded defaultExtension f' = do+getIncludedToks :: PandocMonad m+                => FilePath+                -> FilePath+                -> LP m TokStream+getIncludedToks defaultExtension f' = do   let f = case takeExtension f' of                 ".tex" -> f'                 ".sty" -> f'@@ -723,9 +761,18 @@                    Nothing -> do                      report $ CouldNotLoadIncludeFile (T.pack f) pos                      return ""-  getInput >>= setInput . (tokenize (initialPos f) contents ++)   updateState dropLatestIncludeFile+  return $ TokStream False $ tokenize (initialPos f) contents +insertIncluded :: PandocMonad m+               => FilePath+               -> FilePath+               -> LP m ()+insertIncluded defaultExtension f' = do+  contents <- getIncludedToks defaultExtension f'+  ts <- getInput+  setInput $ contents <> ts+ authors :: PandocMonad m => LP m () authors = try $ do   bgroup@@ -916,7 +963,7 @@    , ("include", rawBlockOr "include" $ include "include")    , ("input", rawBlockOr "input" $ include "input")    , ("subfile", rawBlockOr "subfile" doSubfile)-   , ("usepackage", rawBlockOr "usepackage" $ include "usepackage")+   , ("usepackage", rawBlockOr "usepackage" usepackage)    -- preamble    , ("PackageError", mempty <$ (braced >> braced >> braced))    -- epigraph package@@ -1241,7 +1288,7 @@  block :: PandocMonad m => LP m Blocks block = do-  Tok _ toktype _ <- lookAhead anyTok+  Tok _ toktype _ <- peekTok   res <- (case toktype of             Newline           -> mempty <$ spaces1             Spaces            -> mempty <$ spaces1
src/Text/Pandoc/Readers/LaTeX/Citation.hs view
@@ -117,7 +117,8 @@     -- now parse the toks as inlines     st <- getState     parsed <- lift $-      runParserT (mconcat <$> many inline) st "bracketed option" toks+      runParserT (mconcat <$> many inline) st "bracketed option"+       (TokStream False toks)     case parsed of       Right result -> return result       Left e       -> throwError $ PandocParsecError (toSources toks) e
src/Text/Pandoc/Readers/LaTeX/Inline.hs view
@@ -94,8 +94,9 @@        Nothing -> return t        Just i  -> do          let (t1, t2) = T.splitAt i txt-         inp <- getInput-         setInput $ Tok (incSourceColumn pos i) Symbol (T.singleton stopchar)+         TokStream macrosExpanded inp <- getInput+         setInput $ TokStream macrosExpanded+                  $ Tok (incSourceColumn pos i) Symbol (T.singleton stopchar)                   : tokenize (incSourceColumn pos (i + 1)) (T.drop 1 t2) ++ inp          return $ Tok pos toktype t1
src/Text/Pandoc/Readers/LaTeX/Math.hs view
@@ -102,6 +102,7 @@   , ("dgroup*", mathEnvWith id (Just "aligned") "dgroup*")   , ("darray", mathEnvWith id (Just "aligned") "darray")   , ("darray*", mathEnvWith id (Just "aligned") "darray*")+  , ("subequations", mathEnvWith id Nothing "subequations")   ]  theoremstyle :: PandocMonad m => LP m Blocks
src/Text/Pandoc/Readers/LaTeX/Parsing.hs view
@@ -1,6 +1,7 @@ {-# LANGUAGE FlexibleInstances     #-} {-# LANGUAGE FlexibleContexts      #-} {-# LANGUAGE LambdaCase            #-}+{-# LANGUAGE BangPatterns          #-} {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE OverloadedStrings     #-} {-# LANGUAGE ScopedTypeVariables   #-}@@ -24,6 +25,7 @@   , LaTeXState(..)   , defaultLaTeXState   , LP+  , TokStream(..)   , withVerbatimMode   , rawLaTeXParser   , applyMacros@@ -34,6 +36,7 @@   , untoken   , toksToString   , satisfyTok+  , peekTok   , parseFromToks   , disablingWithRaw   , doMacros@@ -119,6 +122,7 @@                                         ArgSpec (..), Tok (..), TokType (..)) import Text.Pandoc.Shared import Text.Parsec.Pos+import Text.Parsec (Stream(uncons)) import Text.Pandoc.Walk  newtype DottedNum = DottedNum [Int]@@ -162,6 +166,7 @@                             , sLastHeaderNum :: DottedNum                             , sLastFigureNum :: DottedNum                             , sLastTableNum  :: DottedNum+                            , sLastNoteNum   :: Int                             , sTheoremMap    :: M.Map Text TheoremSpec                             , sLastTheoremStyle :: TheoremStyle                             , sLastLabel     :: Maybe Text@@ -189,6 +194,7 @@                               , sLastHeaderNum = DottedNum []                               , sLastFigureNum = DottedNum []                               , sLastTableNum  = DottedNum []+                              , sLastNoteNum   = 0                               , sTheoremMap    = M.empty                               , sLastTheoremStyle = PlainStyle                               , sLastLabel     = Nothing@@ -241,8 +247,25 @@ instance Default LaTeXState where   def = defaultLaTeXState -type LP m = ParserT [Tok] LaTeXState m+-- The Boolean is True if macros have already been expanded,+-- False if they need expanding.+data TokStream = TokStream !Bool [Tok]+  deriving (Show) +instance Semigroup TokStream where+  (TokStream exp1 ts1) <> (TokStream exp2 ts2) =+    TokStream (if null ts1 then exp2 else exp1) (ts1 <> ts2)++instance Monoid TokStream where+  mempty = TokStream False mempty+  mappend = (<>)++instance Monad m => Stream TokStream m Tok where+  uncons (TokStream _ []) = return Nothing+  uncons (TokStream _ (t:ts)) = return $ Just (t, TokStream False ts)++type LP m = ParserT TokStream LaTeXState m+ withVerbatimMode :: PandocMonad m => LP m a -> LP m a withVerbatimMode parser = do   alreadyVerbatimMode <- sVerbatimMode <$> getState@@ -267,11 +290,12 @@   let preparser = setStartPos >> parser   let rawparser = (,) <$> withRaw valParser <*> getState   res' <- lift $ runParserT (withRaw (preparser >> getPosition))-                            lstate "chunk" toks+                            lstate "chunk" $ TokStream False toks   case res' of        Left _    -> mzero        Right (endpos, toks') -> do-         res <- lift $ runParserT rawparser lstate' "chunk" toks'+         res <- lift $ runParserT rawparser lstate' "chunk"+                     $ TokStream False toks'          case res of               Left _    -> mzero               Right ((val, raw), st) -> do@@ -301,7 +325,8 @@       pstate <- getState       let lstate = def{ sOptions = extractReaderOptions pstate                       , sMacros  = extractMacros pstate :| [] }-      res <- runParserT retokenize lstate "math" (tokenize (initialPos "math") s)+      res <- runParserT retokenize lstate "math" $+                 TokStream False (tokenize (initialPos "math") s)       case res of            Left e   -> Prelude.fail (show e)            Right s' -> return s'@@ -456,7 +481,7 @@ parseFromToks :: PandocMonad m => LP m a -> [Tok] -> LP m a parseFromToks parser toks = do   oldInput <- getInput-  setInput toks+  setInput $ TokStream False toks   oldpos <- getPosition   case toks of      Tok pos _ _ : _ -> setPosition pos@@ -480,20 +505,29 @@     res <- tokenPrim (T.unpack . untoken) updatePos matcher     updateState $ \st ->       if sEnableWithRaw st-         then st{ sRawTokens = IntMap.map (res:) $ sRawTokens st }+         then+           let !newraws = IntMap.map (res:) $! sRawTokens st+            in  st{ sRawTokens = newraws }          else st     return $! res   where matcher t | f t       = Just t                   | otherwise = Nothing-        updatePos :: SourcePos -> Tok -> [Tok] -> SourcePos-        updatePos _spos _ (Tok pos _ _ : _) = pos-        updatePos spos (Tok _ _ t)  []      = incSourceColumn spos (T.length t)+        updatePos :: SourcePos -> Tok -> TokStream -> SourcePos+        updatePos _spos _ (TokStream _ (Tok pos _ _ : _)) = pos+        updatePos spos (Tok _ _ t) _ = incSourceColumn spos (T.length t) +peekTok :: PandocMonad m => LP m Tok+peekTok = do+  doMacros+  lookAhead (satisfyTok (const True))+ doMacros :: PandocMonad m => LP m () doMacros = do-  st <- getState-  unless (sVerbatimMode st) $-    getInput >>= doMacros' 1 >>= setInput+  TokStream macrosExpanded toks <- getInput+  unless macrosExpanded $ do+    st <- getState+    unless (sVerbatimMode st) $+      doMacros' 1 toks >>= setInput . TokStream True  doMacros' :: PandocMonad m => Int -> [Tok] -> LP m [Tok] doMacros' n inp =@@ -566,10 +600,10 @@                              Just o  -> do                                 x <- option o bracketedToks                                 getargs (M.singleton 1 x) $ drop 1 argspecs-                   rest <- getInput+                   TokStream _ rest <- getInput                    return (args, rest)              lstate <- getState-             res <- lift $ runParserT getargs' lstate "args" ts+             res <- lift $ runParserT getargs' lstate "args" $ TokStream False ts              case res of                Left _ -> Prelude.fail $ "Could not parse arguments for " ++                                 T.unpack name@@ -597,7 +631,8 @@  handleIf :: PandocMonad m => Bool -> [Tok] -> LP m [Tok] handleIf b ts = do-  res' <- lift $ runParserT (ifParser b) defaultLaTeXState "tokens" ts+  res' <- lift $ runParserT (ifParser b) defaultLaTeXState "tokens"+               $ TokStream False ts   case res' of     Left _ -> Prelude.fail "Could not parse conditional"     Right ts' -> return ts'@@ -608,7 +643,7 @@                     *> anyTok)   elseToks <- (controlSeq "else" >> manyTill anyTok (controlSeq "fi"))                  <|> ([] <$ controlSeq "fi")-  rest <- getInput+  TokStream _ rest <- getInput   return $ (if b then ifToks else elseToks) ++ rest  startsWithAlphaNum :: Text -> Bool@@ -715,8 +750,9 @@   singleCharFromWord = do     Tok pos toktype t <- disablingWithRaw $ satisfyTok isWordTok     let (t1, t2) = (T.take 1 t, T.drop 1 t)-    inp <- getInput-    setInput $ Tok pos toktype t1 : Tok (incSourceColumn pos 1) toktype t2 : inp+    TokStream macrosExpanded inp <- getInput+    setInput $ TokStream macrosExpanded+             $ Tok pos toktype t1 : Tok (incSourceColumn pos 1) toktype t2 : inp     anyTok  specialChars :: Set.Set Char@@ -800,7 +836,8 @@         Tok (incSourceColumn (incSourceLine pos' (sourceLine pos - 1))              (sourceColumn pos)) toktype' txt'   let newtoks = map updPos $ tokenize pos $ T.tail txt-  getInput >>= setInput . ((Tok pos Symbol "%" : newtoks) ++))+  TokStream macrosExpanded ts <- getInput+  setInput $ TokStream macrosExpanded ((Tok pos Symbol "%" : newtoks) ++ ts))     <|> return ()  bracedOrToken :: PandocMonad m => LP m [Tok]
src/Text/Pandoc/Readers/LaTeX/Table.hs view
@@ -62,10 +62,11 @@ -- Split a Word into individual Symbols (for parseAligns) splitWordTok :: PandocMonad m => LP m () splitWordTok = do-  inp <- getInput+  TokStream macrosExpanded inp <- getInput   case inp of        (Tok spos Word t : rest) ->-         setInput $ map (Tok spos Symbol . T.singleton) (T.unpack t) <> rest+         setInput $ TokStream macrosExpanded+                  $ map (Tok spos Symbol . T.singleton) (T.unpack t) <> rest        _ -> return ()  parseAligns :: PandocMonad m => LP m [(Alignment, ColWidth, ([Tok], [Tok]))]@@ -108,8 +109,9 @@         spaces         spec <- braced         case safeRead ds of-             Just n  ->-               getInput >>= setInput . (mconcat (replicate n spec) ++)+             Just n  -> do+               TokStream _ ts <- getInput+               setInput $ TokStream False (mconcat (replicate n spec) ++ ts)              Nothing -> Prelude.fail $ "Could not parse " <> T.unpack ds <> " as number"   bgroup   spaces
src/Text/Pandoc/Readers/Markdown.hs view
@@ -1,4 +1,5 @@ {-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE BangPatterns        #-} {-# LANGUAGE FlexibleContexts    #-} {-# LANGUAGE TupleSections       #-} {-# LANGUAGE OverloadedStrings   #-}@@ -21,6 +22,7 @@  import Control.Monad import Control.Monad.Except (throwError)+import Data.Bifunctor (second) import Data.Char (isAlphaNum, isPunctuation, isSpace) import Text.DocLayout (realLength) import Data.List (transpose, elemIndex, sortOn, foldl')@@ -44,7 +46,7 @@ import Text.Pandoc.Logging import Text.Pandoc.Options import Text.Pandoc.Walk (walk)-import Text.Pandoc.Parsing hiding (tableWith)+import Text.Pandoc.Parsing hiding (tableCaption) import Text.Pandoc.Readers.HTML (htmlInBalanced, htmlTag, isBlockTag,                                  isCommentTag, isInlineTag, isTextTag) import Text.Pandoc.Readers.LaTeX (applyMacros, rawLaTeXBlock, rawLaTeXInline)@@ -635,7 +637,7 @@ identifierAttr :: PandocMonad m => MarkdownParser m (Attr -> Attr) identifierAttr = try $ do   char '#'-  result <- identifier+  result <- T.pack <$> many1 (alphaNum <|> oneOf "-_:.") -- see #7920   return $ \(_,cs,kvs) -> (result,cs,kvs)  classAttr :: PandocMonad m => MarkdownParser m (Attr -> Attr)@@ -1298,14 +1300,18 @@ -- Parse a simple table with '---' header and one line per row. simpleTable :: PandocMonad m             => Bool  -- ^ Headerless table-            -> MarkdownParser m ([Alignment], [Double], F [Row], F [Row])+            -> MarkdownParser m (F TableComponents) simpleTable headless = do-  (aligns, _widths, heads', lines') <--       tableWith (simpleTableHeader headless) tableLine+  tableComponents <-+       tableWith' NormalizeHeader+              (simpleTableHeader headless) tableLine               (return ())               (if headless then tableFooter else tableFooter <|> blanklines')-  -- Simple tables get 0s for relative column widths (i.e., use default)-  return (aligns, replicate (length aligns) 0, heads', lines')+  -- All columns in simple tables have default widths.+  let useDefaultColumnWidths tc =+        let cs' = map (second (const ColWidthDefault)) $ tableColSpecs tc+        in tc {tableColSpecs = cs'}+  return $ useDefaultColumnWidths <$> tableComponents  -- Parse a multiline table:  starts with row of '-' on top, then header -- (which may be multiline), then the rows,@@ -1313,9 +1319,10 @@ -- ending with a footer (dashed line followed by blank line). multilineTable :: PandocMonad m                => Bool -- ^ Headerless table-               -> MarkdownParser m ([Alignment], [Double], F [Row], F [Row])+               -> MarkdownParser m (F TableComponents) multilineTable headless =-  tableWith (multilineTableHeader headless) multilineRow blanklines tableFooter+  tableWith' NormalizeHeader (multilineTableHeader headless)+             multilineRow blanklines tableFooter  multilineTableHeader :: PandocMonad m                      => Bool -- ^ Headerless table@@ -1355,8 +1362,8 @@ -- which may be grid, separated by blank lines, and -- ending with a footer (dashed line followed by blank line). gridTable :: PandocMonad m => Bool -- ^ Headerless table-          -> MarkdownParser m ([Alignment], [Double], F [Row], F [Row])-gridTable headless = gridTableWith' parseBlocks headless+          -> MarkdownParser m (F TableComponents)+gridTable headless = gridTableWith' NormalizeHeader parseBlocks headless  pipeBreak :: PandocMonad m => MarkdownParser m ([Alignment], [Int]) pipeBreak = try $ do@@ -1364,13 +1371,13 @@   openPipe <- (True <$ char '|') <|> return False   first <- pipeTableHeaderPart   rest <- many $ sepPipe *> pipeTableHeaderPart-  -- surrounding pipes needed for a one-column table:-  guard $ not (null rest && not openPipe)-  optional (char '|')+  closePipe <- (True <$ char '|') <|> return False+  -- at least one pipe needed for a one-column table:+  guard $ not (null rest && not (openPipe || closePipe))   blankline   return $ unzip (first:rest) -pipeTable :: PandocMonad m => MarkdownParser m ([Alignment], [Double], F [Row], F [Row])+pipeTable :: PandocMonad m => MarkdownParser m (F TableComponents) pipeTable = try $ do   nonindentSpaces   lookAhead nonspaceChar@@ -1390,7 +1397,8 @@                   else replicate (length aligns) 0.0   (headCells :: F [Blocks]) <- sequence <$> mapM cellContents heads'   (rows :: F [[Blocks]]) <- sequence <$> mapM (fmap sequence . mapM cellContents) lines''-  return (aligns, widths, toHeaderRow <$> headCells, map toRow <$> rows)+  return $+    toTableComponents' NormalizeHeader aligns widths <$> headCells <*> rows  sepPipe :: PandocMonad m => MarkdownParser m () sepPipe = try $ do@@ -1406,9 +1414,10 @@   -- split into cells   let chunk = void (code <|> math <|> rawHtmlInline <|> escapedChar <|> rawLaTeXInline')        <|> void (noneOf "|\n\r")-  cells <- (snd <$> withRaw (many chunk)) `sepEndBy1` char '|'-  -- surrounding pipes needed for a one-column table:-  guard $ not (length cells == 1 && not openPipe)+  cells <- (snd <$> withRaw (many chunk)) `sepBy1` char '|'+  closePipe <- (True <$ char '|') <|> return False+  -- at least one pipe needed for a one-column table:+  guard $ not (length cells == 1 && not (openPipe || closePipe))   blankline   return cells @@ -1445,29 +1454,10 @@        (_, T.uncons -> Just ('|', _)) -> return ()        _                              -> mzero --- | Parse a table using 'headerParser', 'rowParser',--- 'lineParser', and 'footerParser'.  Variant of the version in--- Text.Pandoc.Parsing.-tableWith :: PandocMonad m-          => MarkdownParser m (F [Blocks], [Alignment], [Int])-          -> ([Int] -> MarkdownParser m (F [Blocks]))-          -> MarkdownParser m sep-          -> MarkdownParser m end-          -> MarkdownParser m ([Alignment], [Double], F [Row], F [Row])-tableWith headerParser rowParser lineParser footerParser = try $ do-    (heads, aligns, indices) <- headerParser-    lines' <- fmap sequence $ rowParser indices `sepEndBy1` lineParser-    footerParser-    numColumns <- getOption readerColumns-    let widths = if null indices-                    then replicate (length aligns) 0.0-                    else widthsFromIndices numColumns indices-    return (aligns, widths, toHeaderRow <$> heads, map toRow <$> lines')- table :: PandocMonad m => MarkdownParser m (F Blocks) table = try $ do   frontCaption <- option Nothing (Just <$> tableCaption)-  (aligns, widths, heads, lns) <-+  tableComponents <-          (guardEnabled Ext_pipe_tables >> try (scanForPipe >> pipeTable)) <|>          (guardEnabled Ext_multiline_tables >> try (multilineTable False)) <|>          (guardEnabled Ext_simple_tables >>@@ -1480,23 +1470,10 @@   caption <- case frontCaption of                   Nothing -> option (return mempty) tableCaption                   Just c  -> return c-  -- renormalize widths if greater than 100%:-  let totalWidth = sum widths-  let widths' = if totalWidth < 1-                   then widths-                   else map (/ totalWidth) widths-  let strictPos w-        | w > 0     = ColWidth w-        | otherwise = ColWidthDefault   return $ do     caption' <- caption-    heads' <- heads-    lns' <- lns-    return $ B.table (B.simpleCaption $ B.plain caption')-                     (zip aligns (strictPos <$> widths'))-                     (TableHead nullAttr heads')-                     [TableBody nullAttr 0 [] lns']-                     (TableFoot nullAttr [])+    (TableComponents _attr _capt colspecs th tb tf) <- tableComponents+    return $ B.table (B.simpleCaption $ B.plain caption') colspecs th tb tf  -- -- inline@@ -1748,7 +1725,7 @@  str :: PandocMonad m => MarkdownParser m (F Inlines) str = do-  result <- mconcat <$> many1+  !result <- mconcat <$> many1              ( T.pack <$> (many1 alphaNum)               <|> "." <$ try (char '.' <* notFollowedBy (char '.')) )   updateLastStrPos@@ -1764,11 +1741,11 @@                         ils' <- ils                         case B.toList ils' of                              [Space] ->-                                 return (B.str result <> B.str "\160")-                             _ -> return (B.str result <> ils'))-                <|> return (return (B.str result))-         else return (return (B.str result)))-     <|> return (return (B.str result))+                                 return $! (B.str result <> B.str "\160")+                             _ -> return $! (B.str result <> ils'))+                <|> return (return $! B.str result)+         else return (return $! B.str result))+     <|> return (return $! B.str result)  -- an endline character that can be treated as a space, not a structural break endline :: PandocMonad m => MarkdownParser m (F Inlines)@@ -2282,9 +2259,3 @@     fmap B.doubleQuoted . trimInlinesF . mconcat <$>       many1Till inline doubleQuoteEnd))     <|> (return (return (B.str "\8220")))--toRow :: [Blocks] -> Row-toRow = Row nullAttr . map B.simpleCell--toHeaderRow :: [Blocks] -> [Row]-toHeaderRow l = [toRow l | not (null l) && not (all null l)]
src/Text/Pandoc/Readers/MediaWiki.hs view
@@ -34,7 +34,7 @@ import Text.Pandoc.Definition import Text.Pandoc.Logging import Text.Pandoc.Options-import Text.Pandoc.Parsing hiding (nested)+import Text.Pandoc.Parsing hiding (nested, tableCaption) import Text.Pandoc.Readers.HTML (htmlTag, isBlockTag, isCommentTag) import Text.Pandoc.Shared (safeRead, stringify, stripTrailingNewlines,                            trim, splitTextBy, tshow)
src/Text/Pandoc/Readers/Org/BlockStarts.hs view
@@ -58,7 +58,6 @@   skipSpaces *> string "\\begin{"              *> latexEnvName              <* string "}"-             <* blankline  where    latexEnvName :: Monad m => OrgParser m Text    latexEnvName = try $ mappend <$> many1Char alphaNum <*> option "" (textStr "*")
src/Text/Pandoc/Readers/Org/Blocks.hs view
@@ -36,7 +36,7 @@ import Data.Char (isSpace) import Data.Default (Default) import Data.Functor (($>))-import Data.List (foldl', intersperse)+import Data.List (foldl') import Data.Maybe (fromMaybe, isJust, isNothing) import Data.Text (Text) import Data.List.NonEmpty (nonEmpty)@@ -509,8 +509,14 @@ --  specialLine :: PandocMonad m => OrgParser m (F Blocks)-specialLine = fmap return . try $ rawExportLine <|> metaLine <|> commentLine+specialLine = fmap return . try $+  rawExportLine <|> printbibliographyLine <|> metaLine <|> commentLine +printbibliographyLine :: PandocMonad m => OrgParser m Blocks+printbibliographyLine = do+  try $ skipSpaces <* string "#+print_bibliography:" <* anyLine+  return $ B.divWith ("refs",[],[]) mempty+ -- | Include the content of a file. include :: PandocMonad m => OrgParser m (F Blocks) include = try $ do@@ -760,19 +766,17 @@   texOpt  <- getExportSetting exportWithLatex   let envStart = "\\begin{" <> envName <> "}"   let envEnd = "\\end{" <> envName <> "}"-  envLines <- do-    content <- manyTill anyLine (latexEnd envName)-    return $ envStart : content ++ [envEnd]+  envContent <- do+    content <- manyTillChar anyChar (latexEnd envName)+    return $ envStart <> content <> envEnd   returnF $ case texOpt of-    TeXExport -> B.rawBlock "latex" . T.unlines $ envLines+    TeXExport -> B.rawBlock "latex" (envContent <> "\n")     TeXIgnore   -> mempty-    TeXVerbatim -> B.para . mconcat . intersperse B.softbreak $-                   map B.str envLines+    TeXVerbatim -> B.para . B.text $ envContent  where   latexEnd :: Monad m => Text -> OrgParser m ()   latexEnd envName = try . void-     $ skipSpaces-    <* textStr ("\\end{" <> envName <> "}")+     $ textStr ("\\end{" <> envName <> "}")     <* blankline
src/Text/Pandoc/Readers/Org/DocumentTree.hs view
@@ -41,6 +41,7 @@              -> OrgParser m (F Inlines)              -> OrgParser m (F Headline) documentTree blocks inline = do+  many commentLine   properties <- option mempty propertiesDrawer   initialBlocks <- blocks   headlines <- sequence <$> manyTill (headline blocks inline 1) eof@@ -59,6 +60,9 @@       , headlineContents = initialBlocks'       , headlineChildren = headlines'       }+  where+    commentLine :: Monad m => OrgParser m ()+    commentLine = commentLineStart <* anyLine  -- | Create a tag containing the given string. toTag :: Text -> Tag@@ -405,7 +409,8 @@     key :: Monad m => OrgParser m PropertyKey    key = fmap toPropertyKey . try $-         skipSpaces *> char ':' *> many1TillChar nonspaceChar (char ':')+         skipSpaces *> char ':' *>+         many1TillChar nonspaceChar (try $ char ':' *> spaceChar)     value :: Monad m => OrgParser m PropertyValue    value = fmap toPropertyValue . try $
src/Text/Pandoc/Readers/Org/Meta.hs view
@@ -78,6 +78,7 @@ keywordHandlers :: PandocMonad m => Map Text (OrgParser m ()) keywordHandlers = Map.fromList   [ "author" ~~> lineOfInlines `parseThen` collectLines "author"+  , "bibliography" ~~> fmap pure anyLine `parseThen` collectAsList "bibliography"   , "creator" ~~> fmap pure anyLine `parseThen` B.setMeta "creator"   , "date" ~~> lineOfInlines `parseThen` B.setMeta "date"   , "description" ~~> lineOfInlines `parseThen` collectLines "description"@@ -156,7 +157,7 @@     MetaBool _bool    -> []  -- | Accumulate the result as a MetaList under the given key.-collectAsList :: Text -> Inlines -> Meta -> Meta+collectAsList :: B.ToMetaValue a => Text -> a -> Meta -> Meta collectAsList key value meta =   let value' = metaListAppend meta (B.toMetaValue value)   in B.setMeta key value' meta
+ src/Text/Pandoc/Readers/RIS.hs view
@@ -0,0 +1,255 @@+{-# LANGUAGE OverloadedStrings #-}+{- |+   Module      : Text.Pandoc.Readers.RIS+   Copyright   : Copyright (C) 2022 John MacFarlane+   License     : GNU GPL, version 2 or above++   Maintainer  : John MacFarlane <jgm@berkeley.edu>+   Stability   : alpha+   Portability : portable++Parses RIS bibliographies into a Pandoc document+with empty body and `references` and `nocite` fields+in the metadata.  A wildcard `nocite` is used so that+if the document is rendered in another format, the+entire bibliography will be printed.+-}+module Text.Pandoc.Readers.RIS+  ( readRIS+  )+where++import Text.Pandoc.Options+import Text.Pandoc.Definition+import Text.Pandoc.Parsing+import Data.Char (isAsciiUpper, isDigit, isSpace, ord, chr)+import Data.List (foldl')+import Citeproc (Reference(..), ItemId(..), Val(..), Date(..), DateParts(..),+                 toVariable)+import Text.Pandoc.Builder as B+import Text.Pandoc.Class (PandocMonad)+import Text.Pandoc.Citeproc.MetaValue (referenceToMetaValue)+import Text.Pandoc.Citeproc.BibTeX (toName)+import Control.Monad.Except (throwError)+import qualified Data.Text as T+import Data.Text (Text)+import Data.Maybe (fromMaybe)+import qualified Data.Map as M+import Safe (readMay)++-- | Read RIS from an input string and return a Pandoc document.+-- The document will have only metadata, with an empty body.+-- The metadata will contain a `references` field with the+-- bibliography entries, and a `nocite` field with the wildcard `[@*]`.+readRIS :: (PandocMonad m, ToSources a)+        => ReaderOptions -> a -> m Pandoc+readRIS _opts inp = do+  parsed <- readWithM risReferences () inp+  case parsed of+    Right refs -> do+      refs' <- mapM (traverse (return . text)) refs+      return $+        setMeta "nocite" (cite [Citation {citationId = "*"+                                         , citationPrefix = []+                                         , citationSuffix = []+                                         , citationMode = NormalCitation+                                         , citationNoteNum = 0+                                         , citationHash = 0}] (str "[@*]")) $+        setMeta "references" (map referenceToMetaValue refs') $+        B.doc mempty+    Left e       -> throwError e++type RISParser m = ParserT Sources () m++risLine :: PandocMonad m => RISParser m (Text, Text)+risLine = do+  key <- T.pack <$> count 2 (satisfy (\c -> isAsciiUpper c || isDigit c))+  _ <- many1 spaceChar+  char '-'+  _ <- many1 spaceChar+  val <- anyLine+  return (key, T.strip val)++risSeparator :: PandocMonad m => RISParser m ()+risSeparator = do+  try $ string "ER"+  _ <- many1 spaceChar+  char '-'+  _ <- anyLine+  return ()++risRecord :: PandocMonad m => RISParser m [(Text, Text)]+risRecord = manyTill risLine risSeparator++risRecordToReference :: [(Text, Text)] -> Reference Text+risRecordToReference keys = addId $ foldr go defref keys+ where+   go (key, val) =+     case key of+       "TY" -> \ref -> ref{ referenceType =+          fromMaybe "misc" (M.lookup val risTypes) }+       "ID" -> \ref -> ref{ referenceId = ItemId val }+       "VL" -> addVar "volume" val+       "KW" -> \ref ->+         ref{ referenceVariables =+               M.alter (\x -> case x of+                           Nothing -> Just $ TextVal val+                           Just (TextVal kws)+                                   -> Just (TextVal (kws <> ", " <> val))+                           _ -> x)+               "keyword"+               (referenceVariables ref) }+       "PB" -> addVar "publisher" val+       "PP" -> addVar "publisher-place" val+       "DO" -> addVar "DOI" val+       "SP" -> \ref ->+         case M.lookup "page" (referenceVariables ref) of+           Nothing -> addVar "page" val ref+           Just (FancyVal eg) -> addVar "page" (val <> eg) ref+           _ -> ref+       "EP" -> \ref ->+         case M.lookup "page" (referenceVariables ref) of+           Nothing -> addVar "page" ("-" <> val) ref+           Just (FancyVal eg) -> addVar "page" (val <> "-" <> eg) ref+           _ -> ref+       "AU" -> addName "author" val+       "A1" -> addName "author" val+       "ED" -> addName "editor" val+       "A2" -> addName "editor" val+       "TI" -> addVar "title" val+       "T1" -> addVar "title" val+       "CT" -> addVar "title" val+       "BT" -> \ref ->+         if referenceType ref == "book"+            then addVar "title" val ref+            else addVar "container-title" val ref+       "JO" -> addVar "container-title" val+       "JF" -> addVar "container-title" val+       "T2" -> addVar "container-title" val+       "ET" -> addVar "edition" val+       "NV" -> addVar "number-of-volumes" val+       "AB" -> addVar "abstract" val+       "PY" -> addYear "issued" val+       "Y1" -> addYear "issued" val+       "IS" -> addVar "issue" val+       "SN" -> addVar "ISSN" val+       "LA" -> addVar "language" val+       "UR" -> addVar "url" val+       "LK" -> addVar "url" val+       _ -> id -- TODO+   addVar k v r = r{ referenceVariables =+                       M.insert (toVariable k) (FancyVal v)+                       (referenceVariables r) }+   addName k v r =+     let new = toName [] . B.toList .  B.text $ v+         f Nothing   = Just (NamesVal new)+         f (Just (NamesVal ns)) = Just (NamesVal (ns ++ new))+         f (Just x) = Just x+      in r{ referenceVariables =+              M.alter f k (referenceVariables r) }+   addYear k v r =+     let d = DateVal $+              case readMay (T.unpack v) of+                Nothing ->+                  Date { dateParts = []+                       , dateCirca = False+                       , dateSeason = Nothing+                       , dateLiteral = Just v }+                Just y ->+                  Date { dateParts = [DateParts [y]]+                       , dateCirca = False+                       , dateSeason = Nothing+                       , dateLiteral = Nothing }+      in r{ referenceVariables = M.insert k d (referenceVariables r) }++   defref = Reference{+       referenceId = mempty+     , referenceType = mempty+     , referenceDisambiguation = Nothing+     , referenceVariables = mempty }+   addId rec =+     if referenceId rec == mempty+        then rec{ referenceId = ItemId (authors <> pubdate) }+        else rec+   authors = T.intercalate "_" $+               [T.takeWhile (\c -> c /= ',' && not (isSpace c)) n+                 | (k, n) <- keys, k == "AU" || k == "A1"]+   pubdate = mconcat ["_" <> d | (k, d) <- keys, k == "PY" || k == "Y1"]++risReferences :: PandocMonad m => RISParser m [Reference Text]+risReferences = do+  recs <- many risRecord+  spaces+  eof+  return $ fixDuplicateIds $ map risRecordToReference recs++fixDuplicateIds :: [Reference Text] -> [Reference Text]+fixDuplicateIds = reverse . snd . foldl' go (mempty, [])+ where+   go (ids_seen, refs) ref =+     case M.lookup (referenceId ref) ids_seen of+       Nothing -> (M.insert (referenceId ref) (ord 'a') ids_seen, ref:refs)+       Just n  -> (M.insert (referenceId ref) (n+1) ids_seen,+                     ref{ referenceId =+                          ItemId . (<> T.singleton (chr n)) . unItemId $+                           referenceId ref }+                    : refs)++risTypes :: M.Map Text Text+risTypes = M.fromList+    [ ("ABST", "article")+    , ("ADVS", "motion-picture")+    , ("AGGR", "dataset")+    , ("ANCIENT", "book")+    , ("ART", "graphic")+    , ("BILL", "bill")+    , ("BLOG", "post-weblog")+    , ("BOOK", "book")+    , ("CASE", "legal_case")+    , ("CHAP", "chapter")+    , ("CHART", "graphic")+    , ("CLSWK", "book")+    , ("COMP", "program")+    , ("CONF", "paper-conference")+    , ("CPAPER", "paper-conference")+    , ("CTLG", "catalog")+    , ("DATA", "dataset")+    , ("DBASE", "dataset")+    , ("DICT", "book")+    , ("EBOOK", "book")+    , ("ECHAP", "chapter")+    , ("EDBOOK", "book")+    , ("EJOUR", "article")+    , ("WEB", "webpage")+    , ("ENCYC", "entry-encyclopedia")+    , ("EQUA", "figure")+    , ("FIGURE", "figure")+    , ("GEN", "entry")+    , ("GOVDOC", "report")+    , ("GRANT", "report")+    , ("HEAR", "report")+    , ("ICOMM", "personal_communication")+    , ("INPR", "article-journal")+    , ("JFULL", "article-journal")+    , ("JOUR", "article-journal")+    , ("LEGAL", "legal_case")+    , ("MANSCPT", "manuscript")+    , ("MAP", "map")+    , ("MGZN", "article-magazine")+    , ("MPCT", "motion-picture")+    , ("MULTI", "webpage")+    , ("MUSIC", "musical_score")+    , ("NEWS", "article-newspaper")+    , ("PAMP", "pamphlet")+    , ("PAT", "patent")+    , ("PCOMM", "personal_communication")+    , ("RPRT", "report")+    , ("SER", "article")+    , ("SLIDE", "graphic")+    , ("SOUND", "musical_score")+    , ("STAND", "report")+    , ("STAT", "legislation")+    , ("THES", "thesis")+    , ("UNBILL", "bill")+    , ("UNPB", "unpublished")+    , ("VIDEO", "graphic") ]
src/Text/Pandoc/Readers/RST.hs view
@@ -720,8 +720,12 @@         "aafig" -> do           let attribs = (name, ["aafig"], map (second trimr) fields)           return $ B.codeBlockWith attribs $ stripTrailingNewlines body-        "math" -> return $ B.para $ mconcat $ map B.displayMath-                         $ toChunks $ top <> "\n\n" <> body+        "math" -> return $ B.para+                  $ (case mkAttr name classes fields of+                       attr | attr == nullAttr -> id+                            | otherwise        -> B.spanWith attr)+                  $ mconcat $ map B.displayMath+                  $ toChunks $ top <> "\n\n" <> body         "figure" -> do            (caption, legend) <- parseFromString' extractCaption body'            let src = escapeURI $ trim top@@ -1027,6 +1031,14 @@                      Just v | not (T.null v) -> [("startFrom", v)]                      _ -> [] +-- | Creates element attributes from a name, list of classes, and fields.+-- Removes fields named @name@, @id@, or @class@.+mkAttr :: Text -> [Text] -> [(Text, Text)] -> Attr+mkAttr ident classes fields = (ident, classes, fields')+  where fields' = [(k, v') | (k, v) <- fields+                           , let v' = trimr v+                           , k /= "name", k /= "id", k /= "class"]+ --- --- note block ---@@ -1304,7 +1316,7 @@   let indices  = scanl (+) 0 lines'   let aligns   = replicate (length lines') AlignDefault   let rawHeads = if headless-                    then replicate (length dashes) ""+                    then []                     else simpleTableSplitLine indices rawContent   heads <- mapM ( parseFromString' (mconcat <$> many plain) . trim) rawHeads   return (heads, aligns, indices)
src/Text/Pandoc/Readers/RTF.hs view
@@ -1,4 +1,5 @@ {-# LANGUAGE FlexibleContexts  #-}+{-# LANGUAGE BangPatterns      #-} {-# LANGUAGE OverloadedStrings #-} {- |    Module      : Text.Pandoc.Readers.RTF@@ -30,7 +31,7 @@ import Text.Pandoc.Definition import Text.Pandoc.Options import Text.Pandoc.Parsing-import Text.Pandoc.Shared (safeRead, tshow)+import Text.Pandoc.Shared (tshow) import Data.Char (isAlphaNum, chr, isAscii, isLetter, isSpace, ord) import qualified Data.ByteString.Lazy as BL import Data.Digest.Pure.SHA (sha1, showDigest)@@ -202,71 +203,83 @@ parseRTF :: PandocMonad m => RTFParser m Pandoc parseRTF = do   skipMany nl-  toks <- many tok-  -- return $! traceShowId toks-  bs <- (case toks of-          -- if we start with {\rtf1...}, parse that and ignore-          -- what follows (which in certain cases can be non-RTF content)-          rtftok@(Tok _ (Grouped (Tok _ (ControlWord "rtf" (Just 1)) : _))) : _-            -> foldM processTok mempty [rtftok]-          _ -> foldM processTok mempty toks)-        >>= emitBlocks+  bs <- many tok >>= foldM processTok mempty >>= emitBlocks   unclosed <- closeContainers   let doc = B.doc $ bs <> unclosed   kvs <- sMetadata <$> getState   pure $ foldr (uncurry B.setMeta) doc kvs -data Tok = Tok SourcePos TokContents+data Tok = Tok !SourcePos !TokContents   deriving (Show, Eq)  data TokContents =-    ControlWord Text (Maybe Int)-  | ControlSymbol Char-  | UnformattedText Text-  | BinData BL.ByteString-  | HexVal Word8+    ControlWord !Text !(Maybe Int)+  | ControlSymbol !Char+  | UnformattedText !Text+  | BinData !BL.ByteString+  | HexVal !Word8   | Grouped [Tok]   deriving (Show, Eq)  tok :: PandocMonad m => RTFParser m Tok tok = do   pos <- getPosition-  Tok pos <$> ((controlThing <|> unformattedText <|> grouped) <* skipMany nl)+  Tok pos <$!> ((controlThing <|> unformattedText <|> grouped) <* skipMany nl)  where   controlThing = do     char '\\' *>-      ( binData-     <|> (ControlWord <$> letterSequence <*> (parameter <* optional delimChar))+      ( controlWord      <|> (HexVal <$> hexVal)      <|> (ControlSymbol <$> anyChar) )-  binData = try $ do-    string "bin" <* notFollowedBy letter-    n <- fromMaybe 0 <$> parameter-    spaces-    -- NOTE: We assume here that if the document contains binary-    -- data, it will not be valid UTF-8 and hence it will have been-    -- read as latin1, so we can recover the data in the following-    -- way.  This is probably not completely reliable, but I don't-    -- know if we can do better without making this reader take-    -- a ByteString input.-    dat <- BL.pack . map (fromIntegral . ord) <$> count n anyChar-    return $ BinData dat+  controlWord = do+    name <- letterSequence+    param <- parameter <* optional delimChar+    case name of+      "bin" -> do+        let n = fromMaybe 0 param+        spaces+        -- NOTE: We assume here that if the document contains binary+        -- data, it will not be valid UTF-8 and hence it will have been+        -- read as latin1, so we can recover the data in the following+        -- way.  This is probably not completely reliable, but I don't+        -- know if we can do better without making this reader take+        -- a ByteString input.+        dat <- BL.pack . map (fromIntegral . ord) <$> count n anyChar+        return $! BinData dat+      _ -> return $! ControlWord name param   parameter = do-    hyph <- string "-" <|> pure ""+    hyph <- option False $ True <$ char '-'     rest <- many digit-    let pstr = T.pack $ hyph <> rest-    return $ safeRead pstr+    if null rest+       then return $! Nothing+       else do+         let pstr = T.pack rest+         case TR.decimal pstr of+           Right (!i,_) ->+                return $! Just $! if hyph+                                     then (-1) * i+                                     else i+           _ -> return $! Nothing   hexVal = do     char '\''     x <- hexDigit     y <- hexDigit     return $ hexToWord (T.pack [x,y])   letterSequence = T.pack <$> many1 (satisfy (\c -> isAscii c && isLetter c))-  unformattedText =-    UnformattedText . T.pack . mconcat <$>-      many1 (   many1 (satisfy (not . isSpecial))-            <|> ("" <$ nl))-  grouped = Grouped <$> (char '{' *> skipMany nl *> manyTill tok (char '}'))+  unformattedText = do+    ts <-  filter (\c -> c /= '\r' && c /= '\n') <$>+           ( many1 (satisfy (\c -> not (isSpecial c) || c == '\r' || c == '\n')))+    return $! UnformattedText $! T.pack ts+  grouped = do+    char '{'+    skipMany nl+    ts <- manyTill tok (char '}')+    case ts of+       Tok _ (ControlWord "rtf" (Just 1)) : _ -> do+         setInput mempty -- discard remaining input: content after the \rtf1+                         -- group can be non-RTF+       _ -> return ()+    return $! Grouped ts  nl :: PandocMonad m => RTFParser m () nl = void (char '\n' <|> char '\r')@@ -332,7 +345,7 @@ addText :: PandocMonad m => Text -> RTFParser m () addText t = do   gs <- sGroupStack <$> getState-  let props = case gs of+  let !props = case gs of                 (x:_) -> x                 _ -> def   updateState (\s -> s{ sTextContent = (props, t) : sTextContent s })
src/Text/Pandoc/Readers/TWiki.hs view
@@ -221,9 +221,9 @@  table :: PandocMonad m => TWParser m B.Blocks table = try $ do-  tableHead <- optionMaybe (unzip <$> many1Till tableParseHeader newline)+  thead <- optionMaybe (unzip <$> many1Till tableParseHeader newline)   rows <- many1 tableParseRow-  return $ buildTable mempty rows $ fromMaybe (align rows, columns rows) tableHead+  return $ buildTable mempty rows $ fromMaybe (align rows, columns rows) thead   where     buildTable caption rows (aligns, heads)                     = B.table (B.simpleCaption $ B.plain caption)
src/Text/Pandoc/SelfContained.hs view
@@ -29,10 +29,12 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad (..), fetchItem,                                       getInputFiles, report, setInputFiles) import Text.Pandoc.Logging+import Text.Pandoc.Error (PandocError(..)) import Text.Pandoc.MIME (MimeType)-import Text.Pandoc.Shared (isURI, renderTags', trim)+import Text.Pandoc.Shared (isURI, renderTags', trim, tshow) import Text.Pandoc.UTF8 (toString, toText, fromText) import Text.Parsec (ParsecT, runParserT)+import Control.Monad.Except (throwError, catchError) import qualified Text.Parsec as P  isOk :: Char -> Bool@@ -60,7 +62,7 @@ convertTags [] = return [] convertTags (t@TagOpen{}:ts)   | fromAttrib "data-external" t == "1" = (t:) <$> convertTags ts-convertTags (t@(TagOpen "script" as):TagClose "script":ts) =+convertTags (t@(TagOpen "script" as):tc@(TagClose "script"):ts) =   case fromAttrib "src" t of        ""  -> (t:) <$> convertTags ts        src -> do@@ -68,10 +70,10 @@            res <- getData typeAttr src            rest <- convertTags ts            case res of-                Left dataUri -> return $ TagOpen "script"+                AlreadyDataURI dataUri -> return $ TagOpen "script"                      (("src",dataUri) : [(x,y) | (x,y) <- as, x /= "src"]) :                      TagClose "script" : rest-                Right (mime, bs)+                Fetched (mime, bs)                   | ("text/javascript" `T.isPrefixOf` mime ||                      "application/javascript" `T.isPrefixOf` mime ||                      "application/x-javascript" `T.isPrefixOf` mime) &&@@ -86,18 +88,19 @@                          (("src",makeDataURI (mime, bs)) :                           [(x,y) | (x,y) <- as, x /= "src"]) :                         TagClose "script" : rest+                CouldNotFetch _ -> return $ t:tc:rest convertTags (t@(TagOpen "link" as):ts) =   case fromAttrib "href" t of        ""  -> (t:) <$> convertTags ts        src -> do            res <- getData (fromAttrib "type" t) src            case res of-                Left dataUri -> do+                AlreadyDataURI dataUri -> do                   rest <- convertTags ts                   return $ TagOpen "link"                      (("href",dataUri) : [(x,y) | (x,y) <- as, x /= "href"]) :                      rest-                Right (mime, bs)+                Fetched (mime, bs)                   | "text/css" `T.isPrefixOf` mime                     && T.null (fromAttrib "media" t)                     && not ("</" `B.isInfixOf` bs) -> do@@ -113,6 +116,9 @@                       return $ TagOpen "link"                        (("href",makeDataURI (mime, bs)) :                          [(x,y) | (x,y) <- as, x /= "href"]) : rest+                CouldNotFetch _ -> do+                      rest <- convertTags ts+                      return $ t:rest convertTags (t@(TagOpen tagname as):ts)   | any (isSourceAttribute tagname) as      = do@@ -122,9 +128,13 @@   where processAttribute (x,y) =            if isSourceAttribute tagname (x,y)               then do-                enc <- getDataURI (fromAttrib "type" t) y-                return (x, enc)+                res <- getData (fromAttrib "type" t) y+                case res of+                  AlreadyDataURI enc -> return (x, enc)+                  Fetched (mt,bs) -> return (x, makeDataURI (mt,bs))+                  CouldNotFetch _ -> return (x, y)               else return (x,y)+ convertTags (t:ts) = (t:) <$> convertTags ts  cssURLs :: PandocMonad m@@ -213,8 +223,8 @@       u ->  do let url' = if isURI (T.pack u) then T.pack u else T.pack (d </> u)                res <- lift $ getData "" url'                case res of-                    Left uri -> return $ Left (fromText $ "url(" <> uri <> ")")-                    Right (mt', raw) -> do+                    AlreadyDataURI uri -> return $ Left (fromText $ "url(" <> uri <> ")")+                    Fetched (mt', raw) -> do                       -- note that the downloaded CSS may                       -- itself contain url(...).                       (mt, b) <- if "text/css" `T.isPrefixOf` mt'@@ -223,20 +233,22 @@                                     then ("text/css",) <$> cssURLs d raw                                     else return (mt', raw)                       return $ Right (mt, b)+                    CouldNotFetch _ -> return $ Left fallback -getDataURI :: PandocMonad m => MimeType -> T.Text -> m T.Text-getDataURI mimetype src = do-  res <- getData mimetype src-  case res of-       Left uri -> return uri-       Right x  -> return $ makeDataURI x+data GetDataResult =+    AlreadyDataURI T.Text+  | CouldNotFetch PandocError+  | Fetched (MimeType, ByteString)+  deriving (Show)  getData :: PandocMonad m         => MimeType -> T.Text-        -> m (Either T.Text (MimeType, ByteString))+        -> m GetDataResult getData mimetype src-  | "data:" `T.isPrefixOf` src = return $ Left src -- already data: uri-  | otherwise = do+  | "data:" `T.isPrefixOf` src = return $ AlreadyDataURI src -- already data: uri+  | otherwise = catchError fetcher handler+ where+   fetcher = do       let ext = T.toLower $ T.pack $ takeExtension $ T.unpack src       (raw, respMime) <- fetchItem src       let raw' = if ext `elem` [".gz", ".svgz"]@@ -254,7 +266,16 @@                   setInputFiles oldInputs                   return res                else return raw'-      return $ Right (mime, result)+      return $ Fetched (mime, result)+   handler e = case e of+                 PandocResourceNotFound r -> do+                   report $ CouldNotFetchResource r ""+                   return $ CouldNotFetch e+                 PandocHttpError u er -> do+                   report $ CouldNotFetchResource u (tshow er)+                   return $ CouldNotFetch e+                 _ -> throwError e+
src/Text/Pandoc/Shared.hs view
@@ -105,6 +105,7 @@ import Data.Char (isAlpha, isLower, isSpace, isUpper, toLower, isAlphaNum,                   generalCategory, GeneralCategory(NonSpacingMark,                   SpacingCombiningMark, EnclosingMark, ConnectorPunctuation))+import Data.Containers.ListUtils (nubOrd) import Data.List (find, intercalate, intersperse, sortOn, foldl') import qualified Data.Map as M import Data.Maybe (mapMaybe, fromMaybe)@@ -152,6 +153,9 @@   | otherwise = let (first, rest) = T.break isSep t                 in  first : splitTextBy isSep (T.dropWhile isSep rest) +-- | Split text at the given widths. Note that the break points are+-- /not/ indices but text widths, which will be different for East Asian+-- characters, emojis, etc. splitTextByIndices :: [Int] -> T.Text -> [T.Text] splitTextByIndices ns = splitTextByRelIndices (zipWith (-) ns (0:ns)) . T.unpack  where@@ -160,22 +164,27 @@     let (first, rest) = splitAt' x cs      in T.pack first : splitTextByRelIndices xs rest --- Note: don't replace this with T.splitAt, which is not sensitive+-- | Returns a pair whose first element is a prefix of @t@ and that has+-- width @n@, and whose second is the remainder of the string.+--+-- Note: Do *not* replace this with 'T.splitAt', which is not sensitive -- to character widths!-splitAt' :: Int -> [Char] -> ([Char],[Char])+splitAt' :: Int {-^ n -} -> [Char] {-^ t -} -> ([Char],[Char]) splitAt' _ []          = ([],[]) splitAt' n xs | n <= 0 = ([],xs) splitAt' n (x:xs)      = (x:ys,zs)   where (ys,zs) = splitAt' (n - charWidth x) xs +-- | Remove duplicates from a list. ordNub :: (Ord a) => [a] -> [a]-ordNub l = go Set.empty l-  where-    go _ [] = []-    go s (x:xs) = if x `Set.member` s then go s xs-                                      else x : go (Set.insert x s) xs+ordNub = nubOrd+{-# INLINE ordNub #-} -findM :: forall m t a. (Monad m, Foldable t) => (a -> m Bool) -> t a -> m (Maybe a)+-- | Returns the first element in a foldable structure for that the+-- monadic predicate holds true, and @Nothing@ if no such element+-- exists.+findM :: forall m t a. (Monad m, Foldable t)+      => (a -> m Bool) -> t a -> m (Maybe a) findM p = foldr go (pure Nothing)   where     go :: a -> m (Maybe a) -> m (Maybe a)@@ -191,6 +200,7 @@ inquotes :: T.Text -> T.Text inquotes txt = T.cons '\"' (T.snoc txt '\"') +-- | Like @'show'@, but returns a 'T.Text' instead of a 'String'. tshow :: Show a => a -> T.Text tshow = T.pack . show @@ -206,6 +216,8 @@ stripTrailingNewlines :: T.Text -> T.Text stripTrailingNewlines = T.dropWhileEnd (== '\n') +-- | Returns 'True' for an ASCII whitespace character, viz. space,+-- carriage return, newline, and horizontal tab. isWS :: Char -> Bool isWS ' '  = True isWS '\r' = True@@ -317,6 +329,7 @@ normalizeDate :: T.Text -> Maybe T.Text normalizeDate = fmap T.pack . normalizeDate' . T.unpack +-- | Like @'normalizeDate'@, but acts on 'String' instead of 'T.Text'. normalizeDate' :: String -> Maybe String normalizeDate' s = fmap (formatTime defaultTimeLocale "%F")   (msum $ map (\fs -> parsetimeWith fs s >>= rejectBadYear) formats :: Maybe Day)@@ -382,10 +395,12 @@         go LineBreak  = [Space]         go _          = [] +-- | Replaces 'Note' elements with empty strings. deNote :: Inline -> Inline deNote (Note _) = Str "" deNote x        = x +-- | Turns links into spans, keeping just the link text. deLink :: Inline -> Inline deLink (Link _ ils _) = Span nullAttr ils deLink x              = x@@ -412,6 +427,8 @@         fixInlines (q@Quoted{}) = deQuote q         fixInlines x = x +-- | Unwrap 'Quoted' inline elements, enclosing the contents with+-- English-style Unicode quotes instead. deQuote :: Inline -> Inline deQuote (Quoted SingleQuote xs) =   Span ("",[],[]) (Str "\8216" : xs ++ [Str "\8217"])@@ -813,7 +830,7 @@     isSingleton _   = Nothing     checkPathSeperator = fmap isPathSeparator . isSingleton --- Convert the path part of a file: URI to a regular path.+-- | Converts the path part of a file: URI to a regular path. -- On windows, @/c:/foo@ should be @c:/foo@. -- On linux, @/foo@ should be @/foo@. uriPathToPath :: T.Text -> FilePath
src/Text/Pandoc/Sources.hs view
@@ -1,3 +1,4 @@+{-# LANGUAGE BangPatterns #-} {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE MultiParamTypeClasses #-}@@ -144,7 +145,7 @@          => (Char -> Bool) -> ParsecT s u m Char satisfy f = P.tokenPrim show updateSourcePos matcher  where-  matcher c = if f c then Just c else Nothing+  matcher !c = if f c then Just c else Nothing  oneOf :: (Monad m, Stream s m Char, UpdateSourcePos s Char)       => [Char] -> ParsecT s u m Char
src/Text/Pandoc/Writers/Custom.hs view
@@ -1,258 +1,55 @@-{-# LANGUAGE FlexibleContexts    #-}-{-# LANGUAGE FlexibleInstances   #-}+{-# LANGUAGE LambdaCase          #-} {-# LANGUAGE OverloadedStrings   #-}-{-# LANGUAGE ScopedTypeVariables #-}-{-# LANGUAGE TypeApplications    #-} {- |    Module      : Text.Pandoc.Writers.Custom-   Copyright   : Copyright (C) 2012-2022 John MacFarlane+   Copyright   : 2012-2022 John MacFarlane,    License     : GNU GPL, version 2 or above-    Maintainer  : John MacFarlane <jgm@berkeley.edu>-   Stability   : alpha-   Portability : portable  Conversion of 'Pandoc' documents to custom markup using a Lua writer. -} module Text.Pandoc.Writers.Custom ( writeCustom ) where-import Control.Applicative (optional)-import Control.Arrow ((***)) import Control.Exception-import Control.Monad (when)-import Data.List (intersperse)-import Data.Maybe (fromMaybe)-import qualified Data.Text as T-import Data.Text (Text, pack)-import HsLua as Lua hiding (Operation (Div))-import HsLua.Aeson (peekViaJSON)-import Text.DocLayout (render, literal)-import Text.DocTemplates (Context)+import Control.Monad ((<=<))+import Data.Text (Text)+import HsLua import Control.Monad.IO.Class (MonadIO)-import Text.Pandoc.Definition-import Text.Pandoc.Lua (Global (..), runLua, setGlobals)-import Text.Pandoc.Lua.Marshal.Attr (pushAttributeList)-import Text.Pandoc.Options import Text.Pandoc.Class (PandocMonad)-import Text.Pandoc.Templates (renderTemplate)-import Text.Pandoc.Writers.Shared---- | List of key-value pairs that is pushed to Lua as AttributeList--- userdata.-newtype AttributeList = AttributeList [(Text, Text)]-instance Pushable AttributeList where-  push (AttributeList kvs) = pushAttributeList kvs--attrToMap :: Attr -> AttributeList-attrToMap (id',classes,keyvals) = AttributeList-    $ ("id", id')-    : ("class", T.unwords classes)-    : keyvals--newtype Stringify a = Stringify a--instance Pushable (Stringify Format) where-  push (Stringify (Format f)) = Lua.push (T.toLower f)--instance Pushable (Stringify [Inline]) where-  push (Stringify ils) = Lua.push =<< inlineListToCustom ils--instance Pushable (Stringify [Block]) where-  push (Stringify blks) = Lua.push =<< blockListToCustom blks--instance Pushable (Stringify MetaValue) where-  push (Stringify (MetaMap m))       = Lua.push (fmap Stringify m)-  push (Stringify (MetaList xs))     = Lua.push (map Stringify xs)-  push (Stringify (MetaBool x))      = Lua.push x-  push (Stringify (MetaString s))    = Lua.push s-  push (Stringify (MetaInlines ils)) = Lua.push (Stringify ils)-  push (Stringify (MetaBlocks bs))   = Lua.push (Stringify bs)--instance Pushable (Stringify Citation) where-  push (Stringify cit) = flip pushAsTable cit-    [ ("citationId", push . citationId)-    , ("citationPrefix",  push . Stringify . citationPrefix)-    , ("citationSuffix",  push . Stringify . citationSuffix)-    , ("citationMode",    push . citationMode)-    , ("citationNoteNum", push . citationNoteNum)-    , ("citationHash",    push . citationHash)-    ]---- | Key-value pair, pushed as a table with @a@ as the only key and @v@ as the--- associated value.-newtype KeyValue a b = KeyValue (a, b)+import Text.Pandoc.Definition (Pandoc (..))+import Text.Pandoc.Lua (Global (..), runLua, setGlobals)+import Text.Pandoc.Options (WriterOptions) -instance (Pushable a, Pushable b) => Pushable (KeyValue a b) where-  push (KeyValue (k, v)) = do-    Lua.newtable-    Lua.push k-    Lua.push v-    Lua.rawset (Lua.nth 3)+import qualified Text.Pandoc.Lua.Writer.Classic as Classic  -- | Convert Pandoc to custom markup. writeCustom :: (PandocMonad m, MonadIO m)             => FilePath -> WriterOptions -> Pandoc -> m Text-writeCustom luaFile opts doc@(Pandoc meta _) = do-  let globals = [ PANDOC_DOCUMENT doc-                , PANDOC_SCRIPT_FILE luaFile-                , PANDOC_WRITER_OPTIONS opts-                ]-  res <- runLua $ do-    setGlobals globals-    stat <- dofileTrace luaFile-    -- check for error in lua script (later we'll change the return type-    -- to handle this more gracefully):-    when (stat /= Lua.OK)-      Lua.throwErrorAsException-    (rendered, context) <- docToCustom opts doc-    metaContext <- metaToContext opts-                   (fmap (literal . pack) . blockListToCustom)-                   (fmap (literal . pack) . inlineListToCustom)-                   meta-    return (pack rendered, context <> metaContext)-  case res of-    Left msg -> throw msg-    Right (body, context) -> return $-      case writerTemplate opts of-        Nothing  -> body-        Just tpl -> render Nothing $-                    renderTemplate tpl $ setField "body" body context--docToCustom :: forall e. LuaError e-            => WriterOptions -> Pandoc -> LuaE e (String, Context Text)-docToCustom opts (Pandoc (Meta metamap) blocks) = do-  body <- blockListToCustom blocks-  -- invoke doesn't work with multiple return values, so we have to call-  -- `Doc` manually.-  Lua.getglobal "Doc"                 -- function-  push body                           -- argument 1-  push (fmap Stringify  metamap)      -- argument 2-  push (writerVariables opts)         -- argument 3-  call 3 2-  rendered  <- peek (nth 2)           -- first return value-  context <- forcePeek . optional $ peekViaJSON top  -- snd return value-  return (rendered, fromMaybe mempty context)---- | Convert Pandoc block element to Custom.-blockToCustom :: forall e. LuaError e-              => Block         -- ^ Block element-              -> LuaE e String--blockToCustom Null = return ""--blockToCustom (Plain inlines) = invoke "Plain" (Stringify inlines)--blockToCustom (Para [Image attr txt (src,tit)]) =-  invoke "CaptionedImage" src tit (Stringify txt) (attrToMap attr)--blockToCustom (Para inlines) = invoke "Para" (Stringify inlines)--blockToCustom (LineBlock linesList) =-  invoke "LineBlock" (map (Stringify) linesList)--blockToCustom (RawBlock format str) =-  invoke "RawBlock" (Stringify format) str--blockToCustom HorizontalRule = invoke "HorizontalRule"--blockToCustom (Header level attr inlines) =-  invoke "Header" level (Stringify inlines) (attrToMap attr)--blockToCustom (CodeBlock attr str) =-  invoke "CodeBlock" str (attrToMap attr)--blockToCustom (BlockQuote blocks) =-  invoke "BlockQuote" (Stringify blocks)--blockToCustom (Table _ blkCapt specs thead tbody tfoot) =-  let (capt, aligns, widths, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot-      aligns' = map show aligns-      capt' = Stringify capt-      headers' = map (Stringify) headers-      rows' = map (map (Stringify)) rows-  in invoke "Table" capt' aligns' widths headers' rows'--blockToCustom (BulletList items) =-  invoke "BulletList" (map (Stringify) items)--blockToCustom (OrderedList (num,sty,delim) items) =-  invoke "OrderedList" (map (Stringify) items) num (show sty) (show delim)--blockToCustom (DefinitionList items) =-  invoke "DefinitionList"-               (map (KeyValue . (Stringify *** map (Stringify))) items)--blockToCustom (Div attr items) =-  invoke "Div" (Stringify items) (attrToMap attr)---- | Convert list of Pandoc block elements to Custom.-blockListToCustom :: forall e. LuaError e-                  => [Block]       -- ^ List of block elements-                  -> LuaE e String-blockListToCustom xs = do-  blocksep <- invoke "Blocksep"-  bs <- mapM blockToCustom xs-  return $ mconcat $ intersperse blocksep bs---- | Convert list of Pandoc inline elements to Custom.-inlineListToCustom :: forall e. LuaError e => [Inline] -> LuaE e String-inlineListToCustom lst = do-  xs <- mapM (inlineToCustom @e) lst-  return $ mconcat xs---- | Convert Pandoc inline element to Custom.-inlineToCustom :: forall e. LuaError e => Inline -> LuaE e String--inlineToCustom (Str str) = invoke "Str" str--inlineToCustom Space = invoke "Space"--inlineToCustom SoftBreak = invoke "SoftBreak"--inlineToCustom (Emph lst) = invoke "Emph" (Stringify lst)--inlineToCustom (Underline lst) = invoke "Underline" (Stringify lst)--inlineToCustom (Strong lst) = invoke "Strong" (Stringify lst)--inlineToCustom (Strikeout lst) = invoke "Strikeout" (Stringify lst)--inlineToCustom (Superscript lst) = invoke "Superscript" (Stringify lst)--inlineToCustom (Subscript lst) = invoke "Subscript" (Stringify lst)--inlineToCustom (SmallCaps lst) = invoke "SmallCaps" (Stringify lst)--inlineToCustom (Quoted SingleQuote lst) =-  invoke "SingleQuoted" (Stringify lst)--inlineToCustom (Quoted DoubleQuote lst) =-  invoke "DoubleQuoted" (Stringify lst)--inlineToCustom (Cite cs lst) =-  invoke "Cite" (Stringify lst) (map (Stringify) cs)--inlineToCustom (Code attr str) =-  invoke "Code" str (attrToMap attr)--inlineToCustom (Math DisplayMath str) =-  invoke "DisplayMath" str--inlineToCustom (Math InlineMath str) =-  invoke "InlineMath" str--inlineToCustom (RawInline format str) =-  invoke "RawInline" (Stringify format) str--inlineToCustom LineBreak = invoke "LineBreak"--inlineToCustom (Link attr txt (src,tit)) =-  invoke "Link" (Stringify txt) src tit (attrToMap attr)--inlineToCustom (Image attr alt (src,tit)) =-  invoke "Image" (Stringify alt) src tit (attrToMap attr)--inlineToCustom (Note contents) = invoke "Note" (Stringify contents)+writeCustom luaFile opts doc = either throw pure <=< runLua $ do+  setGlobals [ PANDOC_DOCUMENT doc+             , PANDOC_SCRIPT_FILE luaFile+             , PANDOC_WRITER_OPTIONS opts+             ]+  dofileTrace luaFile >>= \case+    OK -> pure ()+    _  -> throwErrorAsException+  -- Most classic writers contain code that throws an error if a global+  -- is not present. This would break our check for the existence of a+  -- "Writer" function. We resort to raw access for that reason, but+  -- could also catch the error instead.+  let rawgetglobal x = do+        pushglobaltable+        pushName x+        rawget (nth 2) <* remove (nth 2) -- remove global table -inlineToCustom (Span attr items) =-  invoke "Span" (Stringify items) (attrToMap attr)+  rawgetglobal "Writer" >>= \case+    TypeNil -> do+      pop 1  -- remove nil+      Classic.runCustom opts doc+    _       -> do+      -- Writer on top of the stack. Call it with document and writer+      -- options as arguments.+      push doc+      push opts+      callTrace 2 1+      forcePeek $ peekText top
src/Text/Pandoc/Writers/Docbook.hs view
@@ -94,7 +94,8 @@  -- | Convert Pandoc document to string in Docbook format. writeDocbook :: PandocMonad m => WriterOptions -> Pandoc -> DB m Text-writeDocbook opts (Pandoc meta blocks) = do+writeDocbook opts doc = do+  let Pandoc meta blocks = ensureValidXmlIdentifiers doc   let colwidth = if writerWrapText opts == WrapAuto                     then Just $ writerColumns opts                     else Nothing@@ -429,7 +430,9 @@   | otherwise = do       version <- ask       (if "#" `T.isPrefixOf` src-            then inTags False "link" $ ("linkend", writerIdentifierPrefix opts <> T.drop 1 src) : idAndRole attr+            then let tag = if null txt then "xref" else "link"+                 in  inTags False tag $+                     ("linkend", writerIdentifierPrefix opts <> T.drop 1 src) : idAndRole attr             else if version == DocBook5                     then inTags False "link" $ ("xlink:href", src) : idAndRole attr                     else inTags False "ulink" $ ("url", src) : idAndRole attr )
src/Text/Pandoc/Writers/Docx.hs view
@@ -27,7 +27,7 @@ import Data.List (intercalate, isPrefixOf, isSuffixOf) import Data.String (fromString) import qualified Data.Map as M-import Data.Maybe (fromMaybe, isNothing, mapMaybe, maybeToList)+import Data.Maybe (fromMaybe, isNothing, mapMaybe, maybeToList, isJust) import qualified Data.Set as Set import qualified Data.Text as T import Data.Text (Text)@@ -930,10 +930,12 @@     $ mknode "v:rect" [("style","width:0;height:1.5pt"),                        ("o:hralign","center"),                        ("o:hrstd","t"),("o:hr","t")] () ]-blockToOpenXML' opts (Table attr caption colspecs thead tbodies tfoot) =-  tableToOpenXML opts+blockToOpenXML' opts (Table attr caption colspecs thead tbodies tfoot) = do+  content <- tableToOpenXML opts                  (blocksToOpenXML opts)                  (Grid.toTable attr caption colspecs thead tbodies tfoot)+  let (tableId, _, _) = attr+  wrapBookmark tableId content blockToOpenXML' opts el   | BulletList lst <- el = addOpenXMLList BulletMarker lst   | OrderedList (start, numstyle, numdelim) lst <- el@@ -942,7 +944,10 @@     addOpenXMLList marker lst = do       addList marker       numid  <- getNumId-      l <- asList $ concat `fmap` mapM (listItemToOpenXML opts numid) lst+      exampleid <- case marker of+                        NumberMarker Example _ _ -> gets stExampleId+                        _ -> return Nothing+      l <- asList $ concat `fmap` mapM (listItemToOpenXML opts $ fromMaybe numid exampleid) lst       setFirstPara       return l blockToOpenXML' opts (DefinitionList items) = do@@ -963,7 +968,16 @@ addList :: (PandocMonad m) => ListMarker -> WS m () addList marker = do   lists <- gets stLists-  modify $ \st -> st{ stLists = lists ++ [marker] }+  lastExampleId <- gets stExampleId+  modify $ \st -> st{ stLists = lists ++ case marker of+                                         -- Use only first occurrence of Example for list declaration to avoid overhead+                                         NumberMarker Example _ _ | isJust lastExampleId -> []+                                         _ -> [marker]+                    , stExampleId = case marker of+                                         -- Reuse the same identifier for all other occurrences of Example+                                         NumberMarker Example _ _ -> lastExampleId <|> Just (baseListId + length lists)+                                         _ -> lastExampleId+                  }  listItemToOpenXML :: (PandocMonad m)                   => WriterOptions
src/Text/Pandoc/Writers/Docx/Table.hs view
@@ -13,19 +13,42 @@   ( tableToOpenXML   ) where -import Control.Monad.State.Strict-import Data.Array+import Control.Monad.State.Strict ( modify, gets, unless )+import Data.Array ( elems, (!), assocs, indices ) import Data.Text (Text) import Text.Pandoc.Definition+    ( ColSpec,+      Caption(Caption),+      Format(Format),+      Attr,+      Block(Para, Plain),+      Inline(Str, Span, RawInline),+      Alignment(..),+      RowSpan(..),+      ColSpan(..),+      ColWidth(ColWidth) ) import Text.Pandoc.Class.PandocMonad (PandocMonad, translateTerm) import Text.Pandoc.Writers.Docx.Types-import Text.Pandoc.Shared+    ( WS,+      WriterState(stNextTableNum, stInTable),+      setFirstPara,+      pStyleM,+      withParaProp,+      withParaPropM )+import Text.Pandoc.Shared ( tshow, stringify ) import Text.Pandoc.Options (WriterOptions, isEnabled) import Text.Pandoc.Extensions (Extension(Ext_native_numbering)) import Text.Printf (printf)-import Text.Pandoc.Writers.GridTable hiding (Table)-import Text.Pandoc.Writers.OOXML-import Text.Pandoc.XML.Light as XML hiding (Attr)+import Text.Pandoc.Writers.GridTable+    ( rowArray,+      ColIndex,+      GridCell(..),+      Part(Part, partCellArray, partRowAttrs),+      RowIndex )+import Text.Pandoc.Writers.OOXML ( mknode )+import Text.Pandoc.XML.Light.Proc ( onlyElems )+import Text.Pandoc.XML.Light.Types+    ( Content(Elem), Element(elName), QName(qName) ) import qualified Data.Text as T import qualified Text.Pandoc.Translations as Term import qualified Text.Pandoc.Writers.GridTable as Grid
src/Text/Pandoc/Writers/Docx/Types.hs view
@@ -107,6 +107,7 @@        , stExternalLinks  :: M.Map Text Text        , stImages         :: M.Map FilePath (String, String, Maybe MimeType, B.ByteString)        , stLists          :: [ListMarker]+       , stExampleId      :: Maybe Int        , stInsId          :: Int        , stDelId          :: Int        , stStyleMaps      :: StyleMaps@@ -131,6 +132,7 @@       , stExternalLinks  = M.empty       , stImages         = M.empty       , stLists          = [NoMarker]+      , stExampleId      = Nothing       , stInsId          = 1       , stDelId          = 1       , stStyleMaps      = StyleMaps M.empty M.empty
src/Text/Pandoc/Writers/EPUB.hs view
@@ -35,7 +35,8 @@ import System.FilePath (takeExtension, takeFileName, makeRelative) import Text.HTML.TagSoup (Tag (TagOpen), fromAttrib, parseTags) import Text.Pandoc.Builder (fromList, setMeta)-import Text.Pandoc.Class.PandocMonad (PandocMonad, report)+import Text.Pandoc.Writers.Shared (ensureValidXmlIdentifiers)+import Text.Pandoc.Class (PandocMonad, report) import qualified Text.Pandoc.Class.PandocPure as P import qualified Text.Pandoc.Class.PandocMonad as P import Data.Time@@ -253,7 +254,7 @@   | name == "coverage" = md { epubCoverage = Just $ strContent e }   | name == "rights" = md { epubRights = Just $ strContent e }   | name == "belongs-to-collection" = md { epubBelongsToCollection = Just $ strContent e }-  | name == "group-position" = md { epubGroupPosition = Just $ strContent e }  +  | name == "group-position" = md { epubGroupPosition = Just $ strContent e }   | otherwise = md   where getAttr n = lookupAttr (opfName n) attrs addMetadataFromXML e@(Element (QName "meta" _ _) attrs _ _) md =@@ -435,18 +436,22 @@              -> Pandoc              -> E m B.ByteString pandocToEPUB version opts doc = do+  let doc' = ensureValidXmlIdentifiers doc   -- handle pictures-  Pandoc meta blocks <- walkM (transformInline opts) doc >>=+  Pandoc meta blocks <- walkM (transformInline opts) doc' >>=                         walkM transformBlock   picEntries <- mapMaybe (snd . snd) <$> gets stMediaPaths    epubSubdir <- gets stEpubSubdir   let epub3 = version == EPUB3+   let writeHtml o = fmap (UTF8.fromTextLazy . TL.fromStrict) .                       writeHtmlStringForEPUB version o   metadata <- getEPUBMetadata opts meta -  let plainTitle = case docTitle' meta of+  -- retreive title of document+  let plainTitle :: Text+      plainTitle = case docTitle' meta of                         [] -> case epubTitle metadata of                                    []    -> "UNTITLED"                                    (x:_) -> titleText x@@ -461,14 +466,18 @@         (\bs n -> mkEntry ("styles/stylesheet" ++ show n ++ ".css") bs)         stylesheets [(1 :: Int)..] -  let vars = Context $+  -- writer variables+  let vars :: Context Text+      vars = Context $                M.delete "css" .                M.insert "epub3"                  (toVal' $ if epub3 then "true" else "false") .                M.insert "lang" (toVal' $ epubLanguage metadata)              $ unContext $ writerVariables opts -  let cssvars useprefix = Context $ M.insert "css"+  -- If True create paths relative to parent folder+  let cssvars :: Bool -> Context Text+      cssvars useprefix = Context $ M.insert "css"                            (ListVal $ map                              (\e -> toVal' $                                 (if useprefix then "../" else "") <>@@ -477,7 +486,9 @@                              stylesheetEntries)                              mempty -  let opts' = opts{ writerEmailObfuscation = NoObfuscation+  -- Add additional options for the writer+  let opts' :: WriterOptions+      opts' = opts{ writerEmailObfuscation = NoObfuscation                   , writerSectionDivs = True                   , writerVariables = vars                   , writerHTMLMathMethod =@@ -487,41 +498,7 @@                   , writerWrapText = WrapAuto }    -- cover page-  (cpgEntry, cpicEntry) <--                case epubCoverImage metadata of-                     Nothing   -> return ([],[])-                     Just img  -> do-                       let fp = takeFileName img-                       mediaPaths <- gets (map (fst . snd) . stMediaPaths)-                       coverImageName <-  -- see #4206-                            if ("media/" <> fp) `elem` mediaPaths-                               then getMediaNextNewName (takeExtension fp)-                               else return fp-                       imgContent <- lift $ P.readFileLazy img-                       (coverImageWidth, coverImageHeight) <--                             case imageSize opts' (B.toStrict imgContent) of-                               Right sz  -> return $ sizeInPixels sz-                               Left err' -> (0, 0) <$ report-                                 (CouldNotDetermineImageSize (T.pack img) err')-                       cpContent <- lift $ writeHtml-                            opts'{ writerVariables =-                                   Context (M.fromList [-                                    ("coverpage", toVal' "true"),-                                    ("pagetitle", toVal $-                                      escapeStringForXML plainTitle),-                                    ("cover-image",-                                       toVal' $ T.pack coverImageName),-                                    ("cover-image-width", toVal' $-                                       tshow coverImageWidth),-                                    ("cover-image-height", toVal' $-                                       tshow coverImageHeight)]) <>-                                     cssvars True <> vars }-                            (Pandoc meta [])-                       coverEntry <- mkEntry "text/cover.xhtml" cpContent-                       coverImageEntry <- mkEntry ("media/" ++ coverImageName)-                                             imgContent-                       return ( [ coverEntry ]-                              , [ coverImageEntry ] )+  (cpgEntry, cpicEntry) <- createCoverPage meta metadata opts' vars cssvars writeHtml plainTitle    -- title page   tpContent <- lift $ writeHtml opts'{@@ -535,45 +512,22 @@                                (Pandoc meta [])   tpEntry <- mkEntry "text/title_page.xhtml" tpContent +   -- handle fonts   let matchingGlob f = do         xs <- lift $ P.glob f         when (null xs) $           report $ CouldNotFetchResource (T.pack f) "glob did not match any font files"         return xs-  let mkFontEntry f = mkEntry ("fonts/" ++ takeFileName f) =<<++  let mkFontEntry :: PandocMonad m => FilePath -> StateT EPUBState m Entry+      mkFontEntry f = mkEntry ("fonts/" ++ takeFileName f) =<<                         lift (P.readFileLazy f)   fontFiles <- concat <$> mapM matchingGlob (writerEpubFonts opts')   fontEntries <- mapM mkFontEntry fontFiles -  -- set page progression direction attribution-  let progressionDirection = case epubPageDirection metadata of-                                  Just LTR | epub3 ->-                                    [("page-progression-direction", "ltr")]-                                  Just RTL | epub3 ->-                                    [("page-progression-direction", "rtl")]-                                  _  -> []-   -- body pages -  let chapterHeaderLevel = writerEpubChapterLevel opts--  let isChapterHeader (Div _ (Header n _ _:_)) = n <= chapterHeaderLevel-      isChapterHeader _ = False--  let secsToChapters :: [Block] -> [Chapter]-      secsToChapters [] = []-      secsToChapters (d@(Div attr (h@(Header lvl _ _) : bs)) : rest)-        | chapterHeaderLevel == lvl =-           Chapter [d] : secsToChapters rest-        | chapterHeaderLevel > lvl =-           Chapter [Div attr (h:xs)] :-           secsToChapters ys ++ secsToChapters rest-             where (xs, ys) = break isChapterHeader bs-      secsToChapters bs =-          (if null xs then id else (Chapter xs :)) $ secsToChapters ys-            where (xs, ys) = break isChapterHeader bs-   -- add level 1 header to beginning if none there   let secs = makeSections True Nothing               $ addIdentifiers opts@@ -584,98 +538,26 @@                   _                   -> Header 1 ("",["unnumbered"],[])                                              (docTitle' meta) : blocks -  let chapters' = secsToChapters secs+  -- create the chapters and their reftable from the original options and the sections+  let (chapters, reftable) = createChaptersAndReftable opts secs -  let extractLinkURL' :: Int -> Inline -> [(T.Text, T.Text)]-      extractLinkURL' num (Span (ident, _, _) _)-        | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]-      extractLinkURL' num (Link (ident, _, _) _ _)-        | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]-      extractLinkURL' num (Image (ident, _, _) _ _)-        | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]-      extractLinkURL' num (RawInline fmt raw)-        | isHtmlFormat fmt-        = foldr (\tag ->-                   case tag of-                     TagOpen{} ->-                       case fromAttrib "id" tag of-                         "" -> id-                         x  -> ((x, showChapter num <> "#" <> x):)-                     _ -> id)-            [] (parseTags raw)-      extractLinkURL' _ _ = []+  -- Create the chapter entries from the chapters.+  -- Also requires access to the extended writer options and context+  -- as well as the css Context and html writer+  chapterEntries <- createChapterEntries opts' vars cssvars writeHtml chapters -  let extractLinkURL :: Int -> Block -> [(T.Text, T.Text)]-      extractLinkURL num (Div (ident, _, _) _)-        | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]-      extractLinkURL num (Header _ (ident, _, _) _)-        | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]-      extractLinkURL num (Table (ident,_,_) _ _ _ _ _)-        | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]-      extractLinkURL num (RawBlock fmt raw)-        | isHtmlFormat fmt-        = foldr (\tag ->-                   case tag of-                     TagOpen{} ->-                       case fromAttrib "id" tag of-                         "" -> id-                         x  -> ((x, showChapter num <> "#" <> x):)-                     _ -> id)-            [] (parseTags raw)-      extractLinkURL num b = query (extractLinkURL' num) b -  let reftable = concat $ zipWith (\(Chapter bs) num ->-                                    query (extractLinkURL num) bs)-                          chapters' [1..] -  let fixInternalReferences :: Inline -> Inline-      fixInternalReferences (Link attr lab (src, tit))-        | Just ('#', xs) <- T.uncons src = case lookup xs reftable of-             Just ys -> Link attr lab (ys, tit)-             Nothing -> Link attr lab (src, tit)-      fixInternalReferences x = x--  -- internal reference IDs change when we chunk the file,-  -- so that '#my-header-1' might turn into 'chap004.xhtml#my-header'.-  -- this fixes that:-  let chapters = map (\(Chapter bs) ->-                         Chapter $ walk fixInternalReferences bs)-                 chapters'--  let chapToEntry num (Chapter bs) =-        mkEntry ("text/" ++ T.unpack (showChapter num)) =<<-        writeHtml opts'{ writerVariables =-                            Context (M.fromList-                                     [("body-type", toVal' bodyType),-                                      ("pagetitle", toVal' $-                                           showChapter num)])-                            <> cssvars True <> vars } pdoc-         where (pdoc, bodyType) =-                 case bs of-                     (Div (_,"section":_,kvs)-                       (Header _ _ xs : _) : _) ->-                       -- remove notes or we get doubled footnotes-                       (Pandoc (setMeta "title"-                           (walk removeNote $ fromList xs) nullMeta) bs,-                        case lookup "epub:type" kvs of-                             Nothing -> "bodymatter"-                             Just x-                               | x `elem` frontMatterTypes -> "frontmatter"-                               | x `elem` backMatterTypes  -> "backmatter"-                               | otherwise                 -> "bodymatter")-                     _                   -> (Pandoc nullMeta bs, "bodymatter")-               frontMatterTypes = ["prologue", "abstract", "acknowledgments",-                                   "copyright-page", "dedication",-                                   "credits", "keywords", "imprint",-                                   "contributors", "other-credits",-                                   "errata", "revision-history",-                                   "titlepage", "halftitlepage", "seriespage",-                                   "foreword", "preface", "frontispiece",-                                   "seriespage", "titlepage"]-               backMatterTypes = ["appendix", "colophon", "bibliography",-                                  "index"]+  -- contents.opf -  chapterEntries <- zipWithM chapToEntry [1..] chapters+  -- set page progression direction attribution+  let progressionDirection :: [(Text, Text)]+      progressionDirection = case epubPageDirection metadata of+                                  Just LTR | epub3 ->+                                    [("page-progression-direction", "ltr")]+                                  Just RTL | epub3 ->+                                    [("page-progression-direction", "rtl")]+                                  _  -> []    -- incredibly inefficient (TODO):   let containsMathML ent = epub3 &&@@ -686,7 +568,6 @@         B8.unpack (fromEntry ent)   let props ent = ["mathml" | containsMathML ent] ++ ["svg" | containsSVG ent] -  -- contents.opf   let chapterNode ent = unode "item" !                            ([("id", toId $ makeRelative epubSubdir                                          $ eRelativePath ent),@@ -717,12 +598,12 @@                             ("media-type", fromMaybe "" $                                   getMimeType $ eRelativePath ent)] $ () +  -- The tocTitle is either the normal title or a specially configured title.   let tocTitle = maybe plainTitle                    metaValueToString $ lookupMeta "toc-title" meta-  uuid <- case epubIdentifier metadata of-            (x:_) -> return $ identifierText x  -- use first identifier as UUID-            []    -> throwError $ PandocShouldNeverHappenError "epubIdentifier is null"  -- shouldn't happen   currentTime <- lift P.getTimestamp++  -- Construct the contentsData   let contentsData = UTF8.fromTextLazy $ TL.fromStrict $ ppTopElement $         unode "package" !           ([("version", case version of@@ -781,11 +662,13 @@                | isJust (epubCoverImage metadata)              ]           ]+  -- Content should be stored in content.opf   contentsEntry <- mkEntry "content.opf" contentsData    -- toc.ncx   let tocLevel = writerTOCDepth opts +  -- Helper function for both the toc and anv Entries   let navPointNode :: PandocMonad m                    => (Int -> [Inline] -> T.Text -> [Element] -> Element)                    -> Block -> StateT Int m [Element]@@ -810,7 +693,244 @@       navPointNode formatter (Div _ bs) =         concat <$> mapM (navPointNode formatter) bs       navPointNode _ _ = return []+  +  -- Create the tocEntry from the metadata together with the sections and title.+  tocEntry <- createTocEntry meta metadata plainTitle secs navPointNode +  -- Create the navEntry using the metadata, all of the various writer options,+  -- the CSS and HTML helpers, the document and toc title as well as the epub version and all of the sections+  navEntry <- createNavEntry meta metadata opts opts' vars cssvars writeHtml plainTitle tocTitle version secs navPointNode++  -- mimetype+  mimetypeEntry <- mkEntry "mimetype" $+                        UTF8.fromStringLazy "application/epub+zip"++  -- container.xml+  let containerData = B.fromStrict $ UTF8.fromText $ ppTopElement $+       unode "container" ! [("version","1.0")+              ,("xmlns","urn:oasis:names:tc:opendocument:xmlns:container")] $+         unode "rootfiles" $+           unode "rootfile" ! [("full-path",+                    (if null epubSubdir+                        then ""+                        else T.pack epubSubdir <> "/") <> "content.opf")+               ,("media-type","application/oebps-package+xml")] $ ()+  containerEntry <- mkEntry "META-INF/container.xml" containerData++  -- com.apple.ibooks.display-options.xml+  let apple = B.fromStrict $ UTF8.fromText $ ppTopElement $+        unode "display_options" $+          unode "platform" ! [("name","*")] $+            unode "option" ! [("name","specified-fonts")] $ ("true" :: Text)+  appleEntry <- mkEntry "META-INF/com.apple.ibooks.display-options.xml" apple++  -- construct archive+  let archive = foldr addEntryToArchive emptyArchive $+                 [mimetypeEntry, containerEntry, appleEntry,+                  contentsEntry, tocEntry, navEntry, tpEntry] +++                  stylesheetEntries ++ picEntries ++ cpicEntry +++                  cpgEntry ++ chapterEntries ++ fontEntries+  return $ fromArchive archive++-- | Function used during conversion from pandoc to EPUB to create the cover page.+-- The first Entry list is for the cover while the second one is for the cover image.+-- If no cover images are specified, empty lists will be returned.+createCoverPage :: PandocMonad m =>+                   Meta+                   -> EPUBMetadata+                   -> WriterOptions+                   -> Context Text+                   -> (Bool -> Context Text)+                   -> (WriterOptions -> Pandoc -> m B8.ByteString)+                   -> Text+                   -> StateT EPUBState m ([Entry], [Entry])+createCoverPage meta metadata opts' vars cssvars writeHtml plainTitle = +    case epubCoverImage metadata of+        Nothing   -> return ([],[])+        Just img  -> do+          let fp = takeFileName img+          -- retrieve cover image file+          mediaPaths <- gets (map (fst . snd) . stMediaPaths)+          coverImageName <-  -- see #4206+                if ("media/" <> fp) `elem` mediaPaths+                  then getMediaNextNewName (takeExtension fp)+                  else return fp+          -- image dimensions+          imgContent <- lift $ P.readFileLazy img+          (coverImageWidth, coverImageHeight) <-+                case imageSize opts' (B.toStrict imgContent) of+                  Right sz  -> return $ sizeInPixels sz+                  Left err' -> (0, 0) <$ report+                    (CouldNotDetermineImageSize (T.pack img) err')+          -- write the HTML. Use the cssvars, vars and additional writer options.+          cpContent <- lift $ writeHtml+                opts'{ writerVariables =+                      Context (M.fromList [+                        ("coverpage", toVal' "true"),+                        ("pagetitle", toVal $+                          escapeStringForXML plainTitle),+                        ("cover-image",+                          toVal' $ T.pack coverImageName),+                        ("cover-image-width", toVal' $+                          tshow coverImageWidth),+                        ("cover-image-height", toVal' $+                          tshow coverImageHeight)]) <>+                        cssvars True <> vars }+                (Pandoc meta [])++          coverEntry <- mkEntry "text/cover.xhtml" cpContent+          coverImageEntry <- mkEntry ("media/" ++ coverImageName)+                                imgContent++          return ( [ coverEntry ], [ coverImageEntry ] )++-- | Converts the given chapters to entries using the writeHtml function+-- and the various provided options+createChapterEntries :: PandocMonad m =>+                            WriterOptions+                            -> Context Text+                            -> (Bool -> Context Text)+                            -> (WriterOptions -> Pandoc -> StateT EPUBState m B8.ByteString)+                            -> [Chapter]+                            -> StateT EPUBState m [Entry]+createChapterEntries opts' vars cssvars writeHtml chapters = do+  -- Create an entry from the chapter with the provided number.+  -- chapToEntry :: Int -> Chapter -> StateT EPUBState m Entry+  let chapToEntry num (Chapter bs) =+        mkEntry ("text/" ++ T.unpack (showChapter num)) =<<+        -- Combine all provided options+        writeHtml opts'{ writerVariables =+                            Context (M.fromList+                                     [("body-type", toVal' bodyType),+                                      ("pagetitle", toVal' $+                                           showChapter num)])+                            <> cssvars True <> vars } pdoc+         where (pdoc, bodyType) =+                 case bs of+                     (Div (_,"section":_,kvs) (Header _ _ xs : _) : _) ->+                       -- remove notes or we get doubled footnotes+                       (Pandoc (setMeta "title"+                           (walk removeNote $ fromList xs) nullMeta) bs,+                        -- Check if the chapters belongs to the frontmatter, +                        -- backmatter of bodymatter defaulting to the body+                        case lookup "epub:type" kvs of+                             Nothing -> "bodymatter"+                             Just x+                               | x `elem` frontMatterTypes -> "frontmatter"+                               | x `elem` backMatterTypes  -> "backmatter"+                               | otherwise                 -> "bodymatter")+                     _                   -> (Pandoc nullMeta bs, "bodymatter")+               frontMatterTypes = ["prologue", "abstract", "acknowledgments",+                                   "copyright-page", "dedication",+                                   "credits", "keywords", "imprint",+                                   "contributors", "other-credits",+                                   "errata", "revision-history",+                                   "titlepage", "halftitlepage", "seriespage",+                                   "foreword", "preface", "frontispiece",+                                   "seriespage", "titlepage"]+               backMatterTypes = ["appendix", "colophon", "bibliography",+                                  "index"]++  zipWithM chapToEntry [1..] chapters++-- | Splits the blocks into chapters and creates a corresponding reftable+createChaptersAndReftable :: WriterOptions -> [Block] -> ([Chapter], [(Text, Text)])+createChaptersAndReftable opts secs = (chapters, reftable)+  where +    chapterHeaderLevel = writerEpubChapterLevel opts++    isChapterHeader :: Block -> Bool+    isChapterHeader (Div _ (Header n _ _:_)) = n <= chapterHeaderLevel+    isChapterHeader _ = False++    secsToChapters :: [Block] -> [Chapter]+    secsToChapters [] = []+    secsToChapters (d@(Div attr (h@(Header lvl _ _) : bs)) : rest)+      -- If the header is of the same level as chapters, create a chapter+      | chapterHeaderLevel == lvl =+          Chapter [d] : secsToChapters rest+      -- If the header is a level higher than chapters, +      -- create a chapter of everything until the next chapter header.+      | chapterHeaderLevel > lvl =+          Chapter [Div attr (h:xs)] :+          secsToChapters ys ++ secsToChapters rest+            where (xs, ys) = break isChapterHeader bs+    secsToChapters bs =+      -- If this is the last block, keep it as is, +      -- otherwise create a chapter for everything until the next chapter header.+        (if null xs then id else (Chapter xs :)) $ secsToChapters ys+          where (xs, ys) = break isChapterHeader bs+  +    -- Convert the sections to initial chapters+    chapters' = secsToChapters secs++    -- Extract references for the reftable from Inline elements+    extractLinkURL' :: Int -> Inline -> [(T.Text, T.Text)]+    extractLinkURL' num (Span (ident, _, _) _)+      | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]+    extractLinkURL' num (Link (ident, _, _) _ _)+      | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]+    extractLinkURL' num (Image (ident, _, _) _ _)+      | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]+    extractLinkURL' num (RawInline fmt raw)+      | isHtmlFormat fmt+      = foldr (\tag ->+                  case tag of+                    TagOpen{} ->+                      case fromAttrib "id" tag of+                        "" -> id+                        x  -> ((x, showChapter num <> "#" <> x):)+                    _ -> id)+          [] (parseTags raw)+    extractLinkURL' _ _ = []+  +    -- Extract references for the reftable from Block elements+    extractLinkURL :: Int -> Block -> [(T.Text, T.Text)]+    extractLinkURL num (Div (ident, _, _) _)+      | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]+    extractLinkURL num (Header _ (ident, _, _) _)+      | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]+    extractLinkURL num (Table (ident,_,_) _ _ _ _ _)+      | not (T.null ident) = [(ident, showChapter num <> "#" <> ident)]+    extractLinkURL num (RawBlock fmt raw)+      | isHtmlFormat fmt+      = foldr (\tag ->+                  case tag of+                    TagOpen{} ->+                      case fromAttrib "id" tag of+                        "" -> id+                        x  -> ((x, showChapter num <> "#" <> x):)+                    _ -> id)+          [] (parseTags raw)+    extractLinkURL num b = query (extractLinkURL' num) b++    -- Create a reference table for the chapters with appropriate numbering+    reftable = concat $ zipWith (\(Chapter bs) num ->+                                    query (extractLinkURL num) bs)+                          chapters' [1..]++    fixInternalReferences :: Inline -> Inline+    fixInternalReferences (Link attr lab (src, tit))+      | Just ('#', xs) <- T.uncons src = case lookup xs reftable of+            Just ys -> Link attr lab (ys, tit)+            Nothing -> Link attr lab (src, tit)+    fixInternalReferences x = x++    -- internal reference IDs change when we chunk the file,+    -- so that '#my-header-1' might turn into 'chap004.xhtml#my-header'.+    -- this fixes that:+    chapters = map (\(Chapter bs) ->+                        Chapter $ walk fixInternalReferences bs)+                chapters'++createTocEntry :: PandocMonad m =>+                          Meta+                          -> EPUBMetadata+                          -> Text+                          -> [Block]+                          -> ((Int -> [Inline] -> T.Text -> [Element] -> Element) -> Block -> StateT Int m [Element])+                          -> StateT EPUBState m Entry+createTocEntry meta metadata plainTitle secs navPointNode = do   let navMapFormatter :: Int -> [Inline] -> T.Text -> [Element] -> Element       navMapFormatter n tit src subs = unode "navPoint" !                [("id", "navPoint-" <> tshow n)] $@@ -825,6 +945,10 @@    navMap <- lift $ evalStateT              (concat <$> mapM (navPointNode navMapFormatter) secs) 1++  uuid <- case epubIdentifier metadata of+          (x:_) -> return $ identifierText x  -- use first identifier as UUID+          []    -> throwError $ PandocShouldNeverHappenError "epubIdentifier is null"  -- shouldn't happen   let tocData = B.fromStrict $ UTF8.fromText $ ppTopElement $         unode "ncx" ! [("version","2005-1")                        ,("xmlns","http://www.daisy.org/z3986/2005/ncx/")] $@@ -845,29 +969,45 @@           , unode "navMap" $               tpNode : navMap           ]-  tocEntry <- mkEntry "toc.ncx" tocData+  mkEntry "toc.ncx" tocData ++createNavEntry :: PandocMonad m =>+                          Meta+                          -> EPUBMetadata+                          -> WriterOptions+                          -> WriterOptions+                          -> Context Text+                          -> (Bool -> Context Text)+                          -> (WriterOptions -> Pandoc -> m B8.ByteString)+                          -> Text+                          -> Text+                          -> EPUBVersion+                          -> [Block]+                          -> ((Int -> [Inline] -> T.Text -> [Element] -> Element) -> Block -> StateT Int m [Element])+                          -> StateT EPUBState m Entry+createNavEntry meta metadata opts opts' vars cssvars writeHtml plainTitle tocTitle version secs navPointNode = do   let navXhtmlFormatter :: Int -> [Inline] -> T.Text -> [Element] -> Element       navXhtmlFormatter n tit src subs = unode "li" !-                                       [("id", "toc-li-" <> tshow n)] $-                                            (unode "a" !-                                                [("href", "text/" <> src)]-                                             $ titElements)-                                            : case subs of-                                                 []    -> []-                                                 (_:_) -> [unode "ol" ! [("class","toc")] $ subs]+                                    [("id", "toc-li-" <> tshow n)] $+                                        (unode "a" !+                                            [("href", "text/" <> src)]+                                          $ titElements)+                                        : case subs of+                                              []    -> []+                                              (_:_) -> [unode "ol" ! [("class","toc")] $ subs]           where titElements = either (const []) id $                                 parseXMLContents (TL.fromStrict titRendered)                 titRendered = case P.runPure-                               (writeHtmlStringForEPUB version-                                 opts{ writerTemplate = Nothing-                                     , writerVariables =-                                         Context (M.fromList-                                           [("pagetitle", toVal $-                                             escapeStringForXML plainTitle)])-                                       <> writerVariables opts}-                                 (Pandoc nullMeta-                                   [Plain $ walk clean tit])) of+                                (writeHtmlStringForEPUB version+                                  opts{ writerTemplate = Nothing+                                      , writerVariables =+                                          Context (M.fromList+                                            [("pagetitle", toVal $+                                              escapeStringForXML plainTitle)])+                                        <> writerVariables opts}+                                  (Pandoc nullMeta+                                    [Plain $ walk clean tit])) of                                 Left _  -> stringify tit                                 Right x -> x                 -- can't have <a> elements inside generated links...@@ -875,16 +1015,16 @@                 clean (Note _)       = Str ""                 clean x              = x -  let navtag = if epub3 then "nav" else "div"+  let navtag = if version == EPUB3 then "nav" else "div"   tocBlocks <- lift $ evalStateT                  (concat <$> mapM (navPointNode navXhtmlFormatter) secs) 1   let navBlocks = [RawBlock (Format "html")                   $ showElement $ -- prettyprinting introduces bad spaces-                   unode navtag ! ([("epub:type","toc") | epub3] +++                   unode navtag ! ([("epub:type","toc") | version == EPUB3] ++                                    [("id","toc")]) $                     [ unode "h1" ! [("id","toc-title")] $ tocTitle                     , unode "ol" ! [("class","toc")] $ tocBlocks ]]-  let landmarkItems = if epub3+  let landmarkItems = if version == EPUB3                          then unode "li"                                 [ unode "a" ! [("href",                                                   "text/title_page.xhtml")@@ -917,38 +1057,8 @@             (Pandoc (setMeta "title"                      (walk removeNote $ fromList $ docTitle' meta) nullMeta)                (navBlocks ++ landmarks))-  navEntry <- mkEntry "nav.xhtml" navData--  -- mimetype-  mimetypeEntry <- mkEntry "mimetype" $-                        UTF8.fromStringLazy "application/epub+zip"--  -- container.xml-  let containerData = B.fromStrict $ UTF8.fromText $ ppTopElement $-       unode "container" ! [("version","1.0")-              ,("xmlns","urn:oasis:names:tc:opendocument:xmlns:container")] $-         unode "rootfiles" $-           unode "rootfile" ! [("full-path",-                    (if null epubSubdir-                        then ""-                        else T.pack epubSubdir <> "/") <> "content.opf")-               ,("media-type","application/oebps-package+xml")] $ ()-  containerEntry <- mkEntry "META-INF/container.xml" containerData--  -- com.apple.ibooks.display-options.xml-  let apple = B.fromStrict $ UTF8.fromText $ ppTopElement $-        unode "display_options" $-          unode "platform" ! [("name","*")] $-            unode "option" ! [("name","specified-fonts")] $ ("true" :: Text)-  appleEntry <- mkEntry "META-INF/com.apple.ibooks.display-options.xml" apple--  -- construct archive-  let archive = foldr addEntryToArchive emptyArchive $-                 [mimetypeEntry, containerEntry, appleEntry,-                  contentsEntry, tocEntry, navEntry, tpEntry] ++-                  stylesheetEntries ++ picEntries ++ cpicEntry ++-                  cpgEntry ++ chapterEntries ++ fontEntries-  return $ fromArchive archive+  -- Return+  mkEntry "nav.xhtml" navData  metadataElement :: EPUBVersion -> EPUBMetadata -> UTCTime -> Element metadataElement version md currentTime =@@ -1000,7 +1110,7 @@             $ epubCoverImage md         modifiedNodes = [ unode "meta" ! [("property", "dcterms:modified")] $                showDateTimeISO8601 currentTime | version == EPUB3 ]-        belongsToCollectionNodes = +        belongsToCollectionNodes =             maybe []                 (\belongsToCollection -> (unode "meta" !  [("property", "belongs-to-collection"), ("id", "epub-collection-1")] $ belongsToCollection )                 :
src/Text/Pandoc/Writers/FB2.hs view
@@ -40,7 +40,8 @@ import Text.Pandoc.Options (HTMLMathMethod (..), WriterOptions (..), def) import Text.Pandoc.Shared (capitalize, isURI, orderedListMarkers,                            makeSections, tshow, stringify)-import Text.Pandoc.Writers.Shared (lookupMetaString, toLegacyTable)+import Text.Pandoc.Writers.Shared (lookupMetaString, toLegacyTable,+                                   ensureValidXmlIdentifiers) import Data.Generics (everywhere, mkT)  -- | Data to be written at the end of the document:@@ -76,7 +77,8 @@             => WriterOptions             -> Pandoc             -> FBM m Text-pandocToFB2 opts (Pandoc meta blocks) = do+pandocToFB2 opts doc = do+     let Pandoc meta blocks = ensureValidXmlIdentifiers doc      modify (\s -> s { writerOptions = opts })      desc <- description meta      title <- cMapM toXml . docTitle $ meta
src/Text/Pandoc/Writers/GridTable.hs view
@@ -10,7 +10,13 @@  Maintainer  : Albert Krewinkel <albert@zeitkraut.de> -Grid representation of pandoc tables.+Grid representation of pandoc tables. The structures in this module+allow to describe 'Text.Pandoc.Definition.Table' elements without loss+of information. However, they are simpler to use when the grid layout of+a table must be known.++The "grid tables" handled here are conceptually similar to grid tables+in reStructuredText and Markdown, but are more general. -} module Text.Pandoc.Writers.GridTable   ( Table (..)@@ -97,7 +103,7 @@ fromBuilderCell :: BuilderCell -> GridCell fromBuilderCell = \case   FilledCell c -> c-  FreeCell -> error "Found an unassigned cell."+  FreeCell -> error "Found an unassigned cell. Please report this as a bug!"  rowsToPart :: Attr -> [B.Row] -> Part rowsToPart attr = \case
src/Text/Pandoc/Writers/HTML.hs view
@@ -113,7 +113,9 @@ -- Helpers to render HTML with the appropriate function.  strToHtml :: Text -> Html-strToHtml = strToHtml' . T.unpack+strToHtml t+    | T.any isSpecial t = strToHtml' $ T.unpack t+    | otherwise = toHtml t   where     strToHtml' ('\'':xs) = preEscapedString "\'" `mappend` strToHtml' xs     strToHtml' ('"' :xs) = preEscapedString "\"" `mappend` strToHtml' xs@@ -122,11 +124,11 @@                         case xs of                           ('\xFE0E':ys) -> strToHtml' ys                           _             -> strToHtml' xs-    strToHtml' xs@(_:_) = case break (\c -> c == '\'' || c == '"' ||-                                       needsVariationSelector c) xs of+    strToHtml' xs@(_:_) = case break isSpecial xs of                             (_ ,[]) -> toHtml xs                             (ys,zs) -> toHtml ys `mappend` strToHtml' zs     strToHtml' [] = ""+    isSpecial c = c == '\'' || c == '"' || needsVariationSelector c  -- See #5469: this prevents iOS from substituting emojis. needsVariationSelector :: Char -> Bool@@ -149,12 +151,14 @@  -- | Convert Pandoc document to Html 4 string. writeHtml4String :: PandocMonad m => WriterOptions -> Pandoc -> m Text-writeHtml4String = writeHtmlString'-                      defaultWriterState{ stHtml5 = False }+writeHtml4String opts = writeHtmlString'+                         defaultWriterState{ stHtml5 = False } opts .+                        ensureValidXmlIdentifiers  -- | Convert Pandoc document to Html 4 structure. writeHtml4 :: PandocMonad m => WriterOptions -> Pandoc -> m Html-writeHtml4 = writeHtml' defaultWriterState{ stHtml5 = False }+writeHtml4 opts = writeHtml' defaultWriterState{ stHtml5 = False } opts .+                    ensureValidXmlIdentifiers  -- | Convert Pandoc document to Html appropriate for an epub version. writeHtmlStringForEPUB :: PandocMonad m@@ -164,6 +168,8 @@                       defaultWriterState{ stHtml5 = version == EPUB3,                                           stEPUBVersion = Just version }                       o{ writerWrapText = WrapNone }+   -- we don't use ensureValidXmlIdentifiers here because we+   -- do that in the EPUB writer  -- | Convert Pandoc document to Reveal JS HTML slide show. writeRevealJs :: PandocMonad m@@ -173,22 +179,25 @@ -- | Convert Pandoc document to S5 HTML slide show. writeS5 :: PandocMonad m         => WriterOptions -> Pandoc -> m Text-writeS5 = writeHtmlSlideShow' S5Slides+writeS5 opts = writeHtmlSlideShow' S5Slides opts .+               ensureValidXmlIdentifiers  -- | Convert Pandoc document to Slidy HTML slide show. writeSlidy :: PandocMonad m            => WriterOptions -> Pandoc -> m Text-writeSlidy = writeHtmlSlideShow' SlidySlides+writeSlidy opts = writeHtmlSlideShow' SlidySlides opts .+                  ensureValidXmlIdentifiers  -- | Convert Pandoc document to Slideous HTML slide show. writeSlideous :: PandocMonad m               => WriterOptions -> Pandoc -> m Text-writeSlideous = writeHtmlSlideShow' SlideousSlides+writeSlideous opts = writeHtmlSlideShow' SlideousSlides opts .+                     ensureValidXmlIdentifiers  -- | Convert Pandoc document to DZSlides HTML slide show. writeDZSlides :: PandocMonad m               => WriterOptions -> Pandoc -> m Text-writeDZSlides = writeHtmlSlideShow' DZSlides+writeDZSlides opts = writeHtmlSlideShow' DZSlides opts  writeHtmlSlideShow' :: PandocMonad m                     => HTMLSlideVariant -> WriterOptions -> Pandoc -> m Text@@ -982,7 +991,7 @@   let contents' = if writerNumberSections opts && not (T.null secnum)                      && "unnumbered" `notElem` classes                      then (H.span ! A.class_ "header-section-number"-                             $ toHtml secnum) >> strToHtml " " >> contents+                             $ toHtml secnum) >> toHtml ' ' >> contents                      else contents   html5 <- gets stHtml5   let kvs' = if html5@@ -1364,14 +1373,14 @@   html5 <- gets stHtml5   case inline of     (Str str)      -> return $ strToHtml str-    Space          -> return $ strToHtml " "+    Space          -> return $ toHtml ' '     SoftBreak      -> return $ case writerWrapText opts of-                                     WrapNone     -> " "-                                     WrapAuto     -> " "-                                     WrapPreserve -> nl+                                     WrapNone     -> toHtml ' '+                                     WrapAuto     -> toHtml ' '+                                     WrapPreserve -> toHtml '\n'     LineBreak      -> return $ do                         if html5 then H5.br else H.br-                        strToHtml "\n"+                        toHtml '\n'     (Span ("",[cls],[]) ils)         | cls == "csl-block" || cls == "csl-left-margin" ||           cls == "csl-right-inline" || cls == "csl-indent"@@ -1440,10 +1449,10 @@     (Subscript lst)   -> H.sub <$> inlineListToHtml opts lst     (Quoted quoteType lst) ->                         let (leftQuote, rightQuote) = case quoteType of-                              SingleQuote -> (strToHtml "‘",-                                              strToHtml "’")-                              DoubleQuote -> (strToHtml "“",-                                              strToHtml "”")+                              SingleQuote -> (toHtml '‘',+                                              toHtml '’')+                              DoubleQuote -> (toHtml '“',+                                              toHtml '”')                          in if writerHtmlQTags opts                                then do@@ -1465,13 +1474,15 @@       case writerHTMLMathMethod opts of            WebTeX url -> do               let imtag = if html5 then H5.img else H.img+              let str' = T.strip str               let s = case t of                            InlineMath  -> "\\textstyle "                            DisplayMath -> "\\displaystyle "               return $ imtag ! A.style "vertical-align:middle"-                             ! A.src (toValue . (url <>) . urlEncode $ s <> str)-                             ! A.alt (toValue str)-                             ! A.title (toValue str)+                             ! A.src (toValue . (url <>) .+                                 urlEncode $ s <> str')+                             ! A.alt (toValue str')+                             ! A.title (toValue str')                              ! A.class_ mathClass            GladTeX ->               return $
src/Text/Pandoc/Writers/ICML.hs view
@@ -131,7 +131,8 @@  -- | Convert Pandoc document to string in ICML format. writeICML :: PandocMonad m => WriterOptions -> Pandoc -> m Text-writeICML opts (Pandoc meta blocks) = do+writeICML opts doc = do+  let Pandoc meta blocks = ensureValidXmlIdentifiers doc   let colwidth = if writerWrapText opts == WrapAuto                     then Just $ writerColumns opts                     else Nothing
src/Text/Pandoc/Writers/Ipynb.hs view
@@ -31,7 +31,7 @@ import qualified Text.Pandoc.UTF8 as UTF8 import Text.Pandoc.Shared (safeRead, isURI) import Text.Pandoc.Writers.Shared (metaToContext')-import Text.Pandoc.Writers.Markdown (writeMarkdown)+import Text.Pandoc.Writers.Markdown (writePlain, writeMarkdown) import qualified Data.Text.Encoding as TE import qualified Data.ByteString.Lazy as BL import Data.Aeson.Encode.Pretty (Config(..), defConfig,@@ -57,10 +57,11 @@ pandocToNotebook :: PandocMonad m                  => WriterOptions -> Pandoc -> m (Notebook NbV4) pandocToNotebook opts (Pandoc meta blocks) = do-  let blockWriter bs = literal <$> writeMarkdown-           opts{ writerTemplate = Nothing } (Pandoc nullMeta bs)-  let inlineWriter ils = literal . T.stripEnd <$> writeMarkdown-           opts{ writerTemplate = Nothing } (Pandoc nullMeta [Plain ils])+  -- we use writePlain w/ default options because e.g. we don't want+  -- to add backslash escapes or convert en dashes, see #7928+  let blockWriter bs = literal <$> writePlain def (Pandoc nullMeta bs)+  let inlineWriter ils = literal . T.stripEnd <$>+                            writePlain def (Pandoc nullMeta [Plain ils])   let jupyterMeta =         case lookupMeta "jupyter" meta of           Just (MetaMap m) -> Meta m
src/Text/Pandoc/Writers/JATS/References.hs view
@@ -37,7 +37,7 @@                  -> [Reference Inlines]                  -> JATS m (Doc Text) referencesToJATS opts =-  fmap (inTags True "ref-list" [] . vcat) . mapM (referenceToJATS opts)+  fmap vcat . mapM (referenceToJATS opts)  referenceToJATS :: PandocMonad m                 => WriterOptions@@ -65,15 +65,21 @@     , accessed     , "volume"          `varInTag` "volume"     , "issue"           `varInTag` "issue"+    , "edition"         `varInTag` "edition"     , "page-first"      `varInTag` "fpage"-    , "page-last"       `varInTag` "lpage"-    , "pages"           `varInTag` "page-range"     , "ISBN"            `varInTag` "isbn"     , "ISSN"            `varInTag` "issn"     , "URL"             `varInTag` "uri"     , varInTagWith "doi"  "pub-id" [("pub-id-type", "doi")]     , varInTagWith "pmid" "pub-id" [("pub-id-type", "pmid")]-    ]+    ] +++    case lookupVariable "page" ref >>= valToText of+      Nothing -> []+      Just val ->+        let isdash c = c == '-' || c == '\x2013'+            (fpage, lpage) = T.dropWhile isdash <$> T.break isdash val+         in [ inTags' "fpage" [] $ literal $ escapeStringForXML fpage,+              inTags' "lpage" [] $ literal $ escapeStringForXML lpage ]   where     varInTag var tagName = varInTagWith var tagName [] @@ -144,7 +150,9 @@ toNameElements name =   if not (isEmpty nameTags)   then inTags' "name" [] nameTags-  else nameLiteral name `inNameTag` "string-name"+  else if nameLiteral name == Just "others"  -- indicates an "et al."+       then "<etal/>"+       else nameLiteral name `inNameTag` "string-name"     where       inNameTag mVal tag = case mVal of         Nothing  -> empty
src/Text/Pandoc/Writers/JATS/Types.hs view
@@ -39,7 +39,7 @@  -- | Environment containing all information relevant for rendering. data JATSEnv m = JATSEnv-  { jatsTagSet :: JATSTagSet  -- ^ The tag set that's being ouput+  { jatsTagSet :: JATSTagSet  -- ^ The tag set that's being output    , jatsBlockWriter   :: (Block -> Bool)                       -> WriterOptions -> [Block]  -> JATS m (Doc Text)
src/Text/Pandoc/Writers/LaTeX.hs view
@@ -86,7 +86,8 @@               blockListToLaTeX               (fmap chomp . inlineListToLaTeX)               meta-  let chaptersClasses = ["memoir","book","report","scrreprt","scrbook","extreport","extbook","tufte-book"]+  let chaptersClasses = ["memoir","book","report","scrreprt","scrreport",+                        "scrbook","extreport","extbook","tufte-book"]   let frontmatterClasses = ["memoir","book","scrbook","extbook","tufte-book"]   -- these have \frontmatter etc.   beamer <- gets stBeamer@@ -617,7 +618,7 @@      _                       ->        "\\item" <> brackets term'' $$ def' --- | Craft the section header, inserting the secton reference, if supplied.+-- | Craft the section header, inserting the section reference, if supplied. sectionHeader :: PandocMonad m               => [Text]  -- classes               -> Text@@ -741,12 +742,13 @@   let cmds = mapMaybe classToCmd classes ++ mapMaybe kvToCmd kvs ++ langCmds   contents <- inlineListToLaTeX ils   return $-    (case classes of-              ["csl-block"] -> (cr <>)-              ["csl-left-margin"] -> (cr <>)-              ["csl-right-inline"] -> (cr <>)-              ["csl-indent"] -> (cr <>)-              _ -> id) $+    (if "csl-right-inline" `elem` classes+        then ("%" <>) -- see #7932+        else id) $+    (if any (`elem` classes)+            ["csl-block","csl-left-margin","csl-right-inline","csl-indent"]+        then (cr <>)+        else id) $     (if T.null id'         then empty         else "\\protect" <> linkAnchor) <>@@ -928,7 +930,7 @@   | Just _ <- T.stripPrefix "data:" src = do       report $ InlineNotRendered il       return empty-inlineToLaTeX (Image attr _ (source, _)) = do+inlineToLaTeX (Image attr@(_,_,kvs) _ (source, _)) = do   setEmptyLine False   modify $ \s -> s{ stGraphics = True }   opts <- gets stOptions@@ -951,10 +953,13 @@                                 Height | isJust (dimension Width attr) ->                                   [d <> "\\textheight"]                                 _ -> []-      dimList = showDim Width <> showDim Height-      dims = if null dimList-                then empty-                else brackets $ mconcat (intersperse "," dimList)+      optList = showDim Width <> showDim Height <>+                maybe [] (\x -> ["page=" <> literal x]) (lookup "page" kvs) <>+                maybe [] (\x -> ["trim=" <> literal x]) (lookup "trim" kvs) <>+                maybe [] (\_ -> ["clip"]) (lookup "clip" kvs)+      options = if null optList+                   then empty+                   else brackets $ mconcat (intersperse "," optList)       source' = if isURI source                    then source                    else T.pack $ unEscapeString $ T.unpack source@@ -962,7 +967,7 @@   inHeading <- gets stInHeading   return $     (if inHeading then "\\protect\\includegraphics" else "\\includegraphics") <>-    dims <> braces (literal source'')+    options <> braces (literal source'') inlineToLaTeX (Note contents) = do   setEmptyLine False   externalNotes <- gets stExternalNotes
src/Text/Pandoc/Writers/LaTeX/Table.hs view
@@ -52,7 +52,7 @@                else ($$ text "\\endfirsthead") <$>                     headToLaTeX blksToLaTeX thead   head' <- if isEmptyHead thead-           then return "\\toprule"+           then return "\\toprule()"            -- avoid duplicate notes in head and firsthead:            else headToLaTeX blksToLaTeX                 (if isEmpty firsthead@@ -71,7 +71,7 @@     $$ head'     $$ "\\endhead"     $$ vcat rows'-    $$ "\\bottomrule"+    $$ "\\bottomrule()"     $$ "\\end{longtable}"     $$ captNotes     $$ notes@@ -161,7 +161,7 @@ headToLaTeX blocksWriter (Ann.TableHead _attr headerRows) = do   rowsContents <- mapM (rowToLaTeX blocksWriter HeaderCell . headerRowCells)                        headerRows-  return ("\\toprule" $$ vcat rowsContents $$ "\\midrule")+  return ("\\toprule()" $$ vcat rowsContents $$ "\\midrule()")  -- | Converts a row of table cells into a LaTeX row. rowToLaTeX :: PandocMonad m
src/Text/Pandoc/Writers/Markdown.hs view
@@ -23,7 +23,7 @@ import Control.Monad.Reader import Control.Monad.State.Strict import Data.Default-import Data.List (intersperse, sortOn, transpose)+import Data.List (intersperse, sortOn) import Data.List.NonEmpty (nonEmpty, NonEmpty(..)) import qualified Data.Map as M import Data.Maybe (fromMaybe, mapMaybe, isNothing)@@ -43,10 +43,11 @@ import Text.DocTemplates (Val(..), Context(..), FromContext(..)) import Text.Pandoc.Walk import Text.Pandoc.Writers.HTML (writeHtml5String)-import Text.Pandoc.Writers.Markdown.Inline (inlineListToMarkdown, -                                            linkAttributes, -                                            attrsToMarkdown, +import Text.Pandoc.Writers.Markdown.Inline (inlineListToMarkdown,+                                            linkAttributes,+                                            attrsToMarkdown,                                             attrsToMarkua)+import Text.Pandoc.Writers.Markdown.Table (pipeTable, pandocTable) import Text.Pandoc.Writers.Markdown.Types (MarkdownVariant(..),                                            WriterState(..),                                            WriterEnv(..),@@ -356,7 +357,7 @@   variant <- asks envVariant   return $      case () of-         _ | variant == Markua -> +         _ | variant == Markua ->                    case () of                         () | "blurb" `elem` classes' -> prefixed "B> " contents <> blankline                            | "aside" `elem` classes' -> prefixed "A> " contents <> blankline@@ -470,13 +471,15 @@           else return empty   variant <- asks envVariant   -- we calculate the id that would be used by auto_identifiers+  -- or gfm_auto_identifiers   -- so we know whether to print an explicit identifier   ids <- gets stIds   let autoId = uniqueIdent (writerExtensions opts) inlines ids   modify $ \st -> st{ stIds = Set.insert autoId ids }   let attr' = case attr of                    ("",[],[]) -> empty-                   (id',[],[]) | isEnabled Ext_auto_identifiers opts+                   (id',[],[]) | (isEnabled Ext_auto_identifiers opts+                                  || isEnabled Ext_gfm_auto_identifiers opts)                                  && id' == autoId -> empty                    (id',_,_)   | isEnabled Ext_mmd_header_identifiers opts ->                                     space <> brackets (literal id')@@ -515,7 +518,7 @@             -- ghc interprets '#' characters in column 1 as linenum specifiers.             _ | variant == PlainText || isEnabled Ext_literate_haskell opts ->                 contents <> blankline-            _ | variant == Markua -> attr' <> cr <> literal (T.replicate level "#") +            _ | variant == Markua -> attr' <> cr <> literal (T.replicate level "#")                                         <> space <> contents <> blankline             _ -> literal (T.replicate level "#") <> space <> contents <> attr' <> blankline @@ -533,7 +536,7 @@           backticks <> attrs <> cr <> literal str <> cr <> backticks <> blankline            | isEnabled Ext_fenced_code_blocks opts ->           tildes <> attrs <> cr <> literal str <> cr <> tildes <> blankline-     _ | variant == Markua -> blankline <> attrsToMarkua attribs <> cr <> backticks <> cr <> +     _ | variant == Markua -> blankline <> attrsToMarkua attribs <> cr <> backticks <> cr <>                                 literal str <> cr <> backticks <> cr <> blankline        | otherwise -> nest (writerTabStop opts) (literal str) <> blankline    where@@ -626,7 +629,7 @@   return $ nst (tbl $$ caption'') $$ blankline blockToMarkdown' opts (BulletList items) = do   contents <- inList $ mapM (bulletListItemToMarkdown opts) items-  return $ (if isTightList items then vcat else vsep) +  return $ (if isTightList items then vcat else vsep)                   contents <> blankline blockToMarkdown' opts (OrderedList (start,sty,delim) items) = do   variant <- asks envVariant@@ -635,7 +638,7 @@                   else 1   let sty'   = if isEnabled Ext_fancy_lists opts then sty else DefaultStyle   let delim' | isEnabled Ext_fancy_lists opts =-               case variant of +               case variant of                    -- Markua supports 'fancy' enumerators, but no TwoParens                    Markua -> if delim == TwoParens then OneParen else delim                    _ -> delim@@ -667,102 +670,6 @@               where attrs' = ("markdown","1"):[(x,y) | (x,y) <- attrs,                                  x /= "markdown"]        _ -> s--pipeTable :: PandocMonad m-          => WriterOptions-          -> Bool -> [Alignment] -> [Double] -> [Doc Text] -> [[Doc Text]]-          -> MD m (Doc Text)-pipeTable opts headless aligns widths rawHeaders rawRows = do-  let sp = literal " "-  let blockFor AlignLeft   x y = lblock (x + 2) (sp <> y) <> lblock 0 empty-      blockFor AlignCenter x y = cblock (x + 2) (sp <> y <> sp) <> lblock 0 empty-      blockFor AlignRight  x y = rblock (x + 2) (y <> sp) <> lblock 0 empty-      blockFor _           x y = lblock (x + 2) (sp <> y) <> lblock 0 empty-  let contentWidths = map (max 3 . maybe 3 maximum . nonEmpty . map offset) $-                       transpose (rawHeaders : rawRows)-  let colwidth = writerColumns opts-  let numcols = length contentWidths-  let maxwidth = sum contentWidths-  variant <- asks envVariant-  let pipeWidths = if variant == Markdown &&-                      not (all (== 0) widths) &&-                      maxwidth + (numcols + 1) > colwidth-                      then map-                            (floor . (* fromIntegral (colwidth - (numcols +1))))-                            widths-                      else contentWidths-  let torow cs = nowrap $ literal "|" <>-                    hcat (intersperse (literal "|") $-                          zipWith3 blockFor aligns contentWidths (map chomp cs))-                    <> literal "|"-  let toborder a w = literal $ case a of-                          AlignLeft    -> ":" <> T.replicate (w + 1) "-"-                          AlignCenter  -> ":" <> T.replicate w "-" <> ":"-                          AlignRight   -> T.replicate (w + 1) "-" <> ":"-                          AlignDefault -> T.replicate (w + 2) "-"-  -- note:  pipe tables can't completely lack a-  -- header; for a headerless table, we need a header of empty cells.-  -- see jgm/pandoc#1996.-  let header = if headless-                  then torow (replicate (length aligns) empty)-                  else torow rawHeaders-  let border = nowrap $ literal "|" <> hcat (intersperse (literal "|") $-                        zipWith toborder aligns pipeWidths) <> literal "|"-  let body   = vcat $ map torow rawRows-  return $ header $$ border $$ body--pandocTable :: PandocMonad m-            => WriterOptions -> Bool -> Bool -> [Alignment] -> [Double]-            -> [Doc Text] -> [[Doc Text]] -> MD m (Doc Text)-pandocTable opts multiline headless aligns widths rawHeaders rawRows = do-  let isSimple = all (==0) widths-  let alignHeader alignment = case alignment of-                                AlignLeft    -> lblock-                                AlignCenter  -> cblock-                                AlignRight   -> rblock-                                AlignDefault -> lblock-  -- Number of characters per column necessary to output every cell-  -- without requiring a line break.-  -- The @+2@ is needed for specifying the alignment.-  let numChars    = (+ 2) . maybe 0 maximum . nonEmpty . map offset-  -- Number of characters per column necessary to output every cell-  -- without requiring a line break *inside a word*.-  -- The @+2@ is needed for specifying the alignment.-  let minNumChars = (+ 2) . maybe 0 maximum . nonEmpty . map minOffset-  let columns = transpose (rawHeaders : rawRows)-  -- minimal column width without wrapping a single word-  let relWidth w col =-         max (floor $ fromIntegral (writerColumns opts - 1) * w)-             (if writerWrapText opts == WrapAuto-                 then minNumChars col-                 else numChars col)-  let widthsInChars-        | isSimple  = map numChars columns-        | otherwise = zipWith relWidth widths columns-  let makeRow = hcat . intersperse (lblock 1 (literal " ")) .-                   zipWith3 alignHeader aligns widthsInChars-  let rows' = map makeRow rawRows-  let head' = makeRow rawHeaders-  let underline = mconcat $ intersperse (literal " ") $-                  map (\width -> literal (T.replicate width "-")) widthsInChars-  let border-        | multiline = literal (T.replicate (sum widthsInChars +-                        length widthsInChars - 1) "-")-        | headless  = underline-        | otherwise = empty-  let head'' = if headless-                  then empty-                  else border <> cr <> head'-  let body = if multiline-                then vsep rows' $$-                     if length rows' < 2-                        then blankline -- #4578-                        else empty-                else vcat rows'-  let bottom = if headless-                  then underline-                  else border-  return $ head'' $$ underline $$ body $$ bottom  itemEndsWithTightList :: [Block] -> Bool itemEndsWithTightList bs =
src/Text/Pandoc/Writers/Markdown/Inline.hs view
@@ -68,8 +68,16 @@   go [] = []   go (c:cs) =     case c of-       _ | c `elem` ['\\','`','*','_','[',']'] ->-              '\\':c:go cs+       '[' -> '\\':c:go cs+       ']' -> '\\':c:go cs+       '`' -> '\\':c:go cs+       '*' -> '\\':c:go cs+       '_' -> '\\':c:go cs+       '\\' | isEnabled Ext_raw_tex opts -> '\\':c:go cs+            | otherwise ->+              case cs of -- don't escape \ if we don't have to:+                d:_ | isAlphaNum d -> c:go cs+                _ -> '\\':c:go cs        '>' | isEnabled Ext_all_symbols_escapable opts -> '\\':'>':go cs            | otherwise -> "&gt;" ++ go cs        '<' | isEnabled Ext_all_symbols_escapable opts -> '\\':'<':go cs@@ -100,7 +108,7 @@ escapeMarkuaString :: Text -> Text escapeMarkuaString s = foldr (uncurry T.replace) s [("--","~-~-"),                         ("**","~*~*"),("//","~/~/"),("^^","~^~^"),(",,","~,~,")]- + attrsToMarkdown :: Attr -> Doc Text attrsToMarkdown attribs = braces $ hsep [attribId, attribClasses, attribKeys]         where attribId = case attribs of@@ -138,7 +146,7 @@               attrKeyValues = case attributes of                                (_,_,[]) -> []                                (_,_,keyvalues) -> map ((\(k,v) -> escAttr k-                                              <> ": " <> escAttr v) . +                                              <> ": " <> escAttr v) .                                               preprocessKeyValues) keyvalues               escAttr          = mconcat . map escAttrChar . T.unpack               escAttrChar '"'  = literal "\""@@ -162,7 +170,7 @@  -- | Add a (key, value) pair to Pandoc attr type addKeyValueToAttr :: Attr -> (Text,Text) -> Attr-addKeyValueToAttr (ident,classes,kvs) (key,value) +addKeyValueToAttr (ident,classes,kvs) (key,value)     | not (T.null key) && not (T.null value) = (ident,                                                 classes,                                                 (key,value): kvs)@@ -170,7 +178,7 @@  linkAttributes :: WriterOptions -> Attr -> Doc Text linkAttributes opts attr =-  if (isEnabled Ext_link_attributes opts || +  if (isEnabled Ext_link_attributes opts ||         isEnabled Ext_attributes opts) && attr /= nullAttr      then attrsToMarkdown attr      else empty@@ -485,31 +493,35 @@   case () of     _ | variant == Markua -> return $ "`" <> literal str <> "`" <> "$"       | otherwise -> case writerHTMLMathMethod opts of-          WebTeX url -> inlineToMarkdown opts-                          (Image nullAttr [Str str] (url <> urlEncode str, str))+          WebTeX url ->+            let str' = T.strip str+             in inlineToMarkdown opts+                  (Image nullAttr [Str str'] (url <> urlEncode str', str'))           _ | isEnabled Ext_tex_math_dollars opts ->                 return $ "$" <> literal str <> "$"             | isEnabled Ext_tex_math_single_backslash opts ->                 return $ "\\(" <> literal str <> "\\)"             | isEnabled Ext_tex_math_double_backslash opts ->                 return $ "\\\\(" <> literal str <> "\\\\)"-            | otherwise -> +            | otherwise ->                 texMathToInlines InlineMath str >>=                   inlineListToMarkdown opts .                     (if variant == PlainText then makeMathPlainer else id)-     + inlineToMarkdown opts (Math DisplayMath str) = do   variant <- asks envVariant   case () of     _ | variant == Markua -> do-        let attributes = attrsToMarkua (addKeyValueToAttr ("",[],[]) +        let attributes = attrsToMarkua (addKeyValueToAttr ("",[],[])                                                         ("format", "latex"))-        return $ blankline <> attributes <> cr <> literal "```" <> cr +        return $ blankline <> attributes <> cr <> literal "```" <> cr             <> literal str <> cr <> literal "```" <> blankline       | otherwise -> case writerHTMLMathMethod opts of-          WebTeX url -> (\x -> blankline <> x <> blankline) `fmap`-                 inlineToMarkdown opts (Image nullAttr [Str str]-                        (url <> urlEncode str, str))+          WebTeX url ->+            let str' = T.strip str+             in (\x -> blankline <> x <> blankline) `fmap`+                 inlineToMarkdown opts (Image nullAttr [Str str']+                        (url <> urlEncode str', str'))           _ | isEnabled Ext_tex_math_dollars opts ->                 return $ "$$" <> literal str <> "$$"             | isEnabled Ext_tex_math_single_backslash opts ->@@ -670,13 +682,13 @@                else alternate   linkPart <- inlineToMarkdown opts (Link attr txt (source, tit))   alt <- inlineListToMarkdown opts alternate-  let attributes | variant == Markua = attrsToMarkua $ -            addKeyValueToAttr (addKeyValueToAttr attr ("title", tit)) -            ("alt", render (Just (writerColumns opts)) alt) +  let attributes | variant == Markua = attrsToMarkua $+            addKeyValueToAttr (addKeyValueToAttr attr ("title", tit))+            ("alt", render (Just (writerColumns opts)) alt)                  | otherwise = empty   return $ case variant of                 PlainText -> "[" <> linkPart <> "]"-                Markua -> cr <> attributes <> cr <> literal "![](" <> +                Markua -> cr <> attributes <> cr <> literal "![](" <>                             literal source <> ")" <> cr                 _ -> "!" <> linkPart inlineToMarkdown opts (Note contents) = do
+ src/Text/Pandoc/Writers/Markdown/Table.hs view
@@ -0,0 +1,134 @@+{-# LANGUAGE OverloadedStrings   #-}+{- |+   Module      : Text.Pandoc.Writers.Markdown+   Copyright   : © 2006-2022 John MacFarlane+   License     : GPL-2.0-or-later+   Maintainer  : John MacFarlane <jgm@berkeley.edu>++Create Markdown pipe-tables and pandoc-style tables.+-}+module Text.Pandoc.Writers.Markdown.Table+  ( pipeTable+  , pandocTable+  ) where++import Control.Monad.Reader (asks)+import Data.List (intersperse, transpose)+import Data.List.NonEmpty (nonEmpty)+import Data.Text (Text)+import qualified Data.Text as T+import Text.DocLayout+import Text.Pandoc.Class.PandocMonad (PandocMonad)+import Text.Pandoc.Definition (Alignment (..))+import Text.Pandoc.Options (WriterOptions (writerColumns, writerWrapText),+                            WrapOption(WrapAuto))+import Text.Pandoc.Writers.Markdown.Types (MarkdownVariant(Markdown),+                                           WriterEnv(..), MD)++-- | Creates a Markdown pipe table.+pipeTable :: PandocMonad m+          => WriterOptions+          -> Bool            -- ^ headless?+          -> [Alignment]     -- ^ column alignments+          -> [Double]        -- ^ column widhts+          -> [Doc Text]      -- ^ table header cells+          -> [[Doc Text]]    -- ^ table body rows+          -> MD m (Doc Text)+pipeTable opts headless aligns widths rawHeaders rawRows = do+  let sp = literal " "+  let blockFor AlignLeft   x y = lblock (x + 2) (sp <> y) <> lblock 0 empty+      blockFor AlignCenter x y = cblock (x + 2) (sp <> y <> sp) <> lblock 0 empty+      blockFor AlignRight  x y = rblock (x + 2) (y <> sp) <> lblock 0 empty+      blockFor _           x y = lblock (x + 2) (sp <> y) <> lblock 0 empty+  let contentWidths = map (max 3 . maybe 3 maximum . nonEmpty . map offset) $+                       transpose (rawHeaders : rawRows)+  let colwidth = writerColumns opts+  let numcols = length contentWidths+  let maxwidth = sum contentWidths+  variant <- asks envVariant+  let pipeWidths = if variant == Markdown &&+                      not (all (== 0) widths) &&+                      maxwidth + (numcols + 1) > colwidth+                      then map+                            (floor . (* fromIntegral (colwidth - (numcols +1))))+                            widths+                      else contentWidths+  let torow cs = nowrap $ literal "|" <>+                    hcat (intersperse (literal "|") $+                          zipWith3 blockFor aligns contentWidths (map chomp cs))+                    <> literal "|"+  let toborder a w = literal $ case a of+                          AlignLeft    -> ":" <> T.replicate (w + 1) "-"+                          AlignCenter  -> ":" <> T.replicate w "-" <> ":"+                          AlignRight   -> T.replicate (w + 1) "-" <> ":"+                          AlignDefault -> T.replicate (w + 2) "-"+  -- note:  pipe tables can't completely lack a+  -- header; for a headerless table, we need a header of empty cells.+  -- see jgm/pandoc#1996.+  let header = if headless+                  then torow (replicate (length aligns) empty)+                  else torow rawHeaders+  let border = nowrap $ literal "|" <> hcat (intersperse (literal "|") $+                        zipWith toborder aligns pipeWidths) <> literal "|"+  let body   = vcat $ map torow rawRows+  return $ header $$ border $$ body++-- | Write a pandoc-style Markdown table.+pandocTable :: PandocMonad m+            => WriterOptions+            -> Bool            -- ^ whether this is a multiline table+            -> Bool            -- ^ whether the table has a header+            -> [Alignment]     -- ^ column alignments+            -> [Double]        -- ^ column widths+            -> [Doc Text]      -- ^ table header cells+            -> [[Doc Text]]    -- ^ table body rows+            -> MD m (Doc Text)+pandocTable opts multiline headless aligns widths rawHeaders rawRows = do+  let isSimple = all (==0) widths+  let alignHeader alignment = case alignment of+                                AlignLeft    -> lblock+                                AlignCenter  -> cblock+                                AlignRight   -> rblock+                                AlignDefault -> lblock+  -- Number of characters per column necessary to output every cell+  -- without requiring a line break.+  -- The @+2@ is needed for specifying the alignment.+  let numChars    = (+ 2) . maybe 0 maximum . nonEmpty . map offset+  -- Number of characters per column necessary to output every cell+  -- without requiring a line break *inside a word*.+  -- The @+2@ is needed for specifying the alignment.+  let minNumChars = (+ 2) . maybe 0 maximum . nonEmpty . map minOffset+  let columns = transpose (rawHeaders : rawRows)+  -- minimal column width without wrapping a single word+  let relWidth w col =+         max (floor $ fromIntegral (writerColumns opts - 1) * w)+             (if writerWrapText opts == WrapAuto+                 then minNumChars col+                 else numChars col)+  let widthsInChars+        | isSimple  = map numChars columns+        | otherwise = zipWith relWidth widths columns+  let makeRow = hcat . intersperse (lblock 1 (literal " ")) .+                   zipWith3 alignHeader aligns widthsInChars+  let rows' = map makeRow rawRows+  let head' = makeRow rawHeaders+  let underline = mconcat $ intersperse (literal " ") $+                  map (\width -> literal (T.replicate width "-")) widthsInChars+  let border+        | multiline = literal (T.replicate (sum widthsInChars ++                        length widthsInChars - 1) "-")+        | headless  = underline+        | otherwise = empty+  let head'' = if headless+                  then empty+                  else border <> cr <> head'+  let body = if multiline+                then vsep rows' $$+                     if length rows' < 2+                        then blankline -- #4578+                        else empty+                else vcat rows'+  let bottom = if headless+                  then underline+                  else border+  return $ head'' $$ underline $$ body $$ bottom
src/Text/Pandoc/Writers/ODT.hs view
@@ -36,7 +36,8 @@ import Text.DocLayout import Text.Pandoc.Shared (stringify, pandocVersion, tshow) import Text.Pandoc.Writers.Shared (lookupMetaString, lookupMetaBlocks,-                                   fixDisplayMath, getLang)+                                   fixDisplayMath, getLang,+                                   ensureValidXmlIdentifiers) import Text.Pandoc.UTF8 (fromStringLazy, fromTextLazy, toTextLazy) import Text.Pandoc.Walk import Text.Pandoc.Writers.OpenDocument (writeOpenDocument)@@ -58,8 +59,9 @@ writeODT  opts doc =   let initState = ODTState{ stEntries = []                           }+      doc' = ensureValidXmlIdentifiers doc   in-    evalStateT (pandocToODT opts doc) initState+    evalStateT (pandocToODT opts doc') initState  -- | Produce an ODT file from a Pandoc document. pandocToODT :: PandocMonad m
src/Text/Pandoc/Writers/Org.hs view
@@ -146,7 +146,7 @@   let headerStr = text $ if level > 999 then " " else replicate level '*'   let drawerStr = if attr == nullAttr                   then empty-                  else cr <> nest (level + 1) (propertiesDrawer attr)+                  else cr <> propertiesDrawer attr   return $ headerStr <> " " <> contents <> drawerStr <> cr blockToOrg (CodeBlock (_,classes,kvs) str) = do   let startnum = maybe "" (\x -> " " <> trimr x) $ lookup "startFrom" kvs@@ -163,7 +163,7 @@ blockToOrg (BlockQuote blocks) = do   contents <- blockListToOrg blocks   return $ blankline $$ "#+begin_quote" $$-           nest 2 contents $$ "#+end_quote" $$ blankline+           contents $$ "#+end_quote" $$ blankline blockToOrg (Table _ blkCapt specs thead tbody tfoot) =  do   let (caption', _, _, headers, rows) = toLegacyTable blkCapt specs thead tbody tfoot   caption'' <- inlineListToOrg caption'
src/Text/Pandoc/Writers/Shared.hs view
@@ -37,6 +37,7 @@                      , endsWithPlain                      , toLegacyTable                      , splitSentences+                     , ensureValidXmlIdentifiers                      ) where import Safe (lastMay)@@ -44,7 +45,7 @@ import Data.Maybe (fromMaybe, isNothing) import Control.Monad (zipWithM) import Data.Aeson (ToJSON (..), encode)-import Data.Char (chr, ord, isSpace)+import Data.Char (chr, ord, isSpace, isLetter) import Data.List (groupBy, intersperse, transpose, foldl') import Data.List.NonEmpty (NonEmpty(..), nonEmpty) import Data.Text.Conversions (FromText(..))@@ -230,14 +231,15 @@   '\8216' -> "'"   _       -> T.singleton c +-- | Writes a grid table. gridTable :: (Monad m, HasChars a)           => WriterOptions-          -> (WriterOptions -> [Block] -> m (Doc a))-          -> Bool -- ^ headless-          -> [Alignment]-          -> [Double]-          -> [[Block]]-          -> [[[Block]]]+          -> (WriterOptions -> [Block] -> m (Doc a)) -- ^ format Doc writer+          -> Bool             -- ^ headless+          -> [Alignment]      -- ^ column alignments+          -> [Double]         -- ^ column widths+          -> [[Block]]        -- ^ table header row+          -> [[[Block]]]      -- ^ table body rows           -> m (Doc a) gridTable opts blocksToDoc headless aligns widths headers rows = do   -- the number of columns will be used in case of even widths@@ -446,7 +448,7 @@    headerText' = addNumber $ walk (deLink . deNote) ils    headerLink = if T.null ident                    then headerText'-                   else [Link nullAttr headerText' ("#" <> ident, "")]+                   else [Link ("toc-" <> ident, [], []) headerText' ("#" <> ident, "")]    listContents = filter (not . null) $ map (sectionToListItem opts) subsecs sectionToListItem _ _ = [] @@ -547,3 +549,45 @@              Just (_,d) -> d == '.' || d == '!' || d == '?'              _ -> False       _ -> False++-- | Ensure that all identifiers start with a letter,+-- and modify internal links accordingly. (Yes, XML allows an+-- underscore, but HTML 4 doesn't, so we are more conservative.)+ensureValidXmlIdentifiers :: Pandoc -> Pandoc+ensureValidXmlIdentifiers = walk fixLinks . walkAttr fixIdentifiers+ where+  fixIdentifiers (ident, classes, kvs) =+    (case T.uncons ident of+      Nothing -> ident+      Just (c, _) | isLetter c -> ident+      _ -> "id_" <> ident,+     classes, kvs)+  needsFixing src =+    case T.uncons src of+      Just ('#',t) ->+        case T.uncons t of+          Just (c,_) | not (isLetter c) -> Just ("#id_" <> t)+          _ -> Nothing+      _ -> Nothing+  fixLinks (Link attr ils (src, tit))+    | Just src' <- needsFixing src = Link attr ils (src', tit)+  fixLinks (Image attr ils (src, tit))+    | Just src' <- needsFixing src = Image attr ils (src', tit)+  fixLinks x = x++-- | Walk Pandoc document, modifying attributes.+walkAttr :: (Attr -> Attr) -> Pandoc -> Pandoc+walkAttr f = walk goInline . walk goBlock+ where+  goInline (Span attr ils) = Span (f attr) ils+  goInline (Link attr ils target) = Link (f attr) ils target+  goInline (Image attr ils target) = Image (f attr) ils target+  goInline (Code attr txt) = Code (f attr) txt+  goInline x = x++  goBlock (Header lev attr ils) = Header lev (f attr) ils+  goBlock (CodeBlock attr txt) = CodeBlock (f attr) txt+  goBlock (Table attr cap colspecs thead tbodies tfoot) =+    Table (f attr) cap colspecs thead tbodies tfoot+  goBlock (Div attr bs) = Div (f attr) bs+  goBlock x = x
src/Text/Pandoc/Writers/TEI.hs view
@@ -28,7 +28,8 @@  -- | Convert Pandoc document to string in Docbook format. writeTEI :: PandocMonad m => WriterOptions -> Pandoc -> m Text-writeTEI opts (Pandoc meta blocks) = do+writeTEI opts doc = do+  let Pandoc meta blocks = ensureValidXmlIdentifiers doc   let colwidth = if writerWrapText opts == WrapAuto                     then Just $ writerColumns opts                     else Nothing
src/Text/Pandoc/XML/Light.hs view
@@ -36,6 +36,9 @@     -- * Replacement for xml-light's Text.XML.Input   , parseXMLElement   , parseXMLContents+    --  * Versions that allow passing in a custom entity table+  , parseXMLElementWithEntities+  , parseXMLContentsWithEntities   ) where  import qualified Control.Exception as E@@ -48,23 +51,45 @@ import Text.Pandoc.XML.Light.Types import Text.Pandoc.XML.Light.Proc import Text.Pandoc.XML.Light.Output+import qualified Data.XML.Types as XML  -- Drop in replacement for parseXMLDoc in xml-light. parseXMLElement :: TL.Text -> Either T.Text Element-parseXMLElement t =+parseXMLElement = parseXMLElementWithEntities mempty++-- Drop in replacement for parseXMLDoc in xml-light.+parseXMLElementWithEntities :: M.Map T.Text T.Text+                            -> TL.Text -> Either T.Text Element+parseXMLElementWithEntities entityMap t =   elementToElement .  Conduit.documentRoot <$>     either (Left . T.pack . E.displayException) Right-    (Conduit.parseText Conduit.def{ Conduit.psRetainNamespaces = True } t)+    (Conduit.parseText Conduit.def{ Conduit.psRetainNamespaces = True+                                  , Conduit.psDecodeEntities = decodeEnts } t)+ where+   decodeEnts ref = case M.lookup ref entityMap of+                      Nothing -> XML.ContentEntity ref+                      Just t' -> XML.ContentText t'  parseXMLContents :: TL.Text -> Either T.Text [Content]-parseXMLContents t =-  case Conduit.parseText Conduit.def{ Conduit.psRetainNamespaces = True } t of+parseXMLContents = parseXMLContentsWithEntities mempty++parseXMLContentsWithEntities :: M.Map T.Text T.Text+                             -> TL.Text -> Either T.Text [Content]+parseXMLContentsWithEntities entityMap t =+  case Conduit.parseText Conduit.def{ Conduit.psRetainNamespaces = True+                                    , Conduit.psDecodeEntities = decodeEnts+                                    } t of     Left e ->       case E.fromException e of         Just (ContentAfterRoot _) ->-          elContent <$> parseXMLElement ("<wrapper>" <> t <> "</wrapper>")+          elContent <$> parseXMLElementWithEntities entityMap+                          ("<wrapper>" <> t <> "</wrapper>")         _ -> Left . T.pack . E.displayException $ e     Right x -> Right [Elem . elementToElement . Conduit.documentRoot $ x]+ where+   decodeEnts ref = case M.lookup ref entityMap of+                      Nothing -> XML.ContentEntity ref+                      Just t' -> XML.ContentText t'  elementToElement :: Conduit.Element -> Element elementToElement (Conduit.Element name attribMap nodes) =
src/Text/Pandoc/XML/Light/Proc.hs view
@@ -99,23 +99,23 @@ filterElementName     :: (QName -> Bool) -> Element -> Maybe Element filterElementName p e  = listToMaybe (filterElementsName p e) --- | Find all non-nested occurances of an element.+-- | Find all non-nested occurrences of an element. -- (i.e., once we have found an element, we do not search--- for more occurances among the element's children).+-- for more occurrences among the element's children). findElements       :: QName -> Element -> [Element] findElements qn e = filterElementsName (qn==) e  -- | Find all non-nested occurrences of an element wrt. given predicate. -- (i.e., once we have found an element, we do not search--- for more occurances among the element's children).+-- for more occurrences among the element's children). filterElements       :: (Element -> Bool) -> Element -> [Element] filterElements p e  | p e        = [e]  | otherwise  = concatMap (filterElements p) $ onlyElems $ elContent e --- | Find all non-nested occurences of an element wrt a predicate over element names.+-- | Find all non-nested occurrences of an element wrt a predicate over element names. -- (i.e., once we have found an element, we do not search--- for more occurances among the element's children).+-- for more occurrences among the element's children). filterElementsName       :: (QName -> Bool) -> Element -> [Element] filterElementsName p e = filterElements (p.elName) e
stack.yaml view
@@ -7,38 +7,39 @@ packages: - '.' extra-deps:-- skylighting-core-0.12.2-- skylighting-0.12.2-- doctemplates-0.10.0.1+- skylighting-core-0.12.3+- skylighting-0.12.3 - emojis-0.1.2-- doclayout-0.3.1.1-- lpeg-1.0.2-- hslua-2.1.0-- hslua-aeson-2.1.0-- hslua-classes-2.1.0-- hslua-core-2.1.0-- hslua-marshalling-2.1.0-- hslua-module-path-1.0.1-- hslua-module-system-1.0.1-- hslua-module-text-1.0.1-- hslua-module-version-1.0.1-- hslua-objectorientation-2.1.0-- hslua-packaging-2.1.0-- lua-2.1.0-- lua-arbitrary-1.0.0-- tasty-hslua-1.0.1-- tasty-lua-1.0.1-- pandoc-lua-marshal-0.1.4-- pandoc-types-1.22.1+- lpeg-1.0.3+- hslua-2.2.0+- hslua-aeson-2.2.0+- hslua-classes-2.2.0+- hslua-core-2.2.0+- hslua-marshalling-2.2.0+- hslua-module-doclayout-1.0.4+- hslua-module-path-1.0.2+- hslua-module-system-1.0.2+- hslua-module-text-1.0.2+- hslua-module-version-1.0.2+- hslua-objectorientation-2.2.0+- hslua-packaging-2.2.0+- lua-2.2.0+- lua-arbitrary-1.0.1+- tasty-hslua-1.0.2+- tasty-lua-1.0.2+- pandoc-lua-marshal-0.1.5.1 - aeson-pretty-0.8.9-- unicode-transforms-0.4.0-- unicode-data-0.2.0-- texmath-0.12.4-- commonmark-0.2.1.1-- commonmark-extensions-0.2.3+- unicode-transforms-0.4.0.1+- unicode-data-0.3.0 - commonmark-pandoc-0.2.1.2-- citeproc-0.6.0.1 - ipynb-0.2+- pandoc-types-1.22.2+- texmath-0.12.5+- citeproc-0.7+- commonmark-0.2.2+- commonmark-extensions-0.2.3.1+- doclayout-0.4+- doctemplates-0.10.0.2 ghc-options:    "$locals": -fhide-source-paths -Wno-missing-home-modules resolver: lts-18.10
test/Tests/Helpers.hs view
@@ -36,6 +36,7 @@ import Test.Tasty.Golden.Advanced (goldenTest) import Test.Tasty.HUnit import Text.Pandoc.Builder (Blocks, Inlines, doc, plain)+import qualified Text.Pandoc.Builder as B import Text.Pandoc.Class import Text.Pandoc.Definition import Text.Pandoc.Options@@ -149,6 +150,12 @@  instance ToString Blocks where   toString = unpack . purely (writeNative def) . toPandoc++instance ToString [Block] where+  toString = toString . B.fromList++instance ToString Block where+  toString = toString . B.singleton   instance ToString Inlines where   toString = unpack . trimr . purely (writeNative def) . toPandoc
test/Tests/Old.hs view
@@ -211,6 +211,9 @@     ]   , testGroup "rtf"     [ testGroup "writer" $ writerTests' "rtf" ]+  , testGroup "endnotexml"+    [ test' "reader" ["-r", "endnotexml", "-w", "native", "-s"]+        "endnotexml-reader.xml" "endnotexml-reader.native" ]   , testGroup "ipynb"     [ test' "reader" ["-f", "ipynb-raw_html-raw_tex+raw_attribute",                       "-t", "native", "-s"]
test/Tests/Readers/Creole.hs view
@@ -16,6 +16,7 @@ import Data.Text (Text) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -25,7 +26,7 @@ creole = purely $ readCreole def{ readerStandalone = True }  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test creole
test/Tests/Readers/Docx.hs view
@@ -370,6 +370,26 @@             "docx/drop_cap.docx"             "docx/drop_cap.native"           ]+        , testGroup "citations"+          [ testCompare+            "zotero with -citations"+            "docx/zotero_citations.docx"+            "docx/zotero_citations_minus.native"+          , testCompareWithOpts def{readerExtensions =+                  extensionsFromList [Ext_citations]}+            "zotero with +citations"+            "docx/zotero_citations.docx"+            "docx/zotero_citations_plus.native"+          , testCompare+            "mendeley with -citations"+            "docx/mendeley_citations.docx"+            "docx/mendeley_citations_minus.native"+          , testCompareWithOpts def{readerExtensions =+                  extensionsFromList [Ext_citations]}+            "mendeley with +citations"+            "docx/mendeley_citations.docx"+            "docx/mendeley_citations_plus.native"+          ]         , testGroup "track changes"           [ testCompare             "insertion (default)"
test/Tests/Readers/DokuWiki.hs view
@@ -1,5 +1,6 @@ {-# LANGUAGE OverloadedStrings   #-} {-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TupleSections #-} {- |    Module      : Tests.Readers.DokuWiki    Copyright   : © 2018-2020 Alexander Krotov@@ -16,6 +17,7 @@ import Data.Text (Text) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -25,7 +27,7 @@ dokuwiki = purely $ readDokuWiki def{ readerStandalone = True }  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test dokuwiki @@ -224,12 +226,12 @@           , "Ordered list" =:             T.unlines [ "  - The same list but ordered"                       , "  - Another item"-                      , "    - Just use indention for deeper levels"+                      , "    - Just use indentation for deeper levels"                       , "  - That's it"                       ] =?>             orderedList [ plain "The same list but ordered"                         , plain "Another item" <>-                          orderedList [ plain "Just use indention for deeper levels" ]+                          orderedList [ plain "Just use indentation for deeper levels" ]                         , plain "That's it"                         ]           , "Multiline list items" =: -- https://www.dokuwiki.org/faq:lists@@ -300,6 +302,17 @@                     , "| bat | baz |"                     ] =?>           simpleTable [plain "foo", plain "bar"] [[plain "bat", plain "baz"]]+        , "Table with alignment" =:+          T.unlines [ "^ 0  ^  1  ^  2 ^ 3 ^"+                    , "| a  | b   | c  |d  |"+                    ] =?>+          table emptyCaption+                (map (, ColWidthDefault) [AlignLeft, AlignCenter, AlignRight, AlignDefault])+                (TableHead nullAttr+                          [Row nullAttr . map (simpleCell . plain) $ ["0", "1", "2", "3"]])+                [TableBody nullAttr 0 []+                          [Row nullAttr . map (simpleCell . plain) $ ["a", "b", "c", "d"]]]+                (TableFoot nullAttr [])         , "Table with colspan" =:           T.unlines [ "^ 0,0 ^ 0,1 ^ 0,2 ^"                     , "| 1,0 | 1,1 ||"
test/Tests/Readers/HTML.hs view
@@ -120,7 +120,7 @@             =?>             codeBlockWith ("a", ["python"], []) "print('hi')" -          , test html "attributes in pre take precendence" $+          , test html "attributes in pre take precedence" $             "<pre id=\"c\"><code id=\"d\">\nprint('hi mom!')\n</code></pre>"             =?>             codeBlockWith ("c", [], []) "print('hi mom!')"
test/Tests/Readers/Jira.hs view
@@ -16,6 +16,7 @@ import Prelude hiding (unlines) import Data.Text (Text, unlines) import Test.Tasty (TestTree, testGroup)+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers (ToString, purely, test, (=?>)) import Text.Pandoc (def) import Text.Pandoc.Readers.Jira (readJira)@@ -25,7 +26,7 @@ jira = purely $ readJira def  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test jira
test/Tests/Readers/LaTeX.hs view
@@ -15,6 +15,7 @@ import Data.Text (Text) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -25,7 +26,7 @@                    readerExtensions = getDefaultExtensions "latex" }  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test latex @@ -166,7 +167,7 @@                                   , simpleCell (plain "Two")                                   ]                    , Row nullAttr [ simpleCell (plain "Three") ]-                   , Row nullAttr [ simpleCell (plain "Four") +                   , Row nullAttr [ simpleCell (plain "Four")                                   , simpleCell (plain "Five")                                   , simpleCell (plain "Six")                                   , simpleCell (plain "Seven")@@ -174,8 +175,8 @@                    ]           , "Table with multicolumn header" =:             T.unlines [ "\\begin{tabular}{ |l|l| }"-                      , "\\hline\\multicolumn{2}{|c|}{Header}\\\\" -                      , "\\hline key & val\\\\" +                      , "\\hline\\multicolumn{2}{|c|}{Header}\\\\"+                      , "\\hline key & val\\\\"                       , "\\hline\\end{tabular}"                       ] =?>             table emptyCaption
test/Tests/Readers/Man.hs view
@@ -15,6 +15,7 @@  import Data.Text (Text) import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -25,7 +26,7 @@ man = purely $ readMan def  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test man
test/Tests/Readers/Markdown.hs view
@@ -15,6 +15,7 @@ import Data.Text (Text, unpack) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -40,7 +41,7 @@ markdownMMD = purely $ readMarkdown def {                  readerExtensions = multimarkdownExtensions } infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test markdown @@ -371,13 +372,13 @@           , test markdownMMD "normal superscript"             ("x^3^"             =?> para ("x" <> superscript "3"))-          , test markdownMMD "short subscript delimeted by space"+          , test markdownMMD "short subscript delimited by space"             ("O~2 is dangerous"             =?> para ("O" <> subscript "2" <> space <> "is dangerous"))-          , test markdownMMD "short subscript delimeted by newline"+          , test markdownMMD "short subscript delimited by newline"             ("O~2\n"             =?> para ("O" <> subscript "2"))-          , test markdownMMD "short subscript delimeted by EOF"+          , test markdownMMD "short subscript delimited by EOF"             ("O~2"             =?> para ("O" <> subscript "2"))           , test markdownMMD "short subscript delimited by punctuation"@@ -389,13 +390,13 @@           , test markdownMMD "no nesting in short subscripts"             ("y~*2*"             =?> para ("y~" <> emph "2"))-          , test markdownMMD "short superscript delimeted by space"+          , test markdownMMD "short superscript delimited by space"             ("x^2 = y"             =?> para ("x" <> superscript "2" <> space <> "= y"))-          , test markdownMMD "short superscript delimeted by newline"+          , test markdownMMD "short superscript delimited by newline"             ("x^2\n"             =?> para ("x" <> superscript "2"))-          , test markdownMMD "short superscript delimeted by ExF"+          , test markdownMMD "short superscript delimited by ExF"             ("x^2"             =?> para ("x" <> superscript "2"))           , test markdownMMD "short superscript delimited by punctuation"
test/Tests/Readers/Muse.hs view
@@ -17,6 +17,7 @@ import Data.Text (Text) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Test.Tasty.QuickCheck import Test.Tasty.Options (IsOption(defaultValue)) import Tests.Helpers@@ -33,7 +34,7 @@ emacsMuse = purely $ readMuse def { readerExtensions = emptyExtensions }  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test amuse
test/Tests/Readers/Org/Block.hs view
@@ -186,6 +186,27 @@                , "\\end{equation}"                ]) +    , "One-line LaTeX fragment" =:+      "\\begin{equation} 2 + 3 \\end{equation}" =?>+      rawBlock "latex" "\\begin{equation} 2 + 3 \\end{equation}\n"++    , "LaTeX fragment with more arguments" =:+      T.unlines [ "\\begin{tikzcd}[ampersand replacement=\\&]"+                , "  A \\& B \\\\"+                , "  C \\& D"+                , "  \\arrow[from=1-1, to=1-2]"+                , "  \\arrow[\"f\", from=2-1, to=2-2]"+                , "\\end{tikzcd}"+                ] =?>+      rawBlock "latex"+      (T.unlines [ "\\begin{tikzcd}[ampersand replacement=\\&]"+                 , "  A \\& B \\\\"+                 , "  C \\& D"+                 , "  \\arrow[from=1-1, to=1-2]"+                 , "  \\arrow[\"f\", from=2-1, to=2-2]"+                 , "\\end{tikzcd}"+                 ])+     , "Convert blank lines in blocks to single newlines" =:       T.unlines [ "#+begin_html"                 , ""
test/Tests/Readers/Org/Directive.hs view
@@ -250,7 +250,7 @@                     , "\\end{equation}"                     ] =?>           para (str "\\begin{equation}" <> softbreak <>-                str "f(x) = x^2" <> softbreak <>+                text "f(x) = x^2" <> softbreak <>                 str "\\end{equation}")         ]       ]
test/Tests/Readers/Org/Shared.hs view
@@ -21,6 +21,7 @@ import Data.Text (Text) import Tests.Helpers (ToString, purely, test) import Test.Tasty (TestTree)+import Test.Tasty.HUnit (HasCallStack) import Text.Pandoc (Pandoc, ReaderOptions (readerExtensions),                     def, getDefaultExtensions, readOrg) import Text.Pandoc.Builder (Inlines, smallcaps, space, spanWith, str)@@ -29,7 +30,7 @@ org = purely $ readOrg def{ readerExtensions = getDefaultExtensions "org" }  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test org
test/Tests/Readers/RST.hs view
@@ -16,6 +16,7 @@ import Data.Text (Text) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -25,7 +26,7 @@ rst = purely $ readRST def{ readerStandalone = True }  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test rst
test/Tests/Readers/Txt2Tags.hs view
@@ -17,6 +17,7 @@ import Data.Text (Text) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -30,7 +31,7 @@   readTxt2Tags def s  infix 4 =:-(=:) :: ToString c+(=:) :: (ToString c, HasCallStack)      => String -> (Text, c) -> TestTree (=:) = test t2t
test/Tests/Shared.hs view
@@ -14,11 +14,15 @@  import System.FilePath.Posix (joinPath) import Test.Tasty-import Test.Tasty.HUnit (assertBool, testCase, (@?=)) import Text.Pandoc.Arbitrary ()+import Test.Tasty.QuickCheck (testProperty) import Text.Pandoc.Builder import Text.Pandoc.Shared-import Text.Pandoc.Writers.Shared (toLegacyTable)+import Test.Tasty.HUnit+import Tests.Helpers+import Text.Pandoc+import Text.Pandoc.Writers.Shared+import qualified Data.Text as T  tests :: [TestTree] tests = [ testGroup "compactifyDL"@@ -29,7 +33,52 @@           ]         , testGroup "collapseFilePath" testCollapse         , testGroup "toLegacyTable" testLegacyTable+        , testGroup "table of contents" testTOC+        , testGroup "makeSections"+          [ testProperty "makeSections is idempotent" makeSectionsIsIdempotent+          , testCase "makeSections is idempotent for test case" $+              let d = header 1 "H1" <> header 2 "H2" <> header 3 "H3" <>+                      header 2 "H2a" <> header 4 "H4" <> header 1 "H1a"+                  d' = makeSections False Nothing $ toList d+               in assertBool "makeSections is idempotent for test case"+                      (makeSections False Nothing d' == d')+          ]         ]++makeSectionsIsIdempotent :: [Block] -> Bool+makeSectionsIsIdempotent d =+  let d' = makeSections False Nothing d+   in d' == makeSections False Nothing d'++givesTOC :: String -> (Blocks, Blocks) -> TestTree+givesTOC desc (blocks, toc) = test (toTableOfContents def) desc (toList blocks, head . toList $ toc)++linkId :: T.Text -> T.Text -> T.Text -> Inlines -> Inlines+linkId lId = linkWith (lId,[],[])++headerId :: T.Text -> Int -> Inlines -> Blocks+headerId hId = headerWith (hId,[],[])++testTOC :: [TestTree]+testTOC = [ givesTOC "empty case" $ mempty =?> bulletList []+          , givesTOC "no headers" $ horizontalRule =?> bulletList []+          , givesTOC "unlinked header" $+              header 1 "H1" =?>+              bulletList [plain "H1"]+          , givesTOC "linked header" $+              headerId "h1" 1 "H1" =?>+              bulletList [plain $ linkId "toc-h1" "#h1" "" "H1"]+          , givesTOC "nested headlines" $+              header 1 "H1a" <> header 2 "H2" =?>+              bulletList [plain "H1a" <> bulletList [plain "H2"]]+          , givesTOC "only referenced headers" $+              header 1 "H1a" <> headerId "h2" 2 "H2" =?>+              bulletList [plain "H1a" <> +                          bulletList [plain $ linkId "toc-h2" "#h2" "" "H2"]]+          , givesTOC "section id used as backup" $+              divWith ("sec",["section"],[]) (header 1 "H1") =?>+              bulletList [plain $ linkId "toc-sec" "#sec" "" "H1"]             +          ]  testCollapse :: [TestTree] testCollapse = map (testCase "collapse")
test/Tests/Writers/ConTeXt.hs view
@@ -3,6 +3,7 @@  import Data.Text (unpack, pack) import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Test.Tasty.QuickCheck import Tests.Helpers import Text.Pandoc@@ -34,7 +35,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test context
test/Tests/Writers/Docbook.hs view
@@ -32,9 +32,11 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)-     => String -> (a, String) -> TestTree+(=:), testDb4, testDb5 :: (ToString a, ToPandoc a)+                       => String -> (a, String) -> TestTree (=:) = test docbook+testDb4 = test docbook+testDb5 = test docbook5  lineblock :: Blocks lineblock = para ("some text" <> linebreak <>@@ -47,7 +49,19 @@                 ]  tests :: [TestTree]-tests = [ testGroup "line blocks"+tests = [ testGroup "inline elements"+          [ testGroup "links"+            [ testDb4 "db4 external link" $ link "https://example.com" "" "Hello"+                                            =?> "<ulink url=\"https://example.com\">Hello</ulink>"+            , testDb5 "db5 external link" $ link "https://example.com" "" "Hello"+                                            =?> "<link xlink:href=\"https://example.com\">Hello</link>"+            , testDb5 "anchor"            $ link "#foo" "" "Hello"+                                            =?> "<link linkend=\"foo\">Hello</link>"+            , testDb5 "automatic anchor"  $ link "#foo" "" ""+                                            =?> "<xref linkend=\"foo\"></xref>"+            ]+          ]+        , testGroup "line blocks"           [ "none"       =: para "This is a test"                               =?> unlines                                     [ "<para>"
test/Tests/Writers/FB2.hs view
@@ -2,6 +2,7 @@ module Tests.Writers.FB2 (tests) where  import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -12,7 +13,7 @@         "<FictionBook xmlns=\"http://www.gribuser.ru/xml/fictionbook/2.0\" xmlns:l=\"http://www.w3.org/1999/xlink\"><description><title-info><genre>unrecognised</genre></title-info><document-info><program-used>pandoc</program-used></document-info></description><body><title><p /></title><section>" ++ x ++ "</section></body></FictionBook>"  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test (purely (writeFB2 def) . toPandoc)
test/Tests/Writers/HTML.hs view
@@ -4,6 +4,7 @@ import Data.Text (unpack) import qualified Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -33,7 +34,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test html
test/Tests/Writers/JATS.hs view
@@ -3,6 +3,7 @@  import Data.Text (Text) import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -31,7 +32,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, Text) -> TestTree (=:) = test jats
test/Tests/Writers/LaTeX.hs view
@@ -3,6 +3,7 @@  import Data.Text (unpack) import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -33,7 +34,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test latex
test/Tests/Writers/Markdown.hs view
@@ -4,6 +4,7 @@  import Data.Text (unpack) import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -33,7 +34,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test markdown @@ -182,7 +183,7 @@ shortcutLinkRefsTests :: TestTree shortcutLinkRefsTests =   let infix 4 =:-      (=:) :: (ToString a, ToPandoc a)+      (=:) :: (ToString a, ToPandoc a, HasCallStack)          => String -> (a, String) -> TestTree       (=:) = test (purely (writeMarkdown defopts{writerReferenceLinks = True}) . toPandoc)
test/Tests/Writers/Markua.hs view
@@ -2,6 +2,7 @@ module Tests.Writers.Markua (tests) where  import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -20,7 +21,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test (purely (writeMarkua def) . toPandoc)
test/Tests/Writers/Ms.hs view
@@ -2,12 +2,13 @@ module Tests.Writers.Ms (tests) where  import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Builder  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test (purely (writeMs def . toPandoc))
test/Tests/Writers/Muse.hs view
@@ -4,6 +4,7 @@ import Prelude hiding (unlines) import Data.Text (Text, unlines) import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -21,7 +22,7 @@ museWithOpts opts = purely (writeMuse opts) . toPandoc  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, Text) -> TestTree (=:) = test muse
test/Tests/Writers/Org.hs view
@@ -3,13 +3,14 @@  import Data.Text as T import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary () import Text.Pandoc.Builder  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, Text) -> TestTree (=:) = test org
test/Tests/Writers/Plain.hs view
@@ -2,6 +2,7 @@ module Tests.Writers.Plain (tests) where  import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -9,7 +10,7 @@   infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test (purely (writePlain def{ writerExtensions =                           enableExtension Ext_gutenberg plainExtensions }) .
test/Tests/Writers/RST.hs view
@@ -12,7 +12,7 @@ import qualified Data.Text as T  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test (purely (writeRST def . toPandoc))
test/Tests/Writers/TEI.hs view
@@ -2,6 +2,7 @@ module Tests.Writers.TEI (tests) where  import Test.Tasty+import Test.Tasty.HUnit (HasCallStack) import Tests.Helpers import Text.Pandoc import Text.Pandoc.Arbitrary ()@@ -20,7 +21,7 @@ -}  infix 4 =:-(=:) :: (ToString a, ToPandoc a)+(=:) :: (ToString a, ToPandoc a, HasCallStack)      => String -> (a, String) -> TestTree (=:) = test (purely (writeTEI def) . toPandoc)
test/command/2378.md view
@@ -13,15 +13,15 @@ ^D \begin{longtable}[]{@{}ll@{}} \caption{a table}\tabularnewline-\toprule+\toprule() x & y\footnote{a footnote} \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() x & y{} \\-\midrule+\midrule() \endhead 1 & 2 \\-\bottomrule+\bottomrule() \end{longtable} ```
test/command/4527.md view
@@ -1,24 +1,11 @@-# Raw TeX blocks in CommonMark with and without raw_tex-+This command comes through as regular text: ```-% pandoc -f latex+raw_tex -t commonmark-raw_tex-\someunknowncommand--Hello.+% pandoc -f latex+raw_tex -t native+\maketitle ^D-Hello.+[ RawBlock (Format "latex") "\\maketitle" ] ``` -```-% pandoc -f latex+raw_tex -t commonmark+raw_tex-\someunknowncommand--Hello.-^D-\someunknowncommand--Hello.-```  ``` % pandoc -f latex -t native
test/command/5367.md view
@@ -23,20 +23,20 @@ \begin{longtable}[]{@{}   >{\centering\arraybackslash}p{(\columnwidth - 0\tabcolsep) * \real{0.1667}}@{}} \caption[Sample table.]{Sample table.\footnote{caption footnote}}\tabularnewline-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Fruit\footnote{header footnote} \end{minipage} \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Fruit{} \end{minipage} \\-\midrule+\midrule() \endhead Bans\footnote{table cell footnote} \\-\bottomrule+\bottomrule() \end{longtable}  dolly\footnote{doc footnote}
+ test/command/6348.md view
@@ -0,0 +1,42 @@+```+% pandoc -f jats -t native+<?xml version="1.0" encoding="utf-8"?>+<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.0 20120330//EN" "http://jats.nlm.nih.gov/publishing/1.0/JATS-journalpublishing1.dtd">+<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" dtd-version="1.0" article-type="research-article">+  <front>+    <journal-meta>+      <journal-title-group>+        <journal-title>Opinion Research</journal-title>+      </journal-title-group>+    </journal-meta>+    <article-meta>+      <title-group>+        <article-title>Example article</article-title>+      </title-group>+    </article-meta>+  </front>+  <body>+    <sec sec-type="results">+      <title>Results</title>+      <p>A University <xref ref-type="fn" rid="N0001">1</xref></p>+    </sec>+  </body>+  <back>+    <fn-group>+      <fn id="N0001">+        <p>footnote</p>+      </fn>+    </fn-group>+  </back>+</article>+^D+[ Header 1 ( "" , [] , [] ) [ Str "Results" ]+, Para+    [ Str "A"+    , Space+    , Str "University"+    , Space+    , Note [ Para [ Str "footnote" ] ]+    ]+]+```
test/command/7042.md view
@@ -15,10 +15,11 @@   author:   - family: Negidius     given: Numerius+  - literal: others   container-title: Public Library of Tests   id: access-date   issued: 1911-10-03-  title: Entry with access date+  title: Entry with access date and et al.   type: article-journal - author:   - family: Beethoven@@ -49,7 +50,7 @@   id: full-journal-article-entry   issue: 4356   issued: '1953-04-01'-  pages: 737-738+  page: 737-738   pmid: 13054692   title: 'Molecular Structure of Nucleic Acids: A Structure for Deoxyribose Nucleic Acid'   type: article-journal@@ -78,67 +79,66 @@ </body> <back> <ref-list>-  <ref-list>-    <ref id="ref-year-month">-      <element-citation publication-type="article-journal">-        <person-group person-group-type="author">-          <name><surname>Jane</surname><given-names>Doe</given-names></name>-        </person-group>-        <article-title>Year and month</article-title>-        <source>Public Library of Tests</source>-        <year iso-8601-date="1999-08">1999</year><month>08</month>-      </element-citation>-    </ref>-    <ref id="ref-access-date">-      <element-citation publication-type="article-journal">-        <person-group person-group-type="author">-          <name><surname>Negidius</surname><given-names>Numerius</given-names></name>-        </person-group>-        <article-title>Entry with access date</article-title>-        <source>Public Library of Tests</source>-        <year iso-8601-date="1911-10-03">1911</year><month>10</month><day>03</day>-        <date-in-citation content-type="access-date"><year iso-8601-date="1999-01-22">1999</year><month>01</month><day>22</day></date-in-citation>-      </element-citation>-    </ref>-    <ref id="ref-name-particles">-      <element-citation publication-type="article-journal">-        <person-group person-group-type="author">-          <name><surname>Beethoven</surname><given-names>Ludwig van</given-names></name>-          <name><surname>de Bray</surname><given-names>Jan</given-names></name>-        </person-group>-        <article-title>Name particles, dropping and non-dropping</article-title>-        <source>Public Library of Tests</source>-        <year iso-8601-date="1820">1820</year>-      </element-citation>-    </ref>-    <ref id="ref-book-with-japanese-authors">-      <element-citation publication-type="book">-        <person-group person-group-type="author">-          <string-name>宮水 三葉</string-name>-          <string-name>立花 瀧</string-name>-        </person-group>-        <source>Big Book of Tests</source>-        <year iso-8601-date="2016">2016</year>-      </element-citation>-    </ref>-    <ref id="ref-full-journal-article-entry">-      <element-citation publication-type="article-journal">-        <person-group person-group-type="author">-          <name><surname>Watson</surname><given-names>J. D.</given-names></name>-          <name><surname>Crick</surname><given-names>F. H. C.</given-names></name>-        </person-group>-        <article-title>Molecular Structure of Nucleic Acids: A Structure for Deoxyribose Nucleic Acid</article-title>-        <source>Nature</source>-        <year iso-8601-date="1953-04-01">1953</year><month>04</month><day>01</day>-        <volume>171</volume>-        <issue>4356</issue>-        <fpage>737</fpage>-        <page-range>737-738</page-range>-        <pub-id pub-id-type="doi">10.1038/171737a0</pub-id>-        <pub-id pub-id-type="pmid">13054692</pub-id>-      </element-citation>-    </ref>-  </ref-list>+  <ref id="ref-year-month">+    <element-citation publication-type="article-journal">+      <person-group person-group-type="author">+        <name><surname>Jane</surname><given-names>Doe</given-names></name>+      </person-group>+      <article-title>Year and month</article-title>+      <source>Public Library of Tests</source>+      <year iso-8601-date="1999-08">1999</year><month>08</month>+    </element-citation>+  </ref>+  <ref id="ref-access-date">+    <element-citation publication-type="article-journal">+      <person-group person-group-type="author">+        <name><surname>Negidius</surname><given-names>Numerius</given-names></name>+        <etal/>+      </person-group>+      <article-title>Entry with access date and et al.</article-title>+      <source>Public Library of Tests</source>+      <year iso-8601-date="1911-10-03">1911</year><month>10</month><day>03</day>+      <date-in-citation content-type="access-date"><year iso-8601-date="1999-01-22">1999</year><month>01</month><day>22</day></date-in-citation>+    </element-citation>+  </ref>+  <ref id="ref-name-particles">+    <element-citation publication-type="article-journal">+      <person-group person-group-type="author">+        <name><surname>Beethoven</surname><given-names>Ludwig van</given-names></name>+        <name><surname>de Bray</surname><given-names>Jan</given-names></name>+      </person-group>+      <article-title>Name particles, dropping and non-dropping</article-title>+      <source>Public Library of Tests</source>+      <year iso-8601-date="1820">1820</year>+    </element-citation>+  </ref>+  <ref id="ref-book-with-japanese-authors">+    <element-citation publication-type="book">+      <person-group person-group-type="author">+        <string-name>宮水 三葉</string-name>+        <string-name>立花 瀧</string-name>+      </person-group>+      <source>Big Book of Tests</source>+      <year iso-8601-date="2016">2016</year>+    </element-citation>+  </ref>+  <ref id="ref-full-journal-article-entry">+    <element-citation publication-type="article-journal">+      <person-group person-group-type="author">+        <name><surname>Watson</surname><given-names>J. D.</given-names></name>+        <name><surname>Crick</surname><given-names>F. H. C.</given-names></name>+      </person-group>+      <article-title>Molecular Structure of Nucleic Acids: A Structure for Deoxyribose Nucleic Acid</article-title>+      <source>Nature</source>+      <year iso-8601-date="1953-04-01">1953</year><month>04</month><day>01</day>+      <volume>171</volume>+      <issue>4356</issue>+      <pub-id pub-id-type="doi">10.1038/171737a0</pub-id>+      <pub-id pub-id-type="pmid">13054692</pub-id>+      <fpage>737</fpage>+      <lpage>738</lpage>+    </element-citation>+  </ref> </ref-list> </back> </article>
+ test/command/7181.md view
@@ -0,0 +1,11 @@+```+% pandoc -t latex+![Global frog population.](slides.pdf){page=13,trim=1cm,clip,width=4cm}+^D+\begin{figure}+\centering+\includegraphics[page=13,trim=1cm,clip,width=4cm]{slides.pdf}+\caption{Global frog population.}+\end{figure}++```
test/command/7272.md view
@@ -16,12 +16,12 @@ ^D \begin{longtable}[]{@{}   >{\raggedright\arraybackslash}p{(\columnwidth - 0\tabcolsep) * \real{1.0000}}@{}}-\toprule+\toprule() \endhead \begin{minipage}[t]{\linewidth}\raggedright { text\\ text2 }\strut \end{minipage} \\-\bottomrule+\bottomrule() \end{longtable} ```
+ test/command/7884.md view
@@ -0,0 +1,72 @@+```+% pandoc -f html+epub_html_exts -t native+<body epub:type="bodymatter">+<section id="chapter-1" class="level1" data-number="1">+<h1 data-number="1">Chapter 1</h1>+<p>Here is a footnote.<a href="#fn1" class="footnote-ref" id="fnref1" epub:type="noteref">1</a></p>+<p>Here is a second footnote.<a href="#fn2" class="footnote-ref" id="fnref2" epub:type="noteref">2</a></p>+</section>+<section class="footnotes footnotes-end-of-document" epub:type="footnotes">+<hr />+<ol>+<li id="fn1" epub:type="footnote"><p>This is a test<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>+<li id="fn2" epub:type="footnote"><p>This is another test<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>+</ol>+</section>+</body>+^D+[ Div+    ( "chapter-1"+    , [ "section" , "level1" ]+    , [ ( "number" , "1" ) ]+    )+    [ Header+        1+        ( "chapter-1" , [] , [ ( "number" , "1" ) ] )+        [ Str "Chapter" , Space , Str "1" ]+    , Para+        [ Str "Here"+        , Space+        , Str "is"+        , Space+        , Str "a"+        , Space+        , Str "footnote."+        , Note+            [ Para+                [ Str "This"+                , Space+                , Str "is"+                , Space+                , Str "a"+                , Space+                , Str "test"+                ]+            ]+        ]+    , Para+        [ Str "Here"+        , Space+        , Str "is"+        , Space+        , Str "a"+        , Space+        , Str "second"+        , Space+        , Str "footnote."+        , Note+            [ Para+                [ Str "This"+                , Space+                , Str "is"+                , Space+                , Str "another"+                , Space+                , Str "test"+                ]+            ]+        ]+    ]+]++```
+ test/command/7894.md view
@@ -0,0 +1,109 @@+```+% pandoc -f ris -t csljson+TY  - BOOK+ID  - Chang_Keisler_Model_Theory+AU  - Chang, C. C.+AU  - Keisler, H. Jerome+PY  - 1990+ET  - 3+TI  - Model Theory+PU  - North-Holland Press+PP  - Amsterdam+KW  - model theory+KW  - logic+ER  -+TY  - JOUR+AU  - Shannon, Claude E.+PY  - 1948+DA  - July+TI  - A Mathematical Theory of Communication+T2  - Bell System Technical Journal+SP  - 379+EP  - 423+VL  - 27+ER  -+TY  - JOUR+T1  - On computable numbers, with an application to the Entscheidungsproblem+A1  - Turing, Alan Mathison+JO  - Proc. of London Mathematical Society+VL  - 47+IS  - 1+KW  - decidability+KW  - computability+SP  - 230+EP  - 265+Y1  - 1937+ER  -+^D+[+  {+    "author": [+      {+        "family": "Keisler",+        "given": "H. Jerome"+      },+      {+        "family": "Chang",+        "given": "C. C."+      }+    ],+    "edition": "3",+    "id": "Chang_Keisler_Model_Theory",+    "issued": {+      "date-parts": [+        [+          1990+        ]+      ]+    },+    "keyword": "logic, model theory",+    "publisher-place": "Amsterdam",+    "title": "Model Theory",+    "type": "book"+  },+  {+    "author": [+      {+        "family": "Shannon",+        "given": "Claude E."+      }+    ],+    "container-title": "Bell System Technical Journal",+    "id": "Shannon_1948",+    "issued": {+      "date-parts": [+        [+          1948+        ]+      ]+    },+    "page": "379-423",+    "title": "A Mathematical Theory of Communication",+    "type": "article-journal",+    "volume": "27"+  },+  {+    "author": [+      {+        "family": "Turing",+        "given": "Alan Mathison"+      }+    ],+    "container-title": "Proc. of London Mathematical Society",+    "id": "Turing_1937",+    "issue": "1",+    "issued": {+      "date-parts": [+        [+          1937+        ]+      ]+    },+    "keyword": "computability, decidability",+    "page": "230-265",+    "title": "On computable numbers, with an application to the Entscheidungsproblem",+    "type": "article-journal",+    "volume": "47"+  }+]+```
+ test/command/7919.md view
@@ -0,0 +1,118 @@+```+% pandoc -f markdown+single column table | +------------------- | +item 1              |+item 2              |+^D+<table>+<thead>+<tr class="header">+<th>single column table</th>+</tr>+</thead>+<tbody>+<tr class="odd">+<td>item 1</td>+</tr>+<tr class="even">+<td>item 2</td>+</tr>+</tbody>+</table>+```++```+% pandoc -f markdown+| single column table+| -------------------+| item 1+| item 2+^D+<table>+<thead>+<tr class="header">+<th>single column table</th>+</tr>+</thead>+<tbody>+<tr class="odd">+<td>item 1</td>+</tr>+<tr class="even">+<td>item 2</td>+</tr>+</tbody>+</table>+```++```+% pandoc -f markdown+single column table+-------------------+item 1+item 2+^D+<h2 id="single-column-table">single column table</h2>+<p>item 1 item 2</p>+```++```+% pandoc -f gfm+single column table | +------------------- | +item 1              |+item 2              |+^D+<table>+<thead>+<tr class="header">+<th>single column table</th>+</tr>+</thead>+<tbody>+<tr class="odd">+<td>item 1</td>+</tr>+<tr class="even">+<td>item 2</td>+</tr>+</tbody>+</table>+```++```+% pandoc -f gfm+| single column table+| -------------------+| item 1+| item 2+^D+<table>+<thead>+<tr class="header">+<th>single column table</th>+</tr>+</thead>+<tbody>+<tr class="odd">+<td>item 1</td>+</tr>+<tr class="even">+<td>item 2</td>+</tr>+</tbody>+</table>+```++```+% pandoc -f gfm+single column table+-------------------+item 1+item 2+^D+<h2 id="single-column-table">single column table</h2>+<p>item 1 item 2</p>+```+
+ test/command/7920.md view
@@ -0,0 +1,20 @@+```+% pandoc -t native+[hi]{#123}+^D+[ Para [ Span ( "123" , [] , [] ) [ Str "hi" ] ] ]+```++```+% pandoc -t markdown+[hi]{#123}+^D+[hi]{#123}+```++```+% pandoc -t markdown+# Hi {#123}+^D+# Hi {#123}+```
+ test/command/7930.md view
@@ -0,0 +1,98 @@+```+% pandoc -f latex -t native+We discuss foobar in notes \ref{note:X} and \ref{note:Y}.++Foo.\footnote{\label{note:X}A note. See also note~\ref{note:Y}.}++Bar.\footnote{\label{note:Y}Another note. See also+note~\ref{note:X}}+^D+[ Para+    [ Str "We"+    , Space+    , Str "discuss"+    , Space+    , Str "foobar"+    , Space+    , Str "in"+    , Space+    , Str "notes"+    , Space+    , Link+        ( ""+        , []+        , [ ( "reference-type" , "ref" )+          , ( "reference" , "note:X" )+          ]+        )+        [ Str "1" ]+        ( "#note:X" , "" )+    , Space+    , Str "and"+    , Space+    , Link+        ( ""+        , []+        , [ ( "reference-type" , "ref" )+          , ( "reference" , "note:Y" )+          ]+        )+        [ Str "2" ]+        ( "#note:Y" , "" )+    , Str "."+    ]+, Para+    [ Str "Foo."+    , Note+        [ Para+            [ Span ( "note:X" , [] , [ ( "label" , "note:X" ) ] ) []+            , Str "A"+            , Space+            , Str "note."+            , Space+            , Str "See"+            , Space+            , Str "also"+            , Space+            , Str "note\160"+            , Link+                ( ""+                , []+                , [ ( "reference-type" , "ref" )+                  , ( "reference" , "note:Y" )+                  ]+                )+                [ Str "2" ]+                ( "#note:Y" , "" )+            , Str "."+            ]+        ]+    ]+, Para+    [ Str "Bar."+    , Note+        [ Para+            [ Span ( "note:Y" , [] , [ ( "label" , "note:Y" ) ] ) []+            , Str "Another"+            , Space+            , Str "note."+            , Space+            , Str "See"+            , Space+            , Str "also"+            , SoftBreak+            , Str "note\160"+            , Link+                ( ""+                , []+                , [ ( "reference-type" , "ref" )+                  , ( "reference" , "note:X" )+                  ]+                )+                [ Str "1" ]+                ( "#note:X" , "" )+            ]+        ]+    ]+]+```
+ test/command/7939.md view
@@ -0,0 +1,9 @@+```+% pandoc -f latex -t native+\def\|{\verb|}+\vbox{+\hbox{\|\begin{minipage}[t]%|}+}+^D+[ Para [ Code ( "" , [] , [] ) "\\begin{minipage}[t]%" ] ]+```
+ test/command/7941.md view
@@ -0,0 +1,6 @@+```+% pandoc -f commonmark_x -t commonmark_x+# Class setup+^D+# Class setup+```
+ test/command/7953.md view
@@ -0,0 +1,11 @@+```+% pandoc -f latex -t native+{foo\bgroup bar\egroup }+^D+[ Para+    [ Span+        ( "" , [] , [] )+        [ Str "foo" , Span ( "" , [] , [] ) [ Str "bar" ] ]+    ]+]+```
+ test/command/7965.md view
@@ -0,0 +1,54 @@+```+% pandoc -f markdown -t gfm+Watson and Crick (1953)++<div id="refs" class="references csl-bib-body hanging-indent">++<div id="ref-WatsonCrick1953" class="csl-entry">++Watson, J. D., and F. H. C. Crick. 1953. “Molecular Structure of Nucleic+Acids: A Structure for Deoxyribose Nucleic Acid.” *Nature* 171 (4356):+737–38. <https://doi.org/10.1038/171737a0>.++</div>++</div>+^D+Watson and Crick (1953)++<div id="refs" class="references csl-bib-body hanging-indent">++<div id="ref-WatsonCrick1953" class="csl-entry">++Watson, J. D., and F. H. C. Crick. 1953. “Molecular Structure of Nucleic+Acids: A Structure for Deoxyribose Nucleic Acid.” *Nature* 171 (4356):+737–38. <https://doi.org/10.1038/171737a0>.++</div>++</div>+```++```+% pandoc -f markdown -t gfm-raw_html+Watson and Crick (1953)++<div id="refs" class="references csl-bib-body hanging-indent">++<div id="ref-WatsonCrick1953" class="csl-entry">++Watson, J. D., and F. H. C. Crick. 1953. “Molecular Structure of Nucleic+Acids: A Structure for Deoxyribose Nucleic Acid.” *Nature* 171 (4356):+737–38. <https://doi.org/10.1038/171737a0>.++</div>++</div>+^D+Watson and Crick (1953)++Watson, J. D., and F. H. C. Crick. 1953. “Molecular Structure of Nucleic+Acids: A Structure for Deoxyribose Nucleic Acid.” *Nature* 171 (4356):+737–38. <https://doi.org/10.1038/171737a0>.+```+
+ test/command/8003.md view
@@ -0,0 +1,6 @@+```+% pandoc -f markdown+ascii_identifiers+# Işık+^D+<h1 id="isik">Işık</h1>+```
test/command/biblatex-crossref-nested.md view
@@ -2,7 +2,7 @@ % pandoc -f biblatex -t markdown -s @comment{ -Nested crossreferences (see biber manual v 1.7)+Nested cross-references (see biber manual v 1.7)  }
test/command/biblatex-title-and-shorttitle.md view
@@ -9,7 +9,7 @@  CSL “container-title-short” could also be set - from biblatex “shortjournal”-- for inbook, incollection etc. from the “shorttitle” field of the crossreferenced book, collection etc. entry (see item5, item6)+- for inbook, incollection etc. from the “shorttitle” field of the cross-referenced book, collection etc. entry (see item5, item6) ... but it might not really be worth it, “container-title-short” not being used once in my sample of 70+ CSL styles.  }
test/command/chicago-annotated-bibliography.csl view
@@ -104,7 +104,7 @@                   </names>                 </if>               </choose>-              <!--This includes page numers after the container author, e.g. for Introductions -->+              <!--This includes page numbers after the container author, e.g. for Introductions -->               <choose>                 <if variable="container-author author" match="all">                   <group delimiter=". ">
test/command/chicago-fullnote-bibliography.csl view
@@ -102,7 +102,7 @@                   </names>                 </if>               </choose>-              <!--This includes page numers after the container author, e.g. for Introductions -->+              <!--This includes page numbers after the container author, e.g. for Introductions -->               <choose>                 <if variable="container-author author" match="all">                   <group delimiter=". ">
test/command/chicago-note-bibliography.csl view
@@ -109,7 +109,7 @@                   </names>                 </if>               </choose>-              <!--This includes page numers after the container author, e.g. for Introductions -->+              <!--This includes page numbers after the container author, e.g. for Introductions -->               <choose>                 <if variable="container-author author" match="all">                   <group delimiter=". ">
test/command/din-1505-2.csl view
@@ -37,7 +37,7 @@             * correcting errors             known bugs, help appreciated             * Abbreviations (Mitarb.) & (Hrsg. der Reihe) are missing (contributor, seriesEditor labels)-            * "contributor" is not valid as a name variable , nevertheless it works but doesnt validate!+            * "contributor" is not valid as a name variable , nevertheless it works but doesn't validate!             * author etc. first names should not be abbreviated             * reviewed author seems to be not available as variable (newspaper article)             * thesis type does notoffer a PLACE or FACULTY as variable!
test/command/nested-table-to-asciidoc-6942.md view
@@ -1,4 +1,4 @@-A table within a table should be convertet into a table within table+A table within a table should be converted into a table within table  ``` % pandoc -f html -t asciidoc
test/command/tex-group.md view
@@ -8,5 +8,5 @@ hi \end{foo} ^D-<p><em>hi</em></p>+<p><span> <em>hi</em> </span></p> ```
test/command/toc.md view
@@ -16,12 +16,12 @@ ## e ::: ^D--   [A](#a)-    -   [b](#b)--   [B](#b-1)-    -   [b](#b-2)--   [E](#e)-    -   [e](#e-1)+-   [A](#a){#toc-a}+    -   [b](#b){#toc-b}+-   [B](#b-1){#toc-b-1}+    -   [b](#b-2){#toc-b-2}+-   [E](#e){#toc-e}+    -   [e](#e-1){#toc-e-1}  # A
test/docbook-reader.native view
@@ -4,8 +4,7 @@         fromList           [ ( "author"             , MetaList-                [ MetaInlines-                    [ Str "John" , SoftBreak , Str "MacFarlane" ]+                [ MetaInlines [ Str "John" , Space , Str "MacFarlane" ]                 , MetaInlines [ Str "Anonymous" ]                 ]             )
+ test/docx/mendeley_citations.docx view

binary file changed (absent → 17716 bytes)

+ test/docx/mendeley_citations_minus.native view
@@ -0,0 +1,318 @@+Pandoc+  Meta { unMeta = fromList [] }+  [ Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "Mendeley"+      , Space+      , Str "test"+      , Space+      , Str "(prefix"+      , Space+      , Str "Hadwen-Bennett"+      , Space+      , Str "et"+      , Space+      , Str "al.,"+      , Space+      , Str "2018,"+      , Space+      , Str "p."+      , Space+      , Str "123"+      , Space+      , Str "suffix)."+      ]+  , Para+      [ Str "Another"+      , Space+      , Str "test"+      , Space+      , Str "(prefix"+      , Space+      , Str "Seo,"+      , Space+      , Str "2019,"+      , Space+      , Str "pp."+      , Space+      , Str "10\8211\&20"+      , Space+      , Str "suffix)."+      ]+  , Para+      [ Str "The"+      , Space+      , Str "last"+      , Space+      , Str "test"+      , Space+      , Str "(Koh"+      , Space+      , Str "&"+      , Space+      , Str "Abbas,"+      , Space+      , Str "2015;"+      , Space+      , Str "Lee"+      , Space+      , Str "et"+      , Space+      , Str "al.,"+      , Space+      , Str "2011)."+      ]+  , Header 1 ( "references" , [] , [] ) [ Str "References" ]+  , Para+      [ Str "Hadwen-Bennett,"+      , Space+      , Str "A.,"+      , Space+      , Str "Sentance,"+      , Space+      , Str "S.,"+      , Space+      , Str "&"+      , Space+      , Str "Morrison,"+      , Space+      , Str "C."+      , Space+      , Str "(2018)."+      , Space+      , Str "Making"+      , Space+      , Str "Programming"+      , Space+      , Str "Accessible"+      , Space+      , Str "to"+      , Space+      , Str "Learners"+      , Space+      , Str "with"+      , Space+      , Str "Visual"+      , Space+      , Str "Impairments:"+      , Space+      , Str "A"+      , Space+      , Str "Literature"+      , Space+      , Str "Review."+      , Space+      , Emph+          [ Str "International"+          , Space+          , Str "Journal"+          , Space+          , Str "of"+          , Space+          , Str "Computer"+          , Space+          , Str "Science"+          , Space+          , Str "Education"+          , Space+          , Str "in"+          , Space+          , Str "Schools"+          ]+      , Str ","+      , Space+      , Emph [ Str "2" ]+      , Str "(2),"+      , Space+      , Str "3\8211\&13."+      , Space+      , Str+          "https://www.microsoft.com/en-us/research/publication/making-programming-accessible-to-learners-with-visual-impairments-a-literature-review/"+      ]+  , Para+      [ Str "Koh,"+      , Space+      , Str "K.,"+      , Space+      , Str "&"+      , Space+      , Str "Abbas,"+      , Space+      , Str "J."+      , Space+      , Str "(2015)."+      , Space+      , Str "Future"+      , Space+      , Str "of"+      , Space+      , Str "library"+      , Space+      , Str "and"+      , Space+      , Str "museum"+      , Space+      , Str "services"+      , Space+      , Str "supporting"+      , Space+      , Str "teen"+      , Space+      , Str "learning:"+      , Space+      , Str "Perceptions"+      , Space+      , Str "of"+      , Space+      , Str "professionals"+      , Space+      , Str "in"+      , Space+      , Str "learning"+      , Space+      , Str "labs"+      , Space+      , Str "and"+      , Space+      , Str "makerspaces."+      , Space+      , Emph+          [ Str "Journal"+          , Space+          , Str "of"+          , Space+          , Str "Research"+          , Space+          , Str "on"+          , Space+          , Str "Libraries"+          , Space+          , Str "\\&"+          , Space+          , Str "Young"+          , Space+          , Str "Adults"+          ]+      , Str ","+      , Space+      , Emph [ Str "6" ]+      , Str "(4)."+      ]+  , Para+      [ Str "Lee,"+      , Space+      , Str "I.,"+      , Space+      , Str "Martin,"+      , Space+      , Str "F.,"+      , Space+      , Str "Denner,"+      , Space+      , Str "J.,"+      , Space+      , Str "Coulter,"+      , Space+      , Str "B.,"+      , Space+      , Str "Allan,"+      , Space+      , Str "W.,"+      , Space+      , Str "Erickson,"+      , Space+      , Str "J.,"+      , Space+      , Str "Malyn-Smith,"+      , Space+      , Str "J.,"+      , Space+      , Str "&"+      , Space+      , Str "Werner,"+      , Space+      , Str "L."+      , Space+      , Str "(2011)."+      , Space+      , Str "Computational"+      , Space+      , Str "Thinking"+      , Space+      , Str "for"+      , Space+      , Str "Youth"+      , Space+      , Str "in"+      , Space+      , Str "Practice."+      , Space+      , Emph [ Str "ACM" , Space , Str "Inroads" ]+      , Str ","+      , Space+      , Emph [ Str "2" ]+      , Str "(1),"+      , Space+      , Str "32\8211\&37."+      , Space+      , Str "https://doi.org/10.1145/1929887.1929902"+      ]+  , Para+      [ Str "Seo,"+      , Space+      , Str "J."+      , Space+      , Str "(2019)."+      , Space+      , Str "Is"+      , Space+      , Str "the"+      , Space+      , Str "Maker"+      , Space+      , Str "Movement"+      , Space+      , Str "Inclusive"+      , Space+      , Str "of"+      , Space+      , Str "{ANYONE}?:"+      , Space+      , Str "Three"+      , Space+      , Str "Accessibility"+      , Space+      , Str "Considerations"+      , Space+      , Str "to"+      , Space+      , Str "Invite"+      , Space+      , Str "Blind"+      , Space+      , Str "Makers"+      , Space+      , Str "to"+      , Space+      , Str "the"+      , Space+      , Str "Making"+      , Space+      , Str "World."+      , Space+      , Emph [ Str "TechTrends" ]+      , Str ","+      , Space+      , Emph [ Str "63" ]+      , Str "(5),"+      , Space+      , Str "514\8211\&520."+      , Space+      , Str "https://doi.org/10.1007/s11528-019-00377-3"+      ]+  ]
+ test/docx/mendeley_citations_plus.native view
@@ -0,0 +1,369 @@+Pandoc+  Meta+    { unMeta =+        fromList+          [ ( "references"+            , MetaList+                [ MetaMap+                    (fromList+                       [ ( "DOI"+                         , MetaInlines [ Str "10.1145/1929887.1929902" ]+                         )+                       , ( "ISSN" , MetaInlines [ Str "2153-2184" ] )+                       , ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Lee" )+                                    , ( "given" , MetaString "Irene" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Martin" )+                                    , ( "given" , MetaString "Fred" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Denner" )+                                    , ( "given" , MetaString "Jill" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family"+                                      , MetaString "Coulter"+                                      )+                                    , ( "given" , MetaString "Bob" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Allan" )+                                    , ( "given" , MetaString "Walter" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family"+                                      , MetaString "Erickson"+                                      )+                                    , ( "given" , MetaString "Jeri" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family"+                                      , MetaString "Malyn-Smith"+                                      )+                                    , ( "given" , MetaString "Joyce" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Werner" )+                                    , ( "given" , MetaString "Linda" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines+                             [ Str "ACM" , Space , Str "Inroads" ]+                         )+                       , ( "id" , MetaString "ITEM-1" )+                       , ( "issue" , MetaString "1" )+                       , ( "issued" , MetaString "2011-02" )+                       , ( "page" , MetaString "32-37" )+                       , ( "publisher" , MetaInlines [ Str "ACM" ] )+                       , ( "publisher-place"+                         , MetaInlines+                             [ Str "New"+                             , Space+                             , Str "York,"+                             , Space+                             , Str "NY,"+                             , Space+                             , Str "USA"+                             ]+                         )+                       , ( "title"+                         , MetaInlines+                             [ Str "Computational"+                             , Space+                             , Str "Thinking"+                             , Space+                             , Str "for"+                             , Space+                             , Str "Youth"+                             , Space+                             , Str "in"+                             , Space+                             , Str "Practice"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaString "2" )+                       ])+                , MetaMap+                    (fromList+                       [ ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Koh" )+                                    , ( "given"+                                      , MetaString "Kyungwon"+                                      )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "family" , MetaString "Abbas" )+                                    , ( "given" , MetaString "June" )+                                    , ( "non-dropping-particle"+                                      , MetaString ""+                                      )+                                    , ( "suffix" , MetaString "" )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines+                             [ Str "Journal"+                             , Space+                             , Str "of"+                             , Space+                             , Str "Research"+                             , Space+                             , Str "on"+                             , Space+                             , Str "Libraries"+                             , Space+                             , Str "\\&"+                             , Space+                             , Str "Young"+                             , Space+                             , Str "Adults"+                             ]+                         )+                       , ( "id" , MetaString "ITEM-2" )+                       , ( "issue" , MetaString "4" )+                       , ( "issued" , MetaString "2015" )+                       , ( "title"+                         , MetaInlines+                             [ Str "Future"+                             , Space+                             , Str "of"+                             , Space+                             , Str "library"+                             , Space+                             , Str "and"+                             , Space+                             , Str "museum"+                             , Space+                             , Str "services"+                             , Space+                             , Str "supporting"+                             , Space+                             , Str "teen"+                             , Space+                             , Str "learning:"+                             , Space+                             , Str "Perceptions"+                             , Space+                             , Str "of"+                             , Space+                             , Str "professionals"+                             , Space+                             , Str "in"+                             , Space+                             , Str "learning"+                             , Space+                             , Str "labs"+                             , Space+                             , Str "and"+                             , Space+                             , Str "makerspaces"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaString "6" )+                       ])+                ]+            )+          ]+    }+  [ Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "Mendeley"+      , Space+      , Str "test"+      , Space+      , Cite+          [ Citation+              { citationId = "ITEM-1"+              , citationPrefix = [ Str "prefix" ]+              , citationSuffix =+                  [ Str ",", Space , Str "123" , Space , Str "suffix" ]+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          ]+          [ Str "(prefix"+          , Space+          , Str "Hadwen-Bennett"+          , Space+          , Str "et"+          , Space+          , Str "al.,"+          , Space+          , Str "2018,"+          , Space+          , Str "p."+          , Space+          , Str "123"+          , Space+          , Str "suffix)"+          ]+      , Str "."+      ]+  , Para+      [ Str "Another"+      , Space+      , Str "test"+      , Space+      , Cite+          [ Citation+              { citationId = "ITEM-1"+              , citationPrefix = [ Str "prefix" ]+              , citationSuffix =+                  [ Str ",", Space , Str "10-20" , Space , Str "suffix" ]+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          ]+          [ Str "(prefix"+          , Space+          , Str "Seo,"+          , Space+          , Str "2019,"+          , Space+          , Str "pp."+          , Space+          , Str "10\8211\&20"+          , Space+          , Str "suffix)"+          ]+      , Str "."+      ]+  , Para+      [ Str "The"+      , Space+      , Str "last"+      , Space+      , Str "test"+      , Space+      , Cite+          [ Citation+              { citationId = "ITEM-1"+              , citationPrefix = []+              , citationSuffix = []+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          , Citation+              { citationId = "ITEM-2"+              , citationPrefix = []+              , citationSuffix = []+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          ]+          [ Str "(Koh"+          , Space+          , Str "&"+          , Space+          , Str "Abbas,"+          , Space+          , Str "2015;"+          , Space+          , Str "Lee"+          , Space+          , Str "et"+          , Space+          , Str "al.,"+          , Space+          , Str "2011)"+          ]+      , Str "."+      ]+  , Header 1 ( "references" , [] , [] ) [ Str "References" ]+  ]
+ test/docx/zotero_citations.docx view

binary file changed (absent → 14943 bytes)

+ test/docx/zotero_citations_minus.native view
@@ -0,0 +1,335 @@+Pandoc+  Meta { unMeta = fromList [] }+  [ Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "citation"+      , Space+      , Str "(Smith"+      , Space+      , Str "et"+      , Space+      , Str "al.,"+      , Space+      , Str "2013)"+      ]+  , Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "citation"+      , Space+      , Str "with"+      , Space+      , Str "two"+      , Space+      , Str "sources"+      , Space+      , Str "(Boer"+      , Space+      , Str "et"+      , Space+      , Str "al.,"+      , Space+      , Str "2016;"+      , Space+      , Str "Smith"+      , Space+      , Str "et"+      , Space+      , Str "al.,"+      , Space+      , Str "2013)."+      ]+  , Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "citation"+      , Space+      , Str "with"+      , Space+      , Str "additional"+      , Space+      , Str "prefix,"+      , Space+      , Str "suffix"+      , Space+      , Str "and"+      , Space+      , Str "a"+      , Space+      , Str "page"+      , Space+      , Str "number"+      , Space+      , Str "(prefix"+      , Space+      , Str "Bellomo"+      , Space+      , Str "et"+      , Space+      , Str "al.,"+      , Space+      , Str "2016,"+      , Space+      , Str "p."+      , Space+      , Str "2"+      , Space+      , Str "suffix)."+      ]+  , Header+      1+      ( "reference-list" , [] , [] )+      [ Str "Reference" , Space , Str "list" ]+  , Para+      [ Str "Bellomo,"+      , Space+      , Str "K.,"+      , Space+      , Str "Clement,"+      , Space+      , Str "A."+      , Space+      , Str "C.,"+      , Space+      , Str "Murphy,"+      , Space+      , Str "L."+      , Space+      , Str "N.,"+      , Space+      , Str "Polvani,"+      , Space+      , Str "L."+      , Space+      , Str "M.,"+      , Space+      , Str "&"+      , Space+      , Str "Cane,"+      , Space+      , Str "M."+      , Space+      , Str "A."+      , Space+      , Str "(2016)."+      , Space+      , Str "New"+      , Space+      , Str "observational"+      , Space+      , Str "evidence"+      , Space+      , Str "for"+      , Space+      , Str "a"+      , Space+      , Str "positive"+      , Space+      , Str "cloud"+      , Space+      , Str "feedback"+      , Space+      , Str "that"+      , Space+      , Str "amplifies"+      , Space+      , Str "the"+      , Space+      , Str "Atlantic"+      , Space+      , Str "Multidecadal"+      , Space+      , Str "Oscillation."+      , Space+      , Emph+          [ Str "Geophys."+          , Space+          , Str "Res."+          , Space+          , Str "Lett."+          ]+      , Str ","+      , Space+      , Emph [ Str "43" ]+      , Str "(18),"+      , Space+      , Str "9852\8211\&9859."+      , Space+      , Str "https://doi.org/10.1002/2016GL069961"+      ]+  , Para+      [ Str "Boer,"+      , Space+      , Str "G."+      , Space+      , Str "J.,"+      , Space+      , Str "Smith,"+      , Space+      , Str "D."+      , Space+      , Str "M.,"+      , Space+      , Str "Cassou,"+      , Space+      , Str "C.,"+      , Space+      , Str "Doblas-Reyes,"+      , Space+      , Str "F.,"+      , Space+      , Str "Danabasoglu,"+      , Space+      , Str "G.,"+      , Space+      , Str "Kirtman,"+      , Space+      , Str "B.,"+      , Space+      , Str "Kushnir,"+      , Space+      , Str "Y.,"+      , Space+      , Str "Kimoto,"+      , Space+      , Str "M.,"+      , Space+      , Str "Meehl,"+      , Space+      , Str "G."+      , Space+      , Str "A.,"+      , Space+      , Str "Msadek,"+      , Space+      , Str "R.,"+      , Space+      , Str "M\252ller,"+      , Space+      , Str "W."+      , Space+      , Str "A.,"+      , Space+      , Str "Taylor,"+      , Space+      , Str "K."+      , Space+      , Str "E.,"+      , Space+      , Str "Zwiers,"+      , Space+      , Str "F.,"+      , Space+      , Str "Rixen,"+      , Space+      , Str "M.,"+      , Space+      , Str "Ruprich-Robert,"+      , Space+      , Str "Y.,"+      , Space+      , Str "&"+      , Space+      , Str "Eade,"+      , Space+      , Str "R."+      , Space+      , Str "(2016)."+      , Space+      , Str "The"+      , Space+      , Str "Decadal"+      , Space+      , Str "Climate"+      , Space+      , Str "Prediction"+      , Space+      , Str "Project"+      , Space+      , Str "(DCPP)."+      , Space+      , Emph+          [ Str "Geoscientific"+          , Space+          , Str "Model"+          , Space+          , Str "Development"+          ]+      , Str ","+      , Space+      , Emph [ Str "9" ]+      , Str ","+      , Space+      , Str "3751\8211\&3777."+      , Space+      , Str "https://doi.org/10.5194/gmd-9-3751-2016"+      ]+  , Para+      [ Str "Smith,"+      , Space+      , Str "D."+      , Space+      , Str "M.,"+      , Space+      , Str "Eade,"+      , Space+      , Str "R.,"+      , Space+      , Str "&"+      , Space+      , Str "Pohlmann,"+      , Space+      , Str "H."+      , Space+      , Str "(2013)."+      , Space+      , Str "A"+      , Space+      , Str "comparison"+      , Space+      , Str "of"+      , Space+      , Str "full-field"+      , Space+      , Str "and"+      , Space+      , Str "anomaly"+      , Space+      , Str "initialization"+      , Space+      , Str "for"+      , Space+      , Str "seasonal"+      , Space+      , Str "to"+      , Space+      , Str "decadal"+      , Space+      , Str "climate"+      , Space+      , Str "prediction."+      , Space+      , Emph [ Str "Climate" , Space , Str "Dynamics" ]+      , Str ","+      , Space+      , Emph [ Str "41" ]+      , Str "(11\8211\&12),"+      , Space+      , Str "3325\8211\&3338."+      , Space+      , Str "https://doi.org/10.1007/s00382-013-1683-2"+      ]+  ]
+ test/docx/zotero_citations_plus.native view
@@ -0,0 +1,443 @@+Pandoc+  Meta+    { unMeta =+        fromList+          [ ( "references"+            , MetaList+                [ MetaMap+                    (fromList+                       [ ( "DOI"+                         , MetaInlines [ Str "10.1002/2016GL069961" ]+                         )+                       , ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Bellomo"+                                      )+                                    , ( "given" , MetaString "K." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Clement"+                                      )+                                    , ( "given" , MetaString "A.C." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Murphy" )+                                    , ( "given" , MetaString "L.N." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Polvani"+                                      )+                                    , ( "given" , MetaString "L.M." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Cane" )+                                    , ( "given" , MetaString "M.A." )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines+                             [ Str "Geophys."+                             , Space+                             , Str "Res."+                             , Space+                             , Str "Lett."+                             ]+                         )+                       , ( "id" , MetaString "10" )+                       , ( "issue" , MetaString "18" )+                       , ( "issued" , MetaString "2016" )+                       , ( "language" , MetaInlines [ Str "en" ] )+                       , ( "page" , MetaString "9852\8211\&9859" )+                       , ( "title"+                         , MetaInlines+                             [ Str "New"+                             , Space+                             , Str "observational"+                             , Space+                             , Str "evidence"+                             , Space+                             , Str "for"+                             , Space+                             , Str "a"+                             , Space+                             , Str "positive"+                             , Space+                             , Str "cloud"+                             , Space+                             , Str "feedback"+                             , Space+                             , Str "that"+                             , Space+                             , Str "amplifies"+                             , Space+                             , Str "the"+                             , Space+                             , Str "Atlantic"+                             , Space+                             , Str "Multidecadal"+                             , Space+                             , Str "Oscillation"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaString "43" )+                       ])+                , MetaMap+                    (fromList+                       [ ( "DOI"+                         , MetaInlines+                             [ Str "10.1007/s00382-013-1683-2" ]+                         )+                       , ( "ISSN"+                         , MetaInlines+                             [ Str "0930-7575,"+                             , Space+                             , Str "1432-0894"+                             ]+                         )+                       , ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Smith" )+                                    , ( "given" , MetaString "Doug M." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Eade" )+                                    , ( "given" , MetaString "Rosie" )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Pohlmann"+                                      )+                                    , ( "given" , MetaString "Holger" )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines+                             [ Str "Climate" , Space , Str "Dynamics" ]+                         )+                       , ( "id" , MetaString "109" )+                       , ( "issue" , MetaString "11-12" )+                       , ( "issued" , MetaString "2013-12" )+                       , ( "language" , MetaInlines [ Str "en" ] )+                       , ( "page" , MetaString "3325\8211\&3338" )+                       , ( "title"+                         , MetaInlines+                             [ Str "A"+                             , Space+                             , Str "comparison"+                             , Space+                             , Str "of"+                             , Space+                             , Str "full-field"+                             , Space+                             , Str "and"+                             , Space+                             , Str "anomaly"+                             , Space+                             , Str "initialization"+                             , Space+                             , Str "for"+                             , Space+                             , Str "seasonal"+                             , Space+                             , Str "to"+                             , Space+                             , Str "decadal"+                             , Space+                             , Str "climate"+                             , Space+                             , Str "prediction"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaString "41" )+                       ])+                , MetaMap+                    (fromList+                       [ ( "DOI"+                         , MetaInlines [ Str "10.5194/gmd-9-3751-2016" ]+                         )+                       , ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Boer" )+                                    , ( "given" , MetaString "G.J." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Smith" )+                                    , ( "given" , MetaString "D.M." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Cassou" )+                                    , ( "given" , MetaString "C." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Doblas-Reyes"+                                      )+                                    , ( "given" , MetaString "F." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Danabasoglu"+                                      )+                                    , ( "given" , MetaString "G." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Kirtman"+                                      )+                                    , ( "given" , MetaString "B." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Kushnir"+                                      )+                                    , ( "given" , MetaString "Y." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Kimoto" )+                                    , ( "given" , MetaString "M." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Meehl" )+                                    , ( "given" , MetaString "G.A." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Msadek" )+                                    , ( "given" , MetaString "R." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "M\252ller"+                                      )+                                    , ( "given" , MetaString "W.A." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Taylor" )+                                    , ( "given" , MetaString "K.E." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Zwiers" )+                                    , ( "given" , MetaString "F." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Rixen" )+                                    , ( "given" , MetaString "M." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Ruprich-Robert"+                                      )+                                    , ( "given" , MetaString "Y." )+                                    ])+                             , MetaMap+                                 (fromList+                                    [ ( "family" , MetaString "Eade" )+                                    , ( "given" , MetaString "R." )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines+                             [ Str "Geoscientific"+                             , Space+                             , Str "Model"+                             , Space+                             , Str "Development"+                             ]+                         )+                       , ( "id" , MetaString "6" )+                       , ( "issued" , MetaString "2016" )+                       , ( "language" , MetaInlines [ Str "en" ] )+                       , ( "page" , MetaString "3751\8211\&3777" )+                       , ( "title"+                         , MetaInlines+                             [ Str "The"+                             , Space+                             , Str "Decadal"+                             , Space+                             , Str "Climate"+                             , Space+                             , Str "Prediction"+                             , Space+                             , Str "Project"+                             , Space+                             , Str "(DCPP)"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaString "9" )+                       ])+                ]+            )+          ]+    }+  [ Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "citation"+      , Space+      , Cite+          [ Citation+              { citationId = "109"+              , citationPrefix = []+              , citationSuffix = []+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          ]+          [ Str "(Smith"+          , Space+          , Str "et"+          , Space+          , Str "al.,"+          , Space+          , Str "2013)"+          ]+      ]+  , Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "citation"+      , Space+      , Str "with"+      , Space+      , Str "two"+      , Space+      , Str "sources"+      , Space+      , Cite+          [ Citation+              { citationId = "6"+              , citationPrefix = []+              , citationSuffix = []+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          , Citation+              { citationId = "109"+              , citationPrefix = []+              , citationSuffix = []+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          ]+          [ Str "(Boer"+          , Space+          , Str "et"+          , Space+          , Str "al.,"+          , Space+          , Str "2016;"+          , Space+          , Str "Smith"+          , Space+          , Str "et"+          , Space+          , Str "al.,"+          , Space+          , Str "2013)"+          ]+      , Str "."+      ]+  , Para+      [ Str "This"+      , Space+      , Str "is"+      , Space+      , Str "a"+      , Space+      , Str "citation"+      , Space+      , Str "with"+      , Space+      , Str "additional"+      , Space+      , Str "prefix,"+      , Space+      , Str "suffix"+      , Space+      , Str "and"+      , Space+      , Str "a"+      , Space+      , Str "page"+      , Space+      , Str "number"+      , Space+      , Cite+          [ Citation+              { citationId = "10"+              , citationPrefix = [ Str "prefix" ]+              , citationSuffix =+                  [ Str ",", Space , Str "2" , Space , Str "suffix" ]+              , citationMode = NormalCitation+              , citationNoteNum = 0+              , citationHash = 0+              }+          ]+          [ Str "(prefix"+          , Space+          , Str "Bellomo"+          , Space+          , Str "et"+          , Space+          , Str "al.,"+          , Space+          , Str "2016,"+          , Space+          , Str "p."+          , Space+          , Str "2"+          , Space+          , Str "suffix)"+          ]+      , Str "."+      ]+  , Header+      1+      ( "reference-list" , [] , [] )+      [ Str "Reference" , Space , Str "list" ]+  ]
+ test/endnotexml-reader.native view
@@ -0,0 +1,124 @@+Pandoc+  Meta+    { unMeta =+        fromList+          [ ( "nocite"+            , MetaInlines+                [ Cite+                    [ Citation+                        { citationId = "*"+                        , citationPrefix = []+                        , citationSuffix = []+                        , citationMode = NormalCitation+                        , citationNoteNum = 0+                        , citationHash = 0+                        }+                    ]+                    [ Str "[@*]" ]+                ]+            )+          , ( "references"+            , MetaList+                [ MetaMap+                    (fromList+                       [ ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Chapman"+                                      )+                                    , ( "given" , MetaString "A. G." )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines [ Str "Ecology" ]+                         )+                       , ( "id" , MetaString "13264" )+                       , ( "isbn" , MetaInlines [ Str "0012-9658" ] )+                       , ( "issued" , MetaString "1937" )+                       , ( "number" , MetaInlines [ Str "1" ] )+                       , ( "pages" , MetaInlines [ Str "93-105" ] )+                       , ( "title"+                         , MetaInlines+                             [ Str "An"+                             , SoftBreak+                             , Str "ecological"+                             , Space+                             , Str "basis"+                             , Space+                             , Str "for"+                             , Space+                             , Str "reforestation"+                             , Space+                             , Str "of"+                             , Space+                             , Str "submariginal"+                             , Space+                             , Str "lands"+                             , SoftBreak+                             , Str "in"+                             , Space+                             , Str "the"+                             , Space+                             , Str "Central"+                             , Space+                             , Str "Hardwood"+                             , Space+                             , Str "Region"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaInlines [ Str "18" ] )+                       ])+                , MetaMap+                    (fromList+                       [ ( "author"+                         , MetaList+                             [ MetaMap+                                 (fromList+                                    [ ( "family"+                                      , MetaString "Strohecker"+                                      )+                                    , ( "given" , MetaString "H. F." )+                                    ])+                             ]+                         )+                       , ( "container-title"+                         , MetaInlines [ Str "Ecology" ]+                         )+                       , ( "id" , MetaString "13265" )+                       , ( "isbn" , MetaInlines [ Str "0012-9658" ] )+                       , ( "issued" , MetaString "1937" )+                       , ( "number" , MetaInlines [ Str "1" ] )+                       , ( "pages" , MetaInlines [ Str "162-168" ] )+                       , ( "title"+                         , MetaInlines+                             [ Str "A"+                             , Space+                             , Str "survey"+                             , SoftBreak+                             , Str "of"+                             , Space+                             , Str "soil"+                             , Space+                             , Str "temperatures"+                             , Space+                             , Str "in"+                             , Space+                             , Str "the"+                             , Space+                             , Str "Chicago"+                             , Space+                             , Str "area"+                             ]+                         )+                       , ( "type" , MetaString "article-journal" )+                       , ( "volume" , MetaInlines [ Str "18" ] )+                       ])+                ]+            )+          ]+    }+  []
+ test/endnotexml-reader.xml view
@@ -0,0 +1,216 @@+<?xml version="1.0" encoding="utf-8"?>+<xml>+  <records>+    <record>+      <database name="EndNote Ecology.enl"+      path="/Users/LowDecarie/Dropbox/Meta presentation of stats/Data/EndNote Ecology.enl">+      EndNote Ecology.enl</database>+      <source-app name="EndNote" version="15.0">+      EndNote</source-app>+      <rec-number>13264</rec-number>+      <foreign-keys>+        <key app="EN" db-id="0xxzez5ectezp7e9zrnxp0dq5zrafepwxfe2">+        13264</key>+      </foreign-keys>+      <ref-type name="Journal Article">17</ref-type>+      <contributors>+        <authors>+          <author>+            <style face="normal" font="default" size="100%">+            Chapman, A. G.</style>+          </author>+        </authors>+      </contributors>+      <auth-address>+        <style face="normal" font="default" size="100%">Cent States+        Forest Expt Stn, Columbus, OH USA</style>+      </auth-address>+      <titles>+        <title>+          <style face="normal" font="default" size="100%">An+          ecological basis for reforestation of submariginal lands+          in the Central Hardwood Region</style>+        </title>+        <secondary-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </secondary-title>+        <alt-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </alt-title>+      </titles>+      <periodical>+        <full-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </full-title>+        <abbr-1>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </abbr-1>+      </periodical>+      <alt-periodical>+        <full-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </full-title>+        <abbr-1>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </abbr-1>+      </alt-periodical>+      <pages>+        <style face="normal" font="default" size="100%">+        93-105</style>+      </pages>+      <volume>+        <style face="normal" font="default" size="100%">18</style>+      </volume>+      <number>+        <style face="normal" font="default" size="100%">1</style>+      </number>+      <dates>+        <year>+          <style face="normal" font="default" size="100%">+          1937</style>+        </year>+        <pub-dates>+          <date>+            <style face="normal" font="default" size="100%">+            Jan</style>+          </date>+        </pub-dates>+      </dates>+      <isbn>+        <style face="normal" font="default" size="100%">+        0012-9658</style>+      </isbn>+      <accession-num>+        <style face="normal" font="default" size="100%">+        ISI:000200148800007</style>+      </accession-num>+      <notes>+        <style face="normal" font="default" size="100%">V02fg
Times+        Cited:5
Cited References Count:2</style>+      </notes>+      <urls>+        <related-urls>+          <url>+            <style face="normal" font="default" size="100%">&lt;Go+            to ISI&gt;://000200148800007</style>+          </url>+        </related-urls>+      </urls>+      <language>+        <style face="normal" font="default" size="100%">+        English</style>+      </language>+    </record>+    <record>+      <database name="EndNote Ecology.enl"+      path="/Users/LowDecarie/Dropbox/Meta presentation of stats/Data/EndNote Ecology.enl">+      EndNote Ecology.enl</database>+      <source-app name="EndNote" version="15.0">+      EndNote</source-app>+      <rec-number>13265</rec-number>+      <foreign-keys>+        <key app="EN" db-id="0xxzez5ectezp7e9zrnxp0dq5zrafepwxfe2">+        13265</key>+      </foreign-keys>+      <ref-type name="Journal Article">17</ref-type>+      <contributors>+        <authors>+          <author>+            <style face="normal" font="default" size="100%">+            Strohecker, H. F.</style>+          </author>+        </authors>+      </contributors>+      <auth-address>+        <style face="normal" font="default" size="100%">Univ+        Chicago, Chicago, IL USA</style>+      </auth-address>+      <titles>+        <title>+          <style face="normal" font="default" size="100%">A survey+          of soil temperatures in the Chicago area</style>+        </title>+        <secondary-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </secondary-title>+        <alt-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </alt-title>+      </titles>+      <periodical>+        <full-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </full-title>+        <abbr-1>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </abbr-1>+      </periodical>+      <alt-periodical>+        <full-title>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </full-title>+        <abbr-1>+          <style face="normal" font="default" size="100%">+          Ecology</style>+        </abbr-1>+      </alt-periodical>+      <pages>+        <style face="normal" font="default" size="100%">+        162-168</style>+      </pages>+      <volume>+        <style face="normal" font="default" size="100%">18</style>+      </volume>+      <number>+        <style face="normal" font="default" size="100%">1</style>+      </number>+      <dates>+        <year>+          <style face="normal" font="default" size="100%">+          1937</style>+        </year>+        <pub-dates>+          <date>+            <style face="normal" font="default" size="100%">+            Jan</style>+          </date>+        </pub-dates>+      </dates>+      <isbn>+        <style face="normal" font="default" size="100%">+        0012-9658</style>+      </isbn>+      <accession-num>+        <style face="normal" font="default" size="100%">+        ISI:000200148800014</style>+      </accession-num>+      <notes>+        <style face="normal" font="default" size="100%">V02fg
Times+        Cited:5
Cited References Count:14</style>+      </notes>+      <urls>+        <related-urls>+          <url>+            <style face="normal" font="default" size="100%">&lt;Go+            to ISI&gt;://000200148800014</style>+          </url>+        </related-urls>+      </urls>+      <language>+        <style face="normal" font="default" size="100%">+        English</style>+      </language>+    </record>+  </records>+</xml>
test/epub/wasteland.native view
@@ -4230,7 +4230,7 @@     )     [ Div         ( "wasteland-content.xhtml#rearnotes"-        , [ "section" , "rearnotes" ]+        , [ "rearnotes" ]         , []         )         [ Header
test/lhs-test.latex view
@@ -33,12 +33,6 @@   \KOMAoptions{parskip=half}} \makeatother \usepackage{xcolor}-\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available-\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}-\hypersetup{-  hidelinks,-  pdfcreator={LaTeX via pandoc}}-\urlstyle{same} % disable monospaced font for URLs \usepackage{color} \usepackage{fancyvrb} \newcommand{\VerbBar}{|}@@ -84,6 +78,12 @@ \ifLuaTeX   \usepackage{selnolig}  % disable illegal ligatures \fi+\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}+\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available+\urlstyle{same} % disable monospaced font for URLs+\hypersetup{+  hidelinks,+  pdfcreator={LaTeX via pandoc}}  \author{} \date{}
test/lhs-test.latex+lhs view
@@ -33,12 +33,6 @@   \KOMAoptions{parskip=half}} \makeatother \usepackage{xcolor}-\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available-\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}-\hypersetup{-  hidelinks,-  pdfcreator={LaTeX via pandoc}}-\urlstyle{same} % disable monospaced font for URLs \usepackage{listings} \newcommand{\passthrough}[1]{#1} \lstset{defaultdialect=[5.3]Lua}@@ -51,6 +45,12 @@ \ifLuaTeX   \usepackage{selnolig}  % disable illegal ligatures \fi+\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}+\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available+\urlstyle{same} % disable monospaced font for URLs+\hypersetup{+  hidelinks,+  pdfcreator={LaTeX via pandoc}}  \author{} \date{}
test/rst-reader.native view
@@ -1124,15 +1124,7 @@       , ( AlignDefault , ColWidthDefault )       , ( AlignDefault , ColWidthDefault )       ]-      (TableHead-         ( "" , [] , [] )-         [ Row-             ( "" , [] , [] )-             [ Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-             , Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-             , Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-             ]-         ])+      (TableHead ( "" , [] , [] ) [])       [ TableBody           ( "" , [] , [] )           (RowHeadColumns 0)@@ -1564,10 +1556,13 @@       , Math DisplayMath "\\alpha = \\beta"       ]   , Para-      [ Math-          DisplayMath-          "\\begin{aligned}\nE &= mc^2\\\\\nF &= \\pi E\n\\end{aligned}"-      , Math DisplayMath "F &= \\gamma \\alpha^2"+      [ Span+          ( "" , [] , [ ( "label" , "hithere" ) , ( "nowrap" , "" ) ] )+          [ Math+              DisplayMath+              "\\begin{aligned}\nE &= mc^2\\\\\nF &= \\pi E\n\\end{aligned}"+          , Math DisplayMath "F &= \\gamma \\alpha^2"+          ]       ]   , Para [ Str "All" , Space , Str "done." ]   , Header 1 ( "default-role" , [] , [] ) [ Str "Default-Role" ]
test/tables-rstsubset.native view
@@ -755,16 +755,7 @@     , ( AlignDefault , ColWidthDefault )     , ( AlignDefault , ColWidthDefault )     ]-    (TableHead-       ( "" , [] , [] )-       [ Row-           ( "" , [] , [] )-           [ Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-           , Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-           , Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-           , Cell ( "" , [] , [] ) AlignDefault (RowSpan 1) (ColSpan 1) []-           ]-       ])+    (TableHead ( "" , [] , [] ) [])     [ TableBody         ( "" , [] , [] )         (RowHeadColumns 0)
test/tables.latex view
@@ -2,49 +2,49 @@  \begin{longtable}[]{@{}rlcl@{}} \caption{Demonstration of simple table syntax.}\tabularnewline-\toprule+\toprule() Right & Left & Center & Default \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() Right & Left & Center & Default \\-\midrule+\midrule() \endhead 12 & 12 & 12 & 12 \\ 123 & 123 & 123 & 123 \\ 1 & 1 & 1 & 1 \\-\bottomrule+\bottomrule() \end{longtable}  Simple table without caption:  \begin{longtable}[]{@{}rlcl@{}}-\toprule+\toprule() Right & Left & Center & Default \\-\midrule+\midrule() \endhead 12 & 12 & 12 & 12 \\ 123 & 123 & 123 & 123 \\ 1 & 1 & 1 & 1 \\-\bottomrule+\bottomrule() \end{longtable}  Simple table indented two spaces:  \begin{longtable}[]{@{}rlcl@{}} \caption{Demonstration of simple table syntax.}\tabularnewline-\toprule+\toprule() Right & Left & Center & Default \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() Right & Left & Center & Default \\-\midrule+\midrule() \endhead 12 & 12 & 12 & 12 \\ 123 & 123 & 123 & 123 \\ 1 & 1 & 1 & 1 \\-\bottomrule+\bottomrule() \end{longtable}  Multiline table with caption:@@ -55,7 +55,7 @@   >{\raggedleft\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1625}}   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.3500}}@{}} \caption{Here's the caption. It may span multiple lines.}\tabularnewline-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Centered Header \end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright@@ -65,9 +65,9 @@ \end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright Default aligned \end{minipage} \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Centered Header \end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright@@ -77,11 +77,11 @@ \end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright Default aligned \end{minipage} \\-\midrule+\midrule() \endhead First & row & 12.0 & Example of a row that spans multiple lines. \\ Second & row & 5.0 & Here's another one. Note the blank line between rows. \\-\bottomrule+\bottomrule() \end{longtable}  Multiline table without caption:@@ -91,7 +91,7 @@   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1375}}   >{\raggedleft\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1625}}   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.3500}}@{}}-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Centered Header \end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright@@ -101,22 +101,22 @@ \end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright Default aligned \end{minipage} \\-\midrule+\midrule() \endhead First & row & 12.0 & Example of a row that spans multiple lines. \\ Second & row & 5.0 & Here's another one. Note the blank line between rows. \\-\bottomrule+\bottomrule() \end{longtable}  Table without column headers:  \begin{longtable}[]{@{}rlcr@{}}-\toprule+\toprule() \endhead 12 & 12 & 12 & 12 \\ 123 & 123 & 123 & 123 \\ 1 & 1 & 1 & 1 \\-\bottomrule+\bottomrule() \end{longtable}  Multiline table without column headers:@@ -126,9 +126,9 @@   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1375}}   >{\raggedleft\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1625}}   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.3500}}@{}}-\toprule+\toprule() \endhead First & row & 12.0 & Example of a row that spans multiple lines. \\ Second & row & 5.0 & Here's another one. Note the blank line between rows. \\-\bottomrule+\bottomrule() \end{longtable}
test/tables/nordics.latex view
@@ -4,7 +4,7 @@   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2000}}   >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2000}}@{}} \caption{States belonging to the \emph{Nordics.}}\tabularnewline-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Name \end{minipage} & \begin{minipage}[b]{\linewidth}\centering@@ -16,9 +16,9 @@ Area\\ (in km\textsuperscript{2})\strut \end{minipage} \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Name \end{minipage} & \begin{minipage}[b]{\linewidth}\centering@@ -30,7 +30,7 @@ Area\\ (in km\textsuperscript{2})\strut \end{minipage} \\-\midrule+\midrule() \endhead Denmark & Copenhagen & 5,809,502 & 43,094 \\ Finland & Helsinki & 5,537,364 & 338,145 \\@@ -38,5 +38,5 @@ Norway & Oslo & 5,372,191 & 323,802 \\ Sweden & Stockholm & 10,313,447 & 450,295 \\ Total & & 27,376,022 & 1,258,336 \\-\bottomrule+\bottomrule() \end{longtable}
test/tables/planets.latex view
@@ -1,16 +1,16 @@ \begin{longtable}[]{@{}cclrrrrrrrrl@{}} \caption{Data about the planets of our solar system.}\tabularnewline-\toprule+\toprule() \multicolumn{2}{c}{} & Name & Mass (10\^{}24kg) & Diameter (km) & Density (kg/m\^{}3) & Gravity (m/s\^{}2) & Length of day (hours) & Distance from Sun (10\^{}6km) & Mean temperature (C) & Number of moons & Notes \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() \multicolumn{2}{c}{} & Name & Mass (10\^{}24kg) & Diameter (km) & Density (kg/m\^{}3) & Gravity (m/s\^{}2) & Length of day (hours) & Distance from Sun (10\^{}6km) & Mean temperature (C) & Number of moons & Notes \\-\midrule+\midrule() \endhead \multicolumn{2}{c}{\multirow{4}{*}{Terrestrial planets}} & Mercury & 0.330 & 4,879 & 5427 & 3.7 & 4222.6 & 57.9 & 167 & 0 & Closest to the Sun \\@@ -26,5 +26,5 @@ & & Neptune & 102 & 49,528 & 1638 & 11.0 & 16.1 & 4495.1 & -200 & 14 & \\ \multicolumn{2}{c}{Dwarf planets} & Pluto & 0.0146 & 2,370 & 2095 & 0.7 & 153.3 & 5906.4 & -225 & 5 & Declassified as a planet in 2006. \\-\bottomrule+\bottomrule() \end{longtable}
test/tables/students.latex view
@@ -2,21 +2,21 @@   >{\raggedright\arraybackslash}p{(\columnwidth - 2\tabcolsep) * \real{0.5000}}   >{\raggedright\arraybackslash}p{(\columnwidth - 2\tabcolsep) * \real{0.5000}}@{}} \caption{List of Students}\tabularnewline-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Student ID \end{minipage} & \begin{minipage}[b]{\linewidth}\centering Name \end{minipage} \\-\midrule+\midrule() \endfirsthead-\toprule+\toprule() \begin{minipage}[b]{\linewidth}\centering Student ID \end{minipage} & \begin{minipage}[b]{\linewidth}\centering Name \end{minipage} \\-\midrule+\midrule() \endhead \multicolumn{2}{l}{Computer Science} \\ 3741255 & Jones, Martha \\@@ -27,5 +27,5 @@ \multicolumn{2}{l}{Astrophysics} \\ 4100332 & Petrov, Alexandra \\ 4100332 & Toyota, Hiroko \\-\bottomrule+\bottomrule() \end{longtable}
test/writer.latex view
@@ -34,15 +34,6 @@ \makeatother \usepackage{fancyvrb} \usepackage{xcolor}-\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available-\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}-\hypersetup{-  pdftitle={Pandoc Test Suite},-  pdfauthor={John MacFarlane; Anonymous},-  hidelinks,-  pdfcreator={LaTeX via pandoc}}-\urlstyle{same} % disable monospaced font for URLs-\VerbatimFootnotes % allow verbatim text in footnotes \usepackage{graphicx} \makeatletter \def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}@@ -57,8 +48,6 @@ \def\fps@figure{htbp} \makeatother \usepackage[normalem]{ulem}-% Avoid problems with \sout in headers with hyperref-\pdfstringdefDisableCommands{\renewcommand{\sout}{}} \setlength{\emergencystretch}{3em} % prevent overfull lines \providecommand{\tightlist}{%   \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}@@ -66,6 +55,15 @@ \ifLuaTeX   \usepackage{selnolig}  % disable illegal ligatures \fi+\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}+\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available+\urlstyle{same} % disable monospaced font for URLs+\VerbatimFootnotes % allow verbatim text in footnotes+\hypersetup{+  pdftitle={Pandoc Test Suite},+  pdfauthor={John MacFarlane; Anonymous},+  hidelinks,+  pdfcreator={LaTeX via pandoc}}  \title{Pandoc Test Suite} \author{John MacFarlane \and Anonymous}
test/writer.opml view
@@ -50,7 +50,7 @@ </outline> <outline text="Smart quotes, ellipses, dashes" _note="“Hello,” said the spider. “‘Shelob’ is my name.”&#10;&#10;‘A’, ‘B’, and ‘C’ are letters.&#10;&#10;‘Oak,’ ‘elm,’ and ‘beech’ are names of trees. So is ‘pine.’&#10;&#10;‘He said, “I want to go.”’ Were you alive in the 70’s?&#10;&#10;Here is some quoted ‘`code`’ and a “[quoted&#10;link](http://example.com/?foo=1&amp;bar=2)”.&#10;&#10;Some dashes: one—two — three—four — five.&#10;&#10;Dashes between numbers: 5–7, 255–66, 1987–1999.&#10;&#10;Ellipses…and…and….&#10;&#10;------------------------------------------------------------------------"> </outline>-<outline text="LaTeX" _note="-   &#10;-   2 + 2 = 4&#10;-   *x* ∈ *y*&#10;-   *α* ∧ *ω*&#10;-   223&#10;-   *p*-Tree&#10;-   Here’s some display math:&#10;    $$\\frac{d}{dx}f(x)=\\lim\_{h\\to 0}\\frac{f(x+h)-f(x)}{h}$$&#10;-   Here’s one that has a line break in it: *α* + *ω* × *x*².&#10;&#10;These shouldn’t be math:&#10;&#10;-   To get the famous equation, write `$e = mc^2$`.&#10;-   $22,000 is a *lot* of money. So is $34,000. (It worked if “lot” is&#10;    emphasized.)&#10;-   Shoes ($20) and socks ($5).&#10;-   Escaped `$`: $73 *this should be emphasized* 23$.&#10;&#10;Here’s a LaTeX table:&#10;&#10;------------------------------------------------------------------------">+<outline text="LaTeX" _note="-   &#10;-   2 + 2 = 4&#10;-   *x* ∈ *y*&#10;-   *α* ∧ *ω*&#10;-   223&#10;-   *p*-Tree&#10;-   Here’s some display math:&#10;    $$\frac{d}{dx}f(x)=\lim\_{h\to 0}\frac{f(x+h)-f(x)}{h}$$&#10;-   Here’s one that has a line break in it: *α* + *ω* × *x*².&#10;&#10;These shouldn’t be math:&#10;&#10;-   To get the famous equation, write `$e = mc^2$`.&#10;-   $22,000 is a *lot* of money. So is $34,000. (It worked if “lot” is&#10;    emphasized.)&#10;-   Shoes ($20) and socks ($5).&#10;-   Escaped `$`: $73 *this should be emphasized* 23$.&#10;&#10;Here’s a LaTeX table:&#10;&#10;------------------------------------------------------------------------"> </outline> <outline text="Special Characters" _note="Here is some unicode:&#10;&#10;-   I hat: Î&#10;-   o umlaut: ö&#10;-   section: §&#10;-   set membership: ∈&#10;-   copyright: ©&#10;&#10;AT&amp;T has an ampersand in their name.&#10;&#10;AT&amp;T is another way to write it.&#10;&#10;This &amp; that.&#10;&#10;4 &amp;lt; 5.&#10;&#10;6 &amp;gt; 5.&#10;&#10;Backslash: \\&#10;&#10;Backtick: \`&#10;&#10;Asterisk: \*&#10;&#10;Underscore: \_&#10;&#10;Left brace: {&#10;&#10;Right brace: }&#10;&#10;Left bracket: \[&#10;&#10;Right bracket: \]&#10;&#10;Left paren: (&#10;&#10;Right paren: )&#10;&#10;Greater-than: &amp;gt;&#10;&#10;Hash: \#&#10;&#10;Period: .&#10;&#10;Bang: !&#10;&#10;Plus: +&#10;&#10;Minus: -&#10;&#10;------------------------------------------------------------------------"> </outline>
test/writer.org view
@@ -9,51 +9,51 @@ --------------  * Headers-  :PROPERTIES:-  :CUSTOM_ID: headers-  :END:+:PROPERTIES:+:CUSTOM_ID: headers+:END: ** Level 2 with an [[/url][embedded link]]-   :PROPERTIES:-   :CUSTOM_ID: level-2-with-an-embedded-link-   :END:+:PROPERTIES:+:CUSTOM_ID: level-2-with-an-embedded-link+:END: *** Level 3 with /emphasis/-    :PROPERTIES:-    :CUSTOM_ID: level-3-with-emphasis-    :END:+:PROPERTIES:+:CUSTOM_ID: level-3-with-emphasis+:END: **** Level 4-     :PROPERTIES:-     :CUSTOM_ID: level-4-     :END:+:PROPERTIES:+:CUSTOM_ID: level-4+:END: ***** Level 5-      :PROPERTIES:-      :CUSTOM_ID: level-5-      :END:+:PROPERTIES:+:CUSTOM_ID: level-5+:END: * Level 1-  :PROPERTIES:-  :CUSTOM_ID: level-1-  :END:+:PROPERTIES:+:CUSTOM_ID: level-1+:END: ** Level 2 with /emphasis/-   :PROPERTIES:-   :CUSTOM_ID: level-2-with-emphasis-   :END:+:PROPERTIES:+:CUSTOM_ID: level-2-with-emphasis+:END: *** Level 3-    :PROPERTIES:-    :CUSTOM_ID: level-3-    :END:+:PROPERTIES:+:CUSTOM_ID: level-3+:END: with no blank line  ** Level 2-   :PROPERTIES:-   :CUSTOM_ID: level-2-   :END:+:PROPERTIES:+:CUSTOM_ID: level-2+:END: with no blank line  --------------  * Paragraphs-  :PROPERTIES:-  :CUSTOM_ID: paragraphs-  :END:+:PROPERTIES:+:CUSTOM_ID: paragraphs+:END: Here's a regular paragraph.  In Markdown 1.0.0 and earlier. Version 8. This line turns into a list item.@@ -68,40 +68,44 @@ --------------  * Block Quotes-  :PROPERTIES:-  :CUSTOM_ID: block-quotes-  :END:+:PROPERTIES:+:CUSTOM_ID: block-quotes+:END: E-mail style:  #+begin_quote-  This is a block quote. It is pretty short.+This is a block quote. It is pretty short.+ #+end_quote  #+begin_quote-  Code in a block quote:+Code in a block quote: -  #+begin_example-  sub status {-      print "working";-  }-  #+end_example+#+begin_example+sub status {+    print "working";+}+#+end_example -  A list:+A list: -  1. item one-  2. item two+1. item one+2. item two -  Nested block quotes:+Nested block quotes: -  #+begin_quote-    nested-  #+end_quote+#+begin_quote+nested -  #+begin_quote-    nested-  #+end_quote #+end_quote +#+begin_quote+nested++#+end_quote++#+end_quote+ This should not be a block quote: 2 > 1.  And a following paragraph.@@ -109,9 +113,9 @@ --------------  * Code Blocks-  :PROPERTIES:-  :CUSTOM_ID: code-blocks-  :END:+:PROPERTIES:+:CUSTOM_ID: code-blocks+:END: Code:  #+begin_example@@ -135,13 +139,13 @@ --------------  * Lists-  :PROPERTIES:-  :CUSTOM_ID: lists-  :END:+:PROPERTIES:+:CUSTOM_ID: lists+:END: ** Unordered-   :PROPERTIES:-   :CUSTOM_ID: unordered-   :END:+:PROPERTIES:+:CUSTOM_ID: unordered+:END: Asterisks tight:  - asterisk 1@@ -185,9 +189,9 @@ - Minus 3  ** Ordered-   :PROPERTIES:-   :CUSTOM_ID: ordered-   :END:+:PROPERTIES:+:CUSTOM_ID: ordered+:END: Tight:  1. First@@ -227,9 +231,9 @@ 3. Item 3.  ** Nested-   :PROPERTIES:-   :CUSTOM_ID: nested-   :END:+:PROPERTIES:+:CUSTOM_ID: nested+:END: - Tab   - Tab     - Tab@@ -256,9 +260,9 @@ 3. Third  ** Tabs and spaces-   :PROPERTIES:-   :CUSTOM_ID: tabs-and-spaces-   :END:+:PROPERTIES:+:CUSTOM_ID: tabs-and-spaces+:END: - this is a list item indented with tabs  - this is a list item indented with spaces@@ -268,9 +272,9 @@   - this is an example list item indented with spaces  ** Fancy list markers-   :PROPERTIES:-   :CUSTOM_ID: fancy-list-markers-   :END:+:PROPERTIES:+:CUSTOM_ID: fancy-list-markers+:END: 2) [@2] begins with 2  3) and now 3@@ -304,9 +308,9 @@ --------------  * Definition Lists-  :PROPERTIES:-  :CUSTOM_ID: definition-lists-  :END:+:PROPERTIES:+:CUSTOM_ID: definition-lists+:END: Tight using spaces:  - apple :: red fruit@@ -340,7 +344,8 @@   #+end_example    #+begin_quote-    orange block quote+  orange block quote+   #+end_quote  Multiple definitions, tight:@@ -372,9 +377,9 @@   2. sublist  * HTML Blocks-  :PROPERTIES:-  :CUSTOM_ID: html-blocks-  :END:+:PROPERTIES:+:CUSTOM_ID: html-blocks+:END: Simple block on one line:  foo@@ -529,9 +534,9 @@ --------------  * Inline Markup-  :PROPERTIES:-  :CUSTOM_ID: inline-markup-  :END:+:PROPERTIES:+:CUSTOM_ID: inline-markup+:END: This is /emphasized/, and so /is this/.  This is *strong*, and so *is this*.@@ -560,9 +565,9 @@ --------------  * Smart quotes, ellipses, dashes-  :PROPERTIES:-  :CUSTOM_ID: smart-quotes-ellipses-dashes-  :END:+:PROPERTIES:+:CUSTOM_ID: smart-quotes-ellipses-dashes+:END: "Hello," said the spider. "'Shelob' is my name."  'A', 'B', and 'C' are letters.@@ -583,9 +588,9 @@ --------------  * LaTeX-  :PROPERTIES:-  :CUSTOM_ID: latex-  :END:+:PROPERTIES:+:CUSTOM_ID: latex+:END: - \cite[22-23]{smith.1899} - \(2+2=4\) - \(x \in y\)@@ -615,9 +620,9 @@ --------------  * Special Characters-  :PROPERTIES:-  :CUSTOM_ID: special-characters-  :END:+:PROPERTIES:+:CUSTOM_ID: special-characters+:END: Here is some unicode:  - I hat: Î@@ -671,13 +676,13 @@ --------------  * Links-  :PROPERTIES:-  :CUSTOM_ID: links-  :END:+:PROPERTIES:+:CUSTOM_ID: links+:END: ** Explicit-   :PROPERTIES:-   :CUSTOM_ID: explicit-   :END:+:PROPERTIES:+:CUSTOM_ID: explicit+:END: Just a [[/url/][URL]].  [[/url/][URL and title]].@@ -697,9 +702,9 @@ [[][Empty]].  ** Reference-   :PROPERTIES:-   :CUSTOM_ID: reference-   :END:+:PROPERTIES:+:CUSTOM_ID: reference+:END: Foo [[/url/][bar]].  With [[/url/][embedded [brackets]]].@@ -723,9 +728,9 @@ Foo [[/url/][biz]].  ** With ampersands-   :PROPERTIES:-   :CUSTOM_ID: with-ampersands-   :END:+:PROPERTIES:+:CUSTOM_ID: with-ampersands+:END: Here's a [[http://example.com/?foo=1&bar=2][link with an ampersand in the URL]].  Here's a link with an amersand in the link text: [[http://att.com/][AT&T]].@@ -735,9 +740,9 @@ Here's an [[/script?foo=1&bar=2][inline link in pointy braces]].  ** Autolinks-   :PROPERTIES:-   :CUSTOM_ID: autolinks-   :END:+:PROPERTIES:+:CUSTOM_ID: autolinks+:END: With an ampersand: [[http://example.com/?foo=1&bar=2]]  - In a list?@@ -747,7 +752,8 @@ An e-mail address: [[mailto:nobody@nowhere.net][nobody@nowhere.net]]  #+begin_quote-  Blockquoted: [[http://example.com/]]+Blockquoted: [[http://example.com/]]+ #+end_quote  Auto-links should not occur here: =<http://example.com/>=@@ -759,9 +765,9 @@ --------------  * Images-  :PROPERTIES:-  :CUSTOM_ID: images-  :END:+:PROPERTIES:+:CUSTOM_ID: images+:END: From "Voyage dans la Lune" by Georges Melies (1902):  #+caption: lalune@@ -772,15 +778,16 @@ --------------  * Footnotes-  :PROPERTIES:-  :CUSTOM_ID: footnotes-  :END:+:PROPERTIES:+:CUSTOM_ID: footnotes+:END: Here is a footnote reference,[fn:1] and another.[fn:2] This should /not/ be a footnote reference, because it contains a space.[^my note] Here is an inline note.[fn:3]  #+begin_quote-  Notes can go in quotes.[fn:4]+Notes can go in quotes.[fn:4]+ #+end_quote  1. And in list items.[fn:5]
test/writer.rst view
@@ -4,7 +4,7 @@  :Author: John MacFarlane :Author: Anonymous-:Date:   July 17, 2006+:Date: July 17, 2006  .. role:: raw-latex(raw)    :format: latex
test/writers-lang-and-dir.latex view
@@ -33,13 +33,6 @@   \KOMAoptions{parskip=half}} \makeatother \usepackage{xcolor}-\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available-\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}-\hypersetup{-  pdflang={en},-  hidelinks,-  pdfcreator={LaTeX via pandoc}}-\urlstyle{same} % disable monospaced font for URLs \setlength{\emergencystretch}{3em} % prevent overfull lines \providecommand{\tightlist}{%   \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}@@ -68,6 +61,13 @@   \newenvironment{RTL}{\beginR}{\endR}   \newenvironment{LTR}{\beginL}{\endL} \fi+\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}+\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available+\urlstyle{same} % disable monospaced font for URLs+\hypersetup{+  pdflang={en},+  hidelinks,+  pdfcreator={LaTeX via pandoc}}  \author{} \date{}