diff --git a/AUTHORS.md b/AUTHORS.md --- a/AUTHORS.md +++ b/AUTHORS.md @@ -29,6 +29,7 @@ - Andreas Deininger - Andreas Lööw - Andreas Scherer +- Andrzej Rybczak - Andres Freund - Andrew Dunning - Andy Morris @@ -345,6 +346,7 @@ - Owen McGrath - Pablo Rodríguez - Pascal Wagler +- Patrick Creighton - Pau RE - Paul Rivier - Paul Tilley @@ -466,6 +468,7 @@ - William Lupton - William Rusnack - Winnie Hellmann +- Wleeaf - Wout Gevaert - Wrong-Code - Xavier Olive @@ -496,6 +499,7 @@ - dbecher-ito - ebiim - ech0 +- enieuwy - etclub - favonia - gemmaro diff --git a/MANUAL.txt b/MANUAL.txt --- a/MANUAL.txt +++ b/MANUAL.txt @@ -1,7 +1,7 @@ --- title: Pandoc User's Guide author: John MacFarlane -date: 2026-10-07 +date: 2026-10-25 --- # Synopsis @@ -146,7 +146,7 @@ [`babel`] (with `lang`). If `CJKmainfont` is set, [`xeCJK`] is needed if `xelatex` is used, else [`luatexja`] is needed if `lualatex` is used. [`framed`] is required if code is highlighted in a -scheme that use a colored background. The use of `xelatex` or +scheme that uses a colored background. The use of `xelatex` or `lualatex` as the PDF engine requires [`fontspec`]. `lualatex` uses [`selnolig`] and [`lua-ul`]. `xelatex` uses [`bidi`] (with the `dir` variable set). @@ -769,7 +769,7 @@ `all` includes all insertions, deletions, and comments, wrapped in spans with `insertion`, `deletion`, `comment-start`, and `comment-end` classes, respectively. The author and time of - change is included. `all` is useful for scripting: only + change are included. `all` is useful for scripting: only accepting changes from a certain reviewer, say, or before a certain date. If a paragraph is inserted or deleted, `track-changes=all` produces a span with the class @@ -781,7 +781,7 @@ : Extract images and other media contained in or linked from the source document to the path *DIR*, creating it if - necessary, and adjust the images references in the document + necessary, and adjust the image references in the document so they point to the extracted files. Media are downloaded, read from the file system, or extracted from a binary container (e.g. docx), as needed. The original file paths @@ -1255,7 +1255,7 @@ `--listings[=true|false]` : *Deprecated, use `--syntax-highlighting=idiomatic` or - `--syntax-highlighting=default` instead. + `--syntax-highlighting=default` instead.* Use the [`listings`] package for LaTeX code blocks. The package does not support multi-byte encoding for source code. To handle UTF-8 @@ -1504,7 +1504,7 @@ `` (from the document title), `` (from the document authors), `` (from the document date, which should be in [ISO 8601 format]), `` (from the `lang` - variable, or, if is not set, the locale), and `` (a randomly generated UUID). Any of these may be overridden by elements in the metadata file. @@ -1635,7 +1635,7 @@ : Set the `bibliography` field in the document's metadata to *FILE*, overriding any value set in the metadata. If you supply this argument multiple times, each *FILE* will be added to - bibliography. If *FILE* is a URL, it will be fetched + the bibliography. If *FILE* is a URL, it will be fetched via HTTP. If *FILE* is not found relative to the working directory, it will be sought in the resource path (see `--resource-path`). @@ -1689,7 +1689,7 @@ `mathjax`[`:`*URL*] - : Render math using [MathJaX]. TeX math will be put between + : Render math using [MathJax]. TeX math will be put between `\(...\)` (for inline math) or `\[...\]` (for display math) and wrapped in `` tags with class `math`. Then the MathJax JavaScript will render it. The *URL* @@ -2369,7 +2369,7 @@ | ``` | ``` | +----------------------------------+-----------------------------------+ -If the command line option accepts a URL argument, an `url:` field can +If the command line option accepts a URL argument, a `url:` field can be added to `math-method:`. ## Options for wrapper scripts @@ -2794,7 +2794,7 @@ - `roman`: Converts textual values that can be read as an integer into lowercase roman numerals. - This can be used to get lettered enumeration from array indices. + This can be used to get roman-numeral enumeration from array indices. To get uppercase roman, chain with `uppercase`. - `left n "leftborder" "rightborder"`: Renders a textual value @@ -2956,7 +2956,7 @@ `quotebordercolor`, `quotebordercolordark` : sets the CSS `border-color` property for the left-border of -: `blockquote` elements. + `blockquote` elements. `quotecolor`, `quotecolordark` : sets the CSS `color` property for `blockquote` elements. @@ -3492,7 +3492,7 @@ `pdfaintent` : when used in conjunction with `pdfa`, specifies the output intent for - the colors, e.g. `ISO coated v2 300\letterpercent\space (ECI)` + the colors, e.g. `ISO coated v2 300\letterpercent\space (ECI)`. If left unspecified, `sRGB IEC61966-2.1` is used as default. `toc` @@ -3616,7 +3616,7 @@ ### Variables for ms `fontfamily` -: `A` (Avant Garde), `B` (Bookman), `C` (Helvetica), `HN` +: `A` (Avant Garde), `B` (Bookman), `H` (Helvetica), `HN` (Helvetica Narrow), `P` (Palatino), or `T` (Times New Roman). This setting does not affect source code, which is always displayed using monospace Courier. These built-in fonts are @@ -3658,8 +3658,8 @@ The recognized formats for `date` are: `mm/dd/yyyy`, `mm/dd/yy`, `yyyy-mm-dd` (ISO 8601), `dd MM yyyy` (e.g. either `02 Apr 2018` or `02 April 2018`), - `MM dd, yyyy` (e.g. `Apr. 02, 2018` or `April 02, 2018), - `yyyy[mm[dd]]` (e.g. `20180402, `201804` or `2018`). + `MM dd, yyyy` (e.g. `Apr. 02, 2018` or `April 02, 2018`), + `yyyy[mm[dd]]` (e.g. `20180402`, `201804` or `2018`). `header-includes` : contents specified by `-H/--include-in-header` (may have multiple @@ -3771,7 +3771,7 @@ then in reading LaTeX pandoc will parse these characters literally. In writing LaTeX, enabling `smart` tells pandoc to use the ligatures when possible; if `smart` is disabled -pandoc will use unicode quotation mark and dash characters. +pandoc will use Unicode quotation mark and dash characters. ## Headings and sections @@ -4018,9 +4018,9 @@ substituted with cross-references that will use the number of the referenced item. The original link text is discarded. This extension can be combined with `xrefs_name` in which case -the name or caption numbers will appear after the number. +the name or caption will appear after the number. -For the `xrefs_number` to be useful heading numbers must be enabled +For the `xrefs_number` extension to be useful, heading numbers must be enabled in the generated document, also table and figure captions must be enabled using for example the `native_numbering` extension. @@ -4415,7 +4415,7 @@ ### Extension: `fenced_code_attributes` ### -Optionally, you may attach attributes to fenced or backtick code block using +Optionally, you may attach attributes to a fenced or backtick code block using this syntax: ~~~~ {#mycode .haskell .numberLines startFrom="100"} @@ -5090,7 +5090,7 @@ the number of dashes in the line separating the table header from the table body. (For example `---|-` would make the first column 3/4 and the second column 1/4 of the full text width.) -On the other hand, if no lines are wider than column width, then +On the other hand, if no lines are wider than the column width, then cell contents will not be wrapped, and the cells will be sized to their contents. @@ -5595,7 +5595,7 @@ FictionBook2 ~ If `--math-method=webtex` is used, formulas are rendered as images - using CodeCogs or other compatible web service, downloaded + using CodeCogs or another compatible web service, downloaded and embedded in the e-book. Otherwise, they will appear verbatim. HTML, Slidy, DZSlides, S5, EPUB @@ -5622,12 +5622,13 @@ For a more explicit way of including raw HTML in a Markdown document, see the [`raw_attribute` extension][Extension: `raw_attribute`]. -In the CommonMark format, if `raw_html` is enabled, superscripts, +In commonmark output, if `raw_html` is enabled, superscripts, subscripts, strikeouts and small capitals will be represented as HTML. -Otherwise, plain-text fallbacks will be used. Note that even if -`raw_html` is disabled, tables will be rendered with HTML syntax if -they cannot use pipe syntax. +Otherwise, plain-text fallbacks will be used. +Disabling the `raw_html` extension has no effect on `commonmark` (or +`gfm` or `commonmark_x`) *input*. + ### Extension: `markdown_in_html_blocks` ### Original Markdown allows you to include HTML "blocks": blocks @@ -5675,7 +5676,7 @@ ### Extension: `native_spans` ### -Use native pandoc `Span` blocks for content inside `` tags. +Use native pandoc `Span` elements for content inside `` tags. For the most part this should give the same output as `raw_html`, but it makes it easier to write pandoc filters to manipulate groups of inlines. @@ -5816,7 +5817,7 @@ The link consists of link text in square brackets, followed by a label in square brackets. (There cannot be space between the two unless the `spaced_reference_links` extension is enabled.) If the label is empty -(`[]`), then it will be implicitly be taken to be the same as the link text; +(`[]`), then it will implicitly be taken to be the same as the link text; thus `[foo][]` is equivalent to `[foo][foo]`. (If the `shortcut_reference_links` extension is enabled, the empty `[]` may be omitted.) @@ -5828,7 +5829,7 @@ extension is enabled): citations take precedence over link labels. -Here are some examples of reference links and link definitions; +Here are some examples of reference links and link definitions: See [the website *I* built][my website]. @@ -5977,7 +5978,7 @@ itself contains dpi information). - The `%` unit is generally relative to some available space. For example the above example will render to the following. - - HTML: `` + - HTML: `` - LaTeX: `\includegraphics[width=0.5\textwidth,height=\textheight]{file.jpg}` (If you're using a custom template, you need to configure `graphicx` as in the default template.) @@ -6159,7 +6160,7 @@ In complex cases, you can force something to be treated as a locator by enclosing it in curly braces or prevent parsing -the suffix as locator by prepending curly braces: +the suffix as a locator by prepending curly braces: [@smith{ii, A, D-Z}, with a suffix] [@smith, {pp. iv, vi-xi, (xv)-(xvii)} with suffix here] @@ -6250,7 +6251,7 @@ ### Extension: `mark` ### -To highlight out a section of text, begin and end it with +To highlight a section of text, begin and end it with `==`. Thus, for example, This ==is deleted text.== @@ -6493,7 +6494,7 @@ where `FORMAT` is replaced with the name of the format. Note that the list of extensions for `commonmark`, -`gfm`, and `commonmark_x` are defined relative to default +`gfm`, and `commonmark_x` is defined relative to default commonmark. So, for example, `backtick_code_blocks` does not appear as an extension, since it is enabled by default and cannot be disabled. @@ -6518,8 +6519,8 @@ `bibliography` metadata field in a YAML metadata section or the `--bibliography` command line argument. If you want to use multiple bibliography files, you can supply multiple -`--bibliography` arguments or set `bibliography` metadata field -to YAML array. A bibliography may have any of these formats: +`--bibliography` arguments or set the `bibliography` metadata field +to a YAML array. A bibliography may have any of these formats: Format File extension ------------ -------------- @@ -6800,8 +6801,8 @@ For LaTeX output, you can also use [`natbib`] or [`biblatex`] to render the bibliography. In order to do so, specify bibliography -files as outlined above, and add `--natbib` or `--biblatex` -argument to pandoc invocation. Bear in mind that bibliography +files as outlined above, and add the `--natbib` or `--biblatex` +argument to the pandoc invocation. Bear in mind that bibliography files have to be in either BibTeX (for `--natbib`) or BibLaTeX (for `--biblatex`) format. @@ -6815,7 +6816,7 @@ numerical styles only). Defaults to false. `link-bibliography` -: If true, DOIs, PMCIDs, PMID, and URLs in bibliographies will +: If true, DOIs, PMCIDs, PMIDs, and URLs in bibliographies will be rendered as hyperlinks. (If an entry contains a DOI, PMCID, PMID, or URL, but none of these fields are rendered by the style, then the title, or in the absence of a title the whole entry, will @@ -7690,7 +7691,7 @@ # Vimdoc -Vimdoc writer generates Vim help files and makes use of the following metadata +The Vimdoc writer generates Vim help files and makes use of the following metadata variables: ``` yaml @@ -7703,8 +7704,8 @@ vimdoc-prefix: pandoc ``` -Complete header requires `abstract`, `author`, `title` and `filename` to -be set. Compiling file with such metadata produces the following file +A complete header requires `abstract`, `author`, `title` and `filename` to +be set. Compiling a file with such metadata produces the following file (assumes `--standalone`, see [Templates]): ``` vimdoc @@ -7721,7 +7722,7 @@ ``` If `vimdoc-prefix` is set, all non-command tags are prefixed with its -value, it is used to prevent tag collision: all headers have a tag +value; this is used to prevent tag collision: all headers have a tag (either inferred or explicit) and multiple help pages can have the same header names, therefore collision is to be expected. Let our input be the following markdown file: @@ -7759,7 +7760,7 @@ inherently represent anything. ``` -Convert it to vimdoc with metadata variable set (e.g. with +Convert it to vimdoc with the metadata variable set (e.g. with `-M vimdoc-prefix=pandoc`) ``` vimdoc @@ -8073,12 +8074,12 @@ pandoc -t context+tagging doc.md -o doc.pdf A recent `context` version should be used, as older versions -contained a bug that lead to invalid PDF metadata. +contained a bug that led to invalid PDF metadata. ## WeasyPrint The HTML-based engine WeasyPrint includes experimental support for -PDF/A and PDF/UA since version 57. Tagged PDFs can created with +PDF/A and PDF/UA since version 57. Tagged PDFs can be created with pandoc --pdf-engine=weasyprint \ --pdf-engine-opt=--pdf-variant=pdf/ua-1 ... @@ -8151,7 +8152,7 @@ # A note on security 1. Although pandoc itself will not create or modify any files other - than those you explicitly ask it create (with the exception + than those you explicitly ask it to create (with the exception of temporary files used in producing PDFs), a filter or custom writer could in principle do anything on your file system. Please audit filters and custom writers very carefully before using them. @@ -8190,10 +8191,10 @@ For example, in `wkhtmltopdf` the `footer-html` and `header-html` metadata fields will be passed directly to `wkhtmltopdf` as `--footer-html` and `--header-html` arguments, and by using - a `file:` URIs with these, one could exfiltrate local files. + a `file:` URI with these, one could exfiltrate local files. In addition, using `--pdf-engine=wkhtmltopdf` with `-f html+raw_html` enables an SSRF vulnerability in wkhtmltopdf ([CVE-2022-35583]). - Note that [`wkhtmltopdf`) is + Note that [`wkhtmltopdf`] is [deprecated](https://wkhtmltopdf.org/status.html) and its authors recommend not to use it with any untrusted input. However, there may be similar issues with other pdf-engines. diff --git a/changelog.md b/changelog.md --- a/changelog.md +++ b/changelog.md @@ -1,5 +1,195 @@ # Revision history for pandoc +## pandoc 3.13 (2026-10-26) + + * Report the location of errors in defaults files (#8759, Andrzej Rybczak). + An invalid value in a defaults file gave an error at the start of the + file, with no indication of the field. The error now names the option and + points at its value. + + * Use yamlet for YAML parsing. Pandoc parsed YAML with the yaml + package, which depends on the C library libyaml. It now uses + yamlet, a pure Haskell YAML 1.2 parser. Values still go + through the existing aeson instances. Note these behavior changes: + + + `yes`, `no`, `on`, `off`, `y` and `n` are strings, + not booleans, as in YAML 1.2. + + YAML errors now show the file, the line and the column, with the + offending line. In metadata blocks the line is the line in the + document. + + Duplicate keys in a mapping are an error. They were a warning. + + `.5`, `.inf` and `.nan` are numbers. + + * Docx reader: + + + Read `w:fldSimple` fields (#11944, Aslak Hellesøy). + + Keep grid columns that have no width (#11989, Patrick Creighton). + `w:w` is optional on `w:gridCol`, but `elemToTblGrid` skipped the grid + columns that had none. The table then had fewer columns than its rows had + cells, and the extra cells were dropped. Every grid column is now kept. + If any of them has no (readable) width, all columns get the default + width. + + Fix namespaces in notes and comments (#11991, Patrick Creighton). + The contents of footnotes, endnotes and comments were read with the + namespace prefixes declared in word/document.xml rather than in their own + parts. + + Records the indentation of each list level and track the open + list levels; an unnumbered paragraph whose left indentation + matches the text indentation of an open level is treated as a continuation + of that level's current item, rather than as a block quote (cf. #9197). + + Recover from fields with a missing instruction (#11943, enieuwy). + This fixes a bug that could lead to content being omitted. + + Fix regression (from 3.11) in parsing column widths (#11969). + The regression caused only the first column to get + assigned its alignment; every other column received default alignment. + In addition, in 3.11 alignment was taken from the header row if present; + this changed behavior from earlier versions, which took it from the first + body row. We revert to the earlier behavior, using the header row only if + there are no body rows. It's common for the header to have different + alignment than the rest. + + Keep tracked paragraph marks inside notes (#11971, Aslak Hellesøy). + + * RST reader: + + + Allow inline markup and bare links after `:`, `/`, `'`, `"`, `<` + (Wleeaf). Fixes a regression since 3.2.1; these characters could no + longer precede an inline markup start-string or a bare URI or email + address, although the reStructuredText spec allows them. + Also implement the rule that a start-character between an opening + character and its closing counterpart, as in `(*)` or `"*"`, + does not trigger inline markup. + + * RTF reader: + + + Fix tokenization bug (#11981). We were consuming certain + symbols after `\uNNNN` during tokenization. This led to an + *extra* character being consumed as the "fallback," so + letters went missing after encoded unicode characters. + + * LaTeX reader: + + + Detect macro expansion loops across `doMacros` calls (#11088). + + * ODT reader: + + + Fix start numbers with `text:continue-numbering` (#9595). + + * Markdown writer: + + + Write wide tables as grid tables instead of falling back to HTML + (#11945, Aslak Hellesøy). + + Don't drop notes in notes. + + Fix regression with code block language (#11987). We try to + identify which class is a language name and emit that as the gfm + style info string. If none of the classes match a + recognized language, before 3.10.1 we emitted the first + class; fater 3.10.1, we emit nothing. This commit reverts to + the pre-3.10.1 behavior. + + * RST writer: + + + Don't drop notes in notes. + + * Haddock writer: + + + Don't drop notes in notes. + + * Docx writer: + + + Fix multi-paragraph list items (#9197). Previously the docx + writer gave the second and subsequent paragraphs of a list + item numbering from a separate list with an invisible bullet. + This looked right, but semantically these paragraphs were + items of another list, which is what Word, screen readers, + and pandoc's own docx reader saw; the list was also split in + three. The writer now gives these paragraphs no numbering, + just an indentation that lines them up with the text of the + list item (as Word's "List Continue" styles do). + + * LaTeX writer: + + + Use braces around term containing `=` (#11954). This is needed for + the new LaTeX list code, which is triggered by using `pdfstandard`. + + * PowerPoint writer: + + + Add `clrMapOvr` to notes slides (#11935, Gaurav Vijay Jadhav). + This makes them importable by Keynote. + + Include `lang` in core document properties. + + * DocBook, JATS, TEI writers: + + + Apply `writerIdentifierPrefix` prefix consistently. + + * Typst writer: + + + Escape `...` and `-?` ligatures (#11979). + + Use `getLang` to determine lang and region. + Previously only the `lang` metadata field was consulted, so + `lang` could not be set properly using a variable. + + * Text.Pandoc.Translations: + + + `readTranslations` now takes the path of the file, for error + messages [API change]. + + * Text.Pandoc.Writers.Shared: + + + Add `wrapColumns`, `renderWithTemplate` [API change]. + These factor out code that was repeated in about 20 writers: computing the + line length from writerWrapText/writerColumns, and rendering the body with + or without the template. + + * Text.Pandoc.Shared: + + + Export `plainToPara` and `paraToPlain`. [API change] + + * Text.Pandoc.PDF: + + + Make `--presentational-hints` default for weasyprint. + This is necessary to get list attributes (start, ordered + list type, etc.) working properly (#11976). + + * cabal.project: + + + Use bundle C option for zlib by default. + This allows pandoc to be built without first installing the C library + zlib. Those who want to use the system zlib can still compile pandoc + that way by overriding this setting in cabal.project.local, or + just installing directly from Hackage. + + Default to `+http` on GHC >= 9.14 (previously done only in + CI), so local builds behave the same; `-f-http` still overrides it. + + Pass wasm link flags for `pandoc-cli` as `ghc-options`. + cabal only passes ld-options along when it finds an `ld` program on the + PATH. In an environment without one (e.g. `nix shell` with just the + ghc-wasm-meta toolchain), the flags were silently dropped and linking + pandoc.wasm failed with undefined `setjmp` and clock symbols. + Use `-optl- ghc-options` instead, which GHC passes to its + linker regardless. + + * Makefile: + + + Add `release` target. + + Improve `make pandoc-templates` so it does the whole job. + + Use bash as `SHELL` (`check-cabal` uses `pushd`, `check-version-sync` + uses `==`). + + Fix `$(id -u)` in `debpkg`, which make expanded to empty. + - Fix missing `echo` in `validate-docx-golden-tests{,2}`. + - Ensure pandoc is built before rules that run it to produce files. + - Use `git ls-files` for `SOURCEFILES`, not the main branch's tree. + - Drop redundant `embed_data_files` flag in `quick-stack`. + + * Use released citeproc 0.14.1.1, texmath 0.13.3.3, + gridtables 0.1.1.1 (#9085). + + * MANUAL.txt: + + + Clarify the effect of `raw_html` in commonmark formats (#8305). + + * Bump bounds on crypton, crypton-x509-system, tasty-bench. + + * Add `tools/circleci-build-release.sh`, `tools/github-build-release.sh`. + ## pandoc 3.12.1 (2026-10-07) * New input and output format: `fodt` (#4010). This is ODF's diff --git a/pandoc.cabal b/pandoc.cabal --- a/pandoc.cabal +++ b/pandoc.cabal @@ -1,6 +1,6 @@ cabal-version: 2.4 name: pandoc -version: 3.12.1 +version: 3.13 build-type: Simple license: GPL-2.0-or-later license-file: COPYING.md @@ -17,18 +17,19 @@ . - light markup formats (many variants of Markdown, reStructuredText, AsciiDoc, Org-mode, Muse, Textile, - txt2tags, djot) - - HTML formats (HTML 4 and 5) + txt2tags, djot, bbcode) + - HTML formats (HTML 4 and 5, chunked HTML) - Ebook formats (EPUB v2 and v3, FB2) - Documentation formats (GNU TexInfo, Haddock, Vimdoc) - - Roff formats (man, ms) + - Roff formats (man, ms, mdoc) - TeX formats (LaTeX, ConTeXt) - Typst + - ANSI-formatted text - XML formats (DocBook 4 and 5, JATS, TEI Simple, OpenDocument) - Outline formats (OPML) - Bibliography formats (BibTeX, BibLaTeX, CSL JSON, CSL YAML, RIS) - - Word processor formats (Docx, RTF, ODT) + - Word processor formats (Docx, RTF, ODT, FODT) - Interactive notebook formats (Jupyter notebook ipynb) - Page layout formats (InDesign ICML) - Wiki markup formats (MediaWiki, DokuWiki, TikiWiki, TWiki, @@ -36,6 +37,8 @@ - Slide show formats (LaTeX Beamer, PowerPoint, Slidy, reveal.js, Slideous, S5, DZSlides) - Data formats (CSV and TSV tables, Excel spreadsheets) + - Serialization formats (Haskell, JSON, or XML representation + of the AST) - PDF (via external programs such as pdflatex or wkhtmltopdf) . Pandoc can convert mathematical content in documents @@ -227,6 +230,9 @@ test/command/11113.docx test/command/11689.docx test/command/11833.docx + test/command/11944.docx + test/command/11989.docx + test/command/11991.docx test/command/11942.js test/command/biblio.bib test/command/averroes.bib @@ -538,12 +544,12 @@ blaze-markup >= 0.8 && < 0.9, bytestring >= 0.9 && < 0.13, case-insensitive >= 1.2 && < 1.3, - citeproc >= 0.14.1 && < 0.15, + citeproc >= 0.14.1.1 && < 0.15, commonmark >= 0.3.1 && < 0.4, commonmark-extensions >= 0.2.7.3 && < 0.3, commonmark-pandoc >= 0.3.0.2 && < 0.4, containers >= 0.6.0.1 && < 0.9, - crypton >= 0.30 && < 2.1, + crypton >= 0.30 && < 2.3, data-default >= 0.4 && < 0.9, deepseq >= 1.3 && < 1.6, directory >= 1.2.5 && < 1.4, @@ -553,7 +559,7 @@ exceptions >= 0.8 && < 0.11, file-embed >= 0.0 && < 0.1, filepath >= 1.1 && < 1.6, - gridtables >= 0.1 && < 0.2, + gridtables >= 0.1.1.1 && < 0.2, haddock-library >= 1.10 && < 1.12, http-types >= 0.8 && < 0.13, ipynb >= 0.2 && < 0.3, @@ -573,15 +579,15 @@ syb >= 0.1 && < 0.8, tagsoup >= 0.14.6 && < 0.15, temporary >= 1.1 && < 1.4, - texmath >= 0.13.3.1 && < 0.14, + texmath >= 0.13.3.3 && < 0.14, text >= 2.0 && < 2.2, text-conversions >= 0.3 && < 0.4, time >= 1.5 && < 1.17, unicode-collation >= 0.1.1 && < 0.2, unicode-data >= 0.6 && < 0.10, unicode-transforms >= 0.3 && < 0.5, - yaml >= 0.11 && < 0.12, - libyaml >= 0.1.4 && < 0.2, + yamlet >= 1.0.0.1 && < 1.1, + yamlet-aeson >= 1.0.0.1 && < 1.1, zip-archive >= 0.5 && < 0.6, zlib >= 0.6 && < 0.8, xml >= 1.3.12 && < 1.4, @@ -599,7 +605,7 @@ cpp-options: -DPANDOC_HTTP_SUPPORT build-depends: crypton-connection >= 0.3.1 && < 0.5, - crypton-x509-system >= 1.6.7 && < 1.10, + crypton-x509-system >= 1.6.7 && < 1.11, http-client >= 0.4.30 && < 0.8, http-client-tls >= 0.2.4 && < 0.5, network >= 2.6 && < 3.3, @@ -944,7 +950,7 @@ main-is: benchmark-pandoc.hs hs-source-dirs: benchmark build-depends: bytestring, - tasty-bench >= 0.4 && <= 0.5, + tasty-bench >= 0.4 && <= 0.6, mtl >= 2.2 && < 2.4, text >= 2.0 && < 2.2, temporary, diff --git a/src/Text/Pandoc/App/Opt.hs b/src/Text/Pandoc/App/Opt.hs --- a/src/Text/Pandoc/App/Opt.hs +++ b/src/Text/Pandoc/App/Opt.hs @@ -1,6 +1,5 @@ {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE PatternSynonyms #-} {-# LANGUAGE TemplateHaskell #-} @@ -67,11 +66,15 @@ import qualified Data.Map as M import qualified Data.ByteString.Char8 as B8 import Text.Pandoc.Definition (Meta(..), MetaValue(..)) -import Data.Aeson (defaultOptions, Options(..), Result(..), - genericToJSON, fromJSON, camelTo2, eitherDecodeStrict) import Data.Aeson.TH (deriveJSON) import Control.Applicative ((<|>)) -import Data.Yaml +import Data.Aeson +import Data.Aeson.Types +import qualified Data.Aeson.Key as K +import qualified Data.List as L +import qualified Data.List.NonEmpty as NE +import qualified Yamlet as Y +import Yamlet.Aeson -- | The type of line-endings to be used when writing plain-text. data LineEnding = LF | CRLF | Native deriving (Show, Generic) @@ -357,11 +360,11 @@ Success (opts :: M.Map Text Value) -> do dataDir <- case M.lookup "data-dir" opts of Nothing -> return Nothing - Just v -> Just . unpack <$> parseJSON v + Just v -> Just . unpack <$> parseJSON v Key "data-dir" f <- parseOptions (M.toList opts) case M.lookup "defaults" opts of Just v -> do - g <- parseDefaults v dataDir + g <- parseDefaults v dataDir Key "defaults" return $ g >=> f >=> resolveVarsInOpt Nothing -> return $ f >=> resolveVarsInOpt parseJSON _ = fail "Expected a mapping" @@ -541,7 +544,7 @@ _ -> doOpt (k,v) doOpt :: (Text, Value) -> Parser (Opt -> Opt) -doOpt (k,v) = do +doOpt (k,v) = ( Key (K.fromText k)) $ do case k of "tab-stop" -> parseJSON v >>= \x -> return (\o -> o{ optTabStop = x }) @@ -938,11 +941,11 @@ Left err' -> throwError $ PandocParseError $ T.pack $ "Error parsing " <> file <> ":\n" <> err' else - case decodeEither' (B8.unlines . takeWhile (/= "...") . B8.lines $ inp) of - Right f -> f opt - Left err' -> throwError $ PandocParseError $ T.pack $ + case Y.decode (B8.unlines . takeWhile (/= "...") . B8.lines $ inp) of + Right (ViaAeson f) -> f opt + Left errs -> throwError $ PandocParseError $ T.pack $ "Error parsing " <> file <> ":\n" <> - Data.Yaml.prettyPrintParseException err' + L.intercalate "\n" (map (Y.prettyError file) (NE.toList errs)) fullDefaultsPath :: (PandocMonad m, MonadIO m) => Maybe FilePath diff --git a/src/Text/Pandoc/App/OutputSettings.hs b/src/Text/Pandoc/App/OutputSettings.hs --- a/src/Text/Pandoc/App/OutputSettings.hs +++ b/src/Text/Pandoc/App/OutputSettings.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE LambdaCase #-} {-# LANGUAGE OverloadedStrings #-} diff --git a/src/Text/Pandoc/Extensions.hs b/src/Text/Pandoc/Extensions.hs --- a/src/Text/Pandoc/Extensions.hs +++ b/src/Text/Pandoc/Extensions.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE OverloadedStrings #-} diff --git a/src/Text/Pandoc/Filter.hs b/src/Text/Pandoc/Filter.hs --- a/src/Text/Pandoc/Filter.hs +++ b/src/Text/Pandoc/Filter.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE OverloadedStrings #-} {- | diff --git a/src/Text/Pandoc/Image.hs b/src/Text/Pandoc/Image.hs --- a/src/Text/Pandoc/Image.hs +++ b/src/Text/Pandoc/Image.hs @@ -1,4 +1,4 @@ -{-# LANGUAGE OverloadedStrings, ScopedTypeVariables, CPP #-} +{-# LANGUAGE OverloadedStrings, ScopedTypeVariables #-} {- | Module : Text.Pandoc.Image Copyright : Copyright (C) 2020-2024 John MacFarlane diff --git a/src/Text/Pandoc/MediaBag.hs b/src/Text/Pandoc/MediaBag.hs --- a/src/Text/Pandoc/MediaBag.hs +++ b/src/Text/Pandoc/MediaBag.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-} diff --git a/src/Text/Pandoc/Options.hs b/src/Text/Pandoc/Options.hs --- a/src/Text/Pandoc/Options.hs +++ b/src/Text/Pandoc/Options.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE LambdaCase #-} diff --git a/src/Text/Pandoc/PDF.hs b/src/Text/Pandoc/PDF.hs --- a/src/Text/Pandoc/PDF.hs +++ b/src/Text/Pandoc/PDF.hs @@ -112,7 +112,7 @@ >>= writer opts' verbosity <- getVerbosity - let compileHTML mkOutArgs = do + let compileHTML defaultOpts mkOutArgs = do -- check to see if there is anything in mediabag, and if so, -- make the HTML self contained mediabag <- getMediaBag @@ -120,12 +120,12 @@ [] -> pure source _ -> makeSelfContained source liftIO $ - toPdfViaTempFile verbosity program pdfargs mkOutArgs ".html" source' + toPdfViaTempFile verbosity program (defaultOpts ++ pdfargs) mkOutArgs ".html" source' case takeBaseName program of "wkhtmltopdf" -> makeWithWkhtmltopdf program pdfargs writer opts' doc - "pagedjs-cli" -> compileHTML (\f -> ["-o", f]) - "prince" -> compileHTML (\f -> ["-o", f]) - "weasyprint" -> compileHTML (:[]) + "pagedjs-cli" -> compileHTML [] (\f -> ["-o", f]) + "prince" -> compileHTML [] (\f -> ["-o", f]) + "weasyprint" -> compileHTML ["--presentational-hints"] (\f -> [f]) "typst" -> liftIO $ toPdfViaTempFile verbosity program ("compile":pdfargs) (:[]) ".typ" source "pdfroff" -> do diff --git a/src/Text/Pandoc/Readers/Djot.hs b/src/Text/Pandoc/Readers/Djot.hs --- a/src/Text/Pandoc/Readers/Djot.hs +++ b/src/Text/Pandoc/Readers/Djot.hs @@ -25,7 +25,8 @@ import Text.Parsec.Pos (newPos) import Text.Pandoc.Options import Text.Pandoc.Definition -import Text.Pandoc.Shared (addPandocAttributes, tshow, compactifyTable) +import Text.Pandoc.Shared (addPandocAttributes, tshow, compactifyTable, + paraToPlain) import qualified Text.Pandoc.UTF8 as UTF8 import Djot (ParseOptions(..), SourcePosOption(..), parseDoc, Pos(..)) import qualified Djot.AST as D @@ -259,9 +260,6 @@ toTight :: Blocks -> Blocks toTight (Many bls) = Many $ paraToPlain <$> bls - where - paraToPlain (Para ils) = Plain ils - paraToPlain x = x toTaskListItem :: PandocMonad m => (D.TaskStatus, D.Blocks) -> ReaderT Env m Blocks diff --git a/src/Text/Pandoc/Readers/DocBook.hs b/src/Text/Pandoc/Readers/DocBook.hs --- a/src/Text/Pandoc/Readers/DocBook.hs +++ b/src/Text/Pandoc/Readers/DocBook.hs @@ -46,7 +46,7 @@ import Text.Pandoc.Class.PandocMonad (PandocMonad, report) import Text.Pandoc.Options import Text.Pandoc.Logging (LogMessage(..)) -import Text.Pandoc.Shared (safeRead, addPandocAttributes) +import Text.Pandoc.Shared (safeRead, addPandocAttributes, paraToPlain) import Text.Pandoc.Sources (ToSources(..), sourcesToText) import Text.Pandoc.Transforms (headerShift) import Text.TeXMath (readMathML, writeTeX) @@ -1481,10 +1481,6 @@ -- | Set the prefix of a name to 'Nothing' removePrefix :: QName -> QName removePrefix elname = elname { qPrefix = Nothing } - -paraToPlain :: Block -> Block -paraToPlain (Para ils) = Plain ils -paraToPlain x = x docbookEntityMap :: M.Map Text Text docbookEntityMap = M.fromList diff --git a/src/Text/Pandoc/Readers/Docx.hs b/src/Text/Pandoc/Readers/Docx.hs --- a/src/Text/Pandoc/Readers/Docx.hs +++ b/src/Text/Pandoc/Readers/Docx.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE PatternGuards #-} {-# LANGUAGE FlexibleContexts #-} @@ -60,6 +59,7 @@ ) where import Codec.Archive.Zip +import Control.Applicative ((<|>)) import Control.Monad ( liftM, unless ) import Control.Monad.Reader ( asks, @@ -132,6 +132,11 @@ , docxListState :: M.Map (T.Text, T.Text) Integer , docxPrevPara :: Inlines , docxReferences :: M.Map ItemId (Reference Inlines) + -- the open list levels, innermost first, as + -- (text indentation, level, numId); used to + -- recognize unnumbered paragraphs continuing + -- a list item + , docxOpenListLevels :: [(Integer, T.Text, T.Text)] } instance Default DState where @@ -144,6 +149,7 @@ , docxListState = M.empty , docxPrevPara = mempty , docxReferences = mempty + , docxOpenListLevels = [] } data DEnv = DEnv { docxOptions :: ReaderOptions @@ -270,6 +276,8 @@ T.concat $ map parPartToText children parPartToText (ExternalHyperLink _ _ children) = T.concat $ map parPartToText children +parPartToText (Field _ children) = + T.concat $ map parPartToText children parPartToText _ = "" blacklistedCharStyles :: [CharStyleName] @@ -343,8 +351,8 @@ let ils = smushInlines (map runElemToInlines runElems) transform <- runStyleToTransform rPr return $ transform ils -runToInlines (Footnote bps) = note . smushBlocks <$> mapM bodyPartToBlocks bps -runToInlines (Endnote bps) = note . smushBlocks <$> mapM bodyPartToBlocks bps +runToInlines (Footnote bps) = note <$> notePartsToBlocks bps +runToInlines (Endnote bps) = note <$> notePartsToBlocks bps runToInlines (InlineDrawing fp title alt bs ext) = do (lift . lift) $ P.insertMedia fp Nothing bs return $ imageWith (extentToAttr ext) (T.pack fp) title $ text alt @@ -358,6 +366,19 @@ showDim d = tshow (d / 914400) <> "in" extentToAttr _ = nullAttr +-- A note is read in the middle of the paragraph that references it. +-- Text held back from a paragraph whose mark is a tracked change +-- (docxPrevPara) must therefore neither flow into the note from the +-- body, nor out of the note into the referencing paragraph. +notePartsToBlocks :: PandocMonad m => [BodyPart] -> DocxContext m Blocks +notePartsToBlocks bps = do + outerPrevPara <- gets docxPrevPara + modify $ \s -> s { docxPrevPara = mempty } + blks <- smushBlocks <$> mapM bodyPartToBlocks bps + leftover <- gets docxPrevPara + modify $ \s -> s { docxPrevPara = outerPrevPara } + return $ if null leftover then blks else blks <> para leftover + blocksToInlinesWarn :: PandocMonad m => T.Text -> Blocks -> DocxContext m Inlines blocksToInlinesWarn cmtId blks = do let paraOrPlain :: Block -> Bool @@ -694,7 +715,17 @@ | otherwise = c bodyPartToBlocks :: PandocMonad m => BodyPart -> DocxContext m Blocks -bodyPartToBlocks (Heading n style pPr numId lvl mblvlInfo parparts) = do +bodyPartToBlocks bp = + case bp of + Paragraph{} -> bodyPartToBlocks' bp + ListItem{} -> bodyPartToBlocks' bp + -- anything else ends the open lists: + _ -> closeListLevels *> bodyPartToBlocks' bp <* closeListLevels + where + closeListLevels = modify $ \s -> s{ docxOpenListLevels = [] } + +bodyPartToBlocks' :: PandocMonad m => BodyPart -> DocxContext m Blocks +bodyPartToBlocks' (Heading n style pPr numId lvl mblvlInfo parparts) = do ils <- local (\s-> s{docxInHeaderBlock=True}) (smushInlines <$> mapM parPartToInlines parparts) let classes = map normalizeToClassName . delete style @@ -712,7 +743,7 @@ let start = case M.lookup (numId, lvl) listState of Nothing -> case mblvlInfo of Nothing -> 1 - Just (Level _ _ _ z) -> fromMaybe 1 z + Just (Level _ _ _ z _) -> fromMaybe 1 z Just z -> z + 1 modify $ \st -> st{ docxListState = -- expire all the continuation data for lists of level > this one: @@ -722,77 +753,28 @@ in M.insert (numId, lvl) start (M.filterWithKey notExpired listState) } makeHeaderAnchor $ headerWith ("", addNum classes, []) n ils -bodyPartToBlocks (Paragraph pPr parparts) = do - let getMainPar - | Just True <- pBidi pPr = do - let pPr' = pPr { pBidi = Nothing } - local (\s -> s{ docxInBidi = True }) - (bodyPartToBlocks (Paragraph pPr' parparts)) - | isCodeDiv pPr = do - transform <- paragraphStyleToTransform pPr - return $ - transform $ - codeBlock $ - T.concat $ - map parPartToText parparts - | otherwise = do - ils <- trimSps . smushInlines <$> mapM parPartToInlines parparts - prevParaIls <- gets docxPrevPara - dropIls <- gets docxDropCap - let ils' = dropIls <> ils - let (paraOrPlain, pPr') - | hasStylesInheritedFrom ["Compact"] pPr = (plain, removeStyleNamed "Compact" pPr) - | otherwise = (para, pPr) - if dropCap pPr' - then do modify $ \s -> s { docxDropCap = ils' } - return mempty - else do modify $ \s -> s { docxDropCap = mempty } - let ils'' = (if null prevParaIls then mempty - else prevParaIls <> space) <> ils' - handleInsertion = do - modify $ \s -> s {docxPrevPara = mempty} - transform <- paragraphStyleToTransform pPr' - return $ transform $ paraOrPlain ils'' - opts <- asks docxOptions - case (pChange pPr', readerTrackChanges opts) of - _ | null ils'', not (isEnabled Ext_empty_paragraphs opts) -> - return mempty - (Just (TrackedChange Insertion _), AcceptChanges) -> - handleInsertion - (Just (TrackedChange Insertion _), RejectChanges) -> do - modify $ \s -> s {docxPrevPara = ils''} - return mempty - (Just (TrackedChange Insertion (ChangeInfo _ cAuthor cDate)) - , AllChanges) -> do - let attr = ("", ["paragraph-insertion"], addAuthorAndDate cAuthor cDate) - insertMark = spanWith attr mempty - transform <- paragraphStyleToTransform pPr' - return $ transform $ - paraOrPlain $ ils'' <> insertMark - (Just (TrackedChange Deletion _), AcceptChanges) -> do - modify $ \s -> s {docxPrevPara = ils''} - return mempty - (Just (TrackedChange Deletion _), RejectChanges) -> - handleInsertion - (Just (TrackedChange Deletion (ChangeInfo _ cAuthor cDate)) - , AllChanges) -> do - let attr = ("", ["paragraph-deletion"], addAuthorAndDate cAuthor cDate) - insertMark = spanWith attr mempty - transform <- paragraphStyleToTransform pPr' - return $ transform $ - paraOrPlain $ ils'' <> insertMark - _ -> handleInsertion - (if pBottomBorder pPr - then (<> horizontalRule) - else id) - <$> getMainPar -bodyPartToBlocks (ListItem pPr numId lvl isRestart (Just levelInfo) parparts) = do +bodyPartToBlocks' (Paragraph pPr parparts) = do + openLevels <- gets docxOpenListLevels + let leftIndent = (indentation pPr >>= leftParIndent) <|> + (pStyleIndentation pPr >>= leftParIndent) + -- An unnumbered paragraph indented like the text of an open list + -- item continues that item (this is how pandoc's docx writer and + -- Word's "List Continue" styles do multi-paragraph list items): + case break (\(ind, _, _) -> Just ind == leftIndent) openLevels of + (_, levels@((_, lvl, numId) : _)) | not (numbered pPr) -> do + modify $ \s -> s{ docxOpenListLevels = levels } + divWith ("", ["list-continuation"], [("level", lvl), ("num-id", numId)]) + <$> paragraphToBlocks pPr{ indentation = Nothing } parparts + _ -> do + modify $ \s -> s{ docxOpenListLevels = [] } + paragraphToBlocks pPr parparts +bodyPartToBlocks' (ListItem pPr numId lvl isRestart (Just levelInfo) parparts) = do -- We check whether this current abstract numbering id has previously been -- used, since Docx expects us to pick up where we left off -- unless this -- item explicitly restarts numbering (see #8367). listState <- gets docxListState let startFromState = M.lookup (numId, lvl) listState - Level _ fmt txt startFromLevelInfo = levelInfo + Level _ fmt txt startFromLevelInfo lvlIndent = levelInfo start = case startFromState of Just n | not isRestart -> n + 1 _ -> fromMaybe 1 startFromLevelInfo @@ -807,13 +789,20 @@ -- a new level 1 list item resets continuation for level 2+ let notExpired (_, lvl') _ = lvl' <= lvl in M.insert (numId, lvl) start (M.filterWithKey notExpired listState) } - blks <- bodyPartToBlocks (Paragraph pPr parparts) + -- record the indentation of the item's text, so that we can recognize + -- unnumbered paragraphs continuing this item: + let textIndent = (indentation pPr >>= leftParIndent) <|> lvlIndent <|> + (pStyleIndentation pPr >>= leftParIndent) + modify $ \st -> st{ docxOpenListLevels = + maybe id (\ind -> ((ind, lvl, numId) :)) textIndent + (filter (\(_, lvl', _) -> lvl' < lvl) (docxOpenListLevels st)) } + blks <- paragraphToBlocks pPr parparts return $ divWith ("", ["list-item"], kvs) blks -bodyPartToBlocks (ListItem pPr _ _ _ _ parparts) = +bodyPartToBlocks' (ListItem pPr _ _ _ _ parparts) = let pPr' = pPr {pStyle = constructBogusParStyleData "list-paragraph": pStyle pPr} in bodyPartToBlocks $ Paragraph pPr' parparts -bodyPartToBlocks (Captioned parstyle parparts bpart) = do +bodyPartToBlocks' (Captioned parstyle parparts bpart) = do bs <- bodyPartToBlocks bpart captContents <- bodyPartToBlocks (Paragraph parstyle parparts) let capt = Caption Nothing (toList captContents) @@ -830,9 +819,9 @@ -> toCaptioned attr' bls' _ -> captContents pure $ toCaptioned nullAttr (toList bs) -bodyPartToBlocks (Tbl _ _ _ _ []) = +bodyPartToBlocks' (Tbl _ _ _ _ []) = return mempty -bodyPartToBlocks (Tbl mbsty cap grid look parts) = do +bodyPartToBlocks' (Tbl mbsty cap grid look parts) = do let fullCaption = if T.null cap then mempty else plain (text cap) let shortCaption = if T.null cap then Nothing else Just (toList (text cap)) cap' = caption shortCaption fullCaption @@ -847,11 +836,10 @@ -- Horizontal column alignment is taken from the first row's cells. numcols = length widths getAlignment (Docx.Cell al colspan _ _) = - take numcols $ replicate (fromIntegral colspan) (convertAlign al) ++ - repeat Pandoc.AlignDefault + replicate (fromIntegral colspan) (convertAlign al) getAlignments (Docx.Row _ cs) = concatMap getAlignment cs alignments = take numcols $ - case hdr ++ rows of + case rows ++ hdr of [] -> repeat Pandoc.AlignDefault r : _ -> getAlignments r @@ -865,7 +853,75 @@ (TableHead nullAttr headerCells) [TableBody nullAttr 0 [] bodyCells] (TableFoot nullAttr []) -bodyPartToBlocks HRule = pure Pandoc.horizontalRule +bodyPartToBlocks' HRule = pure Pandoc.horizontalRule + +-- | Convert a paragraph (also used for the text of list items). +paragraphToBlocks :: PandocMonad m + => ParagraphStyle -> [ParPart] -> DocxContext m Blocks +paragraphToBlocks pPr parparts = do + let getMainPar + | Just True <- pBidi pPr = do + let pPr' = pPr { pBidi = Nothing } + local (\s -> s{ docxInBidi = True }) + (paragraphToBlocks pPr' parparts) + | isCodeDiv pPr = do + transform <- paragraphStyleToTransform pPr + return $ + transform $ + codeBlock $ + T.concat $ + map parPartToText parparts + | otherwise = do + ils <- trimSps . smushInlines <$> mapM parPartToInlines parparts + prevParaIls <- gets docxPrevPara + dropIls <- gets docxDropCap + let ils' = dropIls <> ils + let (paraOrPlain, pPr') + | hasStylesInheritedFrom ["Compact"] pPr = (plain, removeStyleNamed "Compact" pPr) + | otherwise = (para, pPr) + if dropCap pPr' + then do modify $ \s -> s { docxDropCap = ils' } + return mempty + else do modify $ \s -> s { docxDropCap = mempty } + let ils'' = (if null prevParaIls then mempty + else prevParaIls <> space) <> ils' + handleInsertion = do + modify $ \s -> s {docxPrevPara = mempty} + transform <- paragraphStyleToTransform pPr' + return $ transform $ paraOrPlain ils'' + opts <- asks docxOptions + case (pChange pPr', readerTrackChanges opts) of + _ | null ils'', not (isEnabled Ext_empty_paragraphs opts) -> + return mempty + (Just (TrackedChange Insertion _), AcceptChanges) -> + handleInsertion + (Just (TrackedChange Insertion _), RejectChanges) -> do + modify $ \s -> s {docxPrevPara = ils''} + return mempty + (Just (TrackedChange Insertion (ChangeInfo _ cAuthor cDate)) + , AllChanges) -> do + let attr = ("", ["paragraph-insertion"], addAuthorAndDate cAuthor cDate) + insertMark = spanWith attr mempty + transform <- paragraphStyleToTransform pPr' + return $ transform $ + paraOrPlain $ ils'' <> insertMark + (Just (TrackedChange Deletion _), AcceptChanges) -> do + modify $ \s -> s {docxPrevPara = ils''} + return mempty + (Just (TrackedChange Deletion _), RejectChanges) -> + handleInsertion + (Just (TrackedChange Deletion (ChangeInfo _ cAuthor cDate)) + , AllChanges) -> do + let attr = ("", ["paragraph-deletion"], addAuthorAndDate cAuthor cDate) + insertMark = spanWith attr mempty + transform <- paragraphStyleToTransform pPr' + return $ transform $ + paraOrPlain $ ils'' <> insertMark + _ -> handleInsertion + (if pBottomBorder pPr + then (<> horizontalRule) + else id) + <$> getMainPar -- replace targets with generated anchors. rewriteLink' :: PandocMonad m => Inline -> DocxContext m Inline diff --git a/src/Text/Pandoc/Readers/Docx/Lists.hs b/src/Text/Pandoc/Readers/Docx/Lists.hs --- a/src/Text/Pandoc/Readers/Docx/Lists.hs +++ b/src/Text/Pandoc/Readers/Docx/Lists.hs @@ -32,6 +32,11 @@ isListItem (Div (_, classes, _) _) | "list-item" `elem` classes = True isListItem _ = False +-- | An unnumbered paragraph continuing a list item. +isListContinuation :: Block -> Bool +isListContinuation (Div (_, classes, _) _) = "list-continuation" `elem` classes +isListContinuation _ = False + getLevel :: Block -> Maybe Integer getLevel (Div (_, _, kvs) _) = lookup "level" kvs >>= safeRead getLevel _ = Nothing @@ -115,6 +120,8 @@ separateBlocks' b@(OrderedList _ _) acc = init acc ++ [last acc ++ [b]] -- The following is for the invisible bullet lists. This is how -- pandoc-generated ooxml does multiparagraph item lists. +separateBlocks' b acc | isListContinuation b = + init acc ++ [last acc ++ [b]] separateBlocks' b acc | fmap trim (getText b) == Just "" = init acc ++ [last acc ++ [b]] separateBlocks' b acc = acc ++ [[b]] @@ -219,6 +226,8 @@ case delete "list-item" classes of [] -> blks classes' -> [Div (ident, classes', kvs) blks] +removeListDivs' b@(Div _ blks) + | isListContinuation b = blks removeListDivs' (Div (ident, classes, kvs) blks) | not $ null $ listParagraphDivs `intersect` classes = case classes \\ listParagraphDivs of diff --git a/src/Text/Pandoc/Readers/Docx/Parse.hs b/src/Text/Pandoc/Readers/Docx/Parse.hs --- a/src/Text/Pandoc/Readers/Docx/Parse.hs +++ b/src/Text/Pandoc/Readers/Docx/Parse.hs @@ -249,8 +249,8 @@ data AbstractNumb = AbstractNumb T.Text [Level] deriving Show --- ilvl format string start -data Level = Level T.Text T.Text T.Text (Maybe Integer) +-- ilvl format string start left indent +data Level = Level T.Text T.Text T.Text (Maybe Integer) (Maybe Integer) deriving Show data DocumentLocation = InDocument | InFootnote | InEndnote @@ -620,9 +620,9 @@ case lvlOverride of Just (LevelOverride _ _ (Just lvl')) -> Just lvl' Just (LevelOverride _ (Just strt) _) -> - lookup ilvl $ map (\(Level i fmt s _) -> (i, Level i fmt s (Just strt))) lvls + lookup ilvl $ map (\(Level i fmt s _ ind) -> (i, Level i fmt s (Just strt) ind)) lvls _ -> - lookup ilvl $ map (\l@(Level i _ _ _) -> (i, l)) lvls + lookup ilvl $ map (\l@(Level i _ _ _ _) -> (i, l)) lvls -- | Given a @numId@ and level, return the abstract numbering id (the stable -- identifier shared by all numbering instances based on the same abstract @@ -690,7 +690,8 @@ let start = findChildByName ns "w" "start" element >>= findAttrByName ns "w" "val" >>= stringToInteger - return (Level ilvl fmt txt start) + let ind = getIndentation ns element >>= leftParIndent + return (Level ilvl fmt txt start ind) levelElemToLevel _ _ = Nothing archiveToNumbering' :: Archive -> Maybe Numbering @@ -763,9 +764,13 @@ case sum xs of tot | tot > 1.0 -> map (/ tot) xs _ -> xs - normalizeFractions <$> - mapD (\e -> maybeToD (findAttrByName ns "w" "w" e >>= - fmap toFraction . safeRead)) cols + -- w:w is optional. If a column has no width, we keep it but leave + -- every column's width unspecified (0), since writers don't handle + -- a mix of specified and unspecified widths well. + return $ + case mapM (findAttrByName ns "w" "w" >=> safeRead) cols of + Just widths -> normalizeFractions (map toFraction widths) + Nothing -> replicate (length cols) 0 elemToTblGrid _ _ = throwError WrongElem elemToTblLook :: NameSpaces -> Element -> D TblLook @@ -1109,18 +1114,28 @@ info <- eitherToD $ parseFieldInfo t modify $ \st -> st {stateFldCharState = FldCharContent info [] : ancestors} return [] + -- A field with no instruction (e.g. a dead cross-reference, or + -- one whose w:instrText sits inside a tracked change): + FldCharOpen : ancestors | fldCharType == "separate" -> do + modify $ \st -> st {stateFldCharState = FldCharContent UnknownField [] : ancestors} + return [] -- Some fields have no content, e.g. index XE: FldCharFieldInfo t : ancestors | fldCharType == "end" -> do modify $ \st -> st {stateFldCharState = ancestors} info <- eitherToD $ parseFieldInfo t return [Field info []] - [FldCharContent info children] | fldCharType == "end" -> do - modify $ \st -> st {stateFldCharState = []} - return [Field info $ reverse children] + FldCharOpen : ancestors | fldCharType == "end" -> do + modify $ \st -> st {stateFldCharState = ancestors} + return [Field UnknownField []] FldCharContent info children : FldCharContent parentInfo siblings : ancestors | fldCharType == "end" -> let parent = FldCharContent parentInfo $ (Field info (reverse children)) : siblings in do modify $ \st -> st {stateFldCharState = parent : ancestors} return [] + -- Always pop the field, even if the enclosing field is not + -- (yet) in its content, so that the state cannot get stuck: + FldCharContent info children : ancestors | fldCharType == "end" -> do + modify $ \st -> st {stateFldCharState = ancestors} + return [Field info $ reverse children] _ -> throwError WrongElem elemToParPart ns element | isElem ns "w" "r" element @@ -1212,6 +1227,12 @@ runs <- mconcat <$> mapD (elemToParPart' ns) (elChildren element) return [ChangedRuns change runs] elemToParPart' ns element + | isElem ns "w" "fldSimple" element + , Just instr <- findAttrByName ns "w" "instr" element = do + info <- eitherToD $ parseFieldInfo instr + children <- mconcat <$> mapD (elemToParPart ns) (elChildren element) + return [Field info children] +elemToParPart' ns element | isElem ns "w" "bookmarkStart" element , Just bmId <- findAttrByName ns "w" "id" element , Just bmName <- findAttrByName ns "w" "name" element = @@ -1239,9 +1260,9 @@ elemToParPart' ns element | isElem ns "w" "commentRangeStart" element , Just cmtId <- findAttrByName ns "w" "id" element = do - (Comments _ commentMap) <- asks envComments + (Comments cmtNs commentMap) <- asks envComments case M.lookup cmtId commentMap of - Just cmtElem -> elemToCommentStart ns cmtElem + Just cmtElem -> elemToCommentStart cmtNs cmtElem Nothing -> throwError WrongElem elemToParPart' ns element | isElem ns "w" "commentRangeEnd" element @@ -1310,17 +1331,17 @@ childElemToRun ns element | isElem ns "w" "footnoteReference" element , Just fnId <- findAttrByName ns "w" "id" element = do - notes <- asks envNotes + notes@(Notes notesNs _ _) <- asks envNotes case lookupFootnote fnId notes of - Just e -> do bps <- local (\r -> r {envLocation=InFootnote}) $ mapD (elemToBodyPart ns) (elChildren e) + Just e -> do bps <- local (\r -> r {envLocation=InFootnote}) $ mapD (elemToBodyPart notesNs) (elChildren e) return [Footnote bps] Nothing -> return [Footnote []] childElemToRun ns element | isElem ns "w" "endnoteReference" element , Just enId <- findAttrByName ns "w" "id" element = do - notes <- asks envNotes + notes@(Notes notesNs _ _) <- asks envNotes case lookupEndnote enId notes of - Just e -> do bps <- local (\r -> r {envLocation=InEndnote}) $ mapD (elemToBodyPart ns) (elChildren e) + Just e -> do bps <- local (\r -> r {envLocation=InEndnote}) $ mapD (elemToBodyPart notesNs) (elChildren e) return [Endnote bps] Nothing -> return [Endnote []] childElemToRun _ _ = throwError WrongElem @@ -1513,11 +1534,14 @@ stripCaptionLabel els = if null afters then els - else filter isBookmark befores <> afters + else filter isBookmark befores <> dropNumber afters where (befores, afters) = break isNumberElt els isBookmark (Element name _ _ _) = qName name == "bookmarkStart" || qName name == "bookmarkEnd" + -- A w:fldSimple carries the number as its result: drop it with the label. + dropNumber (el:rest) | qName (elName el) == "fldSimple" = rest + dropNumber rest = rest isNumberElt el@(Element name attribs _ _) = (qName name == "fldSimple" && case lookupAttrBy ((== "instr") . qName) attribs of diff --git a/src/Text/Pandoc/Readers/HTML.hs b/src/Text/Pandoc/Readers/HTML.hs --- a/src/Text/Pandoc/Readers/HTML.hs +++ b/src/Text/Pandoc/Readers/HTML.hs @@ -66,7 +66,7 @@ import Text.Pandoc.Parsing hiding ((<|>)) import Text.Pandoc.Shared ( addMetaField, extractSpaces, htmlSpanLikeElements, renderTags', - safeRead, tshow, formatCode) + safeRead, tshow, formatCode, plainToPara) import Text.Pandoc.URI (escapeURI) import Text.Pandoc.Walk import Text.TeXMath (readMathML, writeTeX) @@ -485,8 +485,6 @@ isParaish OrderedList{} = not inList isParaish DefinitionList{} = not inList isParaish _ = False - plainToPara (Plain xs) = Para xs - plainToPara x = x bs' = B.toList bs pRawTag :: PandocMonad m => TagParser m Text diff --git a/src/Text/Pandoc/Readers/LaTeX/Parsing.hs b/src/Text/Pandoc/Readers/LaTeX/Parsing.hs --- a/src/Text/Pandoc/Readers/LaTeX/Parsing.hs +++ b/src/Text/Pandoc/Readers/LaTeX/Parsing.hs @@ -191,6 +191,11 @@ , sCaseExclusions :: M.Map Text (Set.Set Text) -- ^ words excluded from case changing -- (keys: @upper@, @lower@, @title@) + , sLastExpansion :: Maybe (Text, SourcePos, Int) + -- ^ name and position of the last macro + -- expanded, and the number of consecutive + -- times it has been expanded there + -- (used to detect expansion loops) } deriving Show @@ -225,6 +230,7 @@ , sRawScopes = 0 , sLigatures = True , sCaseExclusions = M.empty + , sLastExpansion = Nothing } instance PandocMonad m => HasQuoteContext LaTeXState m where @@ -752,6 +758,18 @@ -- macro call, so we continue expanding: Nothing -> trySpecialMacro name ts >>= doMacros' (n' + 1) Just (Macro _scope expansionPoint argspecs optarg newtoks) -> do + -- A macro whose expansion contains itself after an + -- unknown command (e.g. \def\small{\@setfontsize\small..}) + -- would be expanded again and again at the same position, + -- each time in a fresh call to doMacros (#11088): + lastExp <- sLastExpansion <$> getState + let expCount = case lastExp of + Just (name', spos', i) + | name' == name, spos' == spos -> i + 1 + _ -> 1 :: Int + when (expCount > 1000) $ throwError $ PandocMacroLoop name + updateState $ \st -> st{ sLastExpansion = + Just (name, spos, expCount) } let getargs' = do args <- (case expansionPoint of diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs --- a/src/Text/Pandoc/Readers/Markdown.hs +++ b/src/Text/Pandoc/Readers/Markdown.hs @@ -1039,10 +1039,6 @@ then fmap (fmap (fmap paraToPlain)) else id) (sequence contents)) -paraToPlain :: Block -> Block -paraToPlain (Para ils) = Plain ils -paraToPlain x = x - definitionList :: PandocMonad m => MarkdownParser m (F Blocks) definitionList = try $ do guardEnabled Ext_definition_lists diff --git a/src/Text/Pandoc/Readers/Metadata.hs b/src/Text/Pandoc/Readers/Metadata.hs --- a/src/Text/Pandoc/Readers/Metadata.hs +++ b/src/Text/Pandoc/Readers/Metadata.hs @@ -1,5 +1,6 @@ {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE ScopedTypeVariables #-} +{-# LANGUAGE TypeApplications #-} {- | Module : Text.Pandoc.Readers.Metadata Copyright : Copyright (C) 2006-2024 John MacFarlane @@ -20,24 +21,22 @@ import Control.Monad.Except (throwError) import qualified Data.ByteString as B +import qualified Data.List as L import qualified Data.Map as M import qualified Data.Vector as V import Data.Text (Text) import qualified Data.Text as T -import qualified Data.Yaml as Yaml -import qualified Data.Yaml.Internal as Yaml -import qualified Text.Libyaml as Y -import Data.Aeson (Value(..), Object, Result(..), fromJSON, (.:?), withObject, - FromJSON) -import Data.Aeson.Types (formatRelativePath, parse) +import qualified Data.List.NonEmpty as NE +import qualified Yamlet as Y +import Yamlet.Aeson () +import Data.Aeson (Value(..), Object, Result(..), fromJSON, (.:?), withObject) +import Data.Aeson.Types (parse) import Text.Pandoc.Shared (tshow, blocksToInlines) -import Text.Pandoc.Class (PandocMonad (..), report) +import Text.Pandoc.Class (PandocMonad (..)) import Text.Pandoc.Definition import Text.Pandoc.Error -import Text.Pandoc.Logging (LogMessage(YamlWarning)) import Text.Pandoc.Parsing hiding (tableWith, parse) import qualified Text.Pandoc.UTF8 as UTF8 -import System.IO.Unsafe (unsafePerformIO) yamlBsToMeta :: (PandocMonad m, HasLastStrPosition st) => ParsecT Sources st m (Future st MetaValue) @@ -45,37 +44,28 @@ -> ParsecT Sources st m (Future st Meta) yamlBsToMeta pMetaValue bstr = do pos <- getPosition - case decodeAllWithWarnings bstr of - Right (warnings, xs) -> do - mapM_ (\(Yaml.DuplicateKey jpath) -> - report (YamlWarning pos $ "Duplicate key: " <> - T.pack (formatRelativePath jpath))) - warnings + case Y.decodeAll @Value bstr of + Right xs -> case xs of (Object o : _) -> fmap Meta <$> yamlMap pMetaValue o [Null] -> return . return $ mempty [] -> return . return $ mempty _ -> Prelude.fail "expected YAML object" - Left err' -> do - let msg = T.pack $ "Error parsing YAML metadata at " <> - show pos <> ":\n" <> - Yaml.prettyPrintParseException err' - throwError $ PandocParseError $ - if "did not find expected key" `T.isInfixOf` msg - then msg <> - "\nConsider enclosing the entire field in 'single quotes'" - else msg + Left errs -> throwError $ PandocParseError $ T.pack $ + "Error parsing YAML metadata at " <> show pos <> ":\n" <> + renderYamlErrors pos errs -decodeAllWithWarnings :: FromJSON a - => B.ByteString - -> (Either Yaml.ParseException ([Yaml.Warning], [a])) -decodeAllWithWarnings = either Left (\(ws,res) - -> case res of - Left s -> Left (Yaml.AesonException s) - Right v -> Right (ws, v)) - . unsafePerformIO - . Yaml.decodeAllHelper - . Y.decode +-- | Render the errors with their lines in the source, given the position of +-- the first line of the YAML input. +renderYamlErrors :: SourcePos -> NE.NonEmpty Y.Error -> String +renderYamlErrors pos = + L.intercalate "\n" . map (Y.prettyError (sourceName pos) . shift) . NE.toList + where + shift :: Y.Error -> Y.Error + shift err@Y.Error{Y.location = loc@Y.Location{Y.line = l}} + -- Line 0 means that the error has no position. + | l == 0 = err + | otherwise = err{Y.location = loc{Y.line = l + sourceLine pos - 1}} -- Returns filtered list of references. yamlBsToRefs :: (PandocMonad m, HasLastStrPosition st) @@ -83,8 +73,9 @@ -> (Text -> Bool) -- ^ Filter for id -> B.ByteString -> ParsecT Sources st m (Future st [MetaValue]) -yamlBsToRefs pMetaValue idpred bstr = - case Yaml.decodeAllEither' bstr of +yamlBsToRefs pMetaValue idpred bstr = do + pos <- getPosition + case Y.decodeAll @Value bstr of Right (Object m : _) -> do case parse (withObject "metadata" (.:? "references")) (Object m) of Success (Just refs) -> sequence <$> @@ -94,8 +85,8 @@ let refs = filter hasSelectedId $ V.toList v sequence <$> mapM (yamlToMetaValue pMetaValue) (filter hasSelectedId refs) Right _ -> return . return $ [] - Left err' -> throwError $ PandocParseError - $ T.pack $ Yaml.prettyPrintParseException err' + Left errs -> throwError $ PandocParseError + $ T.pack $ renderYamlErrors pos errs where isSelected (String t) = idpred t isSelected _ = False diff --git a/src/Text/Pandoc/Readers/ODT/ContentReader.hs b/src/Text/Pandoc/Readers/ODT/ContentReader.hs --- a/src/Text/Pandoc/Readers/ODT/ContentReader.hs +++ b/src/Text/Pandoc/Readers/ODT/ContentReader.hs @@ -74,18 +74,21 @@ , styleTrace :: [Style] -- | Keeps track of the current depth in nested lists , currentListLevel :: ListLevel - -- | Keeps track of the previous list start counters, - -- so whenever a new list continues numbering, - -- we know what number to start from. - -- If list does not continue numbering, the counter - -- is being reset. - , listContinuationStartCounters :: M.Map ListLevel Int + -- | Keeps track of the previous list start counters + -- for each list style and level, so whenever a new + -- list continues numbering, we know what number to + -- start from. The counters for deeper levels are + -- reset whenever a new list item starts. + , listContinuationStartCounters + :: M.Map (Maybe StyleName, ListLevel) Int -- | Lists may provide their own style, but they don't have -- to. If they do not, the style of a parent list may be used -- or even a default list style from the paragraph style. -- This value keeps track of the closest list style there -- currently is. , currentListStyle :: Maybe ListStyle + -- | The name of 'currentListStyle', if known. + , currentListStyleName :: Maybe StyleName -- | A map from internal anchor names to "pretty" ones. -- The mapping is a purely cosmetic one. , bookmarkAnchors :: M.Map Anchor Anchor @@ -103,7 +106,7 @@ readerState :: Styles -> [(FilePath, B.ByteString)] -> ReaderState readerState styles media = - ReaderState styles [] 0 M.empty Nothing M.empty Set.empty + ReaderState styles [] 0 M.empty Nothing Nothing M.empty Set.empty (M.fromList media) mempty -- @@ -120,10 +123,21 @@ modifyListLevel f state = state { currentListLevel = f (currentListLevel state) } -- -modifyListContinuationStartCounter :: ListLevel -> Int -> (ReaderState -> ReaderState) -modifyListContinuationStartCounter listLevel count state = - state { listContinuationStartCounters = M.insert listLevel count (listContinuationStartCounters state) } +modifyListContinuationStartCounter :: Maybe StyleName -> ListLevel -> Int + -> (ReaderState -> ReaderState) +modifyListContinuationStartCounter mStyleName listLevel count state = + state { listContinuationStartCounters = + M.insert (mStyleName, listLevel) count + (listContinuationStartCounters state) } +-- | Reset the counters of all list levels deeper than the given one. +resetDeeperListContinuationStartCounters :: ListLevel + -> (ReaderState -> ReaderState) +resetDeeperListContinuationStartCounters listLevel state = + state { listContinuationStartCounters = + M.filterWithKey (\(_, lvl) _ -> lvl <= listLevel) + (listContinuationStartCounters state) } + -- shiftListLevel :: ListLevel -> (ReaderState -> ReaderState) shiftListLevel diff = modifyListLevel (+ diff) @@ -169,11 +183,13 @@ lookupListStyle name = getStyles >>= fromMaybeF . lookupListStyleByName name -- -switchCurrentListStyle :: Maybe ListStyle -> ODTReader (Maybe ListStyle) +switchCurrentListStyle :: Maybe (StyleName, ListStyle) + -> ODTReader (Maybe (StyleName, ListStyle)) switchCurrentListStyle mListStyle = do state <- getExtraState - setExtraState $ state { currentListStyle = mListStyle } - return $ currentListStyle state + setExtraState $ state { currentListStyle = snd <$> mListStyle + , currentListStyleName = fst <$> mListStyle } + return $ (,) <$> currentListStyleName state <*> currentListStyle state -- pushStyle :: Style -> ODTReader () @@ -188,9 +204,10 @@ getCurrentListLevel = currentListLevel <$> getExtraState -- -getPreviousListStartCounter :: ListLevel -> ODTReader Int -getPreviousListStartCounter listLevel = - M.findWithDefault 0 listLevel . listContinuationStartCounters +getPreviousListStartCounter :: Maybe StyleName -> ListLevel + -> ODTReader (Maybe Int) +getPreviousListStartCounter mStyleName listLevel = + M.lookup (mStyleName, listLevel) . listContinuationStartCounters <$> getExtraState -- @@ -415,22 +432,28 @@ constructList reader = do modifyExtraState (shiftListLevel 1) listLevel <- getCurrentListLevel - listContinuationStartCounter <- getPreviousListStartCounter listLevel fStyleName <- tryC (findAttr NsText "style-name") fContNumbering <- tryC (findAttr NsText "continue-numbering") + inheritedStyleName <- currentListStyleName <$> getExtraState + -- Numbering is continued from the previous list with the same + -- list style (and level), if there is one: + let mStyleName = either (const inheritedStyleName) Just fStyleName + listContinuationStartCounter <- getPreviousListStartCounter mStyleName + listLevel + let continueNumbering = fContNumbering == Right "true" startNumForListLevelStyle mListLevelStyle - | continueNumbering = listContinuationStartCounter - | isJust mListLevelStyle = listItemStart (fromJust mListLevelStyle) - | otherwise = 1 + | continueNumbering + , Just n <- listContinuationStartCounter = n + | otherwise = maybe 1 listItemStart mListLevelStyle constructWith startNum constructor = do items <- reader modifyExtraState (shiftListLevel (-1)) - modifyExtraState (modifyListContinuationStartCounter listLevel - (startNum + length items)) + modifyExtraState (modifyListContinuationStartCounter mStyleName + listLevel (startNum + length items)) return $ constructor items constructOrderedList = @@ -449,7 +472,8 @@ Right listStyle -> case getListLevelStyle listLevel listStyle of Just listLevelStyle -> do - oldListStyle <- switchCurrentListStyle (Just listStyle) + oldListStyle <- switchCurrentListStyle + (Just (styleName, listStyle)) blocks <- constructListWith listLevelStyle _ <- switchCurrentListStyle oldListStyle return blocks @@ -681,8 +705,14 @@ read_list_element :: ElementName -> Matcher [Blocks] read_list_element listElement = matchingElement NsText listElement - $ compactify . (:[]) - <$> matchBlockContent + $ do + -- a new item restarts the numbering + -- of nested lists: + listLevel <- getCurrentListLevel + modifyExtraState + (resetDeeperListContinuationStartCounters + listLevel) + compactify . (:[]) <$> matchBlockContent ---------------------- -- Sections diff --git a/src/Text/Pandoc/Readers/RST.hs b/src/Text/Pandoc/Readers/RST.hs --- a/src/Text/Pandoc/Readers/RST.hs +++ b/src/Text/Pandoc/Readers/RST.hs @@ -1570,6 +1570,7 @@ inline :: PandocMonad m => RSTParser m Inlines inline = (note -- can start with whitespace, so try before ws + <|> quotedSymbol <|> do notAfterString >>= guard (link <|> inlineAnchor <|> strong <|> emph) <|> code @@ -1599,8 +1600,11 @@ unless (canPrecedeOpener c) updateLastStrPos return $ B.str $ T.singleton c +-- Characters that may precede an inline markup start-string: the ASCII +-- characters - : / ' " < ( [ { and similar non-ASCII punctuation. canPrecedeOpener :: Char -> Bool canPrecedeOpener c = + c == ':' || c == '/' || c == '\'' || c == '"' || c == '<' || generalCategory c `elem` [OpenPunctuation, InitialQuote, FinalQuote, DashPunctuation, OtherSymbol] @@ -1609,6 +1613,21 @@ c <- satisfy isSpecialChar unless (canPrecedeOpener c) updateLastStrPos return $ B.str $ T.singleton c + +-- A start-string between an opening character and the matching closing +-- character, as in (*) or "*", is not inline markup. +quotedSymbol :: Monad m => RSTParser m Inlines +quotedSymbol = try $ do + opener <- oneOf "([{<\"'\8216\8220" + case lookup opener openersClosers of + Nothing -> mzero + Just closer -> + B.text . (\c -> T.pack [opener, c, closer]) + <$> try (oneOf "*`|" <* char closer) + where + openersClosers = [('(', ')'), ('[', ']'), ('{', '}'), ('<', '>'), + ('"', '"'), ('\'', '\''), + ('\8216', '\8217'), ('\8220', '\8221')] -- parses inline code, between codeStart and codeEnd code :: Monad m => RSTParser m Inlines diff --git a/src/Text/Pandoc/Readers/RTF.hs b/src/Text/Pandoc/Readers/RTF.hs --- a/src/Text/Pandoc/Readers/RTF.hs +++ b/src/Text/Pandoc/Readers/RTF.hs @@ -34,7 +34,7 @@ import Text.Pandoc.Parsing import Text.Pandoc.Logging (LogMessage(UnsupportedCodePage)) import Text.Pandoc.Shared (tshow) -import Data.Char (isAlphaNum, chr, isAscii, isLetter, isSpace, ord) +import Data.Char (chr, isAscii, isLetter, isSpace, ord) import qualified Data.ByteString.Lazy as BL import Data.Maybe (mapMaybe, fromMaybe) import Safe (lastMay, initSafe, headDef) @@ -261,7 +261,8 @@ <|> (ControlSymbol <$> anyChar) ) controlWord = do name <- letterSequence - param <- parameter <* optional delimChar + param <- parameter + optional (char ' ') case name of "bin" -> do let n = fromMaybe 0 param @@ -318,9 +319,6 @@ isSpecial '\\' = True isSpecial '\n' = True isSpecial _ = False - -delimChar :: PandocMonad m => RTFParser m Char -delimChar = satisfy (\c -> not (isAlphaNum c || isSpecial c)) modifyGroup :: PandocMonad m => (Properties -> Properties) diff --git a/src/Text/Pandoc/Readers/Vimwiki.hs b/src/Text/Pandoc/Readers/Vimwiki.hs --- a/src/Text/Pandoc/Readers/Vimwiki.hs +++ b/src/Text/Pandoc/Readers/Vimwiki.hs @@ -1,5 +1,4 @@ {-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE CPP #-} {- | Module : Text.Pandoc.Readers.Vimwiki Copyright : Copyright (C) 2017-2020 Yuchen Pei diff --git a/src/Text/Pandoc/Shared.hs b/src/Text/Pandoc/Shared.hs --- a/src/Text/Pandoc/Shared.hs +++ b/src/Text/Pandoc/Shared.hs @@ -1,5 +1,4 @@ {-# LANGUAGE BangPatterns #-} -{-# LANGUAGE CPP #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE ScopedTypeVariables #-} @@ -50,6 +49,8 @@ compactify, compactifyDL, compactifyTable, + plainToPara, + paraToPlain, linesToPara, figureDiv, makeSections, @@ -457,10 +458,15 @@ -> items _ -> map (fmap plainToPara) items +-- | Convert a @Plain@ block to a @Para@; leave other blocks unchanged. plainToPara :: Block -> Block plainToPara (Plain ils) = Para ils plainToPara x = x +-- | Convert a @Para@ block to a @Plain@; leave other blocks unchanged. +paraToPlain :: Block -> Block +paraToPlain (Para ils) = Plain ils +paraToPlain x = x -- | Like @compactify@, but acts on items of definition lists. compactifyDL :: [(Inlines, [Blocks])] -> [(Inlines, [Blocks])] @@ -490,8 +496,6 @@ isSimpleCell [Plain _] = All True isSimpleCell _ = All False isSimpleTable = getAll . query isSimpleCell - paraToPlain (Para ils) = Plain ils - paraToPlain b = b -- walk descends into the notes, so we need to fix them back up: fixNotes (Note [Plain ils]) = Note [Para ils] fixNotes i = i diff --git a/src/Text/Pandoc/Translations.hs b/src/Text/Pandoc/Translations.hs --- a/src/Text/Pandoc/Translations.hs +++ b/src/Text/Pandoc/Translations.hs @@ -1,5 +1,5 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE TypeApplications #-} {- | Module : Text.Pandoc.Translations Copyright : Copyright (C) 2017-2024 John MacFarlane @@ -27,18 +27,19 @@ import Text.Pandoc.Logging (LogMessage(..)) import Control.Monad.Except (catchError) import qualified Data.Text as T -import qualified Data.Yaml as Yaml -import qualified Text.Pandoc.UTF8 as UTF8 -import Data.Yaml (prettyPrintParseException) +import qualified Data.List.NonEmpty as NE +import qualified Yamlet as Y +import Yamlet.Aeson import Text.Collate.Lang (Lang(..), renderLang) --- | Parse YAML translations. -readTranslations :: T.Text -> Either T.Text Translations -readTranslations s = - case Yaml.decodeAllEither' $ UTF8.fromText s of - Left err' -> Left $ T.pack $ prettyPrintParseException err' - Right (t:_) -> Right t - Right [] -> Left "empty YAML document" +-- | Parse YAML translations. The file path is used in error messages. +readTranslations :: FilePath -> T.Text -> Either T.Text Translations +readTranslations fp s = + case Y.decodeAllText @(ViaAeson Translations) s of + Left errs -> Left $ T.intercalate "\n" $ + map (T.pack . Y.prettyError fp) (NE.toList errs) + Right (ViaAeson t:_) -> Right t + Right [] -> Left $ T.pack fp <> ": empty YAML document" -- | Select the language to use with 'translateTerm'. -- Note that this does not read a translation file; @@ -68,12 +69,11 @@ let getTrans [] = return mempty getTrans (fp:fps) = do result <- catchError (Right <$> (readDataFile fp >>= toTextM fp)) - (\_ -> pure (Left "")) - case result >>= readTranslations of + (\_ -> pure (Left (T.pack fp))) + case result >>= readTranslations fp of Left e | null fps -> do - report $ CouldNotLoadTranslations (renderLang lang) - (T.pack fp <> ": " <> e) + report $ CouldNotLoadTranslations (renderLang lang) e -- make sure we don't try again... modifyCommonState $ \st -> st{ stTranslations = Nothing } diff --git a/src/Text/Pandoc/Translations/Types.hs b/src/Text/Pandoc/Translations/Types.hs --- a/src/Text/Pandoc/Translations/Types.hs +++ b/src/Text/Pandoc/Translations/Types.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveGeneric #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-} {-# LANGUAGE OverloadedStrings #-} diff --git a/src/Text/Pandoc/Version.hs b/src/Text/Pandoc/Version.hs --- a/src/Text/Pandoc/Version.hs +++ b/src/Text/Pandoc/Version.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {- | Module : Text.Pandoc.Version Copyright : Copyright (C) 2022-2024 John MacFarlane diff --git a/src/Text/Pandoc/Writers/ANSI.hs b/src/Text/Pandoc/Writers/ANSI.hs --- a/src/Text/Pandoc/Writers/ANSI.hs +++ b/src/Text/Pandoc/Writers/ANSI.hs @@ -79,9 +79,7 @@ metadata <- metaToContext opts (blockListToANSI opts) (inlineListToANSI opts) meta - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing + let colwidth = wrapColumns opts let title = titleBlock colwidth metadata let blocks' = makeSections (writerNumberSections opts) Nothing blocks body <- blockListToANSI opts blocks' diff --git a/src/Text/Pandoc/Writers/AsciiDoc.hs b/src/Text/Pandoc/Writers/AsciiDoc.hs --- a/src/Text/Pandoc/Writers/AsciiDoc.hs +++ b/src/Text/Pandoc/Writers/AsciiDoc.hs @@ -53,7 +53,6 @@ import Text.Pandoc.Builder (emptyCell) import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Shared import Text.Pandoc.Walk (walk) @@ -111,9 +110,6 @@ pandocToAsciiDoc opts (Pandoc meta blocks) = do let titleblock = not $ null (docTitle meta) && null (docAuthors meta) && null (docDate meta) - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing metadata <- metaToContext opts (blockListToAsciiDoc opts) (fmap chomp . inlineListToAsciiDoc opts) @@ -126,10 +122,7 @@ isJust (writerTemplate opts)) $ defField "math" (hasMath st && not (legacy st)) $ defField "titleblock" titleblock metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main data EscContext = Normal | InTable deriving (Show, Eq) diff --git a/src/Text/Pandoc/Writers/BibTeX.hs b/src/Text/Pandoc/Writers/BibTeX.hs --- a/src/Text/Pandoc/Writers/BibTeX.hs +++ b/src/Text/Pandoc/Writers/BibTeX.hs @@ -26,10 +26,10 @@ import Text.Pandoc.Citeproc.BibTeX as BibTeX import Text.Pandoc.Citeproc.MetaValue (metaValueToReference) import Text.Pandoc.Writers.Shared (lookupMetaString, defField, - addVariablesToContext) -import Text.DocLayout (render, vcat) + addVariablesToContext, + renderWithTemplate) +import Text.DocLayout (vcat) import Text.DocTemplates (Context(..)) -import Text.Pandoc.Templates (renderTemplate) -- | Write BibTeX based on the references metadata from a Pandoc document. writeBibTeX :: PandocMonad m => WriterOptions -> Pandoc -> m Text @@ -50,10 +50,4 @@ let main = vcat $ map (BibTeX.writeBibtexString opts variant mblang) refs let context = defField "body" main $ addVariablesToContext opts (mempty :: Context Text) - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main diff --git a/src/Text/Pandoc/Writers/ConTeXt.hs b/src/Text/Pandoc/Writers/ConTeXt.hs --- a/src/Text/Pandoc/Writers/ConTeXt.hs +++ b/src/Text/Pandoc/Writers/ConTeXt.hs @@ -37,7 +37,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI (isURI) -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Walk (query) import Text.Pandoc.Writers.Shared import Text.Printf (printf) @@ -86,9 +85,6 @@ pandocToConTeXt :: PandocMonad m => WriterOptions -> Pandoc -> WM m Text pandocToConTeXt options (Pandoc meta blocks) = do - let colwidth = if writerWrapText options == WrapAuto - then Just $ writerColumns options - else Nothing metadata <- metaToContext options blockListToConTeXt (fmap chomp . inlineListToConTeXt) @@ -144,10 +140,7 @@ _ -> id) metadata let context' = defField "context-dir" (maybe mempty toContextDir $ getField "dir" context) context - return $ render colwidth $ - case writerTemplate options of - Nothing -> main - Just tpl -> renderTemplate tpl context' + return $ renderWithTemplate options context' main -- change rtl to r2l, ltr to l2r toContextDir :: Doc Text -> Doc Text diff --git a/src/Text/Pandoc/Writers/Djot.hs b/src/Text/Pandoc/Writers/Djot.hs --- a/src/Text/Pandoc/Writers/Djot.hs +++ b/src/Text/Pandoc/Writers/Djot.hs @@ -27,11 +27,11 @@ import qualified Data.Text as T import qualified Data.Map as M import qualified Text.Pandoc.UTF8 as UTF8 -import Text.Pandoc.Writers.Shared ( metaToContext, defField, toLegacyTable ) +import Text.Pandoc.Writers.Shared ( metaToContext, defField, toLegacyTable, + renderWithTemplate ) import Text.Pandoc.Shared (isTightList, tshow, stringifyInlines, onlySimpleTableCells, makeSections) import Text.DocLayout -import Text.DocTemplates (renderTemplate) import Control.Monad.State (StateT(..), gets, modify) import Control.Monad (zipWithM, when) @@ -43,9 +43,6 @@ -- | Convert Pandoc to Djot. writeDjot :: PandocMonad m => WriterOptions -> Pandoc -> m Text writeDjot opts (Pandoc meta blocks) = do - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing let ropts = RenderOptions{ preserveSoftBreaks = writerWrapText opts == WrapPreserve } metadata <- metaToContext opts @@ -56,10 +53,7 @@ main <- renderDjot ropts <$> bodyToDjot opts (makeSections False Nothing blocks) let context = defField "body" main metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main data DjotState = DjotState diff --git a/src/Text/Pandoc/Writers/DocBook.hs b/src/Text/Pandoc/Writers/DocBook.hs --- a/src/Text/Pandoc/Writers/DocBook.hs +++ b/src/Text/Pandoc/Writers/DocBook.hs @@ -66,9 +66,7 @@ authorToDocBook :: PandocMonad m => WriterOptions -> [Inline] -> DB m B.Inlines authorToDocBook opts name' = do name <- render Nothing <$> inlinesToDocBook opts name' - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing + let colwidth = wrapColumns opts return $ B.rawInline "docbook" $ render colwidth $ inTags True "personname" [] $ if T.any (== ',') name @@ -99,9 +97,7 @@ writeDocBook :: PandocMonad m => WriterOptions -> Pandoc -> DB m Text writeDocBook opts doc = do let Pandoc meta blocks = ensureValidXmlIdentifiers doc - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing + let colwidth = wrapColumns opts let startLvl = getStartLvl opts let fromBlocks = blocksToDocBook opts . makeSections False (Just startLvl) @@ -126,11 +122,6 @@ blocksToDocBook :: PandocMonad m => WriterOptions -> [Block] -> DB m (Doc Text) blocksToDocBook opts = fmap vcat . mapM (blockToDocBook opts) --- | Auxiliary function to convert Plain block to Para. -plainToPara :: Block -> Block -plainToPara (Plain x) = Para x -plainToPara x = x - -- | Convert a list of pairs of terms and definitions into a list of -- DocBook varlistentrys. deflistItemsToDocBook :: PandocMonad m @@ -162,8 +153,8 @@ inTagsIndented "listitem" <$> blocksToDocBook opts (map plainToPara item) imageToDocBook :: WriterOptions -> Attr -> Text -> Doc Text -imageToDocBook _ attr src = selfClosingTag "imagedata" $ - ("fileref", src) : idAndRole attr <> dims +imageToDocBook opts attr src = selfClosingTag "imagedata" $ + ("fileref", src) : idAndRole opts attr <> dims where dims = go Width "width" <> go Height "depth" go dir dstr = case dimension dir attr of @@ -206,7 +197,8 @@ return $ inTags True tag attribs $ inTagsSimple "title" title' $$ contents blockToDocBook opts (Div (ident,classes,_) bs) = do version <- ask - let identAttribs = [(idName version, ident) | not (T.null ident)] + let identAttribs = [(idName version, writerIdentifierPrefix opts <> ident) + | not (T.null ident)] admonitions = ["caution","danger","important","note","tip","warning"] case classes of (l:_) | l `elem` admonitions -> do @@ -396,7 +388,8 @@ version <- ask ((if T.null ident then mempty - else selfClosingTag "anchor" [(idName version, ident)]) <>) <$> + else selfClosingTag "anchor" + [(idName version, writerIdentifierPrefix opts <> ident)]) <>) <$> inlinesToDocBook opts ils inlineToDocBook _ (Code _ str) = return $ inTagsSimple "literal" $ literal (escapeStringForXML str) @@ -444,10 +437,10 @@ then let tag = if null txt then "xref" else "link" in inTags False tag $ ("linkend", writerIdentifierPrefix opts <> T.drop 1 src) : - idAndRole attr + idAndRole opts attr else if version == DocBook5 - then inTags False "link" $ ("xlink:href", src) : idAndRole attr - else inTags False "ulink" $ ("url", src) : idAndRole attr ) + then inTags False "link" $ ("xlink:href", src) : idAndRole opts attr + else inTags False "ulink" $ ("url", src) : idAndRole opts attr ) <$> inlinesToDocBook opts txt inlineToDocBook opts (Image attr ils (src, tit)) = return $ let titleDoc = if T.null tit @@ -469,10 +462,10 @@ isMathML MathML = True isMathML _ = False -idAndRole :: Attr -> [(Text, Text)] -idAndRole (id',cls,_) = ident <> role +idAndRole :: WriterOptions -> Attr -> [(Text, Text)] +idAndRole opts (id',cls,_) = ident <> role where - ident = [("id", id') | not (T.null id')] + ident = [("id", writerIdentifierPrefix opts <> id') | not (T.null id')] role = [("role", T.unwords cls) | not (null cls)] -- Used in blockToDocBook for Header (section) to create or extend diff --git a/src/Text/Pandoc/Writers/Docx/OpenXML.hs b/src/Text/Pandoc/Writers/Docx/OpenXML.hs --- a/src/Text/Pandoc/Writers/Docx/OpenXML.hs +++ b/src/Text/Pandoc/Writers/Docx/OpenXML.hs @@ -790,13 +790,19 @@ listLevel <- asks envListLevel numid <- asks envListNumId numIdUsed <- gets stNumIdUsed - -- clear numId after first use to support multiple paragraphs in the same bullet - -- baseListId is the code for no list marker - let numid' = if numIdUsed then baseListId else numid modify $ \st -> st{ stNumIdUsed = True } - let listPr = [mknode "w:numPr" [] - [ mknode "w:ilvl" [("w:val",tshow listLevel)] () - , mknode "w:numId" [("w:val",tshow numid')] () ] | listLevel >= 0 && not displayMathPara] + -- Only the first paragraph of a list item gets numbering. Subsequent + -- paragraphs in the same item are not numbered, but indented to line + -- up with the item's text (see the w:ind in mkLvl), so that they are + -- not mistaken for items of another list (#9197): + let listPr + | listLevel < 0 || displayMathPara = [] + | numIdUsed = [mknode "w:ind" + [ ("w:left", tshow (listLevel * 720 + 720)) + , ("w:firstLine", "0") ] ()] + | otherwise = [mknode "w:numPr" [] + [ mknode "w:ilvl" [("w:val",tshow listLevel)] () + , mknode "w:numId" [("w:val",tshow numid)] () ]] return $ case squashProps pPrTagOrder (EnvProps Nothing listPr <> props) of [] -> [] ps -> [mknode "w:pPr" [] ps] diff --git a/src/Text/Pandoc/Writers/EPUB.hs b/src/Text/Pandoc/Writers/EPUB.hs --- a/src/Text/Pandoc/Writers/EPUB.hs +++ b/src/Text/Pandoc/Writers/EPUB.hs @@ -1,4 +1,3 @@ -{-# LANGUAGE CPP #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE PatternGuards #-} {-# LANGUAGE OverloadedStrings #-} diff --git a/src/Text/Pandoc/Writers/HTML.hs b/src/Text/Pandoc/Writers/HTML.hs --- a/src/Text/Pandoc/Writers/HTML.hs +++ b/src/Text/Pandoc/Writers/HTML.hs @@ -960,8 +960,6 @@ classes' = case slideVariant of NoSlides -> classes _ -> filter (\k -> k /= "incremental" && k /= "nonincremental") classes - let paraToPlain (Para ils) = Plain ils - paraToPlain x = x let bs' = if "csl-entry" `elem` classes' then walk paraToPlain bs else bs diff --git a/src/Text/Pandoc/Writers/Haddock.hs b/src/Text/Pandoc/Writers/Haddock.hs --- a/src/Text/Pandoc/Writers/Haddock.hs +++ b/src/Text/Pandoc/Writers/Haddock.hs @@ -16,7 +16,7 @@ module Text.Pandoc.Writers.Haddock (writeHaddock) where import Control.Monad (zipWithM) import Control.Monad.State.Strict - ( StateT, MonadState(get), modify, evalStateT ) + ( StateT, MonadState(get), gets, modify, evalStateT ) import Data.Char (isAlphaNum) import Data.Default import Data.Text (Text) @@ -28,7 +28,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Shared type Notes = [[Block]] @@ -46,32 +45,35 @@ pandocToHaddock :: PandocMonad m => WriterOptions -> Pandoc -> StateT WriterState m Text pandocToHaddock opts (Pandoc meta blocks) = do - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing body <- blockListToHaddock opts blocks - st <- get - notes' <- notesToHaddock opts (reverse $ stNotes st) + notes' <- notesToHaddock opts let main = body <> (if isEmpty notes' then empty else blankline <> notes') metadata <- metaToContext opts (blockListToHaddock opts) (fmap chomp . inlineListToHaddock opts) meta let context = defField "body" main metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main --- | Return haddock representation of notes. +-- | Return haddock representation of the collected notes. Rendering a +-- note may add further notes to the state (notes nested inside +-- notes); keep going until all of them have been rendered. notesToHaddock :: PandocMonad m - => WriterOptions -> [[Block]] -> StateT WriterState m (Doc Text) -notesToHaddock opts notes = - if null notes - then return empty - else do - contents <- blockToHaddock opts $ OrderedList (1,DefaultStyle,DefaultDelim) notes - return $ text "#notes#" <> blankline <> contents + => WriterOptions -> StateT WriterState m (Doc Text) +notesToHaddock opts = do + contents <- go 0 + return $ if null contents + then empty + else text "#notes#" <> blankline <> vcat contents + where + go done = do + notes <- gets (drop done . reverse . stNotes) + if null notes + then return [] + else do + contents <- blockToHaddock opts $ + OrderedList (done + 1, DefaultStyle, DefaultDelim) notes + (contents :) <$> go (done + length notes) -- | Escape special characters for Haddock. escapeString :: Text -> Text @@ -170,7 +172,7 @@ -> StateT WriterState m (Doc Text) orderedListItemToHaddock opts marker items = do contents <- blockListToHaddock opts items - let sps = case T.length marker - writerTabStop opts of + let sps = case writerTabStop opts - T.length marker of n | n > 0 -> text $ replicate n ' ' _ -> text " " let start = literal marker <> sps diff --git a/src/Text/Pandoc/Writers/ICML.hs b/src/Text/Pandoc/Writers/ICML.hs --- a/src/Text/Pandoc/Writers/ICML.hs +++ b/src/Text/Pandoc/Writers/ICML.hs @@ -138,9 +138,7 @@ writeICML :: PandocMonad m => WriterOptions -> Pandoc -> m Text writeICML opts doc = do let Pandoc meta blocks = ensureValidXmlIdentifiers doc - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing + let colwidth = wrapColumns opts renderBlockMeta f s = fst <$> runStateT (f opts [] s) defaultWriterState renderInlineMeta f s = fst <$> runStateT (f opts [] "" s) defaultWriterState metadata <- metaToContext opts diff --git a/src/Text/Pandoc/Writers/JATS.hs b/src/Text/Pandoc/Writers/JATS.hs --- a/src/Text/Pandoc/Writers/JATS.hs +++ b/src/Text/Pandoc/Writers/JATS.hs @@ -170,9 +170,7 @@ (Header lev (ident,hcls,hkvs) hils : rs) : bs) splitBackBlocks b (fs, bs) = (b:fs, bs) let (bodyblocks, backblocks) = foldr splitBackBlocks ([],[]) blocks - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing + let colwidth = wrapColumns opts metadata <- metaToContext opts (blocksToJATS opts . makeSections False (Just startLvl)) (fmap chomp . inlinesToJATS opts) @@ -240,11 +238,6 @@ then inTags True "p" [("specific-use","wrapper")] inner else inner --- | Auxiliary function to convert Plain block to Para. -plainToPara :: Block -> Block -plainToPara (Plain x) = Para x -plainToPara x = x - -- | Convert a list of pairs of terms and definitions into a list of -- JATS varlistentrys. deflistItemsToJATS :: PandocMonad m @@ -302,7 +295,7 @@ codeAttr :: WriterOptions -> Attr -> (Text, [(Text, Text)]) codeAttr opts (ident,classes,kvs) = (lang, attr) where - attr = [("id", escapeNCName ident) | not (T.null ident)] ++ + attr = idAttr opts ident ++ [("language",lang) | not (T.null lang)] ++ [(k,v) | (k,v) <- kvs, k `elem` ["code-type", "code-version", "executable", @@ -310,6 +303,12 @@ "platforms", "position", "specific-use"]] lang = languageFor opts classes +-- | The id attribute for an element, with the identifier prefix added. +idAttr :: WriterOptions -> Text -> [(Text, Text)] +idAttr opts ident = + [("id", writerIdentifierPrefix opts <> escapeNCName ident) + | not (T.null ident)] + -- is only allowed as a direct child of or or -- <article-title> fixLineBreak :: Inline -> Inline @@ -319,10 +318,8 @@ -- | Convert a Pandoc block element to JATS. blockToJATS :: PandocMonad m => WriterOptions -> Block -> JATS m (Doc Text) blockToJATS opts (Div (id',"section":_,kvs) (Header _lvl (_,_,hkvs) ils : xs)) = do - let idAttr = [ ("id", writerIdentifierPrefix opts <> escapeNCName id') - | not (T.null id')] let otherAttrs = ["sec-type", "specific-use"] - let attribs = idAttr ++ [(k,v) | (k,v) <- kvs, k `elem` otherAttrs] + let attribs = idAttr opts id' ++ [(k,v) | (k,v) <- kvs, k `elem` otherAttrs] title' <- inlinesToJATS opts (map fixLineBreak ils) let label = if writerNumberSections opts then @@ -336,7 +333,7 @@ inTagsSimple "title" title' $$ contents -- Bibliography reference: blockToJATS opts (Div (ident,_,_) [Para lst]) | "ref-" `T.isPrefixOf` ident = - inTags True "ref" [("id", escapeNCName ident)] . + inTags True "ref" (idAttr opts ident) . inTagsSimple "mixed-citation" <$> inlinesToJATS opts lst blockToJATS opts (Div ("refs",_,_) xs) = do @@ -353,7 +350,7 @@ return $ inTagsIndented "ref-list" contents blockToJATS opts (Div (ident,[cls],kvs) bs) | cls `elem` ["fig", "caption", "table-wrap"] = do contents <- blocksToJATS opts bs - let attr = [("id", escapeNCName ident) | not (T.null ident)] ++ + let attr = idAttr opts ident ++ [("xml:lang",l) | ("lang",l) <- kvs] ++ [(k,v) | (k,v) <- kvs, k `elem` ["specific-use", "content-type", "orientation", "position"]] @@ -361,7 +358,7 @@ blockToJATS opts (Div (ident,_,kvs) bs) = do contents <- blocksToJATS opts bs -- Attributes that are allowed on both @<p>@ and @<boxed-text>@ elements - let generic_attr = [("id", escapeNCName ident) | not (T.null ident)] ++ + let generic_attr = idAttr opts ident ++ [("xml:lang",l) | ("lang",l) <- kvs] ++ [(k,v) | (k,v) <- kvs, k `elem` ["specific-use", "content-type"]] @@ -380,10 +377,10 @@ title' <- inlinesToJATS opts (map fixLineBreak title) return $ inTagsSimple "title" title' -- Special cases for bare images, which are rendered as graphics -blockToJATS _opts (Plain [Image attr alt tgt]) = - return $ graphic attr alt tgt -blockToJATS _opts (Para [Image attr alt tgt]) = - return $ graphic attr alt tgt +blockToJATS opts (Plain [Image attr alt tgt]) = + return $ graphic opts attr alt tgt +blockToJATS opts (Para [Image attr alt tgt]) = + return $ graphic opts attr alt tgt -- No Plain, everything needs to be in a block-level tag blockToJATS opts (Plain lst) = blockToJATS opts (Para lst) blockToJATS opts (Para lst) = @@ -476,7 +473,7 @@ else ("fig" , ["fig-type", "orientation", "position", "specific-use"] ) - let xmlattr = [("id", escapeNCName ident) | not (T.null ident)] ++ + let xmlattr = idAttr opts ident ++ [(k,v) | (k,v) <- kvs , k `elem` allowedAttributes] return $ inTags True tag xmlattr $ capt $$ children @@ -544,19 +541,20 @@ let notenum = case notes of (n, _):_ -> n + 1 [] -> 1 - thenote <- inTags True "fn" [("id", "fn" <> tshow notenum)] + thenote <- inTags True "fn" (idAttr opts ("fn" <> tshow notenum)) . (inTagsSimple "label" (literal $ tshow notenum) <>) <$> wrappedBlocksToJATS (not . isPara) opts (walk demoteHeaderAndRefs contents) modify $ \st -> st{ jatsNotes = (notenum, thenote) : notes } return $ inTags False "xref" [("ref-type", "fn"), - ("rid", "fn" <> tshow notenum)] + ("rid", writerIdentifierPrefix opts <> + "fn" <> tshow notenum)] $ text (show notenum) inlineToJATS opts (Cite _ lst) = inlinesToJATS opts lst inlineToJATS opts (Span (ident,classes,kvs) ils) = do contents <- inlinesToJATS opts ils - let commonAttr = [("id", escapeNCName ident) | not (T.null ident)] ++ + let commonAttr = idAttr opts ident ++ [("xml:lang",l) | ("lang",l) <- kvs] ++ [(k,v) | (k,v) <- kvs, k `elem` ["alt", "specific-use"]] -- A named-content element is a good fit for spans, but requires a @@ -615,9 +613,9 @@ return $ inTagsSimple "email" $ literal (escapeStringForXML email) inlineToJATS opts (Link (ident,_,kvs) txt (T.uncons -> Just ('#', src), _)) = do let attr = mconcat - [ [("id", escapeNCName ident) | not (T.null ident)] + [ idAttr opts ident , [("alt", stringifyInlines txt) | not (null txt)] - , [("rid", escapeNCName src)] + , [("rid", writerIdentifierPrefix opts <> escapeNCName src)] , [(k,v) | (k,v) <- kvs, k `elem` ["ref-type", "specific-use"]] , [("ref-type", "bibr") | "ref-" `T.isPrefixOf` src] ] @@ -627,7 +625,7 @@ contents <- inlinesToJATS opts txt return $ inTags False "xref" attr contents inlineToJATS opts (Link (ident,_,kvs) txt (src, tit)) = do - let attr = [("id", escapeNCName ident) | not (T.null ident)] ++ + let attr = idAttr opts ident ++ [("ext-link-type", "uri"), ("xlink:href", src)] ++ [("xlink:title", tit) | not (T.null tit)] ++ @@ -637,23 +635,23 @@ "xlink:type"]] contents <- inlinesToJATS opts txt return $ inTags False "ext-link" attr contents -inlineToJATS _ (Image attr alt tgt) = do - let elattr = graphicAttr attr alt tgt +inlineToJATS opts (Image attr alt tgt) = do + let elattr = graphicAttr opts attr alt tgt return $ case altToJATS alt of Nothing -> selfClosingTag "inline-graphic" elattr Just altTag -> inTags True "inline-graphic" elattr altTag -graphic :: Attr -> [Inline] -> Target -> (Doc Text) -graphic attr alt tgt = - let elattr = graphicAttr attr alt tgt +graphic :: WriterOptions -> Attr -> [Inline] -> Target -> (Doc Text) +graphic opts attr alt tgt = + let elattr = graphicAttr opts attr alt tgt in case altToJATS alt of Nothing -> selfClosingTag "graphic" elattr Just altTag -> inTags True "graphic" elattr altTag -graphicAttr :: Attr -> [Inline] -> Target -> [(Text, Text)] -graphicAttr (ident, _, kvs) _alt (src, tit) = +graphicAttr :: WriterOptions -> Attr -> [Inline] -> Target -> [(Text, Text)] +graphicAttr opts (ident, _, kvs) _alt (src, tit) = let (maintype, subtype) = imageMimeType src kvs - in [("id", escapeNCName ident) | not (T.null ident)] ++ + in idAttr opts ident ++ [ ("mimetype", maintype) , ("mime-subtype", subtype) , ("xlink:href", src) diff --git a/src/Text/Pandoc/Writers/JATS/References.hs b/src/Text/Pandoc/Writers/JATS/References.hs --- a/src/Text/Pandoc/Writers/JATS/References.hs +++ b/src/Text/Pandoc/Writers/JATS/References.hs @@ -26,7 +26,7 @@ import Text.DocLayout (Doc, empty, isEmpty, literal, vcat) import Text.Pandoc.Class.PandocMonad (PandocMonad) import Text.Pandoc.Builder (Inlines) -import Text.Pandoc.Options (WriterOptions) +import Text.Pandoc.Options (WriterOptions (..)) import Text.Pandoc.Shared (tshow) import Text.Pandoc.Writers.JATS.Types import Text.Pandoc.XML (escapeNCName, escapeStringForXML, inTags) @@ -43,11 +43,11 @@ => WriterOptions -> Reference Inlines -> JATS m (Doc Text) -referenceToJATS _opts ref = do +referenceToJATS opts ref = do let refType = referenceType ref let pubType = [("publication-type", refType) | not (T.null refType)] let ident = escapeNCName $ "ref-" <> unItemId (referenceId ref) - let wrap = inTags True "ref" [("id", ident)] + let wrap = inTags True "ref" [("id", writerIdentifierPrefix opts <> ident)] . inTags True "element-citation" pubType return . wrap . vcat $ [ authors diff --git a/src/Text/Pandoc/Writers/JATS/Table.hs b/src/Text/Pandoc/Writers/JATS/Table.hs --- a/src/Text/Pandoc/Writers/JATS/Table.hs +++ b/src/Text/Pandoc/Writers/JATS/Table.hs @@ -21,7 +21,7 @@ import Text.DocLayout (Doc, empty, vcat, ($$)) import Text.Pandoc.Class.PandocMonad (PandocMonad) import Text.Pandoc.Definition -import Text.Pandoc.Options (WriterOptions) +import Text.Pandoc.Options (WriterOptions (..)) import Text.Pandoc.Shared (tshow) import Text.Pandoc.Writers.JATS.Types import Text.Pandoc.XML (escapeNCName, inTags, inTagsIndented, selfClosingTag) @@ -65,7 +65,7 @@ , "frame", "rules", "specific-use", "style", "summary" , "width" ] - let attribs = toAttribs attr validAttribs + let attribs = toAttribs opts attr validAttribs return $ inTags True "table" attribs $ vcat [ colSpecListToJATS colspecs , head' @@ -82,7 +82,7 @@ -> Ann.TableBody -> JATS m (Doc Text) tableBodyToJats opts (Ann.TableBody attr _rowHeadCols inthead rows) = do - let attribs = toAttribs attr validTablePartAttribs + let attribs = toAttribs opts attr validTablePartAttribs intermediateHead <- if null inthead then return mempty else headerRowsToJats opts Thead inthead @@ -117,7 +117,7 @@ Thead -> "thead" Tfoot -> "tfoot" Tbody -> "tbody" -- this would be unexpected - let attribs = toAttribs attr validTablePartAttribs + let attribs = toAttribs opts attr validTablePartAttribs inTags True tag' attribs <$> headerRowsToJats opts tblpart rows where isEmptyRow (Ann.HeaderRow _attr _rownum cells) = all isEmptyCell cells @@ -183,7 +183,7 @@ let validAttribs = [ "align", "char", "charoff", "content-type" , "style", "valign" ] - let attr' = toAttribs attr validAttribs + let attr' = toAttribs opts attr validAttribs let celltype = case tblpart of Thead -> HeaderCell _ -> BodyCell @@ -220,9 +220,11 @@ let align = fst colspec in tableCellToJats opts celltype align cell -toAttribs :: Attr -> [Text] -> [(Text, Text)] -toAttribs (ident, _classes, kvs) knownAttribs = - (if T.null ident then id else (("id", escapeNCName ident) :)) $ +toAttribs :: WriterOptions -> Attr -> [Text] -> [(Text, Text)] +toAttribs opts (ident, _classes, kvs) knownAttribs = + (if T.null ident + then id + else (("id", writerIdentifierPrefix opts <> escapeNCName ident) :)) $ filter ((`elem` knownAttribs) . fst) kvs tableCellToJats :: PandocMonad m @@ -255,7 +257,7 @@ let attribs = maybeCons (alignmentAttrib align') . maybeCons (rowspanAttrib rowspan) . maybeCons (colspanAttrib colspan) - $ toAttribs attr validAttribs + $ toAttribs opts attr validAttribs inTags False tag' attribs <$> cellContents item -- | Whether the JATS produced from this block should be wrapped in a diff --git a/src/Text/Pandoc/Writers/LaTeX.hs b/src/Text/Pandoc/Writers/LaTeX.hs --- a/src/Text/Pandoc/Writers/LaTeX.hs +++ b/src/Text/Pandoc/Writers/LaTeX.hs @@ -40,7 +40,7 @@ import Data.Text (Text) import qualified Data.Text as T import Network.URI (unEscapeString) -import Text.DocTemplates (Context(..), FromContext(lookupContext), Val(..), renderTemplate) +import Text.DocTemplates (Context(..), FromContext(lookupContext), Val(..)) import qualified Data.Map as M import Text.Collate.Lang (renderLang) import Text.Pandoc.Class.PandocMonad (PandocMonad, getPOSIXTime, lookupEnv, @@ -128,9 +128,6 @@ let blocks' = if method == Biblatex || method == Natbib then filter (not . isRefsDiv) blocks else blocks - let colwidth = if writerWrapText options == WrapAuto - then Just $ writerColumns options - else Nothing docLangs <- catMaybes <$> mapM (toLang . Just) (nubOrd (query (extract "lang") blocks)) mblang <- toLang $ case getLang options meta of @@ -326,10 +323,7 @@ $ defField "latex-dir-rtl" ((render Nothing <$> getField "dir" context) == Just ("rtl" :: Text)) context - return $ render colwidth $ - case writerTemplate options of - Nothing -> main - Just tpl -> renderTemplate tpl context' + return $ renderWithTemplate options context' main -- Commas in title-meta need to be put in braces; see #10501 escapeCommas :: Text -> Text @@ -771,12 +765,12 @@ modify $ \s -> s{ stInItem = True } term' <- withExternalNotes $ inlineListToLaTeX term modify $ \s -> s{ stInItem = False } - -- put braces around term if it contains an internal link, - -- since otherwise we get bad bracket interactions: \item[\hyperref[..] - let isInternalLink (Link _ _ (src,_)) + -- put braces around term if it contains an internal link or an =: + let needsBraces (Link _ _ (src,_)) | Just ('#', _) <- T.uncons src = True - isInternalLink _ = False - let term'' = if any isInternalLink term + needsBraces (Str t) = T.any (=='=') t + needsBraces _ = False + let term'' = if any needsBraces term then braces term' else term' def' <- case concat defs of diff --git a/src/Text/Pandoc/Writers/Man.hs b/src/Text/Pandoc/Writers/Man.hs --- a/src/Text/Pandoc/Writers/Man.hs +++ b/src/Text/Pandoc/Writers/Man.hs @@ -31,7 +31,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Math import Text.Pandoc.Writers.Shared import Text.Pandoc.Writers.Roff @@ -48,9 +47,6 @@ -- | Return roff man representation of document. pandocToMan :: PandocMonad m => WriterOptions -> Pandoc -> StateT WriterState m Text pandocToMan opts (Pandoc meta blocks) = do - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing titleText <- inlineListToMan opts $ docTitle meta let title' = render Nothing titleText let setFieldsFromTitle = @@ -80,10 +76,7 @@ $ setFieldsFromTitle $ defField "has-tables" hasTables metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main escString :: WriterOptions -> Text -> Text escString opts = escapeString True (if writerPreferAscii opts diff --git a/src/Text/Pandoc/Writers/Markdown.hs b/src/Text/Pandoc/Writers/Markdown.hs --- a/src/Text/Pandoc/Writers/Markdown.hs +++ b/src/Text/Pandoc/Writers/Markdown.hs @@ -43,7 +43,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.Writers.Shared -import Text.Pandoc.Templates (renderTemplate) import Text.DocTemplates (Val(..), Context(..), FromContext(..)) import Text.Pandoc.Walk import Text.Pandoc.Writers.HTML (writeHtml5String) @@ -222,9 +221,6 @@ -- | Return markdown representation of document. pandocToMarkdown :: PandocMonad m => WriterOptions -> Pandoc -> MD m Text pandocToMarkdown opts (Pandoc meta blocks) = do - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing variant <- asks envVariant metadata <- metaToContext' (blockListToMarkdown opts) @@ -272,10 +268,7 @@ then id else defField "titleblock" titleblock) $ addVariablesToContext opts metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main -- | Return markdown representation of reference key table. refsToMarkdown :: PandocMonad m => WriterOptions -> Refs -> MD m (Doc Text) @@ -295,13 +288,22 @@ (literal (linkDestination src) <> tit') <+> linkAttributes opts attr --- | Return markdown representation of notes. -notesToMarkdown :: PandocMonad m => WriterOptions -> [[Block]] -> MD m (Doc Text) -notesToMarkdown opts notes = do - n <- gets stNoteNum - notes' <- zipWithM (noteToMarkdown opts) [n..] notes - modify $ \st -> st { stNoteNum = stNoteNum st + length notes } - return $ vsep notes' +-- | Return markdown representation of the collected notes, clearing +-- them from the state. Rendering a note may add further notes to the +-- state (notes nested inside notes); keep going until all of them +-- have been rendered. +notesToMarkdown :: PandocMonad m => WriterOptions -> MD m (Doc Text) +notesToMarkdown opts = vsep <$> go + where + go = do + notes <- gets (reverse . stNotes) + if null notes + then return [] + else do + n <- gets stNoteNum + modify $ \st -> st { stNotes = [], stNoteNum = n + length notes } + docs <- zipWithM (noteToMarkdown opts) [n..] notes + (docs ++) <$> go -- | Return markdown representation of a note. noteToMarkdown :: PandocMonad m => WriterOptions -> Int -> [Block] -> MD m (Doc Text) @@ -347,8 +349,7 @@ notesAndRefs :: PandocMonad m => WriterOptions -> MD m (Doc Text) notesAndRefs opts = do - notes' <- gets stNotes >>= notesToMarkdown opts . reverse - modify $ \s -> s { stNotes = [] } + notes' <- notesToMarkdown opts refs' <- gets stRefs >>= refsToMarkdown opts . reverse modify $ \s -> s { stPrevRefs = stPrevRefs s ++ stRefs s , stRefs = []} @@ -621,7 +622,10 @@ let (_,cls,_) = attribs in case getLangFromClasses opts cls of Just l -> " " <> literal l - Nothing -> empty + Nothing -> -- see #11987 + case cls of + l:_ -> literal l + [] -> empty blockToMarkdown' opts (BlockQuote blocks) = do variant <- asks envVariant -- if we're writing literate haskell, put a space before the bird tracks @@ -689,9 +693,7 @@ isEnabled Ext_multiline_tables opts -> do tbl <- mkTable (pandocTable opts True) return $ nest 2 (tbl $$ caption''') $$ blankline - | isEnabled Ext_grid_tables opts && - (hasColRowSpans || writerColumns opts >= 8 * numcols - || hasFooter) -> do + | isEnabled Ext_grid_tables opts -> do tbl <- gridTable opts blockListToMarkdown specs thead tbody tfoot return $ (tbl $$ caption''') $$ blankline diff --git a/src/Text/Pandoc/Writers/Ms.hs b/src/Text/Pandoc/Writers/Ms.hs --- a/src/Text/Pandoc/Writers/Ms.hs +++ b/src/Text/Pandoc/Writers/Ms.hs @@ -44,7 +44,6 @@ import Text.DocLayout hiding (Color) import Text.DocTemplates (lookupContext) import Text.Pandoc.Shared -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Math import Text.Pandoc.Writers.Shared import Text.Pandoc.Writers.Roff @@ -62,9 +61,6 @@ -- | Return roff ms representation of document. pandocToMs :: PandocMonad m => WriterOptions -> Pandoc -> MS m Text pandocToMs opts (Pandoc meta blocks) = do - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing title <- chomp <$> inlineListToMs' opts (lookupMetaInlines "title" meta) metadata <- metaToContext opts (blockListToMs opts) @@ -91,10 +87,7 @@ $ defField "author-meta" (T.intercalate "; " authorsMeta) $ defField "highlighting-macros" highlightingMacros $ resetField "title" title metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main escapeStr :: WriterOptions -> Text -> Text escapeStr opts = diff --git a/src/Text/Pandoc/Writers/Muse.hs b/src/Text/Pandoc/Writers/Muse.hs --- a/src/Text/Pandoc/Writers/Muse.hs +++ b/src/Text/Pandoc/Writers/Muse.hs @@ -47,7 +47,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Math import Text.Pandoc.Writers.Shared @@ -108,9 +107,6 @@ -> Muse m Text pandocToMuse (Pandoc meta blocks) = do opts <- asks envOptions - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing metadata <- metaToContext opts blockListToMuse (fmap chomp . inlineListToMuse) @@ -119,10 +115,7 @@ notes <- currentNotesToMuse let main = body $+$ notes let context = defField "body" main metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main -- | Helper function for flatBlockListToMuse -- | Render all blocks and insert blank lines between the first two diff --git a/src/Text/Pandoc/Writers/ODT.hs b/src/Text/Pandoc/Writers/ODT.hs --- a/src/Text/Pandoc/Writers/ODT.hs +++ b/src/Text/Pandoc/Writers/ODT.hs @@ -305,10 +305,21 @@ lang <- toLang (getLang opts meta) refArchive <- case writerReferenceDoc opts of - Just f -> lift $ toArchive . B.fromStrict . fst <$> + Just f -> do + arch <- lift $ toArchiveOrFail . B.fromStrict . fst <$> (P.fetchItem (T.pack f)) - Nothing -> lift $ toArchive . B.fromStrict <$> + case arch of + Left err -> throwError $ PandocParseError $ + "Could not parse reference-doc " <> + tshow f <> ": " <> T.pack err + Right x -> pure x + Nothing -> do + arch <- lift $ toArchiveOrFail . B.fromStrict <$> readDataFile "reference.odt" + case arch of + Left err -> throwError $ PandocParseError $ + "Could not parse reference.odt: " <> T.pack err + Right x -> pure x -- handle formulas and pictures -- picEntriesRef <- P.newIORef ([] :: [Entry]) let refTextWidth = referenceTextWidthPt opts refArchive diff --git a/src/Text/Pandoc/Writers/OPML.hs b/src/Text/Pandoc/Writers/OPML.hs --- a/src/Text/Pandoc/Writers/OPML.hs +++ b/src/Text/Pandoc/Writers/OPML.hs @@ -1,5 +1,4 @@ {-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE CPP #-} {- | Module : Text.Pandoc.Writers.OPML Copyright : Copyright (C) 2013-2024 John MacFarlane @@ -30,9 +29,7 @@ -- | Convert Pandoc document to string in OPML format. writeOPML :: PandocMonad m => WriterOptions -> Pandoc -> m Text writeOPML opts (Pandoc meta blocks) = do - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing + let colwidth = wrapColumns opts meta' = B.setMeta "date" (B.str $ convertDate $ docDate meta) meta metadata <- metaToContext opts (fmap literal . writeMarkdown' opts . Pandoc nullMeta) diff --git a/src/Text/Pandoc/Writers/OpenDocument.hs b/src/Text/Pandoc/Writers/OpenDocument.hs --- a/src/Text/Pandoc/Writers/OpenDocument.hs +++ b/src/Text/Pandoc/Writers/OpenDocument.hs @@ -32,8 +32,7 @@ import Text.Pandoc.Logging import Text.Pandoc.Options import Text.DocLayout hiding (link) -import Text.Pandoc.Shared (linesToPara, tshow, blocksToInlines) -import Text.Pandoc.Templates (renderTemplate) +import Text.Pandoc.Shared (linesToPara, plainToPara, tshow, blocksToInlines) import qualified Text.Pandoc.Translations as Term (Term(Figure, Table)) import Text.Pandoc.Walk import Text.Pandoc.Writers.Math @@ -44,11 +43,6 @@ import Text.Pandoc.Highlighting (highlight) import Skylighting (FormatOptions(..), SourceLine, Token) --- | Auxiliary function to convert Plain block to Para. -plainToPara :: Block -> Block -plainToPara (Plain x) = Para x -plainToPara x = x - -- Names of predefined styles in the reference.odt; see data/odt/styles.xml. defaultBulletListStyleName, defaultNumberedListStyleName :: Text defaultBulletListStyleName = "List_20_1" @@ -302,9 +296,6 @@ writeOpenDocument :: PandocMonad m => WriterOptions -> Pandoc -> m Text writeOpenDocument opts (Pandoc meta blocks) = do setupTranslations meta - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing -- The abstract is the one template field that takes block-level -- content, so it is rendered separately; everything else in the -- metadata goes in a context where only inlines are allowed. @@ -358,10 +349,7 @@ then id else defField "abstract" abstract) $ metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> body - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context body withParagraphStyle :: PandocMonad m => WriterOptions -> Text -> [Block] -> OD m (Doc Text) diff --git a/src/Text/Pandoc/Writers/Org.hs b/src/Text/Pandoc/Writers/Org.hs --- a/src/Text/Pandoc/Writers/Org.hs +++ b/src/Text/Pandoc/Writers/Org.hs @@ -33,7 +33,6 @@ import Text.Pandoc.Options import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Citeproc.Locator (parseLocator, LocatorMap(..), LocatorInfo(..)) import Text.Pandoc.Walk (query) import Text.Pandoc.Writers.Shared @@ -60,9 +59,6 @@ pandocToOrg :: PandocMonad m => Pandoc -> Org m Text pandocToOrg (Pandoc meta blocks) = do opts <- gets stOptions - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing metadata <- metaToContext opts blockListToOrg (fmap chomp . inlineListToOrg) @@ -85,10 +81,7 @@ . defField "option-special-strings" (isEnabled Ext_special_strings opts) $ metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main -- | Return Org representation of the collected notes. Rendering a -- note may add further notes to the state (notes nested inside diff --git a/src/Text/Pandoc/Writers/Powerpoint/Output.hs b/src/Text/Pandoc/Writers/Powerpoint/Output.hs --- a/src/Text/Pandoc/Writers/Powerpoint/Output.hs +++ b/src/Text/Pandoc/Writers/Powerpoint/Output.hs @@ -593,13 +593,29 @@ presentationToArchive :: PandocMonad m => WriterOptions -> Meta -> Presentation -> m Archive presentationToArchive opts meta pres = do + -- ensure that the default reference doc isn't built from + -- pptx/ files in the user data directory: + oldUserDataDir <- P.getUserDataDir + P.setUserDataDir Nothing distArchive <- toArchive . BL.fromStrict <$> readDefaultDataFile "reference.pptx" + P.setUserDataDir oldUserDataDir refArchive <- case writerReferenceDoc opts of - Just f -> toArchive . BL.fromStrict . fst - <$> P.fetchItem (T.pack f) - Nothing -> toArchive . BL.fromStrict <$> - readDataFile "reference.pptx" + Just f -> do + arch <- toArchiveOrFail . BL.fromStrict . fst + <$> P.fetchItem (T.pack f) + case arch of + Left err -> throwError $ PandocParseError $ + "Could not parse reference-doc " <> + tshow f <> ": " <> T.pack err + Right x -> pure x + Nothing -> do + arch <- toArchiveOrFail . BL.fromStrict <$> + readDataFile "reference.pptx" + case arch of + Left err -> throwError $ PandocParseError $ + "Could not parse reference.pptx: " <> T.pack err + Right x -> pure x let (referenceLayouts, defaultReferenceLayouts) = (getLayoutsFromArchive refArchive, getLayoutsFromArchive distArchive) @@ -2295,6 +2311,8 @@ , sldNumShape ] ] + -- required by Keynote, see #11935 + , mknode "p:clrMapOvr" [] $ mknode "a:masterClrMapping" [] () ] ----------------------------------------------------------------------- @@ -2768,6 +2786,7 @@ : ( [mknode "dc:subject" [] $ fromMaybe "" $ dcSubject docProps | isJust (dcSubject docProps)]) <> ( [mknode "dc:description" [] $ fromMaybe "" $ dcDescription docProps | isJust (dcDescription docProps)]) <> ( [mknode "cp:category" [] $ fromMaybe "" $ cpCategory docProps | isJust (cpCategory docProps)]) + <> ( [mknode "dc:language" [] $ fromMaybe "" $ dcLanguage docProps | isJust (dcLanguage docProps)]) <> (\x -> [ mknode "dcterms:created" [("xsi:type","dcterms:W3CDTF")] x , mknode "dcterms:modified" [("xsi:type","dcterms:W3CDTF")] x ]) (T.pack $ formatTime defaultTimeLocale "%FT%XZ" utctime) diff --git a/src/Text/Pandoc/Writers/Powerpoint/Presentation.hs b/src/Text/Pandoc/Writers/Powerpoint/Presentation.hs --- a/src/Text/Pandoc/Writers/Powerpoint/Presentation.hs +++ b/src/Text/Pandoc/Writers/Powerpoint/Presentation.hs @@ -188,6 +188,7 @@ , dcKeywords :: Maybe [T.Text] , dcDescription :: Maybe T.Text , cpCategory :: Maybe T.Text + , dcLanguage :: Maybe T.Text , dcDate :: Maybe T.Text , customProperties :: Maybe [(T.Text, T.Text)] } deriving (Show, Eq) @@ -1189,6 +1190,7 @@ , dcKeywords = keywords , dcDescription = description , cpCategory = Shared.stringify <$> lookupMeta "category" meta + , dcLanguage = Shared.stringify <$> lookupMeta "lang" meta , dcDate = let t = Shared.stringifyInlines (docDate meta) in if T.null t diff --git a/src/Text/Pandoc/Writers/RST.hs b/src/Text/Pandoc/Writers/RST.hs --- a/src/Text/Pandoc/Writers/RST.hs +++ b/src/Text/Pandoc/Writers/RST.hs @@ -35,7 +35,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Shared import Text.Pandoc.Walk import Safe (lastMay, headMay) @@ -68,9 +67,6 @@ pandocToRST :: PandocMonad m => Pandoc -> RST m Text pandocToRST (Pandoc meta blocks) = do opts <- gets stOptions - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing let subtit = lookupMetaInlines "subtitle" meta title <- titleToRST (docTitle meta) subtit metadata <- metaToContext opts @@ -80,7 +76,7 @@ body <- blockListToRST' True $ case writerTemplate opts of Just _ -> normalizeHeadings 1 blocks Nothing -> blocks - notes <- gets (reverse . stNotes) >>= notesToRST + notes <- notesToRST -- note that the notes may contain refs, so we do them first refs <- gets (reverse . stLinks) >>= refsToRST pics <- gets (reverse . stImages) >>= pictRefsToRST @@ -95,10 +91,7 @@ $ defField "titleblock" (render Nothing title :: Text) $ defField "math" hasMath $ defField "rawtex" rawTeX metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main where normalizeHeadings lev (Header l a i:bs) = Header lev a i:normalizeHeadings (lev+1) cont ++ normalizeHeadings lev bs' @@ -126,10 +119,19 @@ else label' return $ nowrap $ ".. _" <> label'' <> ": " <> literal src --- | Return RST representation of notes. -notesToRST :: PandocMonad m => [[Block]] -> RST m (Doc Text) -notesToRST notes = - vsep <$> zipWithM noteToRST [1..] notes +-- | Return RST representation of the collected notes. Rendering a +-- note may add further notes to the state (notes nested inside +-- notes); keep going until all of them have been rendered. +notesToRST :: PandocMonad m => RST m (Doc Text) +notesToRST = vsep <$> go 0 + where + go done = do + notes <- gets (drop done . reverse . stNotes) + if null notes + then return [] + else do + docs <- zipWithM noteToRST [done + 1 ..] notes + (docs ++) <$> go (done + length notes) -- | Return RST representation of a note. noteToRST :: PandocMonad m => Int -> [Block] -> RST m (Doc Text) diff --git a/src/Text/Pandoc/Writers/Shared.hs b/src/Text/Pandoc/Writers/Shared.hs --- a/src/Text/Pandoc/Writers/Shared.hs +++ b/src/Text/Pandoc/Writers/Shared.hs @@ -55,6 +55,8 @@ , insertCurrentSpansAtColumn , takePreviousSpansAtColumn , decrementTrailingRowSpans + , wrapColumns + , renderWithTemplate ) where import Safe (lastMay, maximumMay) @@ -83,7 +85,7 @@ import qualified Text.Pandoc.UTF8 as UTF8 import Text.Pandoc.XML (escapeStringForXML, rdfaAttributes, html5Attributes) import Text.DocTemplates (Context(..), Val(..), TemplateTarget, - ToContext(..), FromContext(..)) + ToContext(..), FromContext(..), renderTemplate) import Text.Pandoc.Chunks (tocToList, toTOCTree) import Text.Collate.Lang (Lang (..)) import Text.Pandoc.Class (PandocMonad, toLang) @@ -110,6 +112,24 @@ Nothing -> return mempty Just _ -> addVariablesToContext opts <$> metaToContext' blockWriter inlineWriter meta + +-- | The line length to use in rendering: 'writerColumns' if +-- 'writerWrapText' is 'WrapAuto', otherwise 'Nothing' (no wrapping). +wrapColumns :: WriterOptions -> Maybe Int +wrapColumns opts = + if writerWrapText opts == WrapAuto + then Just $ writerColumns opts + else Nothing + +-- | Render a document body, inserting it into 'writerTemplate' +-- (with the given context) if there is a template, and wrapping +-- lines according to 'wrapColumns'. +renderWithTemplate :: WriterOptions -> Context Text -> Doc Text -> Text +renderWithTemplate opts context main = + render (wrapColumns opts) $ + case writerTemplate opts of + Nothing -> main + Just tpl -> renderTemplate tpl context -- | Like 'metaToContext, but does not include variables and is -- not sensitive to 'writerTemplate'. diff --git a/src/Text/Pandoc/Writers/TEI.hs b/src/Text/Pandoc/Writers/TEI.hs --- a/src/Text/Pandoc/Writers/TEI.hs +++ b/src/Text/Pandoc/Writers/TEI.hs @@ -23,7 +23,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Shared import Text.Pandoc.XML @@ -31,9 +30,6 @@ writeTEI :: PandocMonad m => WriterOptions -> Pandoc -> m Text writeTEI opts doc = do let Pandoc meta blocks = ensureValidXmlIdentifiers doc - let colwidth = if writerWrapText opts == WrapAuto - then Just $ writerColumns opts - else Nothing let startLvl = case writerTopLevelDivision opts of TopLevelPart -> -1 TopLevelChapter -> 0 @@ -49,20 +45,12 @@ $ defField "mathml" (case writerMathMethod opts of MathML -> True _ -> False) metadata - return $ render colwidth $ - case writerTemplate opts of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate opts context main -- | Convert a list of Pandoc blocks to TEI. blocksToTEI :: PandocMonad m => WriterOptions -> [Block] -> m (Doc Text) blocksToTEI opts bs = vcat <$> mapM (blockToTEI opts) bs --- | Auxiliary function to convert Plain block to Para. -plainToPara :: Block -> Block -plainToPara (Plain x) = Para x -plainToPara x = x - -- | Convert a list of pairs of terms and definitions into a TEI -- list with labels and items. deflistItemsToTEI :: PandocMonad m @@ -275,8 +263,12 @@ linktext <- inlinesToTEI opts txt return $ linktext <+> char '(' <> emailLink <> char ')' | otherwise = - inTags False "ref" (("target", src) : idFromAttr opts attr) + inTags False "ref" (("target", src') : idFromAttr opts attr) <$> inlinesToTEI opts txt + where + src' = case T.stripPrefix "#" src of + Just ident -> "#" <> writerIdentifierPrefix opts <> ident + Nothing -> src inlineToTEI opts (Image attr description (src, tit)) = do let titleDoc = if T.null tit then empty diff --git a/src/Text/Pandoc/Writers/Texinfo.hs b/src/Text/Pandoc/Writers/Texinfo.hs --- a/src/Text/Pandoc/Writers/Texinfo.hs +++ b/src/Text/Pandoc/Writers/Texinfo.hs @@ -36,7 +36,6 @@ import Text.DocLayout import Text.Pandoc.Shared import Text.Pandoc.URI -import Text.Pandoc.Templates (renderTemplate) import Text.Pandoc.Writers.Shared import Text.Printf (printf) @@ -105,9 +104,6 @@ blocks <- walkM addNodeText blocks' let titlePage = not $ all null $ docTitle meta : docDate meta : docAuthors meta - let colwidth = if writerWrapText options == WrapAuto - then Just $ writerColumns options - else Nothing metadata <- metaToContext options blockListToTexinfo (fmap chomp .inlineListToTexinfo) @@ -118,10 +114,7 @@ $ defField "toc" (writerTableOfContents options) $ defField "titlepage" titlePage $ defField "strikeout" (stStrikeout st) metadata - return $ render colwidth $ - case writerTemplate options of - Nothing -> body - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate options context body -- | Escape things as needed for Texinfo. stringToTexinfo :: PandocMonad m => Text -> TI m Text diff --git a/src/Text/Pandoc/Writers/Typst.hs b/src/Text/Pandoc/Writers/Typst.hs --- a/src/Text/Pandoc/Writers/Typst.hs +++ b/src/Text/Pandoc/Writers/Typst.hs @@ -32,7 +32,8 @@ import Control.Monad.State ( StateT, evalStateT, gets, modify ) import Text.Pandoc.Writers.Shared ( lookupMetaInlines, lookupMetaString, metaToContext, defField, resetField, - setupTranslations ) + setupTranslations, getLang, + renderWithTemplate ) import Text.Pandoc.Shared (isTightList, orderedListMarkers, tshow, stringifyInlines) import Text.Pandoc.Highlighting (highlight, formatTypstBlock, formatTypstInline, @@ -43,7 +44,6 @@ import Text.Pandoc.Writers.Math (convertMath) import qualified Text.TeXMath as TM import Text.DocLayout -import Text.DocTemplates (renderTemplate) import Text.Pandoc.Extensions (Extension(..)) import Text.Pandoc.Logging (LogMessage(..)) import qualified Text.Pandoc.UTF8 as UTF8 @@ -85,9 +85,6 @@ pandocToTypst :: PandocMonad m => WriterOptions -> Pandoc -> TW m Text pandocToTypst options (Pandoc meta blocks) = do - let colwidth = if writerWrapText options == WrapAuto - then Just $ writerColumns options - else Nothing setupTranslations meta metadata <- metaToContext options blocksToTypst @@ -113,9 +110,9 @@ . defField "nocite-ids" (filter (/= "*") nociteIds) . defField "full-bibliography" ("*" `elem` nociteIds) else id) - $ (case lookupMetaString "lang" meta of - "" -> id - lang -> + $ (case getLang options meta of + Nothing -> id + Just lang -> case parseLang lang of Left _ -> id Right l -> @@ -144,10 +141,7 @@ then defField "section-numbering" ("1.1.1.1.1" :: Text) else id) metadata - return $ render colwidth $ - case writerTemplate options of - Nothing -> main - Just tpl -> renderTemplate tpl context + return $ renderWithTemplate options context main pickTypstAttrs :: [(Text, Text)] -> ([(Text, Text)],[(Text, Text)]) pickTypstAttrs = foldr go ([],[]) @@ -220,7 +214,7 @@ RawBlock fmt str -> case fmt of Format "typst" -> return $ literal str - _ -> return mempty + _ -> mempty <$ report (BlockNotRendered block) CodeBlock (ident,cls,kvs) code -> do let go :: Char -> (Int, Int) -> (Int, Int) go '`' (longest, current) = @@ -522,7 +516,7 @@ RawInline fmt str -> case fmt of Format "typst" -> return $ literal str - _ -> return mempty + _ -> mempty <$ report (InlineNotRendered inline) Strikeout inlines -> textstyle "#strike" inlines Emph inlines -> textstyle "#emph" inlines Underline inlines -> textstyle "#underline" inlines @@ -659,6 +653,8 @@ go (lastc, t') c | needsEscape c = (c, t' <> escapeChar c) | c == '-', lastc == '-', smart = (c, t' <> T.pack ['\\',c]) + | c == '?', lastc == '-', smart = (c, t' <> T.pack ['\\',c]) + | c == '.', lastc == '.', smart = (c, t' <> T.pack ['\\',c]) | c == '/', lastc == '/' = (c, t' <> T.pack ['\\',c]) | otherwise = (c, T.snoc t' c) escapeChar c diff --git a/test/Tests/Readers/Docx.hs b/test/Tests/Readers/Docx.hs --- a/test/Tests/Readers/Docx.hs +++ b/test/Tests/Readers/Docx.hs @@ -136,6 +136,18 @@ "docx/empty_field.docx" "docx/empty_field.native" , testCompare + "field without <w:instrText> does not corrupt later lists" + "docx/field_no_instrText.docx" + "docx/field_no_instrText.native" + , testCompare + "field with <w:instrText> in tracked change does not corrupt later lists" + "docx/field_tracked_instrText.docx" + "docx/field_tracked_instrText.native" + , testCompare + "field without <w:instrText> in a code block keeps its text" + "docx/field_no_instrText_codeblock.docx" + "docx/field_no_instrText_codeblock.native" + , testCompare "pageref hyperlinks in <w:instrText> tag" "docx/pageref.docx" "docx/pageref.native" @@ -459,6 +471,14 @@ "spaces at the edges of changes (reject)" "docx/track_changes_edge_spaces.docx" "docx/track_changes_edge_spaces_reject.native" + , testCompareWithOpts def{readerTrackChanges=AcceptChanges} + "paragraph marks in notes stay in their notes (accept)" + "docx/track_changes_paragraph_mark_in_note.docx" + "docx/track_changes_paragraph_mark_in_note_accept.native" + , testCompareWithOpts def{readerTrackChanges=RejectChanges} + "paragraph marks in notes stay in their notes (reject)" + "docx/track_changes_paragraph_mark_in_note.docx" + "docx/track_changes_paragraph_mark_in_note_reject.native" , testCompareWithOpts def{readerTrackChanges=AcceptChanges} "comments (accept -- no comments)" "docx/comments.docx" diff --git a/test/Tests/Readers/ODT.hs b/test/Tests/Readers/ODT.hs --- a/test/Tests/Readers/ODT.hs +++ b/test/Tests/Readers/ODT.hs @@ -259,6 +259,7 @@ , "inlinedCode" , "listContinueNumbering" , "listContinueNumbering2" + , "listContinueNumbering3" , "orderedListMixed" , "orderedListRoman" , "orderedListSimple" diff --git a/test/Tests/Readers/RST.hs b/test/Tests/Readers/RST.hs --- a/test/Tests/Readers/RST.hs +++ b/test/Tests/Readers/RST.hs @@ -217,6 +217,19 @@ , "bare URI parsing disabled inside emphasis (#4561)" =: "*http://location*" =?> para (emph (text "http://location")) + , "bare URI and email address in angle brackets" =: + "See <https://example.com> or <jane-doe@example.com>." =?> + para ("See <" <> link "https://example.com" "" "https://example.com" <> + "> or <" <> + link "mailto:jane-doe@example.com" "" "jane-doe@example.com" <> + ">.") + , "inline markup after a colon, a slash or a quote" =: + "key:*a* path/*b* \"*c*\"" =?> + para ("key:" <> emph "a" <> " path/" <> emph "b" <> " \"" <> + emph "c" <> "\"") + , "quoted start-strings are not inline markup" =: + "(*) and (*), [*] and [*], \"*\" or \"x*\"" =?> + para (text "(*) and (*), [*] and [*], \"*\" or \"x*\"") , "include newlines" =: "**before\nafter**" =?> para (strong (text "before\nafter")) diff --git a/test/command/11088.md b/test/command/11088.md new file mode 100644 --- /dev/null +++ b/test/command/11088.md @@ -0,0 +1,10 @@ +``` +% pandoc -f latex -t native 2>&1 +\makeatletter +\def\small{\@setfontsize\small\@ixpt{10}} +\makeatother +\small test +^D +Loop encountered in expanding macro small +=> 91 +``` diff --git a/test/command/11944.docx b/test/command/11944.docx new file mode 100644 Binary files /dev/null and b/test/command/11944.docx differ diff --git a/test/command/11944.md b/test/command/11944.md new file mode 100644 --- /dev/null +++ b/test/command/11944.md @@ -0,0 +1,14 @@ +A `w:fldSimple` keeps its result text; a REF field with `\h` becomes a +link to the bookmark it refers to. A caption whose label is not `Figure` +or `Table` keeps its label and number. + +``` +% pandoc command/11944.docx -t html +^D +<p>See figure <a href="#_Ref1">1</a>.</p> +<figure> +<img src="media/image1.png" style="width:1.09702in;height:1.09702in" /> +<figcaption><p><span id="_Ref1" class="anchor"></span>Figur 1 Blue +square.</p></figcaption> +</figure> +``` diff --git a/test/command/11945.md b/test/command/11945.md new file mode 100644 --- /dev/null +++ b/test/command/11945.md @@ -0,0 +1,34 @@ +A table with more columns than `columns / 8` is still written as a grid +table, rather than falling back to raw HTML. + +``` +% pandoc -f html -t markdown-simple_tables-multiline_tables-pipe_tables +<table> +<tr><th>H1</th><th>H2</th><th>H3</th><th>H4</th><th>H5</th><th>H6</th><th>H7</th><th>H8</th><th>H9</th><th>H10</th></tr> +<tr><td>c1</td><td>c2</td><td>c3</td><td>c4</td><td>c5</td><td>c6</td><td>c7</td><td>c8</td><td>c9</td><td>c10</td></tr> +</table> +^D ++----+----+----+----+----+----+----+----+----+-----+ +| H1 | H2 | H3 | H4 | H5 | H6 | H7 | H8 | H9 | H10 | ++====+====+====+====+====+====+====+====+====+=====+ +| c1 | c2 | c3 | c4 | c5 | c6 | c7 | c8 | c9 | c10 | ++----+----+----+----+----+----+----+----+----+-----+ +``` + +This includes a table with block content in a cell, which has no other +markdown representation. + +``` +% pandoc -f html -t markdown +<table> +<tr><th>H1</th><th>H2</th><th>H3</th><th>H4</th><th>H5</th><th>H6</th><th>H7</th><th>H8</th><th>H9</th><th>H10</th></tr> +<tr><td><ul><li>a</li><li>b</li></ul></td><td>c2</td><td>c3</td><td>c4</td><td>c5</td><td>c6</td><td>c7</td><td>c8</td><td>c9</td><td>c10</td></tr> +</table> +^D ++------+------+------+------+------+------+------+------+------+------+ +| H1 | H2 | H3 | H4 | H5 | H6 | H7 | H8 | H9 | H10 | ++======+======+======+======+======+======+======+======+======+======+ +| - a | c2 | c3 | c4 | c5 | c6 | c7 | c8 | c9 | c10 | +| - b | | | | | | | | | | ++------+------+------+------+------+------+------+------+------+------+ +``` diff --git a/test/command/11954.md b/test/command/11954.md new file mode 100644 --- /dev/null +++ b/test/command/11954.md @@ -0,0 +1,12 @@ +``` +% pandoc -t latex +key = value +: Definition +^D +\begin{description} +\tightlist +\item[{key = value}] +Definition +\end{description} + +``` diff --git a/test/command/11979.md b/test/command/11979.md new file mode 100644 --- /dev/null +++ b/test/command/11979.md @@ -0,0 +1,6 @@ +``` +% pandoc -f native -t typst +[Str "-?", Space, Str "...", Space, Str "a-b."] +^D +\-\? \.\.\. a-b. +``` diff --git a/test/command/11981.md b/test/command/11981.md new file mode 100644 --- /dev/null +++ b/test/command/11981.md @@ -0,0 +1,14 @@ +``` +% pandoc -f native -t rtf | pandoc -f rtf -t native +[ Para [ Str "abc\8217def" ] +, Para [ Str "\8220abc\8221" , Space , Str "\8216abc\8217" ] +, Para [ Str "abc\8217def" ] +, Para [ Str "abc\8212def" ] +] +^D +[ Para [ Str "abc\8217def" ] +, Para [ Str "\8220abc\8221" , Space , Str "\8216abc\8217" ] +, Para [ Str "abc\8217def" ] +, Para [ Str "abc\8212def" ] +] +``` diff --git a/test/command/11987.md b/test/command/11987.md new file mode 100644 --- /dev/null +++ b/test/command/11987.md @@ -0,0 +1,9 @@ +If no class matches a language, we just use the first one. +```` +% pandoc -f native -t markdown +[ CodeBlock ( "" , [ "text" ] , [] ) "x" ] +^D +``` text +x +``` +```` diff --git a/test/command/11989.docx b/test/command/11989.docx new file mode 100644 Binary files /dev/null and b/test/command/11989.docx differ diff --git a/test/command/11989.md b/test/command/11989.md new file mode 100644 --- /dev/null +++ b/test/command/11989.md @@ -0,0 +1,19 @@ +A `w:gridCol` need not have a width (`w:w`). Such a column is kept, +and since its width is missing, every column gets the default width. + +``` +% pandoc command/11989.docx -t html +^D +<table> +<tbody> +<tr> +<td>a</td> +<td>b</td> +</tr> +<tr> +<td>c</td> +<td>d</td> +</tr> +</tbody> +</table> +``` diff --git a/test/command/11991.docx b/test/command/11991.docx new file mode 100644 Binary files /dev/null and b/test/command/11991.docx differ diff --git a/test/command/11991.md b/test/command/11991.md new file mode 100644 --- /dev/null +++ b/test/command/11991.md @@ -0,0 +1,37 @@ +Notes and comments are read with the namespace prefixes declared in +their own parts (word/footnotes.xml, word/endnotes.xml, +word/comments.xml). Here word/document.xml declares only `w`; the +footnote's hyperlink uses `r`, and the equations in the endnote and +the comment use `m`. + +``` +% pandoc command/11991.docx -t native --track-changes=all +^D +[ Para + [ Span + ( "" + , [ "comment-start" ] + , [ ( "comment-id" , "0" ) + , ( "author" , "A" ) + , ( "date" , "2026-01-01T00:00:00Z" ) + ] + ) + [ Str "Math:" , Space , Math InlineMath "x" ] + , Str "Text" + , Span + ( "" , [ "comment-end" ] , [ ( "comment-id" , "0" ) ] ) [] + , Note + [ Para + [ Str "See" + , Space + , Link + ( "" , [] , [] ) + [ Str "example" ] + ( "https://example.com/" , "" ) + ] + ] + , Note + [ Para [ Str "Math:" , Space , Math InlineMath "x" ] ] + ] +] +``` diff --git a/test/command/4819.md b/test/command/4819.md --- a/test/command/4819.md +++ b/test/command/4819.md @@ -49,7 +49,9 @@ ... ^D Pandoc - Meta { unMeta = fromList [ ( "foo" , MetaBool False ) ] } + Meta + { unMeta = fromList [ ( "foo" , MetaInlines [ Str "no" ] ) ] + } [] ``` diff --git a/test/command/9197.md b/test/command/9197.md new file mode 100644 --- /dev/null +++ b/test/command/9197.md @@ -0,0 +1,33 @@ +Paragraphs after the first in a list item should not be turned +into items of a separate list, and should round-trip: + +``` +% pandoc -f native -t docx -o - | pandoc -f docx -t native +[ BulletList + [ [ Para [ Str "one" ] + , Para [ Str "sub" ] + , BulletList + [ [ Para [ Str "nested" ] + , Para [ Str "nested" , Space , Str "sub" ] + ] + ] + , Para [ Str "back" ] + ] + , [ Para [ Str "two" ] ] + ] +] +^D +[ BulletList + [ [ Para [ Str "one" ] + , Para [ Str "sub" ] + , BulletList + [ [ Para [ Str "nested" ] + , Para [ Str "nested" , Space , Str "sub" ] + ] + ] + , Para [ Str "back" ] + ] + , [ Para [ Str "two" ] ] + ] +] +``` diff --git a/test/command/nested-notes.md b/test/command/nested-notes.md new file mode 100644 --- /dev/null +++ b/test/command/nested-notes.md @@ -0,0 +1,43 @@ +``` +% pandoc -f native -t markdown +[Para [Str "a", Note [Para [Str "outer", Note [Para [Str "inner"]]]], Str "b", Note [Para [Str "second"]]]] +^D +a[^1]b[^2] + +[^1]: outer[^3] + +[^2]: second + +[^3]: inner +``` + +``` +% pandoc -f native -t rst +[Para [Str "a", Note [Para [Str "outer", Note [Para [Str "inner"]]]], Str "b", Note [Para [Str "second"]]]] +^D +a [1]_b [2]_ + +.. [1] + outer [3]_ + +.. [2] + second + +.. [3] + inner +``` + +``` +% pandoc -f native -t haddock +[Para [Str "a", Note [Para [Str "outer", Note [Para [Str "inner"]]]], Str "b", Note [Para [Str "second"]]]] +^D +a<#notes [1]>b<#notes [2]> + +#notes# + +1. outer<#notes [3]> + +2. second + +3. inner +``` diff --git a/test/command/pandoc-citeproc-57.md b/test/command/pandoc-citeproc-57.md --- a/test/command/pandoc-citeproc-57.md +++ b/test/command/pandoc-citeproc-57.md @@ -30,7 +30,7 @@ :::: {#refs .references .csl-bib-body .hanging-indent entry-spacing="0"} ::: {#ref-Faraday-forthcoming .csl-entry} Faraday, Carry. Forthcoming. "Protean Photography." In *Seven Trips -Beyond the Asteroid Belt*, edited by James Oring. Cape Canaveral, FL: +beyond the Asteroid Belt*, edited by James Oring. Cape Canaveral, FL: Launch Press. ::: :::: diff --git a/test/docx/field_no_instrText.docx b/test/docx/field_no_instrText.docx new file mode 100644 Binary files /dev/null and b/test/docx/field_no_instrText.docx differ diff --git a/test/docx/field_no_instrText.native b/test/docx/field_no_instrText.native new file mode 100644 --- /dev/null +++ b/test/docx/field_no_instrText.native @@ -0,0 +1,24 @@ +[ Para + [ Str "Error!" + , Space + , Str "Reference" + , Space + , Str "source" + , Space + , Str "not" + , Space + , Str "found." + ] +, Para + [ Link + ( "" , [] , [] ) + [ Str "Target" , Space , Str "B" ] + ( "#_Ref2" , "" ) + ] +, BulletList + [ [ Para [ Str "Item" , Space , Str "one" ] ] + , [ Para [ Str "Item" , Space , Str "two" ] ] + ] +, Para + [ Str "After" , Space , Str "the" , Space , Str "list." ] +] diff --git a/test/docx/field_no_instrText_codeblock.docx b/test/docx/field_no_instrText_codeblock.docx new file mode 100644 Binary files /dev/null and b/test/docx/field_no_instrText_codeblock.docx differ diff --git a/test/docx/field_no_instrText_codeblock.native b/test/docx/field_no_instrText_codeblock.native new file mode 100644 --- /dev/null +++ b/test/docx/field_no_instrText_codeblock.native @@ -0,0 +1,1 @@ +[ CodeBlock ( "" , [] , [] ) "int x;" ] diff --git a/test/docx/field_tracked_instrText.docx b/test/docx/field_tracked_instrText.docx new file mode 100644 Binary files /dev/null and b/test/docx/field_tracked_instrText.docx differ diff --git a/test/docx/field_tracked_instrText.native b/test/docx/field_tracked_instrText.native new file mode 100644 --- /dev/null +++ b/test/docx/field_tracked_instrText.native @@ -0,0 +1,14 @@ +[ Para [ Str "Target" , Space , Str "A" ] +, Para + [ Link + ( "" , [] , [] ) + [ Str "Target" , Space , Str "B" ] + ( "#_Ref2" , "" ) + ] +, BulletList + [ [ Para [ Str "Item" , Space , Str "one" ] ] + , [ Para [ Str "Item" , Space , Str "two" ] ] + ] +, Para + [ Str "After" , Space , Str "the" , Space , Str "list." ] +] diff --git a/test/docx/golden/lists.docx b/test/docx/golden/lists.docx Binary files a/test/docx/golden/lists.docx and b/test/docx/golden/lists.docx differ diff --git a/test/docx/golden/lists_div_bullets.docx b/test/docx/golden/lists_div_bullets.docx Binary files a/test/docx/golden/lists_div_bullets.docx and b/test/docx/golden/lists_div_bullets.docx differ diff --git a/test/docx/golden/task_list.docx b/test/docx/golden/task_list.docx Binary files a/test/docx/golden/task_list.docx and b/test/docx/golden/task_list.docx differ diff --git a/test/docx/lists_level_override.native b/test/docx/lists_level_override.native --- a/test/docx/lists_level_override.native +++ b/test/docx/lists_level_override.native @@ -77,93 +77,91 @@ , Space , Str "Documentation" ] + , Para + [ Strong + [ Str "Goal:" + , Space + , Str "Baseline" + , Space + , Str "and" + , Space + , Str "ongoing" + , Space + , Str "metrics" + , Space + , Str "tracking" + , Space + , Str "doc" + , Space + , Str "usefulness" + , Space + , Str "and" + , Space + , Str "completeness." + ] + ] + , Para [ Str "\160" ] ] ] - , BlockQuote - [ Para - [ Strong - [ Str "Goal:" - , Space - , Str "Baseline" - , Space - , Str "and" - , Space - , Str "ongoing" - , Space - , Str "metrics" - , Space - , Str "tracking" - , Space - , Str "doc" - , Space - , Str "usefulness" - , Space - , Str "and" - , Space - , Str "completeness." - ] - ] - , Para [ Str "\160" ] - ] , OrderedList ( 2 , Decimal , Period ) - [ [ Para [ Str "Content" , Space , Str "Migration" ] ] ] - , BlockQuote - [ Para - [ Str "Goal:" - , Space - , Str "Content" - , Space - , Str "is" - , Space - , Str "accessible" - , Space - , Str "to" - , Space - , Str "new" - , Space - , Str "employees" - , Space - , Str "and" - , Space - , Str "is" - , Space - , Str "better" - , Space - , Str "organized/archived." - ] - , Para [ Str "\160" ] + [ [ Para [ Str "Content" , Space , Str "Migration" ] + , Para + [ Str "Goal:" + , Space + , Str "Content" + , Space + , Str "is" + , Space + , Str "accessible" + , Space + , Str "to" + , Space + , Str "new" + , Space + , Str "employees" + , Space + , Str "and" + , Space + , Str "is" + , Space + , Str "better" + , Space + , Str "organized/archived." + ] + , Para [ Str "\160" ] + ] ] , OrderedList ( 3 , Decimal , Period ) - [ [ Para [ Str "Wiki" , Space , Str "(xl)" ] ] ] - , BlockQuote - [ Para - [ Strong - [ Str "Goal:" - , Space - , Str "Useful" - , Space - , Str "documentation" - , Space - , Str "that" - , Space - , Str "is" - , Space - , Str "archived," - , Space - , Str "searchable" - , Space - , Str "and" - , Space - , Str "easy" - , Space - , Str "to" - , Space - , Str "create" - ] - ] - , Para [ Str "\160\160" ] + [ [ Para [ Str "Wiki" , Space , Str "(xl)" ] + , Para + [ Strong + [ Str "Goal:" + , Space + , Str "Useful" + , Space + , Str "documentation" + , Space + , Str "that" + , Space + , Str "is" + , Space + , Str "archived," + , Space + , Str "searchable" + , Space + , Str "and" + , Space + , Str "easy" + , Space + , Str "to" + , Space + , Str "create" + ] + ] + , Para [ Str "\160\160" ] + ] ] , OrderedList ( 4 , Decimal , Period ) @@ -178,89 +176,87 @@ , Space , Str "(XLCRBot)." ] + , Para + [ Strong + [ Str "Goal:" + , Space + , Str "Feedback" + , Space + , Str "on" + , Space + , Str "basic" + , Space + , Str "violations" + , Space + , Str "in" + , Space + , Str "seconds" + , Space + , Str "or" + , Space + , Str "minutes" + , Space + , Str "at" + , Space + , Str "most" + , Space + , Str "in" + , Space + , Str "either" + , Space + , Str "VS" + , Space + , Str "or" + , Space + , Str "Codeflow." + ] + ] ] ] - , BlockQuote - [ Para - [ Strong - [ Str "Goal:" - , Space - , Str "Feedback" - , Space - , Str "on" - , Space - , Str "basic" - , Space - , Str "violations" - , Space - , Str "in" - , Space - , Str "seconds" - , Space - , Str "or" - , Space - , Str "minutes" - , Space - , Str "at" - , Space - , Str "most" - , Space - , Str "in" - , Space - , Str "either" - , Space - , Str "VS" - , Space - , Str "or" - , Space - , Str "Codeflow." - ] - ] - ] , OrderedList ( 5 , Decimal , Period ) - [ [ Para [ Str "Code" , Space , Str "documentation" ] ] ] - , BlockQuote - [ Para - [ Strong - [ Str "Goal:" - , Space - , Str "Useful," - , Space - , Str "consistent," - , Space - , Str "tool" - , Space - , Str "supported" - , Space - , Str "comments" - ] - ] + [ [ Para [ Str "Code" , Space , Str "documentation" ] + , Para + [ Strong + [ Str "Goal:" + , Space + , Str "Useful," + , Space + , Str "consistent," + , Space + , Str "tool" + , Space + , Str "supported" + , Space + , Str "comments" + ] + ] + ] ] , Para [ Strong [ Str "\160" ] , Str "\160" ] , OrderedList ( 6 , Decimal , Period ) - [ [ Para [ Str "Education" , Space , Str "efforts" ] ] ] - , BlockQuote - [ Para - [ Strong - [ Str "Goal:" - , Space - , Str "Broad," - , Space - , Str "discoverable" - , Space - , Str "channels" - , Space - , Str "for" - , Space - , Str "updates" - , Space - , Str "and" - , Space - , Str "news" - ] - ] + [ [ Para [ Str "Education" , Space , Str "efforts" ] + , Para + [ Strong + [ Str "Goal:" + , Space + , Str "Broad," + , Space + , Str "discoverable" + , Space + , Str "channels" + , Space + , Str "for" + , Space + , Str "updates" + , Space + , Str "and" + , Space + , Str "news" + ] + ] + ] ] , Para [ Strong [ Str "\160" ] ] ] diff --git a/test/docx/sdt_elements.native b/test/docx/sdt_elements.native --- a/test/docx/sdt_elements.native +++ b/test/docx/sdt_elements.native @@ -3,7 +3,7 @@ [ Table ( "" , [] , [] ) (Caption Nothing []) - [ ( AlignCenter , ColWidth 0.16167023554603854 ) + [ ( AlignDefault , ColWidth 0.16167023554603854 ) , ( AlignDefault , ColWidth 0.16167023554603854 ) , ( AlignDefault , ColWidth 0.40920770877944324 ) ] diff --git a/test/docx/table_header_rowspan.native b/test/docx/table_header_rowspan.native --- a/test/docx/table_header_rowspan.native +++ b/test/docx/table_header_rowspan.native @@ -3,7 +3,7 @@ [ Table ( "" , [] , [] ) (Caption Nothing []) - [ ( AlignDefault , ColWidth 0.30701754385964913 ) + [ ( AlignLeft , ColWidth 0.30701754385964913 ) , ( AlignDefault , ColWidth 0.13645224171539963 ) , ( AlignDefault , ColWidth 0.10009746588693959 ) , ( AlignDefault , ColWidth 9.707602339181289e-2 ) diff --git a/test/docx/track_changes_paragraph_mark_in_note.docx b/test/docx/track_changes_paragraph_mark_in_note.docx new file mode 100644 Binary files /dev/null and b/test/docx/track_changes_paragraph_mark_in_note.docx differ diff --git a/test/docx/track_changes_paragraph_mark_in_note_accept.native b/test/docx/track_changes_paragraph_mark_in_note_accept.native new file mode 100644 --- /dev/null +++ b/test/docx/track_changes_paragraph_mark_in_note_accept.native @@ -0,0 +1,58 @@ +[ Para + [ Str "A" + , Space + , Str "note" + , Space + , Str "whose" + , Space + , Str "paragraph" + , Space + , Str "mark" + , Space + , Str "is" + , Space + , Str "deleted" + , Note [ Para [ Str "First" , Space , Str "note." ] ] + , Str "." + ] +, Para + [ Str "A" + , Space + , Str "note" + , Space + , Str "whose" + , Space + , Str "paragraph" + , Space + , Str "mark" + , Space + , Str "is" + , Space + , Str "inserted" + , Note [ Para [ Str "Second" , Space , Str "note." ] ] + , Str "." + ] +, Para + [ Str "A" + , Space + , Str "paragraph" + , Space + , Str "whose" + , Space + , Str "mark" + , Space + , Str "is" + , Space + , Str "deleted." + , Space + , Str "The" + , Space + , Str "paragraph" + , Space + , Str "it" + , Space + , Str "joins" + , Note [ Para [ Str "Third" , Space , Str "note." ] ] + , Str "." + ] +] diff --git a/test/docx/track_changes_paragraph_mark_in_note_reject.native b/test/docx/track_changes_paragraph_mark_in_note_reject.native new file mode 100644 --- /dev/null +++ b/test/docx/track_changes_paragraph_mark_in_note_reject.native @@ -0,0 +1,59 @@ +[ Para + [ Str "A" + , Space + , Str "note" + , Space + , Str "whose" + , Space + , Str "paragraph" + , Space + , Str "mark" + , Space + , Str "is" + , Space + , Str "deleted" + , Note [ Para [ Str "First" , Space , Str "note." ] ] + , Str "." + ] +, Para + [ Str "A" + , Space + , Str "note" + , Space + , Str "whose" + , Space + , Str "paragraph" + , Space + , Str "mark" + , Space + , Str "is" + , Space + , Str "inserted" + , Note [ Para [ Str "Second" , Space , Str "note." ] ] + , Str "." + ] +, Para + [ Str "A" + , Space + , Str "paragraph" + , Space + , Str "whose" + , Space + , Str "mark" + , Space + , Str "is" + , Space + , Str "deleted." + ] +, Para + [ Str "The" + , Space + , Str "paragraph" + , Space + , Str "it" + , Space + , Str "joins" + , Note [ Para [ Str "Third" , Space , Str "note." ] ] + , Str "." + ] +] diff --git a/test/odt/native/listContinueNumbering3.native b/test/odt/native/listContinueNumbering3.native new file mode 100644 --- /dev/null +++ b/test/odt/native/listContinueNumbering3.native @@ -0,0 +1,814 @@ +[ Para + [ Str "An" , Space , Str "example" , Space , Str "list" ] +, Para [] +, OrderedList + ( 1 , Decimal , Period ) + [ [ Plain + [ Str "This" + , Space + , Str "is" + , Space + , Str "a" + , Space + , Str "list," + ] + ] + , [ Para [ Str "With" , Space , Str "numbers" ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Plain + [ Str "A" + , Space + , Str "sublist" + , Space + , Str "from" + , Space + , Str "word," + , Space + , Str "starts" + , Space + , Str "with" + ] + ] + , [ Para [ Str "Alphabetic" , Space , Str "characters" ] + , OrderedList + ( 1 , LowerRoman , Period ) + [ [ Plain + [ Str "A" + , Space + , Str "sub-sub-list" + , Space + , Str "will" + , Space + , Str "be" + ] + ] + , [ Para + [ Str "With" + , Space + , Str "lowercase" + , Space + , Str "roman" + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Plain + [ Str "And" + , Space + , Str "here" + , Space + , Str "we" + , Space + , Str "are" + , Space + , Str "back" + , Space + , Str "with" + , Space + , Str "numbers" + ] + ] + , [ Para + [ Str "And" + , Space + , Str "another" + , Space + , Str "number" + ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Plain + [ Str "And" + , Space + , Str "back" + , Space + , Str "to" + , Space + , Str "lowercase" + , Space + , Str "alphabetical" + ] + ] + ] + ] + ] + ] + ] + ] + ] + ] + , [ Plain + [ Str "BACK" + , Space + , Str "TO" + , Space + , Str "ORIGINAL" + , Space + , Str "LEVEL" + ] + ] + ] +, Para [] +, Para + [ Str "Wow" + , Space + , Str "that" + , Space + , Str "was" + , Space + , Str "cool!" + , Space + , Str "AMAZING" + ] +, Para [] +, OrderedList + ( 1 , UpperAlpha , Period ) + [ [ Plain + [ Str "But" + , Space + , Str "we" + , Space + , Str "also" + , Space + , Str "have" + , Space + , Str "lists" + ] + ] + , [ Para + [ Str "With" + , Space + , Str "uppercase" + , Space + , Str "alphabet" + ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Plain + [ Str "And" + , Space + , Str "the" + , Space + , Str "sublist" + , Space + , Str "here" + , Space + , Str "again" + ] + ] + , [ Para + [ Str "Is" + , Space + , Str "lowercase" + , Space + , Str "alphabet" + ] + , OrderedList + ( 1 , LowerRoman , Period ) + [ [ Plain + [ Str "And" + , Space + , Str "lowercase" + , Space + , Str "roman" + ] + ] + ] + ] + ] + ] + , [ Plain + [ Str "BACK" + , Space + , Str "TO" + , Space + , Str "ORIGINAL" + , Space + , Str "LEVEL" + ] + ] + ] +, Para [] +, Para + [ Str "And" + , Space + , Str "foremost," + , Space + , Str "we" + , Space + , Str "have" + , Space + , Str "lists" + , Space + , Str "starting" + , Space + , Str "with" + , Space + , Str "uppercase" + , Space + , Str "roman." + ] +, Para [] +, OrderedList + ( 1 , UpperRoman , Period ) + [ [ Plain [ Str "Like" , Space , Str "this" ] ] + , [ Plain + [ Str "This" + , Space + , Str "list" + , Space + , Str "has" + , Space + , Str "uppercase" + , Space + , Str "roman" + ] + ] + , [ Para + [ Str "And" + , Space + , Str "that" + , Space + , Str "is" + , Space + , Str "very" + , Space + , Str "OK" + ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Para + [ Str "And" + , Space + , Str "again," + , Space + , Str "the" + , Space + , Str "sublist" + , Space + , Str "starts" + , Space + , Str "with" + , Space + , Str "lowercase" + , Space + , Str "alphabet" + ] + , OrderedList + ( 1 , LowerRoman , Period ) + [ [ Para + [ Str "And" + , Space + , Str "a" + , Space + , Str "sub-sub-list" + , Space + , Str "with" + , Space + , Str "lowercase" + , Space + , Str "roman" + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Plain + [ Str "But" + , Space + , Str "the" + , Space + , Str "sub-sub-sub-list" + , Space + , Str "starts" + , Space + , Str "with" + , Space + , Str "numbers" + , Space + , Str "here" + ] + ] + ] + ] + ] + ] + ] + ] + , [ Plain + [ Str "BACK" + , Space + , Str "TO" + , Space + , Str "ORIGINAL" + , Space + , Str "LEVEL" + ] + ] + ] +, Para [] +, Para + [ Str "And" + , Space + , Str "a" + , Space + , Str "list" + , Space + , Str "in" + , Space + , Str "lowercase" + , Space + , Str "roman" + ] +, Para [] +, OrderedList + ( 1 , LowerRoman , Period ) + [ [ Para + [ Str "This" + , Space + , Str "is" + , Space + , Str "an" + , Space + , Str "example" + , Space + , Str "of" + , Space + , Str "that" + ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Para + [ Str "And" + , Space + , Str "this" + , Space + , Str "will" + , Space + , Str "be" + , Space + , Str "lowercase" + , Space + , Str "alphabetic" + ] + , OrderedList + ( 1 , LowerRoman , Period ) + [ [ Para + [ Str "And" + , Space + , Str "lowercase" + , Space + , Str "roman??" + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Plain [ Str "And" , Space , Str "numeric" ] ] + ] + ] + ] + ] + ] + ] + , [ Plain + [ Str "BACK" + , Space + , Str "TO" + , Space + , Str "ORIGINAL" + , Space + , Str "LEVEL" + ] + ] + ] +, Para [] +, Para + [ Str "And" + , Space + , Str "an" + , Space + , Str "odd" + , Space + , Str "list," + , Space + , Str "using" + , Space + , Str "1)," + , Space + , Str "2)," + , Space + , Str "etc:" + ] +, Para [] +, OrderedList + ( 1 , Decimal , OneParen ) + [ [ Para [ Str "Hello" , Space , Str "there" ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Para + [ Str "This" + , Space + , Str "is" + , Space + , Str "a" + , Space + , Str "sublist" + , Space + , Str "(lowercase" + , Space + , Str "alphabetic)" + ] + , OrderedList + ( 1 , LowerRoman , Period ) + [ [ Para + [ Str "Sub-sub-list" + , Space + , Str "(lowercase" + , Space + , Str "roman)" + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Plain + [ Str "Numeric," + , Space + , Str "but" + , Space + , Str "without" + , Space + , Str "the" + , Space + , Str ")" + , Space + , Str "but" + , Space + , Str "with" + , Space + , Str "the" + , Space + , Str "." + ] + ] + ] + ] + , [ Plain [ Str "Lower" , Space , Str "level" ] ] + ] + ] + , [ Plain [ Str "Lower" , Space , Str "level" ] ] + ] + ] + , [ Plain + [ Str "BACK" + , Space + , Str "TO" + , Space + , Str "ORIGINAL" + , Space + , Str "LEVEL" + ] + ] + ] +, Para [] +, Para + [ Str "Some" + , Space + , Str "list" + , Space + , Str "do" + , Space + , Str "the" + , Space + , Str "same" + , Space + , Str "but" + , Space + , Str "with" + , Space + , Str "alphabetic:" + , Space + , Str "a)," + , Space + , Str "b)," + , Space + , Str "etc." + ] +, Para [] +, OrderedList + ( 1 , LowerAlpha , OneParen ) + [ [ Para [ Str "Hello" , Space , Str "there" ] + , OrderedList + ( 1 , LowerAlpha , Period ) + [ [ Para + [ Str "Sub-list," + , Space + , Str "normal" + , Space + , Str "alphabetic" + , Space + , Str "with" + , Space + , Str "dot" + ] + , OrderedList + ( 1 , LowerRoman , Period ) + [ [ Plain + [ Str "Sub-sub-list," + , Space + , Str "lowercase" + , Space + , Str "roman" + , Space + , Str "again!" + ] + ] + ] + ] + ] + ] + , [ Plain + [ Str "BACK" + , Space + , Str "TO" + , Space + , Str "THE" + , Space + , Str "ORIGINAL" + ] + ] + ] +, Para [] +, Para + [ Str "We" + , Space + , Str "also" + , Space + , Str "can" + , Space + , Str "have" + , Space + , Str "list" + , Space + , Str "functioning" + , Space + , Str "with" + , Space + , Str "1.," + , Space + , Str "1.1.," + , Space + , Str "1.1.1," + , Space + , Str "etc." + ] +, Para [] +, OrderedList + ( 1 , Decimal , Period ) + [ [ Para + [ Str "This" + , Space + , Str "is" + , Space + , Str "an" + , Space + , Str "example" + , Space + , Str "of" + , Space + , Str "that" + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Plain + [ Str "This" + , Space + , Str "is" + , Space + , Str "the" + , Space + , Str "1.1" + ] + ] + , [ Para + [ Str "And" + , Space + , Str "this" + , Space + , Str "is" + , Space + , Str "the" + , Space + , Str "1.2." + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Para + [ Str "And" + , Space + , Str "this" + , Space + , Str "is" + , Space + , Str "a" + , Space + , Str "1.2.1." + ] + , OrderedList + ( 1 , Decimal , Period ) + [ [ Plain + [ Str "And" + , Space + , Str "so" + , Space + , Str "forth," + , Space + , Str "this" + , Space + , Str "will" + , Space + , Str "be" + , Space + , Str "a" + , Space + , Str "1.2.1.1" + ] + ] + ] + ] + ] + ] + ] + ] + , [ Plain + [ Str "Back" + , Space + , Str "to" + , Space + , Str "the" + , Space + , Str "original" + , Space + , Str "leven" + , Space + , Str "(2.)" + ] + ] + ] +, Para [] +, Para + [ Str "Some" + , Space + , Str "other" + , Space + , Str "interesting" + , Space + , Str "list" + , Space + , Str "as" + , Space + , Str "follows," + , Space + , Str "these" + , Space + , Str "are" + , Space + , Str "headings," + , Space + , Str "but" + , Space + , Str "not" + , Space + , Str "really" + , Space + , Str "lists," + , Space + , Str "but" + , Space + , Str "Microsoft" + , Space + , Str "Word" + , Space + , Str "pretends" + , Space + , Str "they" + , Space + , Str "are.." + ] +, Para [] +, Header + 1 + ( "it-seems-like-this-list-has-headings-1." , [] , [] ) + [ Str "It" + , Space + , Str "seems" + , Space + , Str "like" + , Space + , Str "this" + , Space + , Str "list" + , Space + , Str "has" + , Space + , Str "headings??" + , Space + , Str "(1.)" + ] +, Para [] +, Header + 2 + ( "i-wonder-how-this-is-converted-1.1" , [] , [] ) + [ Str "I" + , Space + , Str "wonder" + , Space + , Str "how" + , Space + , Str "this" + , Space + , Str "is" + , Space + , Str "converted" + , Space + , Str "(1.1)" + ] +, Para [] +, Para [] +, Para [ Strong [ Str "And" , Space , Str "finally:" ] ] +, Para [] +, Para + [ Str "We" + , Space + , Str "can" + , Space + , Str "also" + , Space + , Str "have" + , Space + , Str "lists" + , Space + , Str "with" + , Space + , Str "text" + , Space + , Str "in" + , Space + , Str "between." + , Space + ] +, Para [] +, Para + [ Str "Things" + , Space + , Str "in" + , Space + , Str "life" + , Space + , Str "I" + , Space + , Str "like:" + ] +, Para [] +, OrderedList + ( 1 , Decimal , Period ) [ [ Plain [ Str "Cookies" ] ] ] +, Para [] +, Para + [ Str "Cookies" + , Space + , Str "are" + , Space + , Str "just" + , Space + , Str "delicious" + ] +, Para [] +, OrderedList + ( 2 , Decimal , Period ) [ [ Plain [ Str "Alcohol" ] ] ] +, Para [] +, Para + [ Str "It\8217s" + , Space + , Str "really" + , Space + , Str "nice." + ] +, Para [] +, OrderedList + ( 3 , Decimal , Period ) [ [ Plain [ Str "CENSORED" ] ] ] +, Para [] +, Para + [ Str "Yeah" + , Space + , Str "we" + , Space + , Str "all" + , Space + , Str "know" + , Space + , Str "what" + , Space + , Str "I" + , Space + , Str "mean" + , Space + , Str "by" + , Space + , Str "that." + ] +] diff --git a/test/odt/odt/listContinueNumbering3.odt b/test/odt/odt/listContinueNumbering3.odt new file mode 100644 Binary files /dev/null and b/test/odt/odt/listContinueNumbering3.odt differ diff --git a/test/pptx/background-image/output.pptx b/test/pptx/background-image/output.pptx Binary files a/test/pptx/background-image/output.pptx and b/test/pptx/background-image/output.pptx differ diff --git a/test/pptx/background-image/templated.pptx b/test/pptx/background-image/templated.pptx Binary files a/test/pptx/background-image/templated.pptx and b/test/pptx/background-image/templated.pptx differ diff --git a/test/pptx/blanks/just-speaker-notes/output.pptx b/test/pptx/blanks/just-speaker-notes/output.pptx Binary files a/test/pptx/blanks/just-speaker-notes/output.pptx and b/test/pptx/blanks/just-speaker-notes/output.pptx differ diff --git a/test/pptx/blanks/just-speaker-notes/templated.pptx b/test/pptx/blanks/just-speaker-notes/templated.pptx Binary files a/test/pptx/blanks/just-speaker-notes/templated.pptx and b/test/pptx/blanks/just-speaker-notes/templated.pptx differ diff --git a/test/pptx/document-properties/output.pptx b/test/pptx/document-properties/output.pptx Binary files a/test/pptx/document-properties/output.pptx and b/test/pptx/document-properties/output.pptx differ diff --git a/test/pptx/document-properties/templated.pptx b/test/pptx/document-properties/templated.pptx Binary files a/test/pptx/document-properties/templated.pptx and b/test/pptx/document-properties/templated.pptx differ diff --git a/test/pptx/footer/basic/output.pptx b/test/pptx/footer/basic/output.pptx Binary files a/test/pptx/footer/basic/output.pptx and b/test/pptx/footer/basic/output.pptx differ diff --git a/test/pptx/footer/fixed-date/output.pptx b/test/pptx/footer/fixed-date/output.pptx Binary files a/test/pptx/footer/fixed-date/output.pptx and b/test/pptx/footer/fixed-date/output.pptx differ diff --git a/test/pptx/footer/higher-slide-number/output.pptx b/test/pptx/footer/higher-slide-number/output.pptx Binary files a/test/pptx/footer/higher-slide-number/output.pptx and b/test/pptx/footer/higher-slide-number/output.pptx differ diff --git a/test/pptx/footer/no-title-slide/output.pptx b/test/pptx/footer/no-title-slide/output.pptx Binary files a/test/pptx/footer/no-title-slide/output.pptx and b/test/pptx/footer/no-title-slide/output.pptx differ diff --git a/test/pptx/layouts/deleted.pptx b/test/pptx/layouts/deleted.pptx Binary files a/test/pptx/layouts/deleted.pptx and b/test/pptx/layouts/deleted.pptx differ diff --git a/test/pptx/layouts/moved.pptx b/test/pptx/layouts/moved.pptx Binary files a/test/pptx/layouts/moved.pptx and b/test/pptx/layouts/moved.pptx differ diff --git a/test/pptx/metadata-speaker-notes/output.pptx b/test/pptx/metadata-speaker-notes/output.pptx Binary files a/test/pptx/metadata-speaker-notes/output.pptx and b/test/pptx/metadata-speaker-notes/output.pptx differ diff --git a/test/pptx/metadata-speaker-notes/templated.pptx b/test/pptx/metadata-speaker-notes/templated.pptx Binary files a/test/pptx/metadata-speaker-notes/templated.pptx and b/test/pptx/metadata-speaker-notes/templated.pptx differ diff --git a/test/pptx/reference-no-slides/with-notes/output.pptx b/test/pptx/reference-no-slides/with-notes/output.pptx Binary files a/test/pptx/reference-no-slides/with-notes/output.pptx and b/test/pptx/reference-no-slides/with-notes/output.pptx differ diff --git a/test/pptx/speaker-notes-after-metadata/output.pptx b/test/pptx/speaker-notes-after-metadata/output.pptx Binary files a/test/pptx/speaker-notes-after-metadata/output.pptx and b/test/pptx/speaker-notes-after-metadata/output.pptx differ diff --git a/test/pptx/speaker-notes-after-metadata/templated.pptx b/test/pptx/speaker-notes-after-metadata/templated.pptx Binary files a/test/pptx/speaker-notes-after-metadata/templated.pptx and b/test/pptx/speaker-notes-after-metadata/templated.pptx differ diff --git a/test/pptx/speaker-notes-afterheader/output.pptx b/test/pptx/speaker-notes-afterheader/output.pptx Binary files a/test/pptx/speaker-notes-afterheader/output.pptx and b/test/pptx/speaker-notes-afterheader/output.pptx differ diff --git a/test/pptx/speaker-notes-afterheader/templated.pptx b/test/pptx/speaker-notes-afterheader/templated.pptx Binary files a/test/pptx/speaker-notes-afterheader/templated.pptx and b/test/pptx/speaker-notes-afterheader/templated.pptx differ diff --git a/test/pptx/speaker-notes-afterseps/output.pptx b/test/pptx/speaker-notes-afterseps/output.pptx Binary files a/test/pptx/speaker-notes-afterseps/output.pptx and b/test/pptx/speaker-notes-afterseps/output.pptx differ diff --git a/test/pptx/speaker-notes-afterseps/templated.pptx b/test/pptx/speaker-notes-afterseps/templated.pptx Binary files a/test/pptx/speaker-notes-afterseps/templated.pptx and b/test/pptx/speaker-notes-afterseps/templated.pptx differ diff --git a/test/pptx/speaker-notes/output.pptx b/test/pptx/speaker-notes/output.pptx Binary files a/test/pptx/speaker-notes/output.pptx and b/test/pptx/speaker-notes/output.pptx differ diff --git a/test/pptx/speaker-notes/templated.pptx b/test/pptx/speaker-notes/templated.pptx Binary files a/test/pptx/speaker-notes/templated.pptx and b/test/pptx/speaker-notes/templated.pptx differ