pandoc 3.5 → 3.6
raw patch · 86 files changed
+4200/−818 lines, 86 filesdep +libyamldep ~Diffdep ~commonmark-extensionsdep ~commonmark-pandoc
Dependencies added: libyaml
Dependency ranges changed: Diff, commonmark-extensions, commonmark-pandoc, data-default, skylighting, skylighting-core, texmath, typst
Files
- AUTHORS.md +3/−0
- MANUAL.txt +25/−11
- README.md +2/−0
- changelog.md +220/−1
- data/epub.css +0/−1
- data/templates/default.beamer +19/−18
- data/templates/default.context +7/−7
- data/templates/default.latex +1/−0
- data/templates/default.typst +7/−1
- data/templates/definitions.typst +0/−8
- data/templates/font-settings.latex +103/−0
- data/templates/fonts.latex +0/−103
- data/templates/template.typst +11/−11
- data/translations/ua.yaml +1/−1
- pandoc.cabal +17/−10
- src/Text/Pandoc/App.hs +13/−14
- src/Text/Pandoc/App/OutputSettings.hs +18/−12
- src/Text/Pandoc/Extensions.hs +3/−2
- src/Text/Pandoc/Format.hs +0/−1
- src/Text/Pandoc/ImageSize.hs +76/−1
- src/Text/Pandoc/Logging.hs +9/−0
- src/Text/Pandoc/PDF.hs +7/−15
- src/Text/Pandoc/Parsing/State.hs +1/−1
- src/Text/Pandoc/Readers.hs +3/−0
- src/Text/Pandoc/Readers/CommonMark.hs +2/−1
- src/Text/Pandoc/Readers/Docx.hs +18/−1
- src/Text/Pandoc/Readers/Docx/Fields.hs +56/−39
- src/Text/Pandoc/Readers/Docx/Parse.hs +19/−7
- src/Text/Pandoc/Readers/HTML.hs +14/−7
- src/Text/Pandoc/Readers/JATS.hs +1/−1
- src/Text/Pandoc/Readers/LaTeX.hs +2/−2
- src/Text/Pandoc/Readers/Mdoc.hs +1469/−0
- src/Text/Pandoc/Readers/Mdoc/Lex.hs +225/−0
- src/Text/Pandoc/Readers/Mdoc/Macros.hs +183/−0
- src/Text/Pandoc/Readers/Mdoc/Standards.hs +69/−0
- src/Text/Pandoc/Readers/MediaWiki.hs +28/−32
- src/Text/Pandoc/Readers/Metadata.hs +38/−9
- src/Text/Pandoc/Readers/RST.hs +216/−132
- src/Text/Pandoc/Readers/Roff.hs +57/−221
- src/Text/Pandoc/Readers/Roff/Escape.hs +222/−0
- src/Text/Pandoc/Readers/Typst.hs +11/−7
- src/Text/Pandoc/Readers/Typst/Math.hs +14/−0
- src/Text/Pandoc/Writers/ANSI.hs +6/−2
- src/Text/Pandoc/Writers/Docx/OpenXML.hs +1/−0
- src/Text/Pandoc/Writers/EPUB.hs +1/−5
- src/Text/Pandoc/Writers/HTML.hs +25/−16
- src/Text/Pandoc/Writers/LaTeX.hs +2/−1
- src/Text/Pandoc/Writers/Markdown.hs +17/−8
- src/Text/Pandoc/Writers/Powerpoint/Output.hs +1/−0
- src/Text/Pandoc/Writers/TEI.hs +3/−3
- src/Text/Pandoc/Writers/Typst.hs +24/−9
- test/Tests/Readers/Mdoc.hs +321/−0
- test/Tests/Readers/RST.hs +12/−12
- test/ansi-test.ansi +4/−0
- test/ansi-test.txt +4/−0
- test/command/10271.md +29/−0
- test/command/10279.md +9/−0
- test/command/10281.md +68/−0
- test/command/10318.md +84/−0
- test/command/10328.md +37/−0
- test/command/10390.md +23/−0
- test/command/2649.md +1/−1
- test/command/5081.md +8/−0
- test/command/512.md +1/−1
- test/command/6588.md +10/−0
- test/command/6992.md +76/−0
- test/command/8966.md +2/−2
- test/command/9196.md +6/−0
- test/command/bits-title-supress.md +2/−2
- test/command/jabberwocky.md +22/−0
- test/command/latex-color.md +12/−6
- test/command/man-defines.md +15/−0
- test/command/mdoc-An.md +17/−0
- test/command/mdoc-Bd-unfilled.md +24/−0
- test/command/mdoc-Bl-column.md +22/−0
- test/command/mdoc-Bl-tag.md +36/−0
- test/command/rst_block_subst.md +20/−0
- test/docx/empty_field.native +1/−1
- test/jats-reader.native +8/−3
- test/jats-reader.xml +3/−3
- test/rst-reader.native +7/−7
- test/tables.typst +4/−4
- test/test-pandoc.hs +2/−0
- test/typst-reader.native +7/−17
- test/undergradmath.typ +1/−1
- test/writer.typst +32/−37
AUTHORS.md view
@@ -23,6 +23,7 @@ - Amy de Buitléir - Anabra - Anders Waldenborg+- Andreas Deininger - Andreas Lööw - Andreas Scherer - Andres Freund@@ -240,6 +241,7 @@ - Lucas Escot - Lucas Viana - Lucas V. R+- Luis Rivera - Luke Plant - Marc Schreiber - Marcin Serwin@@ -366,6 +368,7 @@ - Sukil Etxenike - Sukka - Sumit Sahrawat+- Suraj Patil - Sven Wick - TEC - Tarik Graba
MANUAL.txt view
@@ -1,7 +1,7 @@ --- title: Pandoc User's Guide author: John MacFarlane-date: October 5, 2024+date: December 7, 2024 --- # Synopsis@@ -263,6 +263,7 @@ - `markdown_strict` (original unextended [Markdown]) - `mediawiki` ([MediaWiki markup]) - `man` ([roff man])+ - `mdoc` ([mdoc] manual page markup) - `muse` ([Muse]) - `native` (native Haskell) - `odt` ([OpenOffice text document][ODT])@@ -535,6 +536,7 @@ [Markua]: https://leanpub.com/markua/read [EndNote XML bibliography]: https://support.clarivate.com/Endnote/s/article/EndNote-XML-Document-Type-Definition [typst]: https://typst.app+[mdoc]: https://mandoc.bsd.lv/man/mdoc.7.html ## Reader options {.options} @@ -770,9 +772,11 @@ `-V` *KEY*[`=`*VAL*], `--variable=`*KEY*[`:`*VAL*] -: Set the template variable *KEY* to the value *VAL* when rendering the- document in standalone mode. If no *VAL* is specified, the- key will be given the value `true`.+: Set the template variable *KEY* to the string value *VAL* when rendering+ the document in standalone mode. If no *VAL* is specified, the+ key will be given the value `true`. Structured values (lists, maps)+ cannot be assigned using this option, but they can be assigned in+ the `variables` section of a [defaults file][Defaults files]. `--sandbox[=true|false]` @@ -2241,15 +2245,20 @@ by putting a file `templates/default.*FORMAT*` in the user data directory (see `--data-dir`, above). *Exceptions:* -- For `odt` output, customize the `default.opendocument`- template.+- For `odt` output, customize the `default.opendocument` template.+- For `docx` output, customize the `default.openxml` template. - For `pdf` output, customize the `default.latex` template (or the `default.context` template, if you use `-t context`, or the `default.ms` template, if you use `-t ms`, or the `default.html` template, if you use `-t html`).-- `docx` and `pptx` have no template (however, you can use- `--reference-doc` to customize the output).+- `pptx` has no template. +Note that `docx`, `odt`, and `pptx` output can also be customized+using `--reference-doc`. Use a reference doc to adjust the styles+in your document; use a template to handle variable interpolation and+customize the presentation of metadata, the position of the table+of contents, boilerplate text, etc.+ Templates contain *variables*, which allow for the inclusion of arbitrary information at any point in the file. They may be set at the command line using the `-V/--variable` option. If a variable is not set,@@ -2489,9 +2498,9 @@ or partial: ```-${months[, ]}$+${months[, ]} -${articles:bibentry()[; ]$+${articles:bibentry()[; ]} ``` The separator in this case is literal and (unlike with `sep`@@ -3323,6 +3332,10 @@ `section-numbering` : Schema to use for numbering sections, e.g. `1.A.1`. +`page-numbering`+: Schema to use for numbering pages, e.g. `1` or `i`, or+ an empty string to omit page numbering.+ `columns` : Number of columns for body text. @@ -6918,7 +6931,8 @@ EPUB metadata may be specified using the `--epub-metadata` option, but if the source document is Markdown, it is better to use a [YAML metadata-block][Extension: `yaml_metadata_block`]. Here is an example:+block][Extension: `yaml_metadata_block`]. Here is an example of+a YAML metadata block with EPUB metadata: --- title:
README.md view
@@ -81,6 +81,8 @@ - `mediawiki` ([MediaWiki markup](https://www.mediawiki.org/wiki/Help:Formatting)) - `man` ([roff man](https://man.cx/groff_man(7)))+- `mdoc` ([mdoc](https://mandoc.bsd.lv/man/mdoc.7.html) manual page+ markup) - `muse` ([Muse](https://amusewiki.org/library/manual)) - `native` (native Haskell) - `odt` ([OpenOffice text
changelog.md view
@@ -1,5 +1,224 @@ # Revision history for pandoc +## pandoc 3.6 (2024-12-07)++ * Add `mdoc` as input format (Evan Silberman). This change+ introduces a reader for mdoc, a roff-derived semantic markup+ language for manual pages. This reader has been developed almost+ exclusively against mandoc's documentation and implementation of+ mdoc as a reference, and the real-world manual pages tested+ against are those from the OpenBSD base system. Of ~3500 manuals+ in mdoc format shipped with a fresh OpenBSD install, 17 cause the+ mdoc reader to exit with a parse error. Any further chasing of+ edge cases is deferred to future work.++ * New module: Text.Pandoc.Readers.Mdoc, exporting `readMdoc` [API change].++ * Issue warnings for duplicate YAML metadata keys (#10312).++ * Ensure that `--sandbox` affects `--embed-resources`. Previously it+ did not (contrary to what was implied by the manual), which means+ that an image with URL `/etc/passwd` would leak an encoded version+ of that file to HTML output with `--self-contained` or+ `--embed-resources`, even if `--sandbox` was used. Thanks to+ Samuel Mortenson for pointing out the issue.++ * Text.Pandoc.App.OutputSettings: add `sandbox'` function.+ This computes the sandboxed files from Opt and avoids code repetition.++ * Docx reader:++ + Parse index references as empty spans with attributes (#10171).+ Attributes included are `entry`, and optionally `bold`, `italic`,+ `yomi`, `see`.+ + Don't create multiple paragraphs for title or subtitle (#10359).+ If there are multiple paragraphs with Title or Subtitle style,+ use only the first for metadata.+ + Handle case where Zotero `itemData` has different id from the+ `citationItem` id. In this case we use the `citationItemId`+ in the bibliography as well, overriding the `referenceId`+ in the itemData (#10366).++ * LaTeX reader:++ + Put parsed minipage in specially marked Div (#10266).++ * HTML reader:++ + Parse footnotes defined by dpub-aria roles (#5294).++ * MediaWiki reader:++ + Fix indented tables with caption (#10390).+ + Fix parsing of col/rowspan (#6992).++ * Typst reader:++ + Avoid generating empty paragraphs.+ + Support `underparen`, `overparen`.+ + Fix `#quote` attribution. If attribution is not present, don't print+ the `--` (#10320).+ + Fix typo in unicode code point for em dash (see #10320).++ * Commonmark reader:++ + `implicit_figures` should check for empty caption+ and not produce an implicit figure in this case (#10429).++ * RST reader:++ + Use a new one-pass parsing strategy. Instead of having an initial pass where+ we collect reference definitions, we create links with target+ `##SUBST##something` or `##REF##something` or `##NOTE##something`,+ and resolve these in a pass over the parsed AST. This allows us to handle+ link references that are not at the top level (#10281).+ + Ignore newlines in URL in explicit link (#10279).+ + Handle block level substitutions.+ + Support `:file:` on raw directive (#8584).+ + Implement option lists (#10318).+ + Avoid putting metadata in Para (#7766). Create MetaInlines when possible,+ just as with markdown input. MetaBlocks is still used when there are+ multiple paragraphs or non-paragraph content.+ This change also affects field lists.+ + Fix linked substitutions (#6588). E.g. `|Python|_`.+ + Support inline anchors (#9196).+ + Explicit links define references (#5081). For example,+ ``Go to `g`_ `g <www.example.com>`_.`` should produce two links to+ www.example.com.++ * EPUB writer:++ + Use standardized filename for cover image instead of the original name+ (#10404). This avoids problems with e.g. filenames containing spaces.++ * Markdown writer:++ + Issue INFO warning when not rendering table, e.g., when `raw_html` is+ disabled and the table can't be fit into a supported markdown table format+ (#10407).+ + Respect empty LineBlock lines in `plain` output (Evan Silberman).+ The plain writer behaved as a markdown variant with `Ext_line_blocks`+ turned off, and so empty lines in a line block would get eliminated.++ * LaTeX writer:++ Ensure that beamer footnotes go on frame, not column (#5769).++ * HTML writer:++ + Unwrap empty incremental divs (#10328, Albert Krewinkel).+ Divs are unwrapped if the only purpose of the div seems to be to control+ whether lists are presented incrementally on slides.++ * Typst writer:++ + Make template sensitive to a `page-numbering` variable (#10370).+ This can be set to an empty string (or, in metadata, to false) for+ no page numbers.+ + Make `smart` extension work (#10271). If `smart` is not enabled, a command in+ the default template will disable smartquote substitutions.+ When `smart` is enabled, render curly apostrophes as straight+ and escape straight apostrophes. When `smart` is disabled,+ render curly apostrophes as curly and don't escape straight+ apostrophes. Similarly for quotes, em and en dashes. This should+ give more idiomatic typst output, with fewer unnecessary+ escapes.++ * ANSI writer:++ + Respect empty LineBlock lines (Evan Silberman).++ * JATS writer:++ + Correct spelling of suppress attribute (#10350, Andreas Deininger).++ * Typst template:++ + Remove `definitions.typst` partial.+ + Remove unnecessary definition of `endnote`.+ + Incorporate the one remaining definition into `default.typst`.+ + Use typst 0.12 code for two column layout (#10294, Luis Rivera).+ + Note: the new templates presuppose typst 0.12; if you try to use an+ earlier version of typst, an error will be raised.++ * LaTeX/Beamer template:++ + Split `fonts.latex` partial into two parts: `fonts.latex` and+ `font-settings.latex`.+ + In beamer template, load beamer theme between `fonts.latex` and+ `font-settings.latex`. This allows a theme (such as metropolis)+ to set its own default font, while still allowing the user to+ override it. This fixes a regression in pandoc 3.5 (#10297).+ + Note: Users who have custom templates based on pandoc 3.5 templates will+ need to add `font-settings.latex()` after `fonts.latex()` in the+ latex template. In a beamer template, the beamer theme-setting+ code needs to be moved between these two partials.++ * ConTeXt template: Ensure that font names don't wrap (#10305).++ * `epub.css`: remove background-color (#10264, Suraj Patil).+ With this greyish background color, epubs look bad on a Kindle (#10263).++ * Text.Pandoc.ImageSize: add WebP support (Evan Silberman, #10397).+ Add `Webp` constructor on ImageType [API change].++ * Text.Pandoc.Readers.Roff and a new unexported module+ Text.Pandoc.Readers.Roff.Escape: parameterize Roff+ escaping (Evan Silberman) [API change]. This allows code to+ be reused between the mdoc and man readers, despite the+ differing Token types.++ * Text.Pandoc.PDF:++ + PDF via LaTeX: always do max runs if `toc` is present (#10308).+ The old method (checking to see if toc hash had changed) is not+ completely reliable.+ + Use `.source` extension, not `.html`, in `toPdfViaTempFile` (#10314).++ * Text.Pandoc.Logging: add `YamlWarning` constructor to `LogMessage`+ [API change] (#10312).++ * Text.Pandoc.Format: remove duplicate typst entry (#10388, Caleb Mclennan).++ * Fix a typo in the `ua.yaml` localization for 'See' (Jens).++ * Lua subsystem (Albert Krewinkel):++ + Remove prefixes from Lua type names (#8574). Lua type names were+ inconsistent with regard to the use of prefixes; all prefixes+ are removed now, and Lua types now have the same name as the+ Haskell types. The use of app-specific prefixes is suggested by+ the Lua manual to avoid collisions. However, this shouldn't be a+ problem with pandoc, as it cannot be used as a Lua package.++ * doc/libraries.md: Add newly developed Haskell packages.+ Sort list alphabetically (Albert Krewinkel).++ * doc/lua-filters.md: document `pandoc.List:iter` method (Albert Krewinkel).+ List objects have a new function `iter` that returns an iterator+ function that returns the next list item on each call.++ * MANUAL.txt:++ + Clarify what the example of YAML EPUB metadata shows (#10405).+ + Fix typo in template syntax (#10265, Pascal Wagler).+ + Update manual with information on openxml template (#10273).+ + Clarify that `--variable` can only assign string values (#10298).++ * Fix comments in TEI writer referring to DocBook (#10430, Evan Silberman).++ * Fix several typos in documentation (#10349, Andreas Deininger).++ * Allow Diff 1.0.++ * Add font-settings.latex partial to pandoc.cabal (#10379).++ * Bump upper bound for data-default.++ * Use latest typst, texmath, pandoc-lua-marshal, commonmark-pandoc,+ commonmark-extensions, skylighting, skylighting-format-blaze-html.++ ## pandoc 3.5 (2024-10-04) * Add command-line options `--list-of-figures/--lof` and@@ -3311,7 +3530,7 @@ * HTML, Markdown writers: filter out empty class attributes (#8251). These should not be generated by any pandoc readers, but they- might be produced programatically.+ might be produced programmatically. * Markdown writer:
data/epub.css view
@@ -19,7 +19,6 @@ line-height: 1.2; font-family: Georgia, serif; color: #1a1a1a;- background-color: #fdfdfd; } p { text-indent: 0;
data/templates/default.beamer view
@@ -62,6 +62,24 @@ $for(beameroption)$ \setbeameroption{$beameroption$} $endfor$+% Prevent slide breaks in the middle of a paragraph+\widowpenalties 1 10000+\raggedbottom+$if(section-titles)$+\AtBeginPart{+ \frame{\partpage}+}+\AtBeginSection{+ \ifbibliography+ \else+ \frame{\sectionpage}+ \fi+}+\AtBeginSubsection{+ \frame{\subsectionpage}+}+$endif$+$fonts.latex()$ $-- Set Beamer theme before user font settings so they can override theme $if(theme)$ \usetheme[$for(themeoptions)$$themeoptions$$sep$,$endfor$]{$theme$}@@ -81,24 +99,7 @@ $if(outertheme)$ \useoutertheme[$for(outerthemeoptions)$$outerthemeoptions$$sep$,$endfor$]{$outertheme$} $endif$-% Prevent slide breaks in the middle of a paragraph-\widowpenalties 1 10000-\raggedbottom-$if(section-titles)$-\AtBeginPart{- \frame{\partpage}-}-\AtBeginSection{- \ifbibliography- \else- \frame{\sectionpage}- \fi-}-\AtBeginSubsection{- \frame{\subsectionpage}-}-$endif$-$fonts.latex()$+$font-settings.latex()$ $common.latex()$ $for(header-includes)$ $header-includes$
data/templates/default.context view
@@ -60,19 +60,19 @@ $-- set up font fallbacks $for(mainfontfallback)$-\definefallbackfamily[mainface][rm][$mainfontfallback$][range=0x0000-0xFFFF, check=yes, force=no]+\definefallbackfamily[mainface][rm][$mainfontfallback/nowrap$][range=0x0000-0xFFFF, check=yes, force=no] $endfor$ $for(sansfontfallback)$-\definefallbackfamily[mainface][ss][$sansfontfallback$][range=0x0000-0xFFFF, check=yes, force=no]+\definefallbackfamily[mainface][ss][$sansfontfallback/nowrap$][range=0x0000-0xFFFF, check=yes, force=no] $endfor$ $for(monofontfallback)$-\definefallbackfamily[mainface][tt][$monofontfallback$][range=0x0000-0xFFFF, check=yes, force=no]+\definefallbackfamily[mainface][tt][$monofontfallback/nowrap$][range=0x0000-0xFFFF, check=yes, force=no] $endfor$ \definefallbackfamily[mainface][rm][CMU Serif][preset=range:greek, force=yes]-\definefontfamily[mainface][rm][$if(mainfont)$$mainfont$$else$Latin Modern Roman$endif$]-\definefontfamily[mainface][mm][$if(mathfont)$$mathfont$$else$Latin Modern Math$endif$]-\definefontfamily[mainface][ss][$if(sansfont)$$sansfont$$else$Latin Modern Sans$endif$]-\definefontfamily[mainface][tt][$if(monofont)$$monofont$$else$Latin Modern Typewriter$endif$][features=none]+\definefontfamily[mainface][rm][$if(mainfont)$$mainfont/nowrap$$else$Latin Modern Roman$endif$]+\definefontfamily[mainface][mm][$if(mathfont)$$mathfont/nowrap$$else$Latin Modern Math$endif$]+\definefontfamily[mainface][ss][$if(sansfont)$$sansfont/nowrap$$else$Latin Modern Sans$endif$]+\definefontfamily[mainface][tt][$if(monofont)$$monofont/nowrap$$else$Latin Modern Typewriter$endif$][features=none] \setupbodyfont[mainface$if(fontsize)$,$fontsize$$endif$] \setupwhitespace[$if(whitespace)$$whitespace$$else$medium$endif$]
data/templates/default.latex view
@@ -27,6 +27,7 @@ \setcounter{secnumdepth}{-\maxdimen} % remove section numbering $endif$ $fonts.latex()$+$font-settings.latex()$ $common.latex()$ $for(header-includes)$ $header-includes$
data/templates/default.typst view
@@ -1,4 +1,4 @@-$definitions.typst()$+#let horizontalrule = line(start: (25%,0%), end: (75%,0%)) #show terms: it => { it.children@@ -28,6 +28,11 @@ $template.typst()$ $endif$ +$if(smart)$+$else$+#set smartquote(enabled: false)++$endif$ $for(header-includes)$ $header-includes$ @@ -84,6 +89,7 @@ $if(section-numbering)$ sectionnumbering: "$section-numbering$", $endif$+ pagenumbering: $if(page-numbering)$"$page-numbering$"$else$none$endif$, cols: $if(columns)$$columns$$else$1$endif$, doc, )
− data/templates/definitions.typst
@@ -1,8 +0,0 @@-// Some definitions presupposed by pandoc's typst output.-#let horizontalrule = [- #line(start: (25%,0%), end: (75%,0%))-]--#let endnote(num, contents) = [- #stack(dir: ltr, spacing: 3pt, super[#num], contents)-]
+ data/templates/font-settings.latex view
@@ -0,0 +1,103 @@+$-- User font settings (must come after default font and Beamer theme)+$if(fontfamily)$+\usepackage[$for(fontfamilyoptions)$$fontfamilyoptions$$sep$,$endfor$]{$fontfamily$}+$endif$+\ifPDFTeX\else+ % xetex/luatex font selection+$if(mainfont)$+ $if(mainfontfallback)$+ \ifLuaTeX+ \usepackage{luaotfload}+ \directlua{luaotfload.add_fallback("mainfontfallback",{+ $for(mainfontfallback)$"$mainfontfallback$"$sep$,$endfor$+ })}+ \fi+ $endif$+ \setmainfont[$for(mainfontoptions)$$mainfontoptions$$sep$,$endfor$$if(mainfontfallback)$,RawFeature={fallback=mainfontfallback}$endif$]{$mainfont$}+$endif$+$if(sansfont)$+ $if(sansfontfallback)$+ \ifLuaTeX+ \usepackage{luaotfload}+ \directlua{luaotfload.add_fallback("sansfontfallback",{+ $for(sansfontfallback)$"$sansfontfallback$"$sep$,$endfor$+ })}+ \fi+ $endif$+ \setsansfont[$for(sansfontoptions)$$sansfontoptions$$sep$,$endfor$$if(sansfontfallback)$,RawFeature={fallback=sansfontfallback}$endif$]{$sansfont$}+$endif$+$if(monofont)$+ $if(monofontfallback)$+ \ifLuaTeX+ \usepackage{luaotfload}+ \directlua{luaotfload.add_fallback("monofontfallback",{+ $for(monofontfallback)$"$monofontfallback$"$sep$,$endfor$+ })}+ \fi+ $endif$+ \setmonofont[$for(monofontoptions)$$monofontoptions$$sep$,$endfor$$if(monofontfallback)$,RawFeature={fallback=monofontfallback}$endif$]{$monofont$}+$endif$+$for(fontfamilies)$+ \newfontfamily{$fontfamilies.name$}[$for(fontfamilies.options)$$fontfamilies.options$$sep$,$endfor$]{$fontfamilies.font$}+$endfor$+$if(mathfont)$+$if(mathspec)$+ \ifXeTeX+ \setmathfont(Digits,Latin,Greek)[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}+ \else+ \setmathfont[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}+ \fi+$else$+ \setmathfont[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}+$endif$+$endif$+$if(CJKmainfont)$+ \ifXeTeX+ \usepackage{xeCJK}+ \setCJKmainfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmainfont$}+ $if(CJKsansfont)$+ \setCJKsansfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKsansfont$}+ $endif$+ $if(CJKmonofont)$+ \setCJKmonofont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmonofont$}+ $endif$+ \fi+$endif$+$if(luatexjapresetoptions)$+ \ifLuaTeX+ \usepackage[$for(luatexjapresetoptions)$$luatexjapresetoptions$$sep$,$endfor$]{luatexja-preset}+ \fi+$endif$+$if(CJKmainfont)$+ \ifLuaTeX+ \usepackage[$for(luatexjafontspecoptions)$$luatexjafontspecoptions$$sep$,$endfor$]{luatexja-fontspec}+ \setmainjfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmainfont$}+ \fi+$endif$+\fi+$if(zero-width-non-joiner)$+%% Support for zero-width non-joiner characters.+\makeatletter+\def\zerowidthnonjoiner{%+ % Prevent ligatures and adjust kerning, but still support hyphenating.+ \texorpdfstring{%+ \TextOrMath{\nobreak\discretionary{-}{}{\kern.03em}%+ \ifvmode\else\nobreak\hskip\z@skip\fi}{}%+ }{}%+}+\makeatother+\ifPDFTeX+ \DeclareUnicodeCharacter{200C}{\zerowidthnonjoiner}+\else+ \catcode`^^^^200c=\active+ \protected\def ^^^^200c{\zerowidthnonjoiner}+\fi+%% End of ZWNJ support+$endif$+% Use upquote if available, for straight quotes in verbatim environments+\IfFileExists{upquote.sty}{\usepackage{upquote}}{}+\IfFileExists{microtype.sty}{% use microtype if available+ \usepackage[$for(microtypeoptions)$$microtypeoptions$$sep$,$endfor$]{microtype}+ \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts+}{}+
data/templates/fonts.latex view
@@ -21,106 +21,3 @@ $-- Set default font before Beamer theme so the theme can override it \usepackage{lmodern} $endif$-$-- User font settings (must come after default font and Beamer theme)-$if(fontfamily)$-\usepackage[$for(fontfamilyoptions)$$fontfamilyoptions$$sep$,$endfor$]{$fontfamily$}-$endif$-\ifPDFTeX\else- % xetex/luatex font selection-$if(mainfont)$- $if(mainfontfallback)$- \ifLuaTeX- \usepackage{luaotfload}- \directlua{luaotfload.add_fallback("mainfontfallback",{- $for(mainfontfallback)$"$mainfontfallback$"$sep$,$endfor$- })}- \fi- $endif$- \setmainfont[$for(mainfontoptions)$$mainfontoptions$$sep$,$endfor$$if(mainfontfallback)$,RawFeature={fallback=mainfontfallback}$endif$]{$mainfont$}-$endif$-$if(sansfont)$- $if(sansfontfallback)$- \ifLuaTeX- \usepackage{luaotfload}- \directlua{luaotfload.add_fallback("sansfontfallback",{- $for(sansfontfallback)$"$sansfontfallback$"$sep$,$endfor$- })}- \fi- $endif$- \setsansfont[$for(sansfontoptions)$$sansfontoptions$$sep$,$endfor$$if(sansfontfallback)$,RawFeature={fallback=sansfontfallback}$endif$]{$sansfont$}-$endif$-$if(monofont)$- $if(monofontfallback)$- \ifLuaTeX- \usepackage{luaotfload}- \directlua{luaotfload.add_fallback("monofontfallback",{- $for(monofontfallback)$"$monofontfallback$"$sep$,$endfor$- })}- \fi- $endif$- \setmonofont[$for(monofontoptions)$$monofontoptions$$sep$,$endfor$$if(monofontfallback)$,RawFeature={fallback=monofontfallback}$endif$]{$monofont$}-$endif$-$for(fontfamilies)$- \newfontfamily{$fontfamilies.name$}[$for(fontfamilies.options)$$fontfamilies.options$$sep$,$endfor$]{$fontfamilies.font$}-$endfor$-$if(mathfont)$-$if(mathspec)$- \ifXeTeX- \setmathfont(Digits,Latin,Greek)[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}- \else- \setmathfont[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}- \fi-$else$- \setmathfont[$for(mathfontoptions)$$mathfontoptions$$sep$,$endfor$]{$mathfont$}-$endif$-$endif$-$if(CJKmainfont)$- \ifXeTeX- \usepackage{xeCJK}- \setCJKmainfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmainfont$}- $if(CJKsansfont)$- \setCJKsansfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKsansfont$}- $endif$- $if(CJKmonofont)$- \setCJKmonofont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmonofont$}- $endif$- \fi-$endif$-$if(luatexjapresetoptions)$- \ifLuaTeX- \usepackage[$for(luatexjapresetoptions)$$luatexjapresetoptions$$sep$,$endfor$]{luatexja-preset}- \fi-$endif$-$if(CJKmainfont)$- \ifLuaTeX- \usepackage[$for(luatexjafontspecoptions)$$luatexjafontspecoptions$$sep$,$endfor$]{luatexja-fontspec}- \setmainjfont[$for(CJKoptions)$$CJKoptions$$sep$,$endfor$]{$CJKmainfont$}- \fi-$endif$-\fi-$if(zero-width-non-joiner)$-%% Support for zero-width non-joiner characters.-\makeatletter-\def\zerowidthnonjoiner{%- % Prevent ligatures and adjust kerning, but still support hyphenating.- \texorpdfstring{%- \TextOrMath{\nobreak\discretionary{-}{}{\kern.03em}%- \ifvmode\else\nobreak\hskip\z@skip\fi}{}%- }{}%-}-\makeatother-\ifPDFTeX- \DeclareUnicodeCharacter{200C}{\zerowidthnonjoiner}-\else- \catcode`^^^^200c=\active- \protected\def ^^^^200c{\zerowidthnonjoiner}-\fi-%% End of ZWNJ support-$endif$-% Use upquote if available, for straight quotes in verbatim environments-\IfFileExists{upquote.sty}{\usepackage{upquote}}{}-\IfFileExists{microtype.sty}{% use microtype if available- \usepackage[$for(microtypeoptions)$$microtypeoptions$$sep$,$endfor$]{microtype}- \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts-}{}-
data/templates/template.typst view
@@ -24,6 +24,7 @@ font: (), fontsize: 11pt, sectionnumbering: none,+ pagenumbering: "1", doc, ) = { set document(@@ -34,8 +35,9 @@ set page( paper: paper, margin: margin,- numbering: "1",- )+ numbering: pagenumbering,+ columns: cols,+ ) set par(justify: true) set text(lang: lang, region: region,@@ -43,7 +45,8 @@ size: fontsize) set heading(numbering: sectionnumbering) - if title != none {+ place(top, float: true, scope: "parent", clearance: 4mm)[+ #if title != none { align(center)[#block(inset: 2em)[ #text(weight: "bold", size: 1.5em)[#title] #(if subtitle != none {@@ -53,7 +56,7 @@ ]] } - if authors != none and authors != [] {+ #if authors != none and authors != [] { let count = authors.len() let ncols = calc.min(count, 3) grid(@@ -69,21 +72,18 @@ ) } - if date != none {+ #if date != none { align(center)[#block(inset: 1em)[ #date ]] } - if abstract != none {+ #if abstract != none { block(inset: 2em)[ #text(weight: "semibold")[Abstract] #h(1em) #abstract ] }+ ] - if cols == 1 {- doc- } else {- columns(cols, doc)- }+ doc }
data/translations/ua.yaml view
@@ -16,7 +16,7 @@ Preface: Передмова Proof: Докази References: Література-See.: див.+See: див. SeeAlso: див. також Table: Таблиця To: До
pandoc.cabal view
@@ -1,6 +1,6 @@ cabal-version: 2.4 name: pandoc-version: 3.5+version: 3.6 build-type: Simple license: GPL-2.0-or-later license-file: COPYING.md@@ -102,12 +102,12 @@ data/templates/affiliations.jats data/templates/default.markua data/templates/default.typst- data/templates/definitions.typst data/templates/template.typst data/templates/common.latex data/templates/hypersetup.latex data/templates/passoptions.latex data/templates/fonts.latex+ data/templates/font-settings.latex data/templates/after-header-includes.latex -- translations@@ -491,12 +491,12 @@ case-insensitive >= 1.2 && < 1.3, citeproc >= 0.8.1.1 && < 0.9, commonmark >= 0.2.6.1 && < 0.3,- commonmark-extensions >= 0.2.5.5 && < 0.3,- commonmark-pandoc >= 0.2.2.2 && < 0.3,+ commonmark-extensions >= 0.2.5.6 && < 0.3,+ commonmark-pandoc >= 0.2.2.3 && < 0.3, containers >= 0.6.0.1 && < 0.8, crypton >= 0.30 && < 1.1, crypton-connection >= 0.3.1 && < 0.5,- data-default >= 0.4 && < 0.8,+ data-default >= 0.4 && < 0.9, deepseq >= 1.3 && < 1.6, directory >= 1.2.3 && < 1.4, doclayout >= 0.5 && < 0.6,@@ -524,23 +524,24 @@ random >= 1 && < 1.3, safe >= 0.3.18 && < 0.4, scientific >= 0.3 && < 0.4,- skylighting >= 0.14.3 && < 0.15,- skylighting-core >= 0.14.3 && < 0.15,+ skylighting >= 0.14.4 && < 0.15,+ skylighting-core >= 0.14.4 && < 0.15, split >= 0.2 && < 0.3, syb >= 0.1 && < 0.8, tagsoup >= 0.14.6 && < 0.15, temporary >= 1.1 && < 1.4,- texmath >= 0.12.8.11 && < 0.13,+ texmath >= 0.12.8.12 && < 0.13, text >= 1.1.1.0 && < 2.2, text-conversions >= 0.3 && < 0.4, time >= 1.5 && < 1.15, unicode-collation >= 0.1.1 && < 0.2, unicode-transforms >= 0.3 && < 0.5, yaml >= 0.11 && < 0.12,+ libyaml >= 0.1.4 && < 0.2, zip-archive >= 0.4.3.1 && < 0.5, zlib >= 0.5 && < 0.8, xml >= 1.3.12 && < 1.4,- typst >= 0.6 && < 0.6.1,+ typst >= 0.6.1 && < 0.6.2, vector >= 0.12 && < 0.14, djot >= 0.1.2.2 && < 0.2, tls >= 2.0.1 && < 2.2,@@ -595,6 +596,7 @@ Text.Pandoc.Readers.EPUB, Text.Pandoc.Readers.Muse, Text.Pandoc.Readers.Man,+ Text.Pandoc.Readers.Mdoc, Text.Pandoc.Readers.FB2, Text.Pandoc.Readers.DokuWiki, Text.Pandoc.Readers.Ipynb,@@ -706,6 +708,9 @@ Text.Pandoc.Readers.LaTeX.Parsing, Text.Pandoc.Readers.LaTeX.SIunitx, Text.Pandoc.Readers.LaTeX.Table,+ Text.Pandoc.Readers.Mdoc.Lex,+ Text.Pandoc.Readers.Mdoc.Macros,+ Text.Pandoc.Readers.Mdoc.Standards, Text.Pandoc.Readers.Typst.Parsing, Text.Pandoc.Readers.Typst.Math, Text.Pandoc.Readers.ODT.Base,@@ -730,6 +735,7 @@ Text.Pandoc.Readers.Org.Shared, Text.Pandoc.Readers.Metadata, Text.Pandoc.Readers.Roff,+ Text.Pandoc.Readers.Roff.Escape, Text.Pandoc.Writers.Docx.OpenXML, Text.Pandoc.Writers.Docx.StyleMap, Text.Pandoc.Writers.Docx.Table,@@ -777,7 +783,7 @@ main-is: test-pandoc.hs hs-source-dirs: test build-depends: pandoc,- Diff >= 0.2 && < 0.6,+ Diff >= 0.2 && < 1.1, Glob >= 0.7 && < 0.11, bytestring >= 0.9 && < 0.13, containers >= 0.4.2.1 && < 0.8,@@ -829,6 +835,7 @@ Tests.Readers.Muse Tests.Readers.Creole Tests.Readers.Man+ Tests.Readers.Mdoc Tests.Readers.FB2 Tests.Readers.DokuWiki Tests.Writers.Native
src/Text/Pandoc/App.hs view
@@ -56,7 +56,8 @@ import Text.Pandoc.App.CommandLineOptions (parseOptions, parseOptionsFromArgs, options, handleOptInfo) import Text.Pandoc.App.Input (InputParameters (..), readInput)-import Text.Pandoc.App.OutputSettings (OutputSettings (..), optToOutputSettings)+import Text.Pandoc.App.OutputSettings (OutputSettings (..), optToOutputSettings,+ sandbox') import Text.Pandoc.Transforms (applyTransforms, filterIpynbOutput, headerShift, eastAsianLineBreakFilter) import Text.Collate.Lang (Lang (..), parseLang)@@ -155,17 +156,11 @@ return $ defFlavor "markdown" let makeSandboxed pureReader =- let files = maybe id (:) (optReferenceDoc opts) .- maybe id (:) (optEpubMetadata opts) .- maybe id (:) (optEpubCoverImage opts) .- maybe id (:) (optCSL opts) .- maybe id (:) (optCitationAbbreviations opts) $- optEpubFonts opts ++- optBibliography opts- in case pureReader of- TextReader r -> TextReader $ \o t -> sandbox files (r o t)- ByteStringReader r- -> ByteStringReader $ \o t -> sandbox files (r o t)+ case pureReader of+ TextReader r+ -> TextReader $ \o t -> sandbox' opts (r o t)+ ByteStringReader r+ -> ByteStringReader $ \o t -> sandbox' opts (r o t) (reader, readerExts) <- if ".lua" `T.isSuffixOf` readerNameBase@@ -328,8 +323,12 @@ | T.null t || T.last t /= '\n' = t <> T.singleton '\n' | otherwise = t textOutput <- ensureNl <$> f writerOptions doc- if (optSelfContained opts || optEmbedResources opts) && htmlFormat format- then TextOutput <$> makeSelfContained textOutput+ if htmlFormat format &&+ (optSelfContained opts || optEmbedResources opts)+ then if optSandbox opts+ then sandbox' opts $+ TextOutput <$> makeSelfContained textOutput+ else TextOutput <$> makeSelfContained textOutput else return $ TextOutput textOutput reports <- getLog return (output, reports)
src/Text/Pandoc/App/OutputSettings.hs view
@@ -18,6 +18,7 @@ module Text.Pandoc.App.OutputSettings ( OutputSettings (..) , optToOutputSettings+ , sandbox' ) where import qualified Data.Map as M import qualified Data.Text as T@@ -97,17 +98,9 @@ report $ Deprecated "asciidoctor" "use asciidoc instead" let makeSandboxed pureWriter =- let files = maybe id (:) (optReferenceDoc opts) .- maybe id (:) (optEpubMetadata opts) .- maybe id (:) (optEpubCoverImage opts) .- maybe id (:) (optCSL opts) .- maybe id (:) (optCitationAbbreviations opts) $- optEpubFonts opts ++- optBibliography opts- in case pureWriter of- TextWriter w -> TextWriter $ \o d -> sandbox files (w o d)- ByteStringWriter w ->- ByteStringWriter $ \o d -> sandbox files (w o d)+ case pureWriter of+ TextWriter w -> TextWriter $ \o d -> sandbox' opts (w o d)+ ByteStringWriter w -> ByteStringWriter $ \o d -> sandbox' opts (w o d) let standalone = optStandalone opts || isBinaryFormat format || pdfOutput let templateOrThrow = \case@@ -140,7 +133,7 @@ templ <- processCustomTemplate $ case customTemplate components of Nothing -> throwError $ PandocNoTemplateError format- Just t -> (runWithDefaultPartials $ compileTemplate path t) >>=+ Just t -> runWithDefaultPartials (compileTemplate path t) >>= templateOrThrow return (w, wexts, templ) else@@ -322,3 +315,16 @@ isBinaryFormat :: T.Text -> Bool isBinaryFormat s = s `elem` ["odt","docx","epub2","epub3","epub","pptx","pdf","chunkedhtml"]++-- Like 'sandbox', but computes the list of files to preserve from+-- 'Opt'.+sandbox' :: (PandocMonad m, MonadIO m) => Opt -> PandocPure a -> m a+sandbox' opts = sandbox sandboxedFiles+ where+ sandboxedFiles = maybe id (:) (optReferenceDoc opts) .+ maybe id (:) (optEpubMetadata opts) .+ maybe id (:) (optEpubCoverImage opts) .+ maybe id (:) (optCSL opts) .+ maybe id (:) (optCitationAbbreviations opts) $+ optEpubFonts opts +++ optBibliography opts
src/Text/Pandoc/Extensions.hs view
@@ -471,7 +471,8 @@ getDefaultExtensions "opml" = pandocExtensions -- affects notes getDefaultExtensions "markua" = extensionsFromList []-getDefaultExtensions "typst" = extensionsFromList [Ext_citations]+getDefaultExtensions "typst" = extensionsFromList [Ext_citations,+ Ext_smart] getDefaultExtensions "dokuwiki" = extensionsFromList [Ext_smart] getDefaultExtensions _ = extensionsFromList [Ext_auto_identifiers]@@ -656,6 +657,6 @@ getAll "mediawiki" = autoIdExtensions <> extensionsFromList [ Ext_smart ]- getAll "typst" = extensionsFromList [Ext_citations]+ getAll "typst" = extensionsFromList [Ext_citations, Ext_smart] getAll "djot" = extensionsFromList [Ext_sourcepos] getAll _ = mempty
src/Text/Pandoc/Format.hs view
@@ -222,7 +222,6 @@ ".text" -> defFlavor "markdown" ".textile" -> defFlavor "textile" ".tsv" -> defFlavor "tsv"- ".typ" -> defFlavor "typst" ".txt" -> defFlavor "markdown" ".typ" -> defFlavor "typst" ".wiki" -> defFlavor "mediawiki"
src/Text/Pandoc/ImageSize.hs view
@@ -37,6 +37,9 @@ import qualified Data.ByteString.Char8 as B import qualified Data.ByteString.Lazy as BL import Data.Binary.Get+import Data.Bits ((.&.), shiftR, shiftL)+import Data.Word (bitReverse32)+import Data.Maybe (isJust, fromJust) import Data.Char (isDigit) import Control.Monad import Text.Pandoc.Shared (safeRead)@@ -50,6 +53,7 @@ import qualified Data.Text.Lazy as TL import qualified Data.Text.Encoding as TE import Control.Applicative+import qualified Data.Attoparsec.ByteString as AW import qualified Data.Attoparsec.ByteString.Char8 as A import qualified Codec.Picture.Metadata as Metadata import Codec.Picture (decodeImageWithMetadata)@@ -57,7 +61,7 @@ -- quick and dirty functions to get image sizes -- algorithms borrowed from wwwis.pl -data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf | Tiff+data ImageType = Png | Gif | Jpeg | Svg | Pdf | Eps | Emf | Tiff | Webp deriving Show data Direction = Width | Height instance Show Direction where@@ -124,6 +128,9 @@ -> return Emf "\xEF\xBB\xBF<" -- BOM before svg -> imageType (B.drop 3 img)+ "RIFF"+ | B.take 4 (B.drop 8 img) == "WEBP"+ -> return Webp _ -> mzero findSvgTag :: ByteString -> Bool@@ -140,6 +147,7 @@ Just Eps -> mbToEither "could not determine EPS size" $ epsSize img Just Pdf -> mbToEither "could not determine PDF size" $ pdfSize img Just Emf -> mbToEither "could not determine EMF size" $ emfSize img+ Just Webp -> mbToEither "could not determine WebP size" $ webpSize opts img Nothing -> Left "could not determine image type" where mbToEither msg Nothing = Left msg mbToEither _ (Just x) = Right x@@ -376,3 +384,70 @@ case parseheader . BL.fromStrict $ img of Left _ -> Nothing Right (_, _, size) -> Just size++-- See https://developers.google.com/speed/webp/docs/riff_container+-- and RFC 6386+pWebpSize :: AW.Parser ImageSize+pWebpSize = do+ AW.string "RIFF"+ AW.take 4+ AW.string "WEBP"+ (w, h) <- lossy <|> lossless <|> extended+ return $ def+ { pxX = w+ , pxY = h+ }+ where+ bitsToMaybe = either (const Nothing) (\((_, _, s)) -> Just s)+ decode d = bitsToMaybe . d . BL.fromStrict+ lossySize = runGetOrFail $ do+ word <- getWord16le+ return $ word .&. 0x3FFF+ lossy = do+ AW.string "VP8 "+ AW.take 4 -- length in bytes of VP8 Lossy stream size+ keyFrame <- AW.anyWord8+ guard $ keyFrame .&. 1 == 0+ AW.take 2 -- remaining bytes of frame header+ AW.word8 0x9d -- VP8 keyframe magic+ AW.word8 0x01+ AW.word8 0x2a+ width16 <- AW.take 2+ height16 <- AW.take 2+ let w = toInteger <$> decode lossySize width16+ h = toInteger <$> decode lossySize height16+ guard $ isJust w && isJust h+ return (fromJust w, fromJust h)+ losslessSizes = runGetOrFail $ do+ bitReverse32 <$> getWord32le+ losslessSize word = 1 + (word .&. 0x3FFF)+ lossless = do+ AW.string "VP8L"+ AW.take 4 -- length in bytes of VP8 Lossless chunk size+ AW.word8 0x2f -- webp lossless stream magic+ sizes <- AW.take 4+ let mbword = decode losslessSizes sizes+ guard $ isJust mbword+ let word = fromJust mbword+ let w = toInteger $ losslessSize word+ h = toInteger $ losslessSize (word `shiftR` 14)+ return (w, h)+ extendedSize = runGetOrFail $ do+ low <- toInteger <$> getWord16le+ high <- toInteger <$> getWord8+ return $ 1 + (high `shiftL` 16) + (low)+ extended = do+ AW.string "VP8X"+ AW.take 8 -- VP8X chunk length, flags and reserved area+ width24 <- AW.take 3+ height24 <- AW.take 3+ let w = decode extendedSize width24+ h = decode extendedSize height24+ guard $ isJust w && isJust h+ return (fromJust w, fromJust h)++webpSize :: WriterOptions -> ByteString -> Maybe ImageSize+webpSize opts img =+ case AW.parseOnly pWebpSize img of+ Left _ -> Nothing+ Right sz -> Just sz { dpiX = fromIntegral $ writerDpi opts, dpiY = fromIntegral $ writerDpi opts}
src/Text/Pandoc/Logging.hs view
@@ -104,6 +104,7 @@ | MakePDFWarning Text | UnclosedDiv SourcePos SourcePos | UnsupportedCodePage Int+ | YamlWarning SourcePos Text deriving (Show, Eq, Data, Ord, Typeable, Generic) instance ToJSON LogMessage where@@ -283,6 +284,12 @@ ] UnsupportedCodePage cpg -> ["codepage" .= cpg]+ YamlWarning pos msg ->+ [ "source" .= sourceName pos+ , "line" .= toJSON (sourceLine pos)+ , "column" .= toJSON (sourceColumn pos)+ , "message" .= msg+ ] showPos :: SourcePos -> Text showPos pos = Text.pack $ sn ++ "line " ++@@ -430,6 +437,7 @@ " unclosed at " <> showPos closepos <> ", closing implicitly." UnsupportedCodePage cpg -> "Unsupported code page " <> tshow cpg <> ". Text will likely be garbled."+ YamlWarning pos m -> "YAML warning (" <> showPos pos <> "): " <> m messageVerbosity :: LogMessage -> Verbosity messageVerbosity msg =@@ -488,3 +496,4 @@ MakePDFWarning{} -> WARNING UnclosedDiv{} -> WARNING UnsupportedCodePage{} -> WARNING+ YamlWarning{} -> WARNING
src/Text/Pandoc/PDF.hs view
@@ -411,11 +411,11 @@ , k /= "TEXINPUTS" && k /= "TEXMFOUTPUT"] liftIO (UTF8.readFile file) >>= showVerboseInfo (Just tmpDir) program programArgs env''- go env'' programArgs (1 :: Int) Nothing+ go env'' programArgs (1 :: Int) where file = tmpDir ++ "/input.tex" outfile = outDir ++ "/input.pdf"- go env'' programArgs runNumber oldTocHash = do+ go env'' programArgs runNumber = do let maxruns = 4 -- stop if warnings present after 4 runs report $ MakePDFInfo ("LaTeX run number " <> tshow runNumber) mempty (exit, out) <- liftIO $ E.catch@@ -429,23 +429,15 @@ then BL.fromStrict <$> readFileStrict logFile else return mempty let rerunWarnings = checkForRerun logContents- tocHash <- do- let tocFile = replaceExtension outfile ".toc"- tocFileExists <- fileExists tocFile- if tocFileExists- then do- tocContents <- readFileStrict tocFile- pure $ Just $! hashWith SHA1 tocContents- else pure Nothing- -- compare hash of toc to former hash to see if it changed (#9295)- let rerunWarnings' = rerunWarnings ++- ["TOC changed" | tocHash /= oldTocHash ]+ tocFileExists <- fileExists (replaceExtension outfile ".toc")+ -- if we have a TOC we always need 3 runs, see #10308+ let rerunWarnings' = rerunWarnings ++ ["TOC is present" | tocFileExists] if not (null rerunWarnings') && runNumber < maxruns then do report $ MakePDFInfo "Rerun needed" (T.intercalate "\n" (map (UTF8.toText . BC.toStrict) rerunWarnings'))- go env'' programArgs (runNumber + 1) tocHash+ go env'' programArgs (runNumber + 1) else do (log', pdf) <- getResultingPDF (Just logFile) outfile return (exit, fromMaybe out log', pdf)@@ -480,7 +472,7 @@ -> Text -- ^ Source -> IO (Either ByteString ByteString) toPdfViaTempFile verbosity program args mkOutArgs source =- withTempFile "." "toPdfViaTempFile.html" $ \file h1 ->+ withTempFile "." "toPdfViaTempFile.source" $ \file h1 -> withTempFile "." "toPdfViaTempFile.pdf" $ \pdfFile h2 -> do hClose h1 hClose h2
src/Text/Pandoc/Parsing/State.hs view
@@ -186,4 +186,4 @@ type KeyTable = M.Map Key (Target, Attr) -type SubstTable = M.Map Key Inlines+type SubstTable = M.Map Key Blocks
src/Text/Pandoc/Readers.hs view
@@ -51,6 +51,7 @@ , readEPUB , readMuse , readMan+ , readMdoc , readFB2 , readIpynb , readCSV@@ -106,6 +107,7 @@ import Text.Pandoc.Readers.Txt2Tags import Text.Pandoc.Readers.Vimwiki import Text.Pandoc.Readers.Man+import Text.Pandoc.Readers.Mdoc import Text.Pandoc.Readers.CSV import Text.Pandoc.Readers.CslJson import Text.Pandoc.Readers.BibTeX@@ -168,6 +170,7 @@ ,("rtf" , TextReader readRTF) ,("typst" , TextReader readTypst) ,("djot" , TextReader readDjot)+ ,("mdoc" , TextReader readMdoc) ] -- | Retrieve reader, extensions based on format spec (format+extensions).
src/Text/Pandoc/Readers/CommonMark.hs view
@@ -71,7 +71,8 @@ readCommonMarkBody opts sources toks makeFigures :: Block -> Block-makeFigures (Para [Image (ident,classes,kvs) alt (src,tit)]) =+makeFigures (Para [Image (ident,classes,kvs) alt (src,tit)])+ | not (null alt) = Figure (ident,[],[]) (Caption Nothing [Plain alt]) [Plain [Image ("",classes,kvs) alt (src,tit)]]
src/Text/Pandoc/Readers/Docx.hs view
@@ -201,6 +201,8 @@ inlines <- smushInlines <$> mapM parPartToInlines parParts remaining <- bodyPartsToMeta' bps let+ -- for titles, we just take the first one and ignore the rest, #10359:+ f _ x | metaField == "title" || metaField == "subtitle" = x f (MetaInlines ils) (MetaInlines ils') = MetaBlocks [Para ils, Para ils'] f (MetaInlines ils) (MetaBlocks blks) = MetaBlocks (Para ils : blks) f m (MetaList mv) = MetaList (m : mv)@@ -470,6 +472,13 @@ parPartToInlines' (Field info children) = case info of HyperlinkField url -> parPartToInlines' $ ExternalHyperLink url children+ IndexrefField ie ->+ pure $ spanWith ("",["indexref"],+ (("entry", entryTitle ie) :+ maybe [] (\x -> [("crossref",x)]) (entrySee ie)+ ++ maybe [] (\x -> [("yomi",x)]) (entryYomi ie)+ ++ [("bold","") | entryBold ie]+ ++ [("italic","") | entryItalic ie])) mempty PagerefField fieldAnchor True -> parPartToInlines' $ InternalHyperLink fieldAnchor children EndNoteCite t -> do formattedCite <- smushInlines <$> mapM parPartToInlines' children@@ -539,7 +548,15 @@ let items = Citeproc.citationItems citation let cs = map toPandocCitation items refs <- mapM (traverse (return . text)) $- mapMaybe Citeproc.citationItemData items+ mapMaybe (\item ->+ case Citeproc.citationItemData item of+ Nothing -> Nothing+ Just itemData ->+ -- see #10366, sometimes itemData has a different+ -- id and we need to use the same one:+ Just $ itemData{ referenceId =+ Citeproc.citationItemId item })+ items modify $ \st -> st{ docxReferences = foldr (\ref -> M.insert (referenceId ref) ref)
src/Text/Pandoc/Readers/Docx/Fields.hs view
@@ -12,19 +12,30 @@ ECMA-376-1:2016, §17.16.5 -} module Text.Pandoc.Readers.Docx.Fields ( FieldInfo(..)+ , IndexEntry(..) , parseFieldInfo ) where import Data.Functor (($>), void) import qualified Data.Text as T import Text.Pandoc.Parsing+import Data.Maybe (isJust) type URL = T.Text type Anchor = T.Text +data IndexEntry = IndexEntry+ { entryTitle :: T.Text+ , entrySee :: Maybe T.Text+ , entryYomi :: Maybe T.Text+ , entryBold :: Bool+ , entryItalic :: Bool }+ deriving (Show)+ data FieldInfo = HyperlinkField URL -- The boolean indicates whether the field is a hyperlink. | PagerefField Anchor Bool+ | IndexrefField IndexEntry | CslCitation T.Text | CslBibliography | EndNoteCite T.Text@@ -38,26 +49,27 @@ parseFieldInfo = parse fieldInfo "" fieldInfo :: Parser FieldInfo-fieldInfo =- try (HyperlinkField <$> hyperlink)- <|>- try ((uncurry PagerefField) <$> pageref)- <|>- try addIn- <|>- return UnknownField+fieldInfo = do+ spaces+ hyperlink+ <|>+ pageref+ <|>+ indexref+ <|>+ addIn+ <|>+ return UnknownField addIn :: Parser FieldInfo addIn = do- spaces string "ADDIN" spaces try cslCitation <|> cslBibliography <|> endnoteCite <|> endnoteRefList cslCitation :: Parser FieldInfo cslCitation = do- optional (string "ZOTERO_ITEM")- spaces+ optional (string "ZOTERO_ITEM" *> spaces) string "CSL_CITATION" spaces CslCitation <$> getInput@@ -95,44 +107,49 @@ unquotedString = T.pack <$> manyTill anyChar (try $ void (lookAhead space) <|> eof) fieldArgument :: Parser T.Text-fieldArgument = quotedString <|> unquotedString---- there are other switches, but this is the only one I've seen in the wild so far, so it's the first one I'll implement. See §17.16.5.25-hyperlinkSwitch :: Parser (T.Text, T.Text)-hyperlinkSwitch = do- sw <- string "\\l"- spaces- farg <- fieldArgument- return (T.pack sw, farg)+fieldArgument = do+ notFollowedBy (char '\\') -- switch+ quotedString <|> unquotedString -hyperlink :: Parser URL+hyperlink :: Parser FieldInfo hyperlink = do- many space string "HYPERLINK" spaces farg <- option "" $ notFollowedBy (char '\\') *> fieldArgument- switches <- spaces *> many hyperlinkSwitch- let url = case switches of- ("\\l", s) : _ -> farg <> "#" <> s- _ -> farg- return url+ switches <- many fieldSwitch+ let url = case [s | ('l',s) <- switches] of+ [s] -> farg <> "#" <> s+ _ -> farg+ return $ HyperlinkField url -- See §17.16.5.45-pagerefSwitch :: Parser (T.Text, T.Text)-pagerefSwitch = do- sw <- string "\\h"+fieldSwitch :: Parser (Char, T.Text)+fieldSwitch = try $ do spaces- farg <- fieldArgument- return (T.pack sw, farg)+ char '\\'+ c <- anyChar+ spaces+ farg <- option mempty fieldArgument+ return (c, farg) -pageref :: Parser (Anchor, Bool)+pageref :: Parser FieldInfo pageref = do- many space string "PAGEREF" spaces farg <- fieldArgument- switches <- spaces *> many pagerefSwitch- let isLink = case switches of- ("\\h", _) : _ -> True- _ -> False- return (farg, isLink)+ switches <- many fieldSwitch+ let isLink = any ((== 'h') . fst) switches+ return $ PagerefField farg isLink++-- second element of tuple is optional "see".+indexref :: Parser FieldInfo+indexref = do+ string "XE"+ spaces+ farg <- fieldArgument+ switches <- spaces *> many fieldSwitch+ return $ IndexrefField $ IndexEntry{ entryTitle = farg+ , entrySee = lookup 't' switches+ , entryYomi = lookup 'y' switches+ , entryBold = isJust (lookup 'b' switches)+ , entryItalic = isJust (lookup 'i' switches) }
src/Text/Pandoc/Readers/Docx/Parse.hs view
@@ -43,6 +43,7 @@ , ChangeType(..) , ChangeInfo(..) , FieldInfo(..)+ , IndexEntry(..) , Level(..) , ParaStyleName , CharStyleName@@ -118,7 +119,7 @@ deriving Show data FldCharState = FldCharOpen- | FldCharFieldInfo FieldInfo+ | FldCharFieldInfo T.Text | FldCharContent FieldInfo [ParPart] deriving (Show) @@ -941,6 +942,10 @@ <w:fldChar w:fldCharType="end"/> </w:r> +Note that there may be mulitple w:instrText elements in a row.+For example, you might first have ` XE "`, then `Kay, Alan`, then `"`.+The texts in all of them should be concatenated before it is processed!+ So we do this in a number of steps. If we encounter the fldchar begin tag, we start open a fldchar state variable (see state above). We add the instrtext to it as FieldInfo. Then we close that and start adding@@ -961,13 +966,15 @@ _ | fldCharType == "begin" -> do modify $ \st -> st {stateFldCharState = FldCharOpen : fldCharState} return []- FldCharFieldInfo info : ancestors | fldCharType == "separate" -> do+ FldCharFieldInfo t : ancestors | fldCharType == "separate" -> do+ info <- eitherToD $ parseFieldInfo t modify $ \st -> st {stateFldCharState = FldCharContent info [] : ancestors} return []- -- Some fields have no content, since Pandoc doesn't understand any of those fields, we can just close it.- FldCharFieldInfo _ : ancestors | fldCharType == "end" -> do+ -- Some fields have no content, e.g. index XE:+ FldCharFieldInfo t : ancestors | fldCharType == "end" -> do modify $ \st -> st {stateFldCharState = ancestors}- return []+ info <- eitherToD $ parseFieldInfo t+ return [Field info []] [FldCharContent info children] | fldCharType == "end" -> do modify $ \st -> st {stateFldCharState = []} return [Field info $ reverse children]@@ -982,8 +989,13 @@ fldCharState <- gets stateFldCharState case fldCharState of FldCharOpen : ancestors -> do- info <- eitherToD $ parseFieldInfo $ strContent instrText- modify $ \st -> st {stateFldCharState = FldCharFieldInfo info : ancestors}+ modify $ \st -> st {stateFldCharState =+ FldCharFieldInfo (strContent instrText) : ancestors}+ return []+ FldCharFieldInfo t : ancestors -> do+ modify $ \st -> st {stateFldCharState =+ FldCharFieldInfo (t <> strContent instrText) :+ ancestors} return [] _ -> return [] {-
src/Text/Pandoc/Readers/HTML.hs view
@@ -192,6 +192,7 @@ TagOpen name attr -> let type' = fromMaybe "" $ lookup "type" attr <|> lookup "epub:type" attr+ role = fromMaybe "" $ lookup "role" attr epubExts = extensionEnabled Ext_epub_html_exts exts in case name of@@ -211,6 +212,8 @@ _ | "titlepage" `T.isInfixOf` type' , name `elem` ("section" : groupingContent) -> mempty <$ eTitlePage+ _ | role == "doc-endnotes"+ -> eFootnotes "p" -> pPara "h1" -> pHeader "h2" -> pHeader@@ -281,14 +284,14 @@ eFootnote :: PandocMonad m => TagParser m () eFootnote = do- guardEnabled Ext_epub_html_exts+ inNotes <- inFootnotes <$> getState TagOpen tag attr' <- lookAhead $ pSatisfy (\case TagOpen _ attr' -> case lookup "type" attr' <|> lookup "epub:type" attr' of Just "footnote" -> True Just "rearnote" -> True- _ -> False+ _ -> inNotes _ -> False) let attr = toStringAttr attr' let ident = fromMaybe "" (lookup "id" attr)@@ -299,11 +302,12 @@ eFootnotes :: PandocMonad m => TagParser m Blocks eFootnotes = try $ do let notes = ["footnotes", "rearnotes"]- guardEnabled Ext_epub_html_exts (TagOpen tag attr') <- lookAhead pAny let attr = toStringAttr attr'- guard $ maybe False (`elem` notes)- (lookup "type" attr <|> lookup "epub:type" attr)+ guard (lookup "role" attr == Just "doc-endnotes") <|>+ (guardEnabled Ext_epub_html_exts >>+ guard (maybe False (`elem` notes)+ (lookup "type" attr <|> lookup "epub:type" attr))) updateState $ \s -> s{ inFootnotes = True } result <- pInTags tag block updateState $ \s -> s{ inFootnotes = False }@@ -316,12 +320,12 @@ eNoteref :: PandocMonad m => TagParser m Inlines eNoteref = try $ do- guardEnabled Ext_epub_html_exts TagOpen tag attr <- pSatisfy (\case TagOpen _ as -> (lookup "type" as <|> lookup "epub:type" as)- == Just "noteref"+ == Just "noteref" ||+ lookup "role" as == Just "doc-noteref" _ -> False) ident <- case lookup "href" attr >>= T.uncons of Just ('#', rest) -> return rest@@ -679,6 +683,9 @@ case name of "a" | extensionEnabled Ext_epub_html_exts exts , Just "noteref" <- lookup "type" attr <|> lookup "epub:type" attr+ , Just ('#',_) <- lookup "href" attr >>= T.uncons+ -> eNoteref+ | Just "doc-noteref" <- lookup "role" attr , Just ('#',_) <- lookup "href" attr >>= T.uncons -> eNoteref | otherwise -> pLink
src/Text/Pandoc/Readers/JATS.hs view
@@ -388,7 +388,7 @@ Just t -> read $ T.unpack t Nothing -> if isBook || n == 0 then n + 1 else n headerText <- case filterChild (named "title") e of- Just t -> case maybeAttrValue "supress" t of+ Just t -> case maybeAttrValue "suppress" t of Just s -> if s == "no" then getInlines t else return mempty
src/Text/Pandoc/Readers/LaTeX.hs view
@@ -1013,8 +1013,8 @@ , ("abstract", mempty <$ (env "abstract" blocks >>= addMeta "abstract")) , ("sloppypar", env "sloppypar" blocks) , ("letter", env "letter" letterContents)- , ("minipage", env "minipage" $- skipopts *> spaces *> optional braced *> spaces *> blocks)+ , ("minipage", divWith ("",["minipage"],[]) <$>+ env "minipage" (skipopts *> spaces *> optional braced *> spaces *> blocks)) , ("figure", env "figure" $ skipopts *> figure') , ("subfigure", env "subfigure" $ skipopts *> tok *> figure') , ("center", divWith ("", ["center"], []) <$> env "center" blocks)
+ src/Text/Pandoc/Readers/Mdoc.hs view
@@ -0,0 +1,1469 @@+{-# LANGUAGE CPP #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE LambdaCase #-}+{-# LANGUAGE ViewPatterns #-}+{- |+ Module : Text.Pandoc.Readers.Mdoc+ Copyright : © 2024 Evan Silberman+ License : GNU GPL, version 2 or above++ Maintainer : Evan Silberman <evan@jklol.net>+ Stability : WIP+ Portability : portable++Conversion of mdoc to 'Pandoc' document.+-}+module Text.Pandoc.Readers.Mdoc (readMdoc) where++import Data.Char (isAsciiLower, toUpper)+import Data.Default (Default)+import Data.Either (fromRight)+import Data.Functor (($>))+import Data.Maybe (catMaybes)+import Control.Monad (mplus, guard, void, when, unless)+import Control.Monad.Except (throwError)+#if MIN_VERSION_base(4,19,0)+import Data.List (intersperse, unsnoc)+#else+import Data.List (intersperse, uncons)+#endif+import qualified Data.Map.Strict as M+import qualified Data.Text as T+import Text.Pandoc.Definition (Pandoc(Pandoc), Meta)+import Text.Pandoc.Builder (Blocks, Inlines)+import qualified Text.Pandoc.Builder as B+import Text.Pandoc.Class.PandocMonad (PandocMonad(..))+import Text.Pandoc.Options+import Text.Pandoc.Parsing hiding (uncons)+import Text.Pandoc.Logging+import Text.Pandoc.Readers.Mdoc.Lex+import Text.Pandoc.Readers.Mdoc.Standards+import Text.Parsec (modifyState)+import qualified Text.Pandoc.Parsing as P+import qualified Data.Foldable as Foldable+import Text.Pandoc.Shared (stringify)++#if !MIN_VERSION_base(4,19,0)+unsnoc :: [a] -> Maybe ([a], a)+unsnoc xs = (\(hd, tl) -> (reverse tl, hd)) <$> uncons (reverse xs)+#endif++ {- As a general principle, if mandoc -T lint issues a WARNING admonition+ or worse about a construct, I consider it fair game for this reader to+ do something different than what mandoc does with it, including bailing+ out instead of recovering. -}++data MdocSection+ = ShName+ | ShSynopsis+ | ShAuthors+ | ShSeeAlso+ | ShOther+ deriving (Show, Eq)++-- Declaration order is important: this is the order fields of a reference+-- are printed by mandoc+data ReferenceField =+ Author+ | ArticleTitle+ | BookTitle+ | Publisher+ | Journal+ | TechReportTitle+ | IssueNumber+ | VolumeNumber+ | Url+ | Pages+ | Institution+ | PubLocation+ | PubDate+ | Optional+ deriving (Show, Eq, Ord, Enum)++-- mandoc allows specifying multiple of _any_ reference field, and just+-- prints them all out in document order, even though authors are the only+-- field where this is the documented behavior. There's no lint warning+-- about this either. I'd prefer to do last-one-wins for the non-author+-- fields, which would presumably make it easier to transform the+-- bibliographic data into something else, but for now all I'm doing is+-- printing the references out the same way mandoc does.+type MdocReference = M.Map ReferenceField [T.Text]++data MdocState = MdocState+ { readerOptions :: ReaderOptions+ , metadata :: Meta+ , tableCellsPlain :: Bool+ , spacingMode :: Bool+ , authorNameSplit :: Bool+ , inLineEnclosure :: Bool+ , progName :: Maybe T.Text+ , currentSection :: MdocSection+ , currentReference :: MdocReference+ , logMessages :: [LogMessage]+ }+ deriving (Show)++instance Default MdocState where+ def =+ MdocState+ { readerOptions = def+ , metadata = B.nullMeta+ , tableCellsPlain = True+ , spacingMode = True+ , authorNameSplit = False+ , inLineEnclosure = False+ , currentSection = ShOther+ , currentReference = M.empty+ , progName = Nothing+ , logMessages = []+ }++instance HasLogMessages MdocState where+ addLogMessage msg st = st{ logMessages = msg : logMessages st }+ getLogMessages st = reverse $ logMessages st++type MdocParser m = P.ParsecT [MdocToken] MdocState m+++-- | Read mdoc from an input string and return a Pandoc document.+readMdoc :: (PandocMonad m, ToSources a)+ => ReaderOptions+ -> a+ -> m Pandoc+readMdoc opts s = do+ let Sources inps = toSources s+ tokenz <- mconcat <$> mapM (uncurry lexMdoc) inps+ let state = def {readerOptions = opts} :: MdocState+ eitherdoc <- readWithMTokens parseMdoc state+ (Foldable.toList . unMdocTokens $ tokenz)+ either (throwError . fromParsecError (Sources inps)) return eitherdoc+++readWithMTokens :: PandocMonad m+ => ParsecT [MdocToken] MdocState m a -- ^ parser+ -> MdocState -- ^ initial state+ -> [MdocToken] -- ^ input+ -> m (Either ParseError a)+readWithMTokens parser state input =+ runParserT parser state "source" input+++parseMdoc :: PandocMonad m => MdocParser m Pandoc+parseMdoc = do+ optional parsePrologue+ bs <- many parseBlock <* eof+ meta <- metadata <$> getState+ let (Pandoc _ blocks) = B.doc $ mconcat bs+ reportLogMessages+ return $ Pandoc meta blocks++msatisfy :: Monad m+ => (MdocToken -> Bool) -> P.ParsecT [MdocToken] st m MdocToken+msatisfy predic = P.tokenPrim show nextPos testTok+ where+ testTok t = if predic t then Just t else Nothing+ nextPos _ _ (Macro _ pos':_) = pos'+ nextPos _ _ (Lit _ pos':_) = pos'+ nextPos _ _ (Str _ pos':_) = pos'+ nextPos _ _ (Delim _ _ pos':_) = pos'+ nextPos _ _ (Blank pos':_) = pos'+ nextPos a _ (Eol{}:x:xs) = nextPos a x xs+ nextPos pos _ [Eol] = pos+ nextPos pos _ [] = pos++macro :: PandocMonad m => T.Text -> MdocParser m MdocToken+macro name = msatisfy t where+ t (Macro n _) = n == name+ t _ = False++anyMacro :: PandocMonad m => MdocParser m MdocToken+anyMacro = msatisfy t where+ t (Macro _ _) = True+ t _ = False++emptyMacro :: PandocMonad m => T.Text -> MdocParser m MdocToken+emptyMacro n = macro n <* eol++delim :: PandocMonad m => DelimSide -> MdocParser m MdocToken+delim side = msatisfy t where+ t (Delim s _ _) = side == s+ t _ = False++str :: PandocMonad m => MdocParser m MdocToken+str = msatisfy t where+ t Str{} = True+ t _ = False++lit :: PandocMonad m => MdocParser m MdocToken+lit = msatisfy t where+ t Lit{} = True+ t _ = False++arg :: PandocMonad m => MdocParser m MdocToken+arg = msatisfy t where+ t Lit{} = True+ t Macro{} = True+ t _ = False++literal :: PandocMonad m => T.Text -> MdocParser m MdocToken+literal n = msatisfy t where+ t (Lit n' _) = n == n'+ t _ = False++blank :: PandocMonad m => MdocParser m MdocToken+blank = msatisfy t where+ t Blank{} = True+ t _ = False++eol :: PandocMonad m => MdocParser m ()+eol = void $ msatisfy t where+ t Eol{} = True+ t _ = False++newControlContext :: MdocToken -> Bool+newControlContext Eol{} = True+newControlContext Macro{} = True+newControlContext Str{} = True+newControlContext Blank{} = True+newControlContext Lit{} = False+newControlContext Delim{} = False+++inlineContextEnd :: PandocMonad m => MdocParser m ()+inlineContextEnd = eof <|> (void . lookAhead $ msatisfy newControlContext)++sectionEnd :: PandocMonad m => MdocParser m ()+sectionEnd = eof <|> (void . lookAhead $ macro "Sh")++argsToInlines :: PandocMonad m => MdocParser m Inlines+argsToInlines = do+ ls <- manyTill arg eol+ let strs = map (B.str . toString) ls+ spacify strs++parsePrologue :: PandocMonad m => MdocParser m ()+parsePrologue = do+ macro "Dd"+ date <- argsToInlines+ macro "Dt"+ (Lit title _) <- lit+ (Lit section _) <- lit+ arch <- optionMaybe (toString <$> lit)+ eol+ emptyMacro "Os"+ let adjust = B.setMeta "title" (B.str title)+ . B.setMeta "date" date+ . B.setMeta "section" (B.str section)+ . maybe id (B.setMeta "architecture" . B.str) arch+ modifyState $ \s -> s{metadata = adjust $ metadata s}++shToSectionMode :: T.Text -> MdocSection+shToSectionMode "NAME" = ShName+shToSectionMode "SYNOPSIS" = ShSynopsis+shToSectionMode "AUTHORS" = ShAuthors+shToSectionMode "SEE ALSO" = ShSeeAlso+shToSectionMode _ = ShOther++parseHeader :: PandocMonad m => MdocParser m Blocks+parseHeader = do+ (Macro m _) <- lookAhead $ macro "Sh" <|> macro "Ss"+ txt <- lineEnclosure m id+ let lvl = if m == "Sh" then 1 else 2+ when (lvl == 1) $ modifyState $ \s -> s{currentSection = (shToSectionMode . stringify) txt}+ return $ B.header lvl txt++parseNameSection :: PandocMonad m => MdocParser m Blocks+parseNameSection = do+ sec <- currentSection <$> getState+ guard $ sec == ShName+ nms <- mconcat . intersperse B.space <$> many nameNm+ macro "Nd"+ desc <- argsToInlines+ return $ B.para $ nms <> B.space <> "—" <> B.space <> desc+ where+ nameNm = do+ macro "Nm"+ nms <- many1 aNm+ eol+ return $ mconcat $ intersperse B.space nms+ comma = msatisfy $ \case+ (Delim _ "," _) -> True+ _ -> False+ aNm = do+ nm <- toString <$> lit+ c <- option mempty (toString <$> comma)+ modifyState $ \s -> s{progName = mplus (progName s) (Just nm)}+ return $ B.code nm <> B.str c++parseSynopsisSection :: PandocMonad m => MdocParser m Blocks+parseSynopsisSection = do+ sec <- currentSection <$> getState+ guard $ sec == ShSynopsis+ parseSynopsis sectionEnd++parseMiniSynopsis :: PandocMonad m => MdocParser m Blocks+parseMiniSynopsis = do+ macro "nr"+ literal "nS"+ literal "1"+ eol+ parseSynopsis (sectionEnd <|> end)+ where+ end = do+ macro "nr"+ literal "nS"+ literal "0"+ eol+ return ()++parseSynopsis :: PandocMonad m => MdocParser m () -> MdocParser m Blocks+parseSynopsis end = do+ bs <- manyTill synopsisBlock end+ return $ mconcat bs+ where+ synopsisGroup p = B.lineBlock <$> many1 p <* optional (emptyMacro "Pp")+ synopsisBlock = synopsisGroup parseInvocation+ <|> synopsisGroup (parseCd <* optional eol)+ <|> synopsisGroup (parseIn <* optional eol)+ <|> synopsisGroup (parseFd <* optional eol)+ <|> synopsisGroup (parseVt <* optional eol)+ <|> try parseSignature+ <|> parseWeirdSignature+ <|> parseRegularBlock+ parseInvocation = do+ nm <- parseNm+ optional eol+ rest <- many synopsisInline+ spacify (nm:rest)+ parseSignature = do+ ft <- parseFt <* optional eol+ sig <- (parseFn <|> parseFo) <* optional eol+ return $ B.lineBlock [ft, sig <> ";"]+ -- e.g. OpenBSD MB_CUR_MAX(3), mild abuse of notation for Ft+ parseWeirdSignature = do+ ft <- parseFt <* optional eol+ rest <- many synopsisInline+ line <- spacify (ft:rest)+ return $ B.lineBlock [line]+ synopsisInline = parseSmToggle <|> parseStrs <|> (controlLine >>= spacify) <?> "synopsis inlines"+ safeEol = do+ amNested <- inLineEnclosure <$> getState+ unless amNested $ optional eol+ controlLine = many1 ((choice otherInlineMacros <|> litsAndDelimsToInlines) <* safeEol)++parseSeeAlsoSection :: PandocMonad m => MdocParser m Blocks+parseSeeAlsoSection = do+ sec <- currentSection <$> getState+ guard $ sec == ShSeeAlso+ blocks <- many1Till parseSeeAlsoBlock sectionEnd+ return $ mconcat blocks+ where+ parseSeeAlsoBlock = parseRegularBlock <|> (B.para <$> parseRs)++-- roff(7) says "In text lines, whitespace is preserved within a line." I+-- considered following this rule but it really cuts against the grain of what+-- Pandoc writers want to work with, for no clear benefit. This isn't wholly+-- inconsistent with mandoc, because it makes no effort to render multiple+-- consecutive spaces from the source document in HTML. Hence I call B.text+-- instead of B.str+parseStr :: PandocMonad m => MdocParser m Inlines+parseStr = do+ (Str txt _) <- str+ return $ B.text txt++-- It's unclear whether consecutive text lines ought to be affected by the+-- spacing mode. mdoc(7) claims that:+--+-- > By default, spacing is on. When switched off, no white space is+-- > inserted between macro arguments and between the output generated from+-- > adjacent macros, but text lines still get normal spacing between words+-- > **and sentences.**+--+-- (emphasis added)+-- This implied to me that while spacing is off, consecutive text lines+-- would have spacing between them as normal. In fact, in mandoc's+-- implementation, they do not:+--+-- text+-- .Sm off+-- text.+-- text+-- .Sm on+-- text+--+-- renders as+--+-- text text.text text+--+-- (The "." is in there since the allusion in the documentation to+-- sentences made me wonder if that made a difference; it doesn't.)+--+-- I've chosen to adopt my interpretation of the documented behavior, rather+-- than mandoc's implementation. Multiple consecutive strs within a block get+-- spaces between them and then packed up together, and text lines are not+-- affected by the spacing mode.+--+-- Reported at https://inbox.vuxu.org/mandoc-discuss/369KFE6SHMXSE.3PS4387AYEFB5@silby.fyi/T/+parseStrs :: PandocMonad m => MdocParser m Inlines+parseStrs = do+ txt <- many1 parseStr+ return $ mconcat $ intersperse B.space txt++parseDelim :: PandocMonad m => DelimSide -> MdocParser m Inlines+parseDelim pos = do+ (Delim _ txt _) <- delim pos+ return $ B.str txt++litsToText :: PandocMonad m => MdocParser m [T.Text]+litsToText = do+ ls <- many1 lit+ return $ map toString ls++litsToInlines :: PandocMonad m => MdocParser m Inlines+litsToInlines = do+ ls <- many1 lit+ let strs = map (B.str . toString) ls+ spacify strs++litsAndDelimsToInlines :: PandocMonad m => MdocParser m Inlines+litsAndDelimsToInlines = do+ (o, ls, c) <- delimitedArgs $ many lit+ guard $ not (null o && null ls && null c)+ strs <- spacify $ map (B.str . toString) ls+ return $ o <> strs <> c++openingDelimiters :: PandocMonad m => MdocParser m Inlines+openingDelimiters = do+ openDelim <- mconcat <$> many (parseDelim Open)+ omids <- pipes+ addSpace <- spacingMode <$> getState+ let omid | null omids = mempty+ | addSpace = omids <> B.space+ | otherwise = omids+ return $ openDelim <> omid++pipes :: PandocMonad m => MdocParser m Inlines+pipes = many (parseDelim Middle) >>= spacify++closingDelimiters :: PandocMonad m => MdocParser m Inlines+closingDelimiters = do+ cmids <- pipes+ addSpace <- spacingMode <$> getState+ let cmid | null cmids = mempty+ | addSpace = B.space <> cmids+ | otherwise = cmids+ closeDelim <- mconcat <$> many (parseDelim Close)+ return $ cmid <> closeDelim++delimitedArgs :: PandocMonad m => MdocParser m x -> MdocParser m (Inlines, x, Inlines)+delimitedArgs p = do+ openDelim <- openingDelimiters+ inlines <- p+ closeDelim <- closingDelimiters+ return (openDelim, inlines, closeDelim)++simpleInline :: PandocMonad m => T.Text -> (Inlines -> Inlines) -> MdocParser m Inlines+simpleInline nm xform = do+ macro nm+ segs <- manyTill segment inlineContextEnd+ spacify segs+ where+ segment = do+ (openDelim, inlines, closeDelim) <- delimitedArgs $ option mempty litsToInlines+ return $ openDelim <> xform inlines <> closeDelim++codeLikeInline' :: PandocMonad m => T.Text -> T.Text -> MdocParser m Inlines+codeLikeInline' nm cl = simpleInline nm (eliminateEmpty (B.codeWith (cls cl) . stringify))++codeLikeInline :: PandocMonad m => T.Text -> MdocParser m Inlines+codeLikeInline nm = codeLikeInline' nm nm++spanLikeInline :: PandocMonad m => T.Text -> MdocParser m Inlines+spanLikeInline nm = simpleInline nm (eliminateEmpty (B.spanWith (cls nm)))++-- One-line enclosures need a little bit of state so that we don't parse+-- the closing delimiters that follow nested one-line or multiline+-- enclosures; the closing delimiters are meant to go after the close of+-- the outermost enclosure. Hence we respect and set inLineEnclosure.+lineEnclosure :: PandocMonad m => T.Text -> (Inlines -> Inlines) -> MdocParser m Inlines+lineEnclosure nm xform = do+ macro nm+ amNested <- inLineEnclosure <$> getState+ modifyState $ \s -> s{inLineEnclosure = True}+ first <- openingDelimiters+ further <-+ (manyTill+ (parseInlineMacro+ <|> (try (litsAndDelimsToInlines <* notFollowedBy eol))+ <|> litsToInlines+ <|> openingDelimiters)+ lineEnclosureContextEnd)+ further' <- spacify further+ finally <- if amNested then mempty else closingDelimiters <* optional eol+ modifyState $ \s -> s{inLineEnclosure = amNested}+ return $ first <> xform further' <> finally+ where+ lineEnclosureContextEnd =+ try $+ void (lookAhead (macro "Ta"))+ <|> lookAhead (many (macro "Ns" <|> delim Close) *> eol)+++-- The Ns, Ap, and Sm macros affect the automatic insertion of spaces between+-- macro arguments that occurs by default. We parse these macros to RawInlines+-- that we then eliminate in foldNoSpaces. If any of these macros end up+-- in the final AST returned by readMdoc, it's a bug.++noSpace :: Inlines+noSpace = B.rawInline "mdoc" "Ns"++apMacro :: Inlines+apMacro = B.rawInline "mdoc" "Ap"++smOff :: Inlines+smOff = B.rawInline "mdoc" "Sm off"++smOn :: Inlines+smOn = B.rawInline "mdoc" "Sm on"++-- Accumulator for eliminating of Ns, Ap, and Sm macros from a list of 'Inlines'+data SpacifyState = SpacifyState+ { accum :: [Inlines], -- already-folded 'Inlines'+ prev :: Inlines, -- content we might be appending further content to+ ns :: Bool, -- True when we've read an Ns and are waiting to concatenate content to prev+ sm :: Bool -- True when spacing mode is on+ }++instance Default SpacifyState where+ def = SpacifyState [] mempty False True++-- Given a list of 'Inlines'es, concatenate consecutive elements that shouldn't+-- have a 'Space' inserted between them based on changes to the spacing mode,+-- Ap macros, and Ns macros.+foldNoSpaces :: [Inlines] -> [Inlines]+foldNoSpaces xs = (finalize . foldl go def) xs+ where+ go :: SpacifyState -> Inlines -> SpacifyState+ go s x+ | ns s && x == noSpace = s+ | x == apMacro = s{prev = prev s <> "'", ns = True}+ | x == noSpace = s{ns = True}+ | x == smOn = s{sm = True}+ | sm s && x == smOff = s{accum = accum s <> [prev s], prev = mempty, sm = False}+ | ns s = s{prev = prev s <> x, ns = False}+ | not (sm s) = s{prev = prev s <> x}+ | null (prev s) = s{prev = x}+ | otherwise = s{accum = accum s <> [prev s], prev = x}+ finalize s+ | null (prev s) = accum s+ | otherwise = accum s <> [prev s]++-- Add any necessary spaces between individual 'Inlines' in a list.+-- Respects the spacing mode status. This should more or less+-- always get applied to any list of 'Inlines' before doing anything+-- else with it.+spacify :: PandocMonad m => [Inlines] -> MdocParser m Inlines+spacify x = do+ mode <- spacingMode <$> getState+ return (go mode x)+ where+ go True = mconcat . intersperse B.space . foldNoSpaces+ go False = mconcat . foldNoSpaces++-- Compatibility note: mandoc permits, and doesn't warn on, "vertical" macros+-- (Pp, Bl/El, Bd/Ed) inside of "horizontal" block partial-explicit quotations+-- like Do/Dc. However there are no OpenBSD manual pages that employ such markup+-- and it doesn't look right when rendered. We don't attempt to consume anything+-- but pandoc inlines inside of these multiline enclosures.+multilineEnclosure :: PandocMonad m => T.Text -> T.Text -> (Inlines -> Inlines) -> MdocParser m Inlines+multilineEnclosure op cl xform = do+ macro op+ amNested <- inLineEnclosure <$> getState+ -- we're now "protected" from any outer enclosure or .It+ modifyState $ \s -> s{inLineEnclosure = False}+ openDelim <- mconcat <$> many (parseDelim Open)+ optional eol+ contents <- parseInlines+ (macro cl <?> show cl)+ closeDelim <-+ if amNested+ then mempty+ else mconcat <$> many (parseDelim Close) <* optional eol+ modifyState $ \s -> s{inLineEnclosure = amNested}+ return $ openDelim <> xform contents <> closeDelim++parseEo :: PandocMonad m => MdocParser m Inlines+parseEo = do+ macro "Eo"+ odel <- del+ optional eol+ inner <- parseInlines+ macro "Ec"+ cdel <- del+ optional eol+ return $ odel <> inner <> cdel+ where+ del = B.str . toString <$> (arg <|> delim Open <|> delim Middle <|> delim Close)++eliminateEmpty :: (Inlines -> Inlines) -> Inlines -> Inlines+eliminateEmpty x y = if null y then mempty else x y++cls :: T.Text -> B.Attr+cls x = (mempty, [x], mempty)++-- mandoc -T html formats Sy with a <b> tag, since it's not really+-- semantically <strong>, but Strong is our best option in Pandoc+parseSy :: PandocMonad m => MdocParser m Inlines+parseSy = simpleInline "Sy" (eliminateEmpty B.strong)++parseEm :: PandocMonad m => MdocParser m Inlines+parseEm = simpleInline "Em" (eliminateEmpty B.emph)++parseNo :: PandocMonad m => MdocParser m Inlines+parseNo = simpleInline "No" (eliminateEmpty id)++-- Deprecated, mandoc doesn't style this at all+parseTn :: PandocMonad m => MdocParser m Inlines+parseTn = simpleInline "Tn" (eliminateEmpty id)++parseLi :: PandocMonad m => MdocParser m Inlines+parseLi = codeLikeInline "Li"++parseEv :: PandocMonad m => MdocParser m Inlines+parseEv = codeLikeInline "Ev"++parseDv :: PandocMonad m => MdocParser m Inlines+parseDv = codeLikeInline "Dv"++parseAd :: PandocMonad m => MdocParser m Inlines+parseAd = spanLikeInline "Ad"++parseVa :: PandocMonad m => MdocParser m Inlines+parseVa = codeLikeInline' "Va" "variable"++parseVt :: PandocMonad m => MdocParser m Inlines+parseVt = codeLikeInline' "Vt" "variable"++parseAn :: PandocMonad m => MdocParser m Inlines+parseAn = try anSplit <|> anRegular+ where+ anSplit = do+ macro "An"+ mode <- literal "-split" $> True <|> literal "-nosplit" $> False+ modifyState $ \s -> s{authorNameSplit = mode}+ return mempty+ anRegular = do+ an <- spanLikeInline "An"+ spl <- authorNameSplit <$> getState+ return $ (if spl then B.linebreak else mempty) <> an++parseMs :: PandocMonad m => MdocParser m Inlines+parseMs = spanLikeInline "Ms"++-- TODO implement internal reference links+parseSx :: PandocMonad m => MdocParser m Inlines+parseSx = spanLikeInline "Sx"++-- I'm not sure why mandoc inserts a ~ when Mt is missing an argument,+-- but it does, and it doesn't issue a warning, so that quirk is+-- retained.+parseMt :: PandocMonad m => MdocParser m Inlines+parseMt = simpleInline "Mt" mailto+ where mailto x | null x = B.link ("mailto:~") "" "~"+ | otherwise = B.link ("mailto:" <> stringify x) "" x++parsePa :: PandocMonad m => MdocParser m Inlines+parsePa = simpleInline "Pa" p+ where p x | null x = B.spanWith (cls "Pa") "~"+ | otherwise = B.spanWith (cls "Pa") x++-- There's a number of unique-looking cases for Fl parsing so I am just+-- handling them very explicitly instead of trying to generalize anything+-- enough to handle it. Could conceivably be better.+parseFl :: PandocMonad m => MdocParser m Inlines+parseFl = do+ macro "Fl"+ start <- option mempty (emptyWithDelim <|> flfl <|> emptyWithMacro <|> emptyEmpty)+ segs <- manyTill segment inlineContextEnd+ spacify ([start] <> segs)+ where+ emptyWithDelim = do+ lookAhead $ many1 (delim Middle <|> delim Close)+ ds <- closingDelimiters+ return $ fl "-" <> ds+ flfl = do+ lookAhead (macro "Fl")+ x:xs <- B.toList <$> parseFl+ let xx = B.codeWith (cls "Fl") $ "-" <> stringify x+ return $ xx <> B.fromList xs+ emptyWithMacro = do+ lookAhead anyMacro+ rest <- parseInline+ return $ fl "-" <> rest+ emptyEmpty = lookAhead eol $> fl "-"+ segment = do+ (openDelim, inlines, closeDelim) <- delimitedArgs $ option mempty litsToText+ inner <- (spacify . (map fl) . flags) inlines+ return $ openDelim <> inner <> closeDelim+ fl = B.codeWith (cls "Fl")+ flags [] = ["-"]+ flags xs = map ("-" <>) xs++parseAr :: PandocMonad m => MdocParser m Inlines+parseAr = simpleInline "Ar" ar+ where ar x | null x = B.codeWith (cls "variable") "file ..."+ | otherwise = B.codeWith (cls "variable") $ stringify x+++parseCm :: PandocMonad m => MdocParser m Inlines+parseCm = codeLikeInline "Cm"++parseIc :: PandocMonad m => MdocParser m Inlines+parseIc = codeLikeInline "Ic"++parseEr :: PandocMonad m => MdocParser m Inlines+parseEr = codeLikeInline "Er"++parseCd :: PandocMonad m => MdocParser m Inlines+parseCd = codeLikeInline "Cd"++parseQl :: PandocMonad m => MdocParser m Inlines+parseQl = lineEnclosure "Ql" $ B.codeWith (cls "Ql") . stringify++parseDq :: PandocMonad m => MdocParser m Inlines+parseDq = lineEnclosure "Dq" B.doubleQuoted++parseDo :: PandocMonad m => MdocParser m Inlines+parseDo = multilineEnclosure "Do" "Dc" B.doubleQuoted++parseSq :: PandocMonad m => MdocParser m Inlines+parseSq = lineEnclosure "Sq" B.singleQuoted++parseSo :: PandocMonad m => MdocParser m Inlines+parseSo = multilineEnclosure "So" "Sc" B.singleQuoted++parseQq :: PandocMonad m => MdocParser m Inlines+parseQq = lineEnclosure "Qq" $ \x -> "\"" <> x <> "\""++parseQo :: PandocMonad m => MdocParser m Inlines+parseQo = multilineEnclosure "Qo" "Qc" $ \x -> "\"" <> x <> "\""++parsePq :: PandocMonad m => MdocParser m Inlines+parsePq = lineEnclosure "Pq" $ \x -> "(" <> x <> ")"++parsePo :: PandocMonad m => MdocParser m Inlines+parsePo = multilineEnclosure "Po" "Pc" $ \x -> "(" <> x <> ")"++parseBq :: PandocMonad m => MdocParser m Inlines+parseBq = lineEnclosure "Bq" $ \x -> "[" <> x <> "]"++parseBo :: PandocMonad m => MdocParser m Inlines+parseBo = multilineEnclosure "Bo" "Bc" $ \x -> "[" <> x <> "]"++-- For our purposes this probably behaves identically to Bq+-- in most circumstances but I might need to do something+-- special with it in SYNOPSIS+parseOp :: PandocMonad m => MdocParser m Inlines+parseOp = lineEnclosure "Op" $ \x -> "[" <> x <> "]"++parseOo :: PandocMonad m => MdocParser m Inlines+parseOo = multilineEnclosure "Oo" "Oc" $ \x -> "[" <> x <> "]"++parseBrq :: PandocMonad m => MdocParser m Inlines+parseBrq = lineEnclosure "Brq" $ \x -> "{" <> x <> "}"++parseBro :: PandocMonad m => MdocParser m Inlines+parseBro = multilineEnclosure "Bro" "Brc" $ \x -> "{" <> x <> "}"++parseAq :: PandocMonad m => MdocParser m Inlines+parseAq = lineEnclosure "Aq" $ \x -> "⟨" <> x <> "⟩"++parseAo :: PandocMonad m => MdocParser m Inlines+parseAo = multilineEnclosure "Ao" "Ac" $ \x -> "⟨" <> x <> "⟩"++parseDl :: PandocMonad m => MdocParser m Blocks+parseDl = do+ inner <- lineEnclosure "Dl" id+ return $ B.codeBlock (stringify inner)++parseD1 :: PandocMonad m => MdocParser m Blocks+parseD1 = do+ inner <- lineEnclosure "D1" id+ return $ B.divWith (cls "display") $ B.plain inner++parseNm :: PandocMonad m => MdocParser m Inlines+parseNm = do+ macro "Nm"+ mnm <- (progName <$> getState)+ (op, rg, cl) <- delimitedArgs $ option mempty litsToInlines+ return $ case (mnm, rg) of+ (Just nm, x) | null x ->+ op <> ok nm <> cl+ (_, x) ->+ op <> (ok . stringify) x <> cl+ where+ ok = B.codeWith (cls "Nm")+++parseXr :: PandocMonad m => MdocParser m Inlines+parseXr = do+ macro "Xr"+ (open, (name, section), close) <- delimitedArgs f+ let ref = name <> "(" <> section <> ")"+ return $ open <> B.spanWith (cls "Xr") (B.str ref) <> close+ where+ f = do+ n <- lit <?> "Xr manual name"+ s <- lit <?> "Xr manual section"+ return (toString n, toString s)++parseIn :: PandocMonad m => MdocParser m Inlines+parseIn = do+ macro "In"+ openClose <- closingDelimiters+ openOpen <- openingDelimiters+ header <- toString <$> lit+ close <- closingDelimiters+ return $ open openClose openOpen <> B.codeWith (cls "In") ("<" <> header <> ">") <> close+ where+ open a b+ | null a = b+ | null b = a+ | otherwise = a <> B.space <> b++parseFd :: PandocMonad m => MdocParser m Inlines+parseFd = codeLikeInline "Fd"++parseFt :: PandocMonad m => MdocParser m Inlines+parseFt = codeLikeInline' "Ft" "variable"++-- The output here is comparable to mandoc's HTML output, which doesn't tag+-- the commas/parentheses. Is this questionable from a pandoc POV?+formatFunction :: T.Text -> [Inlines] -> Inlines+formatFunction nm args = B.codeWith (cls "Fn") nm <> "(" <> args' <> ")"+ where+ args' = mconcat $ intersperse (", ") args++parseFn :: PandocMonad m => MdocParser m Inlines+parseFn = do+ macro "Fn"+ (op, (nm, args), cl) <- delimitedArgs f+ return $ op <> formatFunction nm (fmap (B.codeWith (cls "variable")) args) <> cl+ where+ f = do+ nm <- toString <$> lit+ args <- option [] litsToText+ return (nm, args)++parseFa :: PandocMonad m => MdocParser m Inlines+parseFa = codeLikeInline' "Fa" "variable"++parseFo :: PandocMonad m => MdocParser m Inlines+parseFo = do+ macro "Fo"+ nm <- toString <$> lit+ eol+ args <- many (parseFa <* eol)+ macro "Fc"+ return $ formatFunction nm args++parseLk :: PandocMonad m => MdocParser m Inlines+parseLk = do+ macro "Lk"+ openClose <- closingDelimiters+ openOpen <- openingDelimiters+ url <- toString <$> lit+ inner <- many segment >>= spacify+ close <- closingDelimiters+ let label | null inner = B.str url+ | otherwise = inner+ return $ open openClose openOpen <> B.link url "" label <> close+ where+ open a b+ | null a = b+ | null b = a+ | otherwise = a <> B.space <> b+ end = msatisfy newControlContext+ segment = do+ a <- openingDelimiters+ m <- option mempty litsToInlines+ z <-+ try (closingDelimiters <* notFollowedBy end)+ <|> option mempty pipes+ guard $ not $ all null [a, m, z]+ return $ a <> m <> z++-- This is a raw roff request but it appears sometimes in mdoc+-- manuals and is easy enough to handle+parsebr :: PandocMonad m => MdocParser m Inlines+parsebr = emptyMacro "br" >> return B.linebreak++parseNs :: PandocMonad m => MdocParser m Inlines+parseNs = macro "Ns" >> return noSpace++-- Per mdoc(7), Pf prefix macro [argument ...] is equivalent to+-- No \&prefix Ns macro [argument ...] and because of the way+-- spacify works, the easiest thing to do is just push an Ns onto+-- the input+parsePf :: PandocMonad m => MdocParser m Inlines+parsePf = do+ macro "Pf"+ t <- toString <$> anyToken+ rest <- getInput+ pos <- getPosition+ setInput $ (Macro "Ns" pos):rest+ return $ B.str t++parseAp :: PandocMonad m => MdocParser m Inlines+parseAp = macro "Ap" >> return apMacro++parseEx :: PandocMonad m => MdocParser m Inlines+parseEx = do+ macro "Ex"+ literal "-std"+ args <- fmap toString <$> many lit+ pn <- progName <$> getState+ eol+ return $ "The"+ <> B.space+ <> utils pn args+ <> B.space+ <> "0 on success, and >0 if an error occurs."+ where+ nm = B.codeWith (cls "Nm")+ sing = "utility exits"+ plur = "utilities exit"+ utils (Just x) [] = nm x <> B.space <> sing+ utils _ [x] = nm x <> B.space <> sing+ utils _ [x,y] = nm x <> B.space <> "and" <> B.space <> nm y <> B.space <> plur+ utils pn xs =+ case (pn, unsnoc xs) of+ (Nothing, Nothing) -> sing+ (_, Just (hd, end)) -> mconcat ((intersperse (", ") . fmap nm) hd) <> ", and " <> nm end <> B.space <> plur+ (Just p, Nothing) -> nm p <> B.space <> sing+++parseRv :: (PandocMonad m) => MdocParser m Inlines+parseRv = do+ macro "Rv"+ literal "-std"+ args <- fmap toString <$> many lit+ pn <- progName <$> getState+ eol+ return $ go pn args+ where+ nm a = B.codeWith (cls "Fn") a <> "()"+ nothing = "Upon successful completion, the value 0 is returned;"+ sing = "function returns"+ plur = "functions return"+ success = "the value 0 if successful;"+ errno =+ "otherwise the value -1 is returned and the global variable"+ <> B.codeWith (cls "variable") "errno"+ <> "is set to indicate the error."+ message conj =+ "The"+ <> B.space+ <> conj+ <> B.space+ <> success+ <> B.space+ <> errno+ go (Just x) [] = message (nm x <> B.space <> sing)+ go _ [x] = message (nm x <> B.space <> sing)+ go _ [x, y] = message (nm x <> B.space <> "and" <> B.space <> nm y <> B.space <> plur)+ go pn xs =+ case (pn, unsnoc xs) of+ (Nothing, Nothing) -> nothing <> B.space <> errno+ (_, Just (hd, end)) -> message (mconcat ((intersperse (", ") . fmap nm) hd) <> ", and " <> nm end <> B.space <> plur)+ (Just p, Nothing) -> message (nm p <> B.space <> sing)++parseSt :: PandocMonad m => MdocParser m Inlines+parseSt = do+ macro "St"+ (Lit std pos) <- lit+ case standard std of+ Nothing -> do+ logMessage $ SkippedContent ("unrecognized argument to St: " <> std) pos+ return mempty+ Just t -> return $ B.text t++-- TODO incorporate well-known library description and linker options+-- from mandoc lib.in expected in FreeBSD LIBRARY section, at minimum.+parseLb :: PandocMonad m => MdocParser m Inlines+parseLb = do+ macro "Lb"+ library <- toString <$> lit+ return $ "library" <> B.space <> B.doubleQuoted (B.str library)++unixVersion :: PandocMonad m => T.Text -> T.Text -> MdocParser m Inlines+unixVersion m s = do+ macro m+ (o, v, c) <- delimitedArgs (option mempty (toString <$> lit))+ return $ o <> B.str s <> f v <> c+ where+ f v | T.null v = mempty+ | otherwise = B.space <> B.str v++parseAt :: PandocMonad m => MdocParser m Inlines+parseAt = do+ macro "At"+ (o, v, c) <- delimitedArgs (optionMaybe (toString <$> lit))+ let v' = maybe "AT&T UNIX" attVer v+ return $ o <> B.text v' <> c+ where+ isVersion x = x `elem` ["1", "2", "3", "4", "5", "6", "7"]+ isRelease x = x `elem` ["1", "2", "3", "4"]+ attVer (T.stripPrefix "v" -> Just ver)+ | isVersion ver = "Version " <> ver <> " AT&T UNIX"+ attVer "32v" = "Version 7 AT&T UNIX/32V"+ attVer "III" = "AT&T System III UNIX"+ attVer (T.stripPrefix "V." -> Just release)+ | isRelease release = "AT&T System V Release " <> release <> " UNIX"+ attVer "V" = "AT&T System V UNIX"+ attVer x = "AT&T UNIX " <> x++parseBsx :: PandocMonad m => MdocParser m Inlines+parseBsx = unixVersion "Bsx" "BSD/OS"++parseBx :: PandocMonad m => MdocParser m Inlines+parseBx = do+ macro "Bx"+ (o, v, c) <- delimitedArgs zeroToTwoLits+ return $ o <> bsd v <> c+ where+ zeroToTwoLits = do+ toks <- try (count 2 lit) <|> count 1 lit <|> count 0 lit+ return $ toString <$> toks+ bsd [] = B.str "BSD"+ bsd [x] = B.str $ x <> "BSD"+ bsd (x:y:_) = B.str (x <> "BSD" <> "-" <> T.toTitle y)++parseDx :: PandocMonad m => MdocParser m Inlines+parseDx = unixVersion "Dx" "DragonFly"++parseFx :: PandocMonad m => MdocParser m Inlines+parseFx = unixVersion "Fx" "FreeBSD"++-- This dance to capitalize a letter at the end of a NetBSD+-- version matches what mandoc does to the argument of .Nx.+-- See mandoc mdoc_validate.c r1.350+-- Curiously, there's little easy-to-find evidence of what+-- these lettered releases actually are, other than+-- references in man page history sections to 0.9A etc.+parseNx :: PandocMonad m => MdocParser m Inlines+parseNx = do+ macro "Nx"+ (o, v, c) <- delimitedArgs (option mempty (toString <$> lit))+ return $ o <> "NetBSD" <> f v <> c+ where+ f v | T.null v = mempty+ | otherwise = B.space <> B.str (fromRight v $ readWith earlyNetBSDVersion () v)+ earlyNetBSDVersion = do+ major <- oneOf "01"+ dot <- char '.'+ minor <- digit+ ltr <- satisfy isAsciiLower+ return $ T.pack [major, dot, minor, toUpper ltr]++parseOx :: PandocMonad m => MdocParser m Inlines+parseOx = unixVersion "Ox" "OpenBSD"++parseUx :: PandocMonad m => MdocParser m Inlines+parseUx = macro "Ux" >> return (B.str "UNIX")++parseInlineMacro :: PandocMonad m => MdocParser m Inlines+parseInlineMacro = choice (synopsisTopicMacros <> otherInlineMacros) <?> "inline macro"++-- These macros always start a new line in SYNOPSIS+synopsisTopicMacros :: PandocMonad m => [MdocParser m Inlines]+synopsisTopicMacros =+ [parseNm, parseCd, parseFd, parseFn, parseFo, parseIn, parseVt, parseFt]++otherInlineMacros :: PandocMonad m => [MdocParser m Inlines]+otherInlineMacros =+ [ parseSy,+ parseEm,+ parseLk,+ parseLi,+ parseEv,+ parseDv,+ parseMt,+ parsePa,+ parseFl,+ parseCm,+ parseIc,+ parseEr,+ parseAd,+ parseVa,+ parseAn,+ parseMs,+ parseSx,+ parseAr,+ parseFa,+ parseNo,+ parseTn,+ parseXr,+ parseQl,+ parseOp,+ parseSq,+ parseDq,+ parseQq,+ parsePq,+ parseBq,+ parseBrq,+ parseAq,+ parseEo,+ parseSo,+ parseDo,+ parseQo,+ parsePo,+ parseBo,+ parseBro,+ parseAo,+ parseOo,+ parseBf,+ parseRsInline,+ parseEx,+ parseRv,+ parseSt,+ parseLb,+ parseAt,+ parseBsx,+ parseBx,+ parseDx,+ parseFx,+ parseNx,+ parseOx,+ parseUx,+ parsebr,+ parseAp,+ parsePf,+ parseNs,+ skipUnsupportedInlines+ ]++parseInline :: PandocMonad m => MdocParser m Inlines+parseInline = parseStrs <|> (controlLine >>= spacify) <?> "text lines or inline macros"+ where+ safeEol = do+ amNested <- inLineEnclosure <$> getState+ unless amNested $ optional eol+ controlLine = many1 ((parseInlineMacro <|> litsAndDelimsToInlines) <* safeEol)++parseInlines :: PandocMonad m => MdocParser m Inlines+parseInlines = many1 (parseSmToggle <|> parseInline) >>= spacify++-- Lp is a deprecated synonym for Pp+parsePara :: PandocMonad m => MdocParser m Blocks+parsePara = B.para . B.trimInlines <$> parseInlines <*+ optional (emptyMacro "Pp" <|> emptyMacro "Lp")++-- Indented display blocks are visually similar to block quotes+-- but rarely carry those semantics. I'm just putting things in+-- divs. Centered is discouraged and rarely seen.+parseDisplay :: PandocMonad m => MdocParser m Blocks+parseDisplay = do+ literal "-filled" <|> literal "-ragged" <|> literal "-centered"+ many $ (literal "-offset" *> lit) <|> (literal "-compact")+ eol+ B.divWith (cls "display") . mconcat <$> many parseRegularBlock++-- This is something of a best-effort interpretation of the -unfilled+-- display block type. The main difference with mandoc is probably+-- that newlines inside of multiline enclosures won't be preserved.+parseUnfilled :: PandocMonad m => MdocParser m Blocks+parseUnfilled = do+ literal "-unfilled"+ many $ (literal "-offset" *> lit) <|> (literal "-compact")+ eol+ lns <- many $ Just <$> parseStrPreserveSpace+ <|> Nothing <$ parseSmToggle+ <|> Just <$> parseInline+ <|> Just "" <$ emptyMacro "Pp"+ return $ B.lineBlock (catMaybes lns)+ where+ parseStrPreserveSpace = (B.str . toString) <$> str <|> (blank *> mempty)++parseCodeBlock :: PandocMonad m => MdocParser m Blocks+parseCodeBlock = do+ literal "-literal"+ many $ (literal "-offset" *> lit) <|> (literal "-compact")+ eol+ lns <- many $ Just . toString <$> (str <|> blank)+ <|> Nothing <$ parseSmToggle+ <|> Just . stringify <$> parseInline+ <|> Just "" <$ emptyMacro "Pp"+ return $ B.codeBlock (T.unlines (catMaybes lns))++parseBd :: PandocMonad m => MdocParser m Blocks+parseBd = do+ macro "Bd"+ blk <- parseCodeBlock <|> parseDisplay <|> parseUnfilled+ emptyMacro "Ed"+ return blk++-- This is a bit of a best effort version. Hypothetically multiple blocks+-- could occur inside a Bf and this should be a stateful thing but I don't+-- know if that's observed in the wild.+parseBf :: PandocMonad m => MdocParser m Inlines+parseBf = do+ macro "Bf"+ xform <- B.strong <$ (literal "Sy" <|> literal "-symbolic")+ <|> B.emph <$ (literal "Em" <|> literal "-emphasis")+ <|> code <$ (literal "Li" <|> literal "-literal")+ eol+ ins <- parseInlines+ emptyMacro "Ef"+ return $ xform ins+ where+ code = B.code . stringify++skipListArgument :: (PandocMonad m) => MdocParser m ()+skipListArgument =+ void $ choice+ [ literal "-width" *> lit,+ literal "-offset" *> lit,+ literal "-compact"+ ]++parseItemList :: PandocMonad m => MdocParser m Blocks+parseItemList = do+ f <- (choice (map literal ["-bullet", "-dash", "-hyphen", "-item"]) $> B.bulletList)+ <|> literal "-enum" $> B.orderedList+ many skipListArgument+ eol+ items <- many bulletItem+ return $ f items+ where+ bulletItem = do+ emptyMacro "It"+ mconcat <$> many parseRegularBlock++-- Despite some ambiguous documentation to the contrary the Xo/Xc macros+-- only seem genuinely useful in an .It head, and it's not clear what if+-- anything it means to use them somewhere else in a contemporary mdoc manual.+-- See https://inbox.vuxu.org/mandoc-discuss/2UKLZW0DL8BSM.2IIO9W4HSUSRR@silby.fyi/T/+-- for more blathering.+parseDefinitionList :: PandocMonad m => MdocParser m Blocks+parseDefinitionList = do+ headParser <- (choice . map literal) ["-hang", "-inset", "-ohang", "-tag"] $> parsedHead <|> literal "-diag" $> diagHead+ many skipListArgument+ eol+ items <- many (parseSmToggle *> mempty <|> dlItem headParser)+ return $ B.definitionList items+ where+ parsedHead = try xoListHead <|> eolListHead+ eolListHead = do+ modifyState $ \s -> s{inLineEnclosure = True}+ inner <- parseInlines+ eol+ modifyState $ \s -> s{inLineEnclosure = False}+ return inner+ diagHead = argsToInlines+ dlItem hed = do+ -- Some manuals have an evidently useless .Pp before .It+ -- e.g. OpenBSD ld(1), just deal with it.+ many ((void . emptyMacro) "Pp" <|> skipUnsupportedMacro "Tg")+ macro "It"+ dt <- hed+ dd <- mconcat <$> many parseRegularBlock+ return (dt, [dd])+ xoListHead = do+ before <- option mempty parseInline+ macro "Xo"+ optional eol+ after <- many1Till parseInlines (emptyMacro "Xc")+ spacify (before:after)++-- TODO support implicit rows:+-- If the first line of the body of a -column list is not an It macro line,+-- It contexts spanning one input line each are implied until an It macro+-- line is encountered+-- and support literal tabs+parseColumnList :: PandocMonad m => MdocParser m Blocks+parseColumnList = do+ literal "-column"+ many skipListArgument+ many $ arg <|> delim Open <|> delim Middle <|> delim Close+ eol+ rows <- many listRow+ return $ B.simpleTable [] rows+ where+ listRow = do+ optional (emptyMacro "Pp")+ macro "It"+ fmap B.plain <$> sepBy (parseInlines <|> pure mempty) (macro "Ta" <* optional eol)++parseBl :: PandocMonad m => MdocParser m Blocks+parseBl = do+ macro "Bl"+ blk <- parseItemList <|> parseDefinitionList <|> parseColumnList+ emptyMacro "El"+ return blk++referenceField :: PandocMonad m => T.Text -> ReferenceField -> MdocParser m ()+referenceField m field = do+ macro m+ reference <- currentReference <$> getState+ contents <- stringify <$> litsAndDelimsToInlines+ eol+ modifyState $ \s -> s{currentReference = M.insertWith (++) field [contents] reference}+ return ()++parsePercentA :: PandocMonad m => MdocParser m ()+parsePercentA = referenceField "%A" Author++parsePercentB :: PandocMonad m => MdocParser m ()+parsePercentB = referenceField "%B" BookTitle++parsePercentC :: PandocMonad m => MdocParser m ()+parsePercentC = referenceField "%C" PubLocation++parsePercentD :: PandocMonad m => MdocParser m ()+parsePercentD = referenceField "%D" PubDate++parsePercentI :: PandocMonad m => MdocParser m ()+parsePercentI = referenceField "%I" Publisher++parsePercentJ :: PandocMonad m => MdocParser m ()+parsePercentJ = referenceField "%J" Journal++parsePercentN :: PandocMonad m => MdocParser m ()+parsePercentN = referenceField "%N" IssueNumber++parsePercentO :: PandocMonad m => MdocParser m ()+parsePercentO = referenceField "%O" Optional++parsePercentP :: PandocMonad m => MdocParser m ()+parsePercentP = referenceField "%P" Pages++parsePercentQ :: PandocMonad m => MdocParser m ()+parsePercentQ = referenceField "%Q" Institution++parsePercentR :: PandocMonad m => MdocParser m ()+parsePercentR = referenceField "%R" TechReportTitle++parsePercentT :: PandocMonad m => MdocParser m ()+parsePercentT = referenceField "%T" ArticleTitle++parsePercentU :: PandocMonad m => MdocParser m ()+parsePercentU = referenceField "%U" Url++parsePercentV :: PandocMonad m => MdocParser m ()+parsePercentV = referenceField "%V" VolumeNumber++parseReferenceField :: PandocMonad m => MdocParser m ()+parseReferenceField =+ choice [+ parsePercentA,+ parsePercentB,+ parsePercentC,+ parsePercentD,+ parsePercentI,+ parsePercentJ,+ parsePercentN,+ parsePercentO,+ parsePercentP,+ parsePercentQ,+ parsePercentR,+ parsePercentT,+ parsePercentU,+ parsePercentV+ ]++parseRsInline :: PandocMonad m => MdocParser m Inlines+parseRsInline = do+ sec <- currentSection <$> getState+ guard $ sec /= ShSeeAlso+ parseRs++parseRs :: PandocMonad m => MdocParser m Inlines+parseRs = do+ emptyMacro "Rs"+ modifyState $ \s -> s{currentReference = M.empty}+ many1 parseReferenceField+ emptyMacro "Re"+ ref <- currentReference <$> getState+ -- TODO formatting fields correctly+ return $ B.text $ (M.foldl f mempty ref) <> "."+ where join v = T.concat (intersperse ", " v)+ f a v | T.null a = join v+ | otherwise = a <> ", " <> join v++-- mandoc's roff(7) says "Blank text lines, which may include whitespace,+-- are only permitted within literal contexts." mandoc -T lint warns about+-- blank lines and inserts a roff `sp` request, which is handled+-- differently depending on the output format. My read is that mandoc+-- considers the handling of a blank line in non-literal context in mdoc(7)+-- to be undefined. The Mdoc reader thus ignores blank input lines outside+-- of -literal and -unfilled displays.+skipBlanks :: PandocMonad m => MdocParser m Blocks+skipBlanks = many1 blank *> mempty++-- By default, mdoc is in "spacing mode", where horizontal space is added+-- between macro contents. The Sm macro turns it off and on. When we encounter+-- the Sm macro, we both modify the parser state and we emit a sentinel value+-- that spacify/foldNoSpaces uses to handle cases where spacing mode gets+-- turned off and on within a stretch of inlines.+parseSmToggle :: PandocMonad m => MdocParser m Inlines+parseSmToggle = do+ macro "Sm"+ cur <- spacingMode <$> getState+ mode <- optionMaybe (literal "on" $> True <|> literal "off" $> False)+ eol+ let newMode = update mode cur+ modifyState $ \s -> s{spacingMode = newMode}+ return $ if newMode then smOn else smOff+ where+ update = \case+ Nothing -> not+ Just x -> const x++skipUnsupportedMacro :: PandocMonad m => T.Text -> MdocParser m ()+skipUnsupportedMacro nm = do+ (Macro _ pos) <- macro nm+ manyTill anyToken eol+ logMessage $ SkippedContent ("unsupported macro: " <> nm) pos+++skipUnsupportedInlines :: PandocMonad m => MdocParser m Inlines+skipUnsupportedInlines = choice+ [ skipUnsupportedMacro "Tg",+ skipUnsupportedMacro "Bk",+ skipUnsupportedMacro "Ek"+ ] *> mempty++skipUnknownMacro :: PandocMonad m => MdocParser m Blocks+skipUnknownMacro = do+ pos <- getPosition+ m <- anyMacro+ manyTill anyToken eol+ logMessage $ SkippedContent ("unsupported macro: " <> toString m) pos+ return mempty+++parseRegularBlock :: PandocMonad m => MdocParser m Blocks+parseRegularBlock =+ choice+ [ parseDl+ , parseD1+ , parsePara+ , emptyMacro "Pp" *> mempty+ , parseBd+ , parseBl+ , skipBlanks+ ]++parseBlock :: (PandocMonad m) => MdocParser m Blocks+parseBlock =+ choice+ [ parseHeader+ , parseNameSection+ , parseSynopsisSection+ , parseSeeAlsoSection+ , parseMiniSynopsis+ , parseRegularBlock+ , skipUnknownMacro+ ]
+ src/Text/Pandoc/Readers/Mdoc/Lex.hs view
@@ -0,0 +1,225 @@+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE GeneralizedNewtypeDeriving #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE TypeFamilies #-}+{- |+ Module : Text.Pandoc.Readers.Mdoc.Lex+ Copyright : Copyright (C) 2024 Evan Silberman+ License : GNU GPL, version 2 or above++ Maintainer : Evan Silberman <evan@jklol.net>+ Stability : WIP+ Portability : portable++Tokenizer for mdoc+-}+module Text.Pandoc.Readers.Mdoc.Lex+ ( MdocToken(..)+ , MdocTokens(..)+ , DelimSide(..)+ , lexMdoc+ , toString+ )+where++import Control.Monad (void, guard, when)+import Control.Monad.Except (throwError)+import Text.Pandoc.Class.PandocMonad (PandocMonad(..))+import Data.Char (isAlphaNum)+import Data.Maybe (isJust)+import qualified Data.Text as T+import Text.Pandoc.Options+import Text.Pandoc.Parsing+import Text.Pandoc.Readers.Roff.Escape+import Text.Pandoc.Readers.Mdoc.Macros+import qualified Data.Sequence as Seq++-- As a higher level language with a wealth of semantic macros, mdoc+-- discourages authors from falling back to low-level roff features like font+-- selection, custom macros, defined strings, etc. Pandoc's mdoc reader is+-- accordingly implemented as a high-level interpreter of mdoc's semantic macros+-- and almost no raw roff requests are supported.+--+-- tbl(7) and eqn(7) macros are rare but not completely unseen in mdoc manuals.+-- they are not yet implemented. most use of tbl macros in mdoc could probably+-- be replaced with .Bl -column++data DelimSide = Open | Middle | Close deriving (Show, Eq)++-- | Tokens for Mdoc documents+data MdocToken = Str T.Text SourcePos -- ^ The contents of a text line+ | Macro T.Text SourcePos -- ^ A macro to be processed+ | Lit T.Text SourcePos -- ^ Literal text on a control line+ | Blank SourcePos -- ^ A blank line+ | Delim DelimSide T.Text SourcePos -- ^ A delimiter character+ | Eol -- ^ The end of a control line+ deriving Show++toString :: MdocToken -> T.Text+toString (Str x _) = x+toString (Macro x _) = x+toString (Lit x _) = x+toString (Delim _ x _) = x+toString Blank{} = mempty+toString Eol = mempty++newtype MdocTokens = MdocTokens { unMdocTokens :: Seq.Seq MdocToken }+ deriving (Show, Semigroup, Monoid)++singleTok :: MdocToken -> MdocTokens+singleTok t = MdocTokens (Seq.singleton t)++type Lexer m = ParsecT Sources () m++instance RoffLikeLexer MdocTokens where+ -- This is a bit confusing. We're lexing to MdocTokens, but for escaping+ -- purposes we just want Texts.+ type Token MdocTokens = T.Text+ -- We don't need a state+ type State MdocTokens = ()+ -- We don't support predefined string expansion+ expandString = return ()+ escString = return mempty+ -- what token type the unescaped text gets wrapped in is decided by other+ -- parts of the lexer.+ emit = id+ -- All escapes are resolved in the lexer and we never need to emit anything,+ -- vs. the roff lexer which has to push the backlashes to the output while+ -- in copy mode.+ backslash = (mempty <* char '\\') <|> (mempty <* string "\\E")+ -- We don't support macro definition and we don't output anything for \A+ checkDefined = const mempty+ -- We don't support copy mode and \E is treated as backslash+ escE = return mempty+ -- We don't support low-level font selection+ escFont = escIgnore 'f' [escapeArg, countChar 1 (satisfy (/='\n'))]++eofline :: (Stream s m Char, UpdateSourcePos s Char) => ParsecT s u m MdocToken+eofline = do+ void newline <|> eof+ return Eol++lexComment :: PandocMonad m => Lexer m MdocTokens+lexComment = do+ try $ string ".\\\""+ skipMany $ noneOf "\n"+ eofline+ return mempty++argText :: PandocMonad m => Lexer m T.Text+argText = do+ beg <- escape <|> regularText+ end <- mconcat <$> many (escape <|> regularText <|> quoteChar)+ return $ beg <> end++spaceTabChar :: PandocMonad m => Lexer m T.Text+spaceTabChar = T.singleton <$> spaceChar++quotedArg :: PandocMonad m => Lexer m T.Text+quotedArg = do+ quoteChar+ t <- mconcat <$> many (try innerQuote <|> escape <|> regularText <|> spaceTabChar)+ quoteChar+ notFollowedBy quoteChar+ return t+ where+ innerQuote = do+ string "\"\""+ return "\""++anyText :: PandocMonad m => Lexer m T.Text+anyText = escape <|> regularText <|> quoteChar <|> spaceTabChar++regularText :: PandocMonad m => Lexer m T.Text+regularText = many1Char $ noneOf "\n\r\t \\\""++quoteChar :: PandocMonad m => Lexer m T.Text+quoteChar = T.singleton <$> char '"'++mdocToken :: PandocMonad m => Lexer m MdocTokens+mdocToken = lexComment <|> lexControlLine <|> lexTextLine++lexMacroName :: PandocMonad m => Lexer m T.Text+lexMacroName = many1Char (satisfy isMacroChar)+ where+ isMacroChar '%' = True+ isMacroChar x = isAlphaNum x++lexMacro :: PandocMonad m => Lexer m MdocToken+lexMacro = do+ pos <- getPosition+ name <- lexMacroName+ eof <|> void (lookAhead (spaceChar <|> newline))+ skipSpaces+ return $ Macro name pos++lexCallableMacro :: PandocMonad m => Lexer m MdocToken+lexCallableMacro = do+ pos <- getPosition+ q <- optionMaybe quoteChar+ name <- lexMacroName+ when (isJust q) (void quoteChar)+ eof <|> void (lookAhead (spaceChar <|> newline))+ skipSpaces+ guard $ isCallableMacro name+ return $ Macro name pos++lexDelim :: (PandocMonad m) => Lexer m MdocToken+lexDelim = do+ pos <- getPosition+ q <- optionMaybe quoteChar+ t <-+ Delim Open <$> oneOfStrings ["(", "["]+ <|> Delim Close <$> oneOfStrings [".", ",", ":", ";", ")", "]", "?", "!"]+ <|> Delim Middle <$> textStr "|"+ when (isJust q) (void quoteChar)+ eof <|> void (lookAhead (spaceChar <|> newline))+ skipSpaces+ return $ t pos++lexLit :: PandocMonad m => Lexer m MdocToken+lexLit = do+ pos <- getPosition+ t <- argText <|> quotedArg+ skipSpaces+ return $ Lit t pos++lexTextLine :: PandocMonad m => Lexer m MdocTokens+lexTextLine = do+ pos <- getPosition+ guard $ sourceColumn pos == 1+ t <- mconcat <$> many anyText+ eofline+ if T.null $ T.strip t+ then return $ singleTok $ Blank pos+ else return $ singleTok $ Str t pos++lexControlLine :: PandocMonad m => Lexer m MdocTokens+lexControlLine = do+ pos <- getPosition+ guard $ sourceColumn pos == 1+ char '.'+ eofline *> mempty <|> do+ m@(Macro name _) <- lexMacro+ -- .Ns macros at the start of a line are ignored. We'd have to look behind+ -- to keep track of the "start of the line" in the parser, so we'll drop+ -- those macros in lexing.+ let start | name == "Ns" = []+ | otherwise = [m]+ let parsed = isParsedMacro name+ (wds, e) <- manyUntil (l parsed) eofline+ return $ MdocTokens $ Seq.fromList $ start <> wds <> [e]+ where+ l True = try lexDelim <|> try lexCallableMacro <|> lexLit+ l False = try lexDelim <|> lexLit++-- | Tokenize a string as a sequence of mdoc tokens.+lexMdoc :: PandocMonad m => SourcePos -> T.Text -> m MdocTokens+lexMdoc pos txt = do+ eithertokens <- readWithM (do setPosition pos+ mconcat <$> manyTill mdocToken eof) def txt+ case eithertokens of+ Left e -> throwError e+ Right tokenz -> return tokenz
+ src/Text/Pandoc/Readers/Mdoc/Macros.hs view
@@ -0,0 +1,183 @@+{-# LANGUAGE OverloadedStrings #-}+{- |+ Module : Text.Pandoc.Readers.Mdoc.Macros+ Copyright : © 2024 Evan Silberman+ License : GNU GPL, version 2 or above++ Maintainer : Evan Silberman <evan@jklol.net>+ Stability : WIP+ Portability : portable++-}+module Text.Pandoc.Readers.Mdoc.Macros (isParsedMacro, isCallableMacro) where++import Data.Set (member, fromList, Set)+import Data.Text++isParsedMacro :: Text -> Bool+isParsedMacro a = member a parsedMacros++isCallableMacro :: Text -> Bool+isCallableMacro a = member a callableMacros++parsedMacros :: Set Text+parsedMacros = fromList [+ "Ac",+ "Ad",+ "An",+ "Ao",+ "Ap",+ "Aq",+ "Ar",+ "At",+ "Bc",+ "Bo",+ "Bq",+ "Brc",+ "Bro",+ "Brq",+ "Bsx",+ "Bx",+ "Cd",+ "Cm",+ "D1",+ "Dc",+ "Dl",+ "Do",+ "Dq",+ "Dv",+ "Dx",+ "Ec",+ "Em",+ "En",+ "Eo",+ "Er",+ "Es",+ "Ev",+ "Fa",+ "Fc",+ "Fl",+ "Fn",+ "Fr",+ "Ft",+ "Fx",+ "Ic",+ "In",+ "It",+ "Li",+ "Lk",+ "Ms",+ "Mt",+ "Nm",+ "No",+ "Ns",+ "Nx",+ "Oc",+ "Oo",+ "Op",+ "Ot",+ "Ox",+ "Pa",+ "Pc",+ "Pf",+ "Po",+ "Pq",+ "Qc",+ "Ql",+ "Qo",+ "Qq",+ "Sc",+ "Sh",+ "So",+ "Sq",+ "Ss",+ "St",+ "Sx",+ "Sy",+ "Ta",+ "Tn",+ "Ux",+ "Va",+ "Vt",+ "Xc",+ "Xo",+ "Xr"]++callableMacros :: Set Text+callableMacros = fromList [+ "Ac",+ "Ad",+ "An",+ "Ao",+ "Ap",+ "Aq",+ "Ar",+ "At",+ "Bc",+ "Bo",+ "Bq",+ "Brc",+ "Bro",+ "Brq",+ "Bsx",+ "Bx",+ "Cd",+ "Cm",+ "Dc",+ "Do",+ "Dq",+ "Dv",+ "Dx",+ "Ec",+ "Em",+ "En",+ "Eo",+ "Er",+ "Es",+ "Ev",+ "Fa",+ "Fc",+ "Fl",+ "Fn",+ "Fo",+ "Fr",+ "Ft",+ "Fx",+ "Ic",+ "In",+ "Li",+ "Lk",+ "Ms",+ "Mt",+ "Nm",+ "No",+ "Ns",+ "Nx",+ "Oc",+ "Oo",+ "Op",+ "Ot",+ "Ox",+ "Pa",+ "Pc",+ "Pf",+ "Po",+ "Pq",+ "Qc",+ "Ql",+ "Qo",+ "Qq",+ "Sc",+ "So",+ "Sq",+ "St",+ "Sx",+ "Sy",+ "Ta",+ "Tn",+ "Ux",+ "Va",+ "Vt",+ "Xc",+ "Xo",+ "Xr"]
+ src/Text/Pandoc/Readers/Mdoc/Standards.hs view
@@ -0,0 +1,69 @@+{-# LANGUAGE OverloadedStrings #-}+{- |+ Module : Text.Pandoc.Readers.Mdoc.Standards+ Copyright : © 2024 Evan Silberman+ License : GNU GPL, version 2 or above++ Maintainer : Evan Silberman <evan@jklol.net>+ Stability : WIP+ Portability : portable++-}+module Text.Pandoc.Readers.Mdoc.Standards (standard) where++import Data.Map (fromList, Map)+import qualified Data.Map as M+import Data.Text++standard :: Text -> Maybe Text+standard = flip M.lookup standards++standards :: Map Text Text+standards = fromList [+ ("-p1003.1-88", "IEEE Std 1003.1-1988 (“POSIX.1”)"),+ ("-p1003.1-90", "IEEE Std 1003.1-1990 (“POSIX.1”)"),+ ("-p1003.1-96", "ISO/IEC 9945-1:1996 (“POSIX.1”)"),+ ("-p1003.1-2001", "IEEE Std 1003.1-2001 (“POSIX.1”)"),+ ("-p1003.1-2004", "IEEE Std 1003.1-2004 (“POSIX.1”)"),+ ("-p1003.1-2008", "IEEE Std 1003.1-2008 (“POSIX.1”)"),+ ("-p1003.1-2024", "IEEE Std 1003.1-2024 (“POSIX.1”)"),+ ("-p1003.1", "IEEE Std 1003.1 (“POSIX.1”)"),+ ("-p1003.1b", "IEEE Std 1003.1b (“POSIX.1b”)"),+ ("-p1003.1b-93", "IEEE Std 1003.1b-1993 (“POSIX.1b”)"),+ ("-p1003.1c-95", "IEEE Std 1003.1c-1995 (“POSIX.1c”)"),+ ("-p1003.1g-2000", "IEEE Std 1003.1g-2000 (“POSIX.1g”)"),+ ("-p1003.1i-95", "IEEE Std 1003.1i-1995 (“POSIX.1i”)"),+ ("-p1003.2", "IEEE Std 1003.2 (“POSIX.2”)"),+ ("-p1003.2-92", "IEEE Std 1003.2-1992 (“POSIX.2”)"),+ ("-p1003.2a-92", "IEEE Std 1003.2a-1992 (“POSIX.2”)"),+ ("-isoC", "ISO/IEC 9899:1990 (“ISO C90”)"),+ ("-isoC-90", "ISO/IEC 9899:1990 (“ISO C90”)"),+ ("-isoC-amd1", "ISO/IEC 9899/AMD1:1995 (“ISO C90, Amendment 1”)"),+ ("-isoC-tcor1", "ISO/IEC 9899/TCOR1:1994 (“ISO C90, Technical Corrigendum 1”)"),+ ("-isoC-tcor2", "ISO/IEC 9899/TCOR2:1995 (“ISO C90, Technical Corrigendum 2”)"),+ ("-isoC-99", "ISO/IEC 9899:1999 (“ISO C99”)"),+ ("-isoC-2011", "ISO/IEC 9899:2011 (“ISO C11”)"),+ ("-iso9945-1-90", "ISO/IEC 9945-1:1990 (“POSIX.1”)"),+ ("-iso9945-1-96", "ISO/IEC 9945-1:1996 (“POSIX.1”)"),+ ("-iso9945-2-93", "ISO/IEC 9945-2:1993 (“POSIX.2”)"),+ ("-ansiC", "ANSI X3.159-1989 (“ANSI C89”)"),+ ("-ansiC-89", "ANSI X3.159-1989 (“ANSI C89”)"),+ ("-ieee754", "IEEE Std 754-1985"),+ ("-iso8802-3", "ISO 8802-3: 1989"),+ ("-iso8601", "ISO 8601"),+ ("-ieee1275-94", "IEEE Std 1275-1994 (“Open Firmware”)"),+ ("-xpg3", "X/Open Portability Guide Issue 3 (“XPG3”)"),+ ("-xpg4", "X/Open Portability Guide Issue 4 (“XPG4”)"),+ ("-xpg4.2", "X/Open Portability Guide Issue 4, Version 2 (“XPG4.2”)"),+ ("-xbd5", "X/Open Base Definitions Issue 5 (“XBD5”)"),+ ("-xcu5", "X/Open Commands and Utilities Issue 5 (“XCU5”)"),+ ("-xsh5", "X/Open System Interfaces and Headers Issue 5 (“XSH5”)"),+ ("-xns5", "X/Open Networking Services Issue 5 (“XNS5”)"),+ ("-xns5.2", "X/Open Networking Services Issue 5.2 (“XNS5.2”)"),+ ("-xcurses4.2", "X/Open Curses Issue 4, Version 2 (“XCURSES4.2”)"),+ ("-susv1", "Version 1 of the Single UNIX Specification (“SUSv1”)"),+ ("-susv2", "Version 2 of the Single UNIX Specification (“SUSv2”)"),+ ("-susv3", "Version 3 of the Single UNIX Specification (“SUSv3”)"),+ ("-susv4", "Version 4 of the Single UNIX Specification (“SUSv4”)"),+ ("-svid4", "System V Interface Definition, Fourth Edition (“SVID4”)")+ ]
src/Text/Pandoc/Readers/MediaWiki.hs view
@@ -209,10 +209,11 @@ Just w -> fromMaybe 1.0 $ parseWidth w Nothing -> 1.0 caption <- option mempty tableCaption+ optional newline optional rowsep hasheader <- option False $ True <$ lookAhead (skipSpaces *> char '!')- (cellspecs',hdr) <- unzip <$> tableRow- let widths = map (tableWidth *) cellspecs'+ (cellwidths,hdr) <- unzip <$> tableRow+ let widths = map (tableWidth *) (concat cellwidths) let restwidth = tableWidth - sum widths let zerocols = length $ filter (==0.0) widths let defaultwidth = if zerocols == 0 || zerocols == length widths@@ -235,10 +236,10 @@ (TableFoot nullAttr []) calculateAlignments :: [Cell] -> [Alignment]-calculateAlignments = map cellAligns+calculateAlignments = concatMap cellAligns where- cellAligns :: Cell -> Alignment- cellAligns (Cell _ align _ _ _) = align+ cellAligns :: Cell -> [Alignment]+ cellAligns (Cell _ align _ (ColSpan colspan) _) = replicate colspan align parseAttrs :: PandocMonad m => MWParser m [(Text,Text)] parseAttrs = many1 parseAttr@@ -269,22 +270,16 @@ <* skipMany htmlComment <* blanklines -cellsep :: PandocMonad m => MWParser m ()+cellsep :: PandocMonad m => MWParser m [(Text,Text)] cellsep = try $ do col <- sourceColumn <$> getPosition- skipSpaces- let pipeSep = do- char '|'- notFollowedBy (oneOf "-}+")- if col == 1- then optional (char '|')- else void (char '|')- let exclSep = do- char '!'- if col == 1- then optional (char '!')- else void (char '!')- pipeSep <|> exclSep+ skipMany spaceChar+ c <- oneOf "|!"+ when (col > 1) $ void $ char c+ notFollowedBy (oneOf "-}+")+ attribs <- option [] (parseAttrs <* skipMany spaceChar <* char '|')+ skipMany spaceChar+ pure attribs tableCaption :: PandocMonad m => MWParser m Inlines tableCaption = try $ do@@ -294,20 +289,17 @@ sym "|+" optional (try $ parseAttrs *> skipSpaces *> char '|' *> blanklines) trimInlines . mconcat <$>- many (notFollowedBy (cellsep <|> rowsep) *> inline)+ many (notFollowedBy (void cellsep <|> rowsep) *> inline) -tableRow :: PandocMonad m => MWParser m [(Double, Cell)]+tableRow :: PandocMonad m => MWParser m [([Double], Cell)] tableRow = try $ skipMany htmlComment *> many tableCell -tableCell :: PandocMonad m => MWParser m (Double, Cell)+-- multiple widths because cell might have colspan+tableCell :: PandocMonad m => MWParser m ([Double], Cell) tableCell = try $ do- cellsep- skipMany spaceChar- attribs <- option [] $ try $ parseAttrs <* skipSpaces <* char '|' <*- notFollowedBy (char '|')- skipMany spaceChar+ attribs <- cellsep pos' <- getPosition- ls <- T.concat <$> many (notFollowedBy (cellsep <|> rowsep <|> tableEnd) *>+ ls <- T.concat <$> many (notFollowedBy (void cellsep <|> rowsep <|> tableEnd) *> ((snd <$> withRaw table) <|> countChar 1 anyChar)) bs <- parseFromString (do setPosition pos' mconcat <$> many block) ls@@ -316,18 +308,22 @@ Just "right" -> AlignRight Just "center" -> AlignCenter _ -> AlignDefault- let width = case lookup "width" attribs of- Just xs -> fromMaybe 0.0 $ parseWidth xs- Nothing -> 0.0 let rowspan = RowSpan . fromMaybe 1 $ safeRead =<< lookup "rowspan" attribs let colspan = ColSpan . fromMaybe 1 $ safeRead =<< lookup "colspan" attribs+ let ColSpan rawcolspan = colspan let handledAttribs = ["align", "colspan", "rowspan"] attribs' = [ (k, v) | (k, v) <- attribs , k `notElem` handledAttribs ]- return (width, B.cellWith (toAttr attribs') align rowspan colspan bs)+ let widths = case lookup "width" attribs of+ Just xs -> maybe (replicate rawcolspan 0.0)+ (\w -> replicate rawcolspan+ (w / fromIntegral rawcolspan))+ (parseWidth xs)+ Nothing -> replicate rawcolspan 0.0+ return (widths, B.cellWith (toAttr attribs') align rowspan colspan bs) parseWidth :: Text -> Maybe Double parseWidth s =
src/Text/Pandoc/Readers/Metadata.hs view
@@ -24,25 +24,38 @@ import Data.Text (Text) import qualified Data.Text as T import qualified Data.Yaml as Yaml-import Data.Aeson (Value(..), Object, Result(..), fromJSON, (.:?), withObject)-import Data.Aeson.Types (parse)+import qualified Data.Yaml.Internal as Yaml+import qualified Text.Libyaml as Y+import Data.Aeson (Value(..), Object, Result(..), fromJSON, (.:?), withObject,+ FromJSON)+import Data.Aeson.Types (formatRelativePath, parse) import Text.Pandoc.Shared (tshow, blocksToInlines)-import Text.Pandoc.Class.PandocMonad (PandocMonad (..))+import Text.Pandoc.Class (PandocMonad (..), report) import Text.Pandoc.Definition import Text.Pandoc.Error+import Text.Pandoc.Logging (LogMessage(YamlWarning)) import Text.Pandoc.Parsing hiding (tableWith, parse) import qualified Text.Pandoc.UTF8 as UTF8+import System.IO.Unsafe (unsafePerformIO) yamlBsToMeta :: (PandocMonad m, HasLastStrPosition st) => ParsecT Sources st m (Future st MetaValue) -> B.ByteString -> ParsecT Sources st m (Future st Meta) yamlBsToMeta pMetaValue bstr = do- case Yaml.decodeAllEither' bstr of- Right (Object o:_) -> fmap Meta <$> yamlMap pMetaValue o- Right [] -> return . return $ mempty- Right [Null] -> return . return $ mempty- Right _ -> Prelude.fail "expected YAML object"+ case decodeAllWithWarnings bstr of+ Right (warnings, xs) -> do+ pos <- getPosition+ mapM_ (\w -> case w of+ Yaml.DuplicateKey jpath ->+ report (YamlWarning pos $ "Duplicate key: " <>+ T.pack (formatRelativePath jpath)))+ warnings+ case xs of+ (Object o : _) -> fmap Meta <$> yamlMap pMetaValue o+ [Null] -> return . return $ mempty+ [] -> return . return $ mempty+ _ -> Prelude.fail "expected YAML object" Left err' -> do let msg = T.pack $ Yaml.prettyPrintParseException err' throwError $ PandocParseError $@@ -51,6 +64,17 @@ "\nConsider enclosing the entire field in 'single quotes'" else msg +decodeAllWithWarnings :: FromJSON a+ => B.ByteString+ -> (Either Yaml.ParseException ([Yaml.Warning], [a]))+decodeAllWithWarnings = either Left (\(ws,res)+ -> case res of+ Left s -> Left (Yaml.AesonException s)+ Right v -> Right (ws, v))+ . unsafePerformIO+ . Yaml.decodeAllHelper+ . Y.decode+ -- Returns filtered list of references. yamlBsToRefs :: (PandocMonad m, HasLastStrPosition st) => ParsecT Sources st m (Future st MetaValue)@@ -142,6 +166,7 @@ => ParsecT Sources st m (Future st MetaValue) -> ParsecT Sources st m (Future st Meta) yamlMetaBlock parser = try $ do+ pos <- getPosition string "---" blankline notFollowedBy blankline -- if --- is followed by a blank it's an HRULE@@ -149,7 +174,11 @@ -- by including --- and ..., we allow yaml blocks with just comments: let rawYaml = T.unlines ("---" : (rawYamlLines ++ ["..."])) optional blanklines- yamlBsToMeta parser $ UTF8.fromText rawYaml+ oldPos <- getPosition+ setPosition pos+ res <- yamlBsToMeta parser $ UTF8.fromText rawYaml+ setPosition oldPos+ pure res stopLine :: Monad m => ParsecT Sources st m () stopLine = try $ (string "---" <|> string "...") >> blankline >> return ()
src/Text/Pandoc/Readers/RST.hs view
@@ -30,7 +30,7 @@ import Text.Printf (printf) import Text.Pandoc.Builder (Blocks, Inlines, fromList, setMeta, trimInlines) import qualified Text.Pandoc.Builder as B-import Text.Pandoc.Class.PandocMonad (PandocMonad, fetchItem, getTimestamp)+import Text.Pandoc.Class (PandocMonad, readFileFromDirs, fetchItem, getTimestamp) import Text.Pandoc.CSV (CSVOptions (..), defaultCSVOptions, parseCSV) import Text.Pandoc.Definition import Text.Pandoc.Error@@ -40,6 +40,7 @@ import Text.Pandoc.Parsing import Text.Pandoc.Shared import Text.Pandoc.URI+import Text.Pandoc.Walk (walkM) import qualified Text.Pandoc.UTF8 as UTF8 import Data.Time.Format import System.FilePath (takeDirectory)@@ -116,18 +117,16 @@ metaFromDefList :: [([Inline], [[Block]])] -> Meta -> Meta metaFromDefList ds meta = adjustAuthors $ foldr f meta ds- where f (k,v) = setMeta (T.toLower $ stringify k) (mconcat $ map fromList v)+ where f (k,v) =+ case v of+ [[Plain ils]] -> setMeta (T.toLower (stringify k)) $ MetaInlines ils+ _ -> setMeta (T.toLower (stringify k)) $ mconcat $ map fromList v adjustAuthors (Meta metamap) = Meta $ M.adjust splitAuthors "author"- $ M.adjust toPlain "date"- $ M.adjust toPlain "title" $ M.mapKeys (\k -> if k == "authors" then "author" else k) metamap- toPlain (MetaBlocks [Para xs]) = MetaInlines xs- toPlain x = x- splitAuthors (MetaBlocks [Para xs])- = MetaList $ map MetaInlines+ splitAuthors (MetaInlines xs) = MetaList $ map MetaInlines $ splitAuthors' xs splitAuthors x = x splitAuthors' = map normalizeSpaces .@@ -150,44 +149,99 @@ parseRST :: PandocMonad m => RSTParser m Pandoc parseRST = do+ standalone <- getOption readerStandalone optional blanklines -- skip blank lines at beginning of file- startPos <- getPosition- -- go through once just to get list of reference keys and notes- -- docMinusKeys is the raw document with blanks where the keys were...- let chunk = referenceKey- <|> anchorDef- <|> noteBlock- <|> citationBlock- <|> (snd <$> withRaw comment)- <|> headerBlock- <|> lineClump- docMinusKeys <- Sources <$>- manyTill (do pos <- getPosition- t <- chunk- return (pos, t)) eof- -- UGLY: we collapse source position information.- -- TODO: fix the parser to use the F monad instead of two passes- setInput docMinusKeys- setPosition startPos- st' <- getState- let reversedNotes = stateNotes st'- updateState $ \s -> s { stateNotes = reverse reversedNotes- , stateIdentifiers = mempty }- -- now parse it for real... blocks <- B.toList <$> parseBlocks citations <- sort . M.toList . stateCitations <$> getState citationItems <- mapM parseCitation citations let refBlock = [Div ("citations",[],[]) $ B.toList $ B.definitionList citationItems | not (null citationItems)]- standalone <- getOption readerStandalone state <- getState let meta = stateMeta state let (blocks', meta') = if standalone then titleTransform (blocks, meta) else (blocks, meta)+ let reversedNotes = stateNotes state+ updateState $ \s -> s { stateNotes = reverse reversedNotes }+ doc <- walkM resolveReferences =<<+ walkM resolveBlockSubstitutions+ (Pandoc meta' (blocks' ++ refBlock)) reportLogMessages- return $ Pandoc meta' (blocks' ++ refBlock)+ return doc +resolveBlockSubstitutions :: PandocMonad m => Block -> RSTParser m Block+resolveBlockSubstitutions x@(Para [Link _attr _ (s,_)])+ | Just ref <- T.stripPrefix "##SUBST##" s = do+ substTable <- stateSubstitutions <$> getState+ let key@(Key key') = toKey $ stripFirstAndLast ref+ case M.lookup key substTable of+ Nothing -> do+ pos <- getPosition+ logMessage $ ReferenceNotFound (tshow key') pos+ return x+ Just target -> case+ B.toList target of+ [bl] -> return bl+ bls -> return $ Div nullAttr bls+resolveBlockSubstitutions x = return x++resolveReferences :: PandocMonad m => Inline -> RSTParser m Inline+resolveReferences x@(Link _ ils (s,_))+ | Just ref <- T.stripPrefix "##REF##" s = do+ let isAnonKey (Key (T.uncons -> Just ('_',_))) = True+ isAnonKey _ = False+ state <- getState+ let keyTable = stateKeys state+ let anonKeys = sort $ filter isAnonKey $ M.keys keyTable+ key <- if ref == "_" -- anonymous key+ then+ case anonKeys of+ [] -> mzero -- TODO log?+ (k:_) -> return k+ else return $ toKey ref+ ((src,tit), attr) <- lookupKey [] key+ -- if anonymous link, remove key so it won't be used again+ when (isAnonKey key) $ updateState $ \st ->+ st{ stateKeys = M.delete key keyTable }+ return $ Link attr ils (src, tit)+ | Just ref <- T.stripPrefix "##NOTE##" s = do+ state <- getState+ let notes = stateNotes state+ case lookup ref notes of+ Nothing -> do+ pos <- getPosition+ logMessage $ ReferenceNotFound ref pos+ return x+ Just raw -> do+ -- We temporarily empty the note list while parsing the note,+ -- so that we don't get infinite loops with notes inside notes...+ -- Note references inside other notes are allowed in reST, but+ -- not yet in this implementation.+ updateState $ \st -> st{ stateNotes = [] }+ contents <- parseFromString' parseBlocks raw+ let newnotes = if ref == "*" || ref == "#" -- auto-numbered+ -- delete the note so the next auto-numbered note+ -- doesn't get the same contents:+ then deleteFirstsBy (==) notes [(ref,raw)]+ else notes+ updateState $ \st -> st{ stateNotes = newnotes }+ return $ Note (B.toList contents)+ | Just ref <- T.stripPrefix "##SUBST##" s = do+ substTable <- stateSubstitutions <$> getState+ let key@(Key key') = toKey $ stripFirstAndLast ref+ case M.lookup key substTable of+ Nothing -> do+ pos <- getPosition+ logMessage $ ReferenceNotFound (tshow key') pos+ return x+ Just target -> case+ B.toList target of+ [Para [t]] -> return t+ [Para xs] -> return $ Span nullAttr xs+ bls -> return $ Span nullAttr $ blocksToInlines bls+ | otherwise = return x+resolveReferences x = return x+ parseCitation :: PandocMonad m => (Text, Text) -> RSTParser m (Inlines, [Blocks]) parseCitation (ref, raw) = do@@ -207,6 +261,10 @@ block = choice [ codeBlock , blockQuote , fieldList+ , optionList+ , referenceKey+ , noteBlock+ , citationBlock , directive , anchor , comment@@ -244,7 +302,10 @@ term <- parseInlineFromText name contents <- parseFromString' parseBlocks raw optional blanklines- return (term, [contents])+ let defn = case B.toList contents of+ [Para ils] -> [B.plain $ B.fromList ils] -- see #7766+ _ -> [contents]+ return (term, defn) fieldList :: PandocMonad m => RSTParser m Blocks fieldList = try $ do@@ -254,6 +315,51 @@ [] -> return mempty items' -> return $ B.definitionList items' +optionList :: PandocMonad m => RSTParser m Blocks+optionList = B.definitionList <$> many1 optionListItem++optionListItem :: PandocMonad m => RSTParser m (Inlines, [Blocks])+optionListItem = try $ do+ opts <- snd <$> withRaw (do+ let anyOpt = shortOpt <|> longOpt <|> dosOpt+ anyOpt+ many $ try (char ',' <* many spaceChar *> anyOpt))+ -- at least two spaces+ rawfirst <- try (char ' ' *> many1 (char ' ') *> anyLineNewline)+ <|> try (mempty <$ skipMany spaceChar <* newline)+ bodyElements <- do+ raw <- option "" indentedBlock+ parseFromString' parseBlocks $ (rawfirst <> raw) <> "\n\n"+ optional blanklines+ pure (B.code opts, [bodyElements])++shortOpt :: PandocMonad m => RSTParser m ()+shortOpt = try $ do+ char '-'+ alphaNum+ optional $ try (optional (char ' ') *> optArg)++optArg :: PandocMonad m => RSTParser m ()+optArg = do+ c <- letter <|> char '<'+ if c == '<'+ then () <$ manyTill (noneOf "<>") (char '>')+ else skipMany (alphaNum <|> char '_' <|> char '-')++longOpt :: PandocMonad m => RSTParser m ()+longOpt = try $ do+ char '-'+ char '-'+ alphaNum+ skipMany1 (alphaNum <|> char '-' <|> char '_')+ optional $ try (oneOf " =" *> optArg)++dosOpt :: PandocMonad m => RSTParser m ()+dosOpt = try $ do+ char '/'+ alphaNum <|> char '?'+ optional $ try (char ' ' *> optArg)+ -- -- line block --@@ -309,7 +415,10 @@ Just ind -> (headerTable, ind + 1) Nothing -> (headerTable ++ [DoubleHeader c], length headerTable + 1) setState (state { stateHeaderTable = headerTable' })- attr <- registerHeader nullAttr txt+ attr@(ident,_,_) <- registerHeader nullAttr txt+ let key = toKey (stringify txt)+ updateState $ \s ->+ s { stateKeys = M.insert key (("#" <> ident,""), nullAttr) $ stateKeys s } return $ B.headerWith attr level txt doubleHeader' :: PandocMonad m => RSTParser m (Inlines, Char)@@ -338,7 +447,10 @@ Just ind -> (headerTable, ind + 1) Nothing -> (headerTable ++ [SingleHeader c], length headerTable + 1) setState (state { stateHeaderTable = headerTable' })- attr <- registerHeader nullAttr txt+ attr@(ident,_,_) <- registerHeader nullAttr txt+ let key = toKey (stringify txt)+ updateState $ \s ->+ s { stateKeys = M.insert key (("#" <> ident,""), nullAttr) $ stateKeys s } return $ B.headerWith attr level txt singleHeader' :: PandocMonad m => RSTParser m (Inlines, Char)@@ -630,7 +742,12 @@ if fieldIndent == 0 then return [] else many $ rawFieldListItem fieldIndent- body <- option "" $ try $ blanklines >> indentedBlock+ let mbfile = trim <$> lookup "file" fields+ body <- case mbfile of+ Just f | label == "raw" -> do+ currentDir <- takeDirectory . sourceName <$> getPosition+ fromMaybe mempty <$> readFileFromDirs [currentDir] (T.unpack f)+ _ -> option "" $ try $ blanklines >> indentedBlock optional blanklines let body' = body <> "\n\n" name = trim $ fromMaybe "" (lookup "name" fields)@@ -1059,27 +1176,24 @@ --- note block --- -noteBlock :: Monad m => RSTParser m Text+noteBlock :: Monad m => RSTParser m Blocks noteBlock = try $ do- (ref, raw, replacement) <- noteBlock' noteMarker+ (ref, raw) <- noteBlock' noteMarker updateState $ \s -> s { stateNotes = (ref, raw) : stateNotes s }- -- return blanks so line count isn't affected- return replacement+ return mempty -citationBlock :: Monad m => RSTParser m Text+citationBlock :: Monad m => RSTParser m Blocks citationBlock = try $ do- (ref, raw, replacement) <- noteBlock' citationMarker+ (ref, raw) <- noteBlock' citationMarker updateState $ \s -> s { stateCitations = M.insert ref raw (stateCitations s), stateKeys = M.insert (toKey ref) (("#" <> ref,""), ("",["citation"],[])) (stateKeys s) }- -- return blanks so line count isn't affected- return replacement+ return mempty noteBlock' :: Monad m- => RSTParser m Text -> RSTParser m (Text, Text, Text)+ => RSTParser m Text -> RSTParser m (Text, Text) noteBlock' marker = try $ do- startPos <- getPosition string ".." spaceChar >> skipMany spaceChar ref <- marker@@ -1087,10 +1201,8 @@ <|> (newline >> return "") blanks <- option "" blanklines rest <- option "" indentedBlock- endPos <- getPosition let raw = first <> "\n" <> blanks <> rest <> "\n"- let replacement = T.replicate (sourceLine endPos - sourceLine startPos) "\n"- return (ref, raw, replacement)+ return (ref, raw) citationMarker :: Monad m => RSTParser m Text citationMarker = do@@ -1132,14 +1244,11 @@ referenceName :: PandocMonad m => RSTParser m Text referenceName = quotedReferenceName <|> simpleReferenceName -referenceKey :: PandocMonad m => RSTParser m Text+referenceKey :: PandocMonad m => RSTParser m Blocks referenceKey = do- startPos <- getPosition choice [substKey, anonymousKey, regularKey] optional blanklines- endPos <- getPosition- -- return enough blanks to replace key- return $ T.replicate (sourceLine endPos - sourceLine startPos) "\n"+ return mempty targetURI :: Monad m => ParsecT Sources st m Text targetURI = do@@ -1163,17 +1272,16 @@ (alt,ref) <- withRaw $ trimInlines . mconcat <$> enclosed (char '|') (char '|') inline res <- B.toList <$> directive'- il <- case res of+ bls <- case res of -- use alt unless :alt: attribute on image: [Para [Image attr [Str "image"] (src,tit)]] ->- return $ B.imageWith attr src tit alt+ return $ B.para $ B.imageWith attr src tit alt [Para [Link _ [Image attr [Str "image"] (src,tit)] (src',tit')]] ->- return $ B.link src' tit' (B.imageWith attr src tit alt)- [Para ils] -> return $ B.fromList ils- _ -> mzero+ return $ B.para $ B.link src' tit' (B.imageWith attr src tit alt)+ _ -> return $ B.fromList res let key = toKey $ stripFirstAndLast ref updateState $ \s -> s{ stateSubstitutions =- M.insert key il $ stateSubstitutions s }+ M.insert key bls $ stateSubstitutions s } anonymousKey :: PandocMonad m => RSTParser m () anonymousKey = try $ do@@ -1217,19 +1325,14 @@ updateState $ \s -> s { stateKeys = M.insert key ((src,""), nullAttr) $ stateKeys s } -anchorDef :: PandocMonad m => RSTParser m Text-anchorDef = try $ do- (refs, raw) <- withRaw $ try (referenceNames <* blanklines)- forM_ refs $ \rawkey ->- updateState $ \s -> s { stateKeys =- M.insert (toKey rawkey) (("#" <> rawkey,""), nullAttr) $ stateKeys s }- -- keep this for 2nd round of parsing, where we'll add the divs (anchor)- return raw- anchor :: PandocMonad m => RSTParser m Blocks anchor = try $ do refs <- referenceNames blanklines+ forM_ refs $ \rawkey ->+ updateState $ \s -> s { stateKeys =+ M.insert (toKey rawkey) (("#" <> rawkey,""), nullAttr)+ (stateKeys s) } b <- block let addDiv ref = B.divWith (ref, [], []) let emptySpanWithId id' = Span (id',[],[]) []@@ -1245,16 +1348,6 @@ -- because it hides them from promoteHeader, see #4240 _ -> return $ foldr addDiv b refs -headerBlock :: PandocMonad m => RSTParser m Text-headerBlock = do- ((txt, _), raw) <- withRaw (doubleHeader' <|> singleHeader')- (ident,_,_) <- registerHeader nullAttr txt- let key = toKey (stringify txt)- updateState $ \s -> s { stateKeys = M.insert key (("#" <> ident,""), nullAttr)- $ stateKeys s }- return raw-- -- -- tables --@@ -1371,6 +1464,7 @@ inline :: PandocMonad m => RSTParser m Inlines inline = choice [ note -- can start with whitespace, so try before ws , link+ , inlineAnchor , strong , emph , code@@ -1603,22 +1697,28 @@ char '`' notFollowedBy (char '`') -- `` marks start of inline code label' <- trimInlines . mconcat <$>- manyTill (notFollowedBy (char '`') >> inlineContent) (char '<')- src <- trim <$> manyTillChar (noneOf ">\n") (char '>')+ manyTill (notFollowedBy (char '`') >> inlineContent) (char '<')+ src <- trim . T.pack . filter (/= '\n') <$> -- see #10279+ manyTill (noneOf ">\n" <|> (char '\n' <* notFollowedBy blankline))+ (char '>') skipSpaces string "`_" optional $ char '_' -- anonymous form- let label'' = if label' == mempty- then B.str src- else label'- -- `link <google_>` is a reference link to _google! ((src',tit),attr) <- if isURI src then return ((src, ""), nullAttr) else case T.unsnoc src of+ -- `link <google_>` is a reference link to _google! Just (xs, '_') -> lookupKey [] (toKey xs) _ -> return ((src, ""), nullAttr)+ let label'' = if label' == mempty+ then B.str src+ else label'+ let key = toKey $ stringify label'+ unless (key == Key mempty) $ do+ updateState $ \s -> s{+ stateKeys = M.insert key ((src',tit), attr) $ stateKeys s } return $ B.linkWith attr (escapeURI src') tit label'' citationName :: PandocMonad m => RSTParser m Text@@ -1626,25 +1726,17 @@ raw <- citationMarker return $ "[" <> raw <> "]" +-- We store the reference link label as the link target,+-- preceded by '##REF##'. This is replaced after the AST+-- has been built by the resolved reference. referenceLink :: PandocMonad m => RSTParser m Inlines referenceLink = try $ do ref <- (referenceName <|> citationName) <* char '_'- let label' = B.text ref- let isAnonKey (Key (T.uncons -> Just ('_',_))) = True- isAnonKey _ = False- state <- getState- let keyTable = stateKeys state- key <- option (toKey ref) $- do char '_'- let anonKeys = sort $ filter isAnonKey $ M.keys keyTable- case anonKeys of- [] -> mzero- (k:_) -> return k- ((src,tit), attr) <- lookupKey [] key- -- if anonymous link, remove key so it won't be used again- when (isAnonKey key) $ updateState $ \s ->- s{ stateKeys = M.delete key keyTable }- return $ B.linkWith attr src tit label'+ isAnonymous <- (True <$ char '_') <|> pure False+ let ref' = if isAnonymous+ then "_"+ else ref+ pure $ B.linkWith nullAttr ("##REF##" <> ref') "" (B.text ref) -- We keep a list of oldkeys so we can detect lookup loops. lookupKey :: PandocMonad m@@ -1664,6 +1756,10 @@ let newkey = toKey rawkey if newkey `elem` oldkeys then do+ -- TODO the pos is not going to be accurate+ -- because we're calling this after the AST is+ -- constructed. Probably good to remove that+ -- parameter form CircularReference at some point. logMessage $ CircularReference rawkey pos return (("",""),nullAttr) else lookupKey (key:oldkeys) newkey@@ -1685,42 +1781,30 @@ subst :: PandocMonad m => RSTParser m Inlines subst = try $ do (_,ref) <- withRaw $ enclosed (char '|') (char '|') inline- state <- getState- let substTable = stateSubstitutions state- let key = toKey $ stripFirstAndLast ref- case M.lookup key substTable of- Nothing -> do- pos <- getPosition- logMessage $ ReferenceNotFound (tshow key) pos- return mempty- Just target -> return target+ let substlink = B.linkWith nullAttr ("##SUBST##" <> ref) "" (B.text ref)+ reflink <- option False (True <$ char '_')+ if reflink+ then do+ let linkref = T.drop 1 $ T.dropEnd 1 ref+ return $ B.linkWith nullAttr ("##REF##" <> linkref) "" substlink+ else return substlink note :: PandocMonad m => RSTParser m Inlines note = try $ do optional whitespace ref <- noteMarker char '_'- state <- getState- let notes = stateNotes state- case lookup ref notes of- Nothing -> do- pos <- getPosition- logMessage $ ReferenceNotFound ref pos- return mempty- Just raw -> do- -- We temporarily empty the note list while parsing the note,- -- so that we don't get infinite loops with notes inside notes...- -- Note references inside other notes are allowed in reST, but- -- not yet in this implementation.- updateState $ \st -> st{ stateNotes = [] }- contents <- parseFromString' parseBlocks raw- let newnotes = if ref == "*" || ref == "#" -- auto-numbered- -- delete the note so the next auto-numbered note- -- doesn't get the same contents:- then deleteFirstsBy (==) notes [(ref,raw)]- else notes- updateState $ \st -> st{ stateNotes = newnotes }- return $ B.note contents+ pure $ B.linkWith nullAttr ("##NOTE##" <> ref) "" (B.text ref) smart :: PandocMonad m => RSTParser m Inlines smart = smartPunctuation inline++inlineAnchor :: PandocMonad m => RSTParser m Inlines+inlineAnchor = do+ char '_'+ name <- quotedReferenceName <|> simpleReferenceName+ let ident = textToIdentifier mempty name+ updateState $ \s ->+ s{ stateKeys = M.insert (toKey name) (("#" <> ident, ""), nullAttr)+ (stateKeys s) }+ pure $ B.spanWith (ident,[],[]) (B.text name)
src/Text/Pandoc/Readers/Roff.hs view
@@ -2,6 +2,7 @@ {-# LANGUAGE GeneralizedNewtypeDeriving #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE ViewPatterns #-}+{-# LANGUAGE TypeFamilies #-} {- | Module : Text.Pandoc.Readers.Roff Copyright : Copyright (C) 2018-2020 Yan Pashkovsky and John MacFarlane@@ -29,11 +30,11 @@ where import Safe (lastDef)-import Control.Monad (void, mzero, mplus, guard)+import Control.Monad (void, guard) import Control.Monad.Except (throwError) import Text.Pandoc.Class.PandocMonad (getResourcePath, readFileFromDirs, PandocMonad(..), report)-import Data.Char (isLower, toLower, toUpper, chr, isAscii, isAlphaNum)+import Data.Char (isLower, toLower, toUpper, isAlphaNum) import Data.Default (Default) import qualified Data.Map as M import Data.List (intercalate)@@ -42,10 +43,9 @@ import Text.Pandoc.Options import Text.Pandoc.Parsing import Text.Pandoc.Shared (safeRead)-import Text.Pandoc.RoffChar (characterCodes, combiningAccents)+import Text.Pandoc.Readers.Roff.Escape import qualified Data.Sequence as Seq import qualified Data.Foldable as Foldable-import qualified Data.Text.Normalize as Normalize -- import Debug.Trace (traceShowId) @@ -65,6 +65,59 @@ | MacroArg Int deriving Show +instance RoffLikeLexer RoffTokens where+ -- The token stream is a list of 'LinePart's+ type Token RoffTokens = [LinePart]+ type State RoffTokens = RoffState+ emit t = [RoffStr t]+ expandString = try $ do+ pos <- getPosition+ char '\\'+ char '*'+ cs <- escapeArg <|> countChar 1 anyChar+ s <- linePartsToText <$> resolveText cs pos+ addToInput s+ escString = try $ do+ pos <- getPosition+ (do cs <- escapeArg <|> countChar 1 anyChar+ resolveText cs pos)+ <|> mempty <$ char 'S'+ backslash = do+ char '\\'+ mode <- roffMode <$> getState+ case mode of+ -- experimentally, it seems you don't always need to double+ -- the backslash in macro defs. It's essential with \\$1,+ -- but not with \\f[I]. So we make the second one optional.+ CopyMode -> optional $ char '\\'+ NormalMode -> return ()+ checkDefined name = do+ macros <- customMacros <$> getState+ case M.lookup name macros of+ Just _ -> return [RoffStr "1"]+ Nothing -> return [RoffStr "0"]+ -- \E is ignored in copy mode+ escE = do+ mode <- roffMode <$> getState+ case mode of+ CopyMode -> return mempty+ NormalMode -> return [RoffStr "\\"]+ escFont = do+ font <- escapeArg <|> countChar 1 alphaNum+ font' <- if T.null font || font == "P"+ then prevFont <$> getState+ else return $ foldr processFontLetter defaultFontSpec $ T.unpack font+ updateState $ \st -> st{ prevFont = currentFont st+ , currentFont = font' }+ return [Font font']+ where+ processFontLetter c fs+ | isLower c = processFontLetter (toUpper c) fs+ processFontLetter 'B' fs = fs{ fontBold = True }+ processFontLetter 'I' fs = fs{ fontItalic = True }+ processFontLetter 'C' fs = fs{ fontMonospace = True }+ processFontLetter _ fs = fs -- do nothing+ type Arg = [LinePart] type TableOption = (T.Text, T.Text)@@ -133,198 +186,6 @@ spacetab :: (Stream s m Char, UpdateSourcePos s Char) => ParsecT s u m Char spacetab = char ' ' <|> char '\t' -characterCodeMap :: M.Map T.Text Char-characterCodeMap =- M.fromList $ map (\(x,y) -> (y,x)) characterCodes--combiningAccentsMap :: M.Map T.Text Char-combiningAccentsMap =- M.fromList $ map (\(x,y) -> (y,x)) combiningAccents--escape :: PandocMonad m => RoffLexer m [LinePart]-escape = try $ do- backslash- escapeGlyph <|> escapeNormal--escapeGlyph :: PandocMonad m => RoffLexer m [LinePart]-escapeGlyph = do- c <- lookAhead (oneOf ['[','('])- escapeArg >>= resolveGlyph c--resolveGlyph :: PandocMonad m => Char -> T.Text -> RoffLexer m [LinePart]-resolveGlyph delimChar glyph = do- let cs = T.replace "_u" " u" glyph -- unicode glyphs separated by _- (case T.words cs of- [] -> mzero- [s] -> case M.lookup s characterCodeMap `mplus` readUnicodeChar s of- Nothing -> mzero- Just c -> return [RoffStr $ T.singleton c]- (s:ss) -> do- basechar <- case M.lookup s characterCodeMap `mplus`- readUnicodeChar s of- Nothing ->- case T.unpack s of- [ch] | isAscii ch && isAlphaNum ch ->- return ch- _ -> mzero- Just c -> return c- let addAccents [] xs = return $ Normalize.normalize Normalize.NFC $- T.reverse xs- addAccents (a:as) xs =- case M.lookup a combiningAccentsMap `mplus` readUnicodeChar a of- Just x -> addAccents as $ T.cons x xs- Nothing -> mzero- addAccents ss (T.singleton basechar) >>= \xs -> return [RoffStr xs])- <|> case delimChar of- '[' -> escUnknown ("\\[" <> glyph <> "]")- '(' -> escUnknown ("\\(" <> glyph)- '\'' -> escUnknown ("\\C'" <> glyph <> "'")- _ -> Prelude.fail "resolveGlyph: unknown glyph delimiter"--readUnicodeChar :: T.Text -> Maybe Char-readUnicodeChar t = case T.uncons t of- Just ('u', cs) | T.length cs > 3 -> chr <$> safeRead ("0x" <> cs)- _ -> Nothing--escapeNormal :: PandocMonad m => RoffLexer m [LinePart]-escapeNormal = do- c <- noneOf "{}"- optional expandString- case c of- ' ' -> return [RoffStr " "]- '"' -> mempty <$ skipMany (satisfy (/='\n')) -- line comment- '#' -> mempty <$ manyTill anyChar newline- '%' -> return mempty -- optional hyphenation- '&' -> return mempty -- nonprintable zero-width- ')' -> return mempty -- nonprintable zero-width- '*' -> escString- ',' -> return mempty -- to fix spacing after roman- '-' -> return [RoffStr "-"]- '.' -> return [RoffStr "."]- '/' -> return mempty -- to fix spacing before roman- '0' -> return [RoffStr "\x2007"] -- digit-width space- ':' -> return mempty -- zero-width break- 'A' -> quoteArg >>= checkDefined- 'B' -> escIgnore 'B' [quoteArg]- 'C' -> quoteArg >>= resolveGlyph '\''- 'D' -> escIgnore 'D' [quoteArg]- 'E' -> do- mode <- roffMode <$> getState- case mode of- CopyMode -> return mempty- NormalMode -> return [RoffStr "\\"]- 'H' -> escIgnore 'H' [quoteArg]- 'L' -> escIgnore 'L' [quoteArg]- 'M' -> escIgnore 'M' [escapeArg, countChar 1 (satisfy (/='\n'))]- 'N' -> escIgnore 'N' [quoteArg]- 'O' -> escIgnore 'O' [countChar 1 (oneOf ['0','1'])]- 'R' -> escIgnore 'R' [quoteArg]- 'S' -> escIgnore 'S' [quoteArg]- 'V' -> escIgnore 'V' [escapeArg, countChar 1 alphaNum]- 'X' -> escIgnore 'X' [quoteArg]- 'Y' -> escIgnore 'Y' [escapeArg, countChar 1 (satisfy (/='\n'))]- 'Z' -> escIgnore 'Z' [quoteArg]- '\'' -> return [RoffStr "'"]- '\n' -> return mempty -- line continuation- '^' -> return [RoffStr "\x200A"] -- 1/12 em space- '_' -> return [RoffStr "_"]- '`' -> return [RoffStr "`"]- 'a' -> return mempty -- "non-interpreted leader character"- 'b' -> escIgnore 'b' [quoteArg]- 'c' -> return mempty -- interrupt text processing- 'd' -> escIgnore 'd' [] -- forward down 1/2em- 'e' -> return [RoffStr "\\"]- 'f' -> escFont- 'g' -> escIgnore 'g' [escapeArg, countChar 1 (satisfy (/='\n'))]- 'h' -> escIgnore 'h' [quoteArg]- 'k' -> escIgnore 'k' [escapeArg, countChar 1 (satisfy (/='\n'))]- 'l' -> escIgnore 'l' [quoteArg]- 'm' -> escIgnore 'm' [escapeArg, countChar 1 (satisfy (/='\n'))]- 'n' -> escIgnore 'm' [escapeArg, countChar 1 (satisfy (/='\n'))]- 'o' -> escIgnore 'o' [quoteArg]- 'p' -> escIgnore 'p' []- 'r' -> escIgnore 'r' []- 's' -> escIgnore 's' [escapeArg, signedNumber]- 't' -> return [RoffStr "\t"]- 'u' -> escIgnore 'u' []- 'v' -> escIgnore 'v' [quoteArg]- 'w' -> escIgnore 'w' [quoteArg]- 'x' -> escIgnore 'x' [quoteArg]- 'z' -> escIgnore 'z' [countChar 1 anyChar]- '|' -> return [RoffStr "\x2006"] --1/6 em space- '~' -> return [RoffStr "\160"] -- nonbreaking space- '\\' -> do- mode <- roffMode <$> getState- case mode of- CopyMode -> char '\\'- NormalMode -> return '\\'- return [RoffStr "\\"]- _ -> return [RoffStr $ T.singleton c]- -- man 7 groff: "If a backslash is followed by a character that- -- does not constitute a defined escape sequence, the backslash- -- is silently ignored and the character maps to itself."--escIgnore :: PandocMonad m- => Char- -> [RoffLexer m T.Text]- -> RoffLexer m [LinePart]-escIgnore c argparsers = do- pos <- getPosition- arg <- snd <$> withRaw (choice argparsers) <|> return ""- report $ SkippedContent ("\\" <> T.cons c arg) pos- return mempty--escUnknown :: PandocMonad m => T.Text -> RoffLexer m [LinePart]-escUnknown s = do- pos <- getPosition- report $ SkippedContent s pos- return [RoffStr "\xFFFD"]--signedNumber :: PandocMonad m => RoffLexer m T.Text-signedNumber = try $ do- sign <- option "" ("-" <$ char '-' <|> "" <$ char '+')- ds <- many1Char digit- return (sign <> ds)---- Parses: [..] or (..-escapeArg :: PandocMonad m => RoffLexer m T.Text-escapeArg = choice- [ char '[' *> optional expandString *>- manyTillChar (noneOf ['\n',']']) (char ']')- , char '(' *> optional expandString *>- countChar 2 (satisfy (/='\n'))- ]--expandString :: PandocMonad m => RoffLexer m ()-expandString = try $ do- pos <- getPosition- char '\\'- char '*'- cs <- escapeArg <|> countChar 1 anyChar- s <- linePartsToText <$> resolveText cs pos- addToInput s---- Parses: '..'-quoteArg :: PandocMonad m => RoffLexer m T.Text-quoteArg = char '\'' *> manyTillChar (noneOf ['\n','\'']) (char '\'')--escFont :: PandocMonad m => RoffLexer m [LinePart]-escFont = do- font <- escapeArg <|> countChar 1 alphaNum- font' <- if T.null font || font == "P"- then prevFont <$> getState- else return $ foldr processFontLetter defaultFontSpec $ T.unpack font- updateState $ \st -> st{ prevFont = currentFont st- , currentFont = font' }- return [Font font']- where- processFontLetter c fs- | isLower c = processFontLetter (toUpper c) fs- processFontLetter 'B' fs = fs{ fontBold = True }- processFontLetter 'I' fs = fs{ fontItalic = True }- processFontLetter 'C' fs = fs{ fontMonospace = True }- processFontLetter _ fs = fs -- do nothing- -- separate function from lexMacro since real man files sometimes do not -- follow the rules lexComment :: PandocMonad m => RoffLexer m RoffTokens@@ -624,20 +485,6 @@ char '"' return [RoffStr "\""] -checkDefined :: PandocMonad m => T.Text -> RoffLexer m [LinePart]-checkDefined name = do- macros <- customMacros <$> getState- case M.lookup name macros of- Just _ -> return [RoffStr "1"]- Nothing -> return [RoffStr "0"]--escString :: PandocMonad m => RoffLexer m [LinePart]-escString = try $ do- pos <- getPosition- (do cs <- escapeArg <|> countChar 1 anyChar- resolveText cs pos)- <|> mempty <$ char 'S'- -- strings and macros share namespace resolveText :: PandocMonad m => T.Text -> SourcePos -> RoffLexer m [LinePart]@@ -667,17 +514,6 @@ linePart :: PandocMonad m => RoffLexer m [LinePart] linePart = macroArg <|> escape <|> regularText <|> quoteChar <|> spaceTabChar--backslash :: PandocMonad m => RoffLexer m ()-backslash = do- char '\\'- mode <- roffMode <$> getState- case mode of- -- experimentally, it seems you don't always need to double- -- the backslash in macro defs. It's essential with \\$1,- -- but not with \\f[I]. So we make the second one optional.- CopyMode -> optional $ char '\\'- NormalMode -> return () macroArg :: PandocMonad m => RoffLexer m [LinePart] macroArg = try $ do
+ src/Text/Pandoc/Readers/Roff/Escape.hs view
@@ -0,0 +1,222 @@+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE TypeFamilyDependencies #-}+module Text.Pandoc.Readers.Roff.Escape+ ( escape,+ escapeArg,+ escIgnore,+ RoffLikeLexer(..),+ )+where+import Text.Pandoc.Class.PandocMonad+ ( PandocMonad(..), report, PandocMonad(..), report )+import Control.Monad+ ( mzero, mplus, mzero, mplus )+import Data.Char (chr, isAscii, isAlphaNum)+import qualified Data.Map as M+import qualified Data.Text as T+import Text.Pandoc.Logging (LogMessage(..))+import Text.Pandoc.Parsing+import Text.Pandoc.Shared (safeRead)+import qualified Data.Text.Normalize as Normalize+import Text.Pandoc.RoffChar (characterCodes, combiningAccents)++-- | Functions and typeclass for escaping special characters in languages+-- that inherit the Roff syntax.+--+-- For various reasons, the mdoc reader doesn't directly reuse the previously+-- existing roff lexer. The main one is to make it possible to simultaneously+-- process roff escapes and to tokenize mdoc macros correctly based on the+-- control line contents. The extracted interface here allows the same `escape`+-- function to work with lexers that target different token types and support+-- different subsets of the original roff language.++type Lexer m x = ParsecT Sources (State x) m++-- | Lexers for Roff macro+class (Monoid (Token x)) => RoffLikeLexer x where+ -- | Type family for the lexer state+ type State x = a | a -> x+ -- | Type family for the token type being lexed+ type Token x = a | a -> x+ -- | Turn a `T.Text` into a token of the output+ emit :: T.Text -> Token x+ -- | Attempt to parse a roff predefined string sequence and push its expansion+ -- onto the input stream.+ expandString :: PandocMonad m => Lexer m x ()+ -- | Parse the name of a defined string and return its expansion as a `Token`+ escString :: PandocMonad m => Lexer m x (Token x)+ -- | Parse the escape character+ backslash :: PandocMonad m => Lexer m x ()+ -- | If the given custom macro is defined in this document, emit a+ -- tokenized "1", otherwise emit a tokenized "0", implementing the roff+ -- escape @\A@.+ checkDefined :: PandocMonad m => T.Text -> Lexer m x (Token x)+ -- | Output an appropriate token for the @\E@ escape sequence. In roff, @\E@+ -- is an "escape character intended to not be interpreted in copy mode".+ -- If the lexer being defined doesn't implement copy mode, @\E@ can just be+ -- lexed by 'backslash'+ escE :: PandocMonad m => Lexer m x (Token x)+ -- | Lex the low-level roff font selection escape @\f@.+ escFont :: PandocMonad m => Lexer m x (Token x)+++characterCodeMap :: M.Map T.Text Char+characterCodeMap =+ M.fromList $ map (\(x,y) -> (y,x)) characterCodes++combiningAccentsMap :: M.Map T.Text Char+combiningAccentsMap =+ M.fromList $ map (\(x,y) -> (y,x)) combiningAccents++escape :: (PandocMonad m, RoffLikeLexer x) => Lexer m x (Token x)+escape = try $ do+ backslash+ escapeGlyph <|> escapeNormal++escapeGlyph :: (PandocMonad m, RoffLikeLexer x) => Lexer m x (Token x)+escapeGlyph = do+ c <- lookAhead (oneOf ['[','('])+ escapeArg >>= resolveGlyph c++resolveGlyph :: (PandocMonad m, RoffLikeLexer x) => Char -> T.Text -> Lexer m x (Token x)+resolveGlyph delimChar glyph = do+ let cs = T.replace "_u" " u" glyph -- unicode glyphs separated by _+ (case T.words cs of+ [] -> mzero+ [s] -> case M.lookup s characterCodeMap `mplus` readUnicodeChar s of+ Nothing -> mzero+ Just c -> return $ emit $ T.singleton c+ (s:ss) -> do+ basechar <- case M.lookup s characterCodeMap `mplus`+ readUnicodeChar s of+ Nothing ->+ case T.unpack s of+ [ch] | isAscii ch && isAlphaNum ch ->+ return ch+ _ -> mzero+ Just c -> return c+ let addAccents [] xs = return $ Normalize.normalize Normalize.NFC $+ T.reverse xs+ addAccents (a:as) xs =+ case M.lookup a combiningAccentsMap `mplus` readUnicodeChar a of+ Just x -> addAccents as $ T.cons x xs+ Nothing -> mzero+ addAccents ss (T.singleton basechar) >>= \xs -> return $ emit xs)+ <|> case delimChar of+ '[' -> escUnknown ("\\[" <> glyph <> "]")+ '(' -> escUnknown ("\\(" <> glyph)+ '\'' -> escUnknown ("\\C'" <> glyph <> "'")+ _ -> Prelude.fail "resolveGlyph: unknown glyph delimiter"++readUnicodeChar :: T.Text -> Maybe Char+readUnicodeChar t = case T.uncons t of+ Just ('u', cs) | T.length cs > 3 -> chr <$> safeRead ("0x" <> cs)+ _ -> Nothing++escapeNormal :: (PandocMonad m, RoffLikeLexer x) => Lexer m x (Token x)+escapeNormal = do+ c <- noneOf "{}"+ optional expandString+ let groffSkip = [escapeArg, countChar 1 (satisfy (/='\n'))]+ case c of+ ' ' -> return $ emit " " -- mandoc_char(7) says this should be a nonbreaking space+ '"' -> mempty <$ skipMany (satisfy (/='\n')) -- line comment+ '#' -> mempty <$ manyTill anyChar newline+ '%' -> return mempty -- optional hyphenation+ '&' -> return mempty -- nonprintable zero-width+ ')' -> return mempty -- nonprintable zero-width+ '*' -> escString+ ',' -> return mempty -- to fix spacing after roman+ '-' -> return $ emit "-"+ '.' -> return $ emit "."+ '/' -> return mempty -- to fix spacing before roman+ '0' -> return $ emit "\x2007" -- digit-width space+ ':' -> return mempty -- zero-width break+ 'A' -> quoteArg >>= checkDefined+ 'B' -> escIgnore 'B' [quoteArg]+ 'C' -> quoteArg >>= resolveGlyph '\''+ 'D' -> escIgnore 'D' [quoteArg]+ 'E' -> escE+ 'F' -> escIgnore 'F' groffSkip+ 'H' -> escIgnore 'H' [quoteArg]+ 'L' -> escIgnore 'L' [quoteArg]+ 'M' -> escIgnore 'M' groffSkip+ 'N' -> escIgnore 'N' [quoteArg]+ 'O' -> escIgnore 'O' [countChar 1 (oneOf ['0','1'])]+ 'R' -> escIgnore 'R' [quoteArg]+ 'S' -> escIgnore 'S' [quoteArg]+ 'V' -> escIgnore 'V' groffSkip+ 'X' -> escIgnore 'X' [quoteArg]+ 'Y' -> escIgnore 'Y' groffSkip+ 'Z' -> escIgnore 'Z' [quoteArg]+ '\'' -> return $ emit "'"+ '\n' -> return mempty -- line continuation+ '^' -> return $ emit "\x200A" -- 1/12 em space+ '_' -> return $ emit "_"+ '`' -> return $ emit "`"+ 'a' -> return mempty -- "non-interpreted leader character"+ 'b' -> escIgnore 'b' [quoteArg]+ 'c' -> return mempty -- interrupt text processing+ 'd' -> escIgnore 'd' [] -- forward down 1/2em+ 'e' -> return $ emit "\\"+ 'f' -> escFont+ 'g' -> escIgnore 'g' groffSkip+ 'h' -> escIgnore 'h' [quoteArg]+ 'k' -> escIgnore 'k' groffSkip+ 'l' -> escIgnore 'l' [quoteArg]+ 'm' -> escIgnore 'm' groffSkip+ 'n' -> escIgnore 'm' groffSkip+ 'o' -> escIgnore 'o' [quoteArg]+ 'p' -> escIgnore 'p' []+ 'r' -> escIgnore 'r' []+ 's' -> escIgnore 's' [escapeArg, signedNumber]+ 't' -> return $ emit "\t"+ 'u' -> escIgnore 'u' []+ 'v' -> escIgnore 'v' [quoteArg]+ 'w' -> escIgnore 'w' [quoteArg]+ 'x' -> escIgnore 'x' [quoteArg]+ 'z' -> escIgnore 'z' [countChar 1 anyChar]+ '|' -> return $ emit "\x2006" --1/6 em space+ '~' -> return $ emit "\160" -- nonbreaking space+ '\\' -> return $ emit "\\"+ _ -> return $ emit $ T.singleton c+ -- man 7 groff: "If a backslash is followed by a character that+ -- does not constitute a defined escape sequence, the backslash+ -- is silently ignored and the character maps to itself."++escIgnore :: (PandocMonad m, RoffLikeLexer x)+ => Char+ -> [Lexer m x T.Text]+ -> Lexer m x (Token x)+escIgnore c argparsers = do+ pos <- getPosition+ arg <- snd <$> withRaw (choice argparsers) <|> return ""+ report $ SkippedContent ("\\" <> T.cons c arg) pos+ return mempty++escUnknown :: (PandocMonad m, RoffLikeLexer x) => T.Text -> Lexer m x (Token x)+escUnknown s = do+ pos <- getPosition+ report $ SkippedContent s pos+ return $ emit "\xFFFD"++signedNumber :: (PandocMonad m, RoffLikeLexer x) => Lexer m x T.Text+signedNumber = try $ do+ sign <- option "" ("-" <$ char '-' <|> "" <$ char '+')+ ds <- many1Char digit+ return (sign <> ds)++-- Parses: [..] or (..+escapeArg :: (PandocMonad m, RoffLikeLexer x) => Lexer m x T.Text+escapeArg = choice+ [ char '[' *> optional expandString *>+ manyTillChar (noneOf ['\n',']']) (char ']')+ , char '(' *> optional expandString *>+ countChar 2 (satisfy (/='\n'))+ ]++-- Parses: '..'+quoteArg :: (PandocMonad m, RoffLikeLexer x) => Lexer m x T.Text+quoteArg = char '\'' *> manyTillChar (noneOf ['\n','\'']) (char '\'')
src/Text/Pandoc/Readers/Typst.hs view
@@ -235,10 +235,11 @@ ,("quote", \_ fields -> do getField "block" fields >>= guard body <- getField "body" fields >>= pWithContents pBlocks- attribution <-- ((\x -> B.para ("\x2104\xa0" <> x)) <$>- (getField "attribution" fields >>= pWithContents pInlines))- <|> pure mempty+ attribution' <- getField "attribution" fields+ attribution <- if attribution' == mempty+ then pure mempty+ else (\x -> B.para ("\x2014\xa0" <> x)) <$>+ (pWithContents pInlines attribution') pure $ B.blockQuote $ body <> attribution) ,("list", \_ fields -> do children <- V.toList <$> getField "children" fields@@ -522,9 +523,12 @@ isParbreak _ = False pPara :: PandocMonad m => P m B.Blocks-pPara =- B.para . B.trimInlines . collapseAdjacentCites . mconcat- <$> (many1 pInline <* optional pParBreak)+pPara = do+ ils <- B.trimInlines . collapseAdjacentCites . mconcat <$> many1 pInline+ optional pParBreak+ pure $ if ils == mempty+ then mempty+ else B.para ils pParBreak :: PandocMonad m => P m () pParBreak =
src/Text/Pandoc/Readers/Typst/Math.hs view
@@ -207,6 +207,20 @@ case mbAnn of Nothing -> pure x Just ann -> EOver False x <$> pMathGrouped ann+ Elt "math.underparen" _ fields -> do+ mbAnn <- getField "annotation" fields+ body <- getField "body" fields >>= pMathGrouped+ let x = EUnder False body (ESymbol TUnder "\9181")+ case mbAnn of+ Nothing -> pure x+ Just ann -> EUnder False x <$> pMathGrouped ann+ Elt "math.overparen" _ fields -> do+ mbAnn <- getField "annotation" fields+ body <- getField "body" fields >>= pMathGrouped+ let x = EOver False body (ESymbol TOver "\9180")+ case mbAnn of+ Nothing -> pure x+ Just ann -> EOver False x <$> pMathGrouped ann Elt "math.scripts" _ fields -> getField "body" fields >>= pMathGrouped Elt "math.limits" _ fields -> getField "body" fields >>= pMathGrouped Elt "math.root" _ fields -> do
src/Text/Pandoc/Writers/ANSI.hs view
@@ -69,7 +69,7 @@ stInTable = False } --- | Return ANSI-styled verison of document+-- | Return ANSI-styled version of document pandocToANSI :: PandocMonad m => WriterOptions -> Pandoc -> TW m Text pandocToANSI opts (Pandoc meta blocks) = do@@ -124,7 +124,11 @@ blockToANSI opts (Para inlines) = inlineListToANSI opts inlines -blockToANSI opts (LineBlock lns) = blockToANSI opts $ linesToPara lns+blockToANSI opts (LineBlock lns) = do+ let go [] = return D.blankline+ go xs = inlineListToANSI opts xs+ lns' <- mapM go lns+ return $ D.vcat lns' blockToANSI _ b@(RawBlock _ _) = do report $ BlockNotRendered b
src/Text/Pandoc/Writers/Docx/OpenXML.hs view
@@ -1017,6 +1017,7 @@ Just Svg -> ".svg" Just Emf -> ".emf" Just Tiff -> ".tiff"+ Just Webp -> ".webp" Nothing -> "" imgpath = "media/" <> ident <> imgext mbMimeType = mt <|> getMimeType (T.unpack imgpath)
src/Text/Pandoc/Writers/EPUB.hs view
@@ -757,11 +757,7 @@ Just img -> do let fp = takeFileName img -- retrieve cover image file- mediaPaths <- gets (map (fst . snd) . stMediaPaths)- coverImageName <- -- see #4206- if ("media/" <> fp) `elem` mediaPaths- then getMediaNextNewName (takeExtension fp)- else return fp+ coverImageName <- getMediaNextNewName (takeExtension fp) -- see #4206 -- image dimensions imgContent <- lift $ P.readFileLazy img (coverImageWidth, coverImageHeight) <-
src/Text/Pandoc/Writers/HTML.hs view
@@ -862,10 +862,15 @@ [("role", "listitem") | isCslBibEntry && html5] let speakerNotes = "notes" `elem` classes -- we don't want incremental output inside speaker notes, see #1394- let opts' = if | speakerNotes -> opts{ writerIncremental = False }- | "incremental" `elem` classes -> opts{ writerIncremental = True }- | "nonincremental" `elem` classes -> opts{ writerIncremental = False }- | otherwise -> opts+ let (opts', isIncrDiv) =+ if | speakerNotes ->+ (opts{ writerIncremental = False }, False)+ | "incremental" `elem` classes ->+ (opts{ writerIncremental = True }, True)+ | "nonincremental" `elem` classes ->+ (opts{ writerIncremental = False }, True)+ | otherwise ->+ (opts, False) -- we remove "incremental" and "nonincremental" if we're in a -- slide presentation format. classes' = case slideVariant of@@ -886,18 +891,22 @@ let (divtag, classes'') = if html5 && "section" `elem` classes' then (H5.section, filter (/= "section") classes') else (H.div, classes')- if speakerNotes- then case slideVariant of- RevealJsSlides -> addAttrs opts' attr $- H5.aside contents'- DZSlides -> do- t <- addAttrs opts' attr $- H5.div contents'- return $ t ! A5.role "note"- NoSlides -> addAttrs opts' attr $- H.div contents'- _ -> return mempty- else addAttrs opts (ident, classes'', kvs) $+ if | isIncrDiv && (ident, classes'', kvs) == nullAttr ->+ -- Unwrap divs that only have (non)increment information+ pure contents+ | speakerNotes ->+ case slideVariant of+ RevealJsSlides -> addAttrs opts' attr $+ H5.aside contents'+ DZSlides -> do+ t <- addAttrs opts' attr $+ H5.div contents'+ return $ t ! A5.role "note"+ NoSlides -> addAttrs opts' attr $+ H.div contents'+ _ -> return mempty+ | otherwise ->+ addAttrs opts (ident, classes'', kvs) $ divtag contents' blockToHtmlInner opts (RawBlock f str) = do ishtml <- isRawHtml f
src/Text/Pandoc/Writers/LaTeX.hs view
@@ -1128,8 +1128,9 @@ let noteContents = nest 2 contents' <> optnl beamer <- gets stBeamer -- in beamer slides, display footnote from current overlay forward+ -- and ensure that the note is on the frame, not e.g. the column (#5769) let beamerMark = if beamer- then text "<.->"+ then text "<.->[frame]" else empty if externalNotes then do
src/Text/Pandoc/Writers/Markdown.hs view
@@ -21,7 +21,7 @@ writeCommonMark, writeMarkua, writePlain) where-import Control.Monad (foldM, zipWithM, MonadPlus(..), when)+import Control.Monad (foldM, zipWithM, MonadPlus(..), when, liftM) import Control.Monad.Reader ( asks, MonadReader(local) ) import Control.Monad.State.Strict ( gets, modify ) import Data.Default@@ -447,12 +447,19 @@ return $ contents <> cr blockToMarkdown' opts (Para inlines) = (<> blankline) `fmap` blockToMarkdown opts (Plain inlines)-blockToMarkdown' opts (LineBlock lns) =- if isEnabled Ext_line_blocks opts- then do- mdLines <- mapM (inlineListToMarkdown opts) lns- return $ vcat (map (hang 2 (literal "| ")) mdLines) <> blankline- else blockToMarkdown opts $ linesToPara lns+blockToMarkdown' opts (LineBlock lns) = do+ variant <- asks envVariant+ case variant of+ PlainText -> do+ let emptyToBlank l = if isEmpty l then blankline else l+ mdLines <- mapM (liftM emptyToBlank . inlineListToMarkdown opts) lns+ return $ vcat mdLines <> blankline+ _ ->+ if isEnabled Ext_line_blocks opts+ then do+ mdLines <- mapM (inlineListToMarkdown opts) lns+ return $ vcat (map (hang 2 (literal "| ")) mdLines) <> blankline+ else blockToMarkdown opts $ linesToPara lns blockToMarkdown' opts b@(RawBlock f str) = do variant <- asks envVariant let Format fmt = f@@ -664,7 +671,9 @@ (Pandoc nullMeta [t]) return $ tbl $$ blankline -- caption is in the HTML table | otherwise- -> return $ (literal "[TABLE]" $$ caption''') $$ blankline+ -> do+ report (BlockNotRendered t)+ return $ (literal "[TABLE]" $$ caption''') $$ blankline blockToMarkdown' opts (BulletList items) = do contents <- inList $ mapM (bulletListItemToMarkdown opts) items return $ (if isTightList items then vcat else vsep)
src/Text/Pandoc/Writers/Powerpoint/Output.hs view
@@ -914,6 +914,7 @@ Just Svg -> Just ".svg" Just Emf -> Just ".emf" Just Tiff -> Just ".tiff"+ Just Webp -> Just ".webp" Nothing -> Nothing let newGlobalId = fromMaybe (maxGlobalId + 1) (M.lookup fp globalIds)
src/Text/Pandoc/Writers/TEI.hs view
@@ -1,7 +1,7 @@ {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE PatternGuards #-} {- |- Module : Text.Pandoc.Writers.Docbook+ Module : Text.Pandoc.Writers.TEI Copyright : Copyright (C) 2006-2024 John MacFarlane License : GNU GPL, version 2 or above @@ -9,7 +9,7 @@ Stability : alpha Portability : portable -Conversion of 'Pandoc' documents to Docbook XML.+Conversion of 'Pandoc' documents to TEI XML -} module Text.Pandoc.Writers.TEI (writeTEI) where import Data.Text (Text)@@ -27,7 +27,7 @@ import Text.Pandoc.Writers.Shared import Text.Pandoc.XML --- | Convert Pandoc document to string in Docbook format.+-- | Convert Pandoc document to string in TEI XML format. writeTEI :: PandocMonad m => WriterOptions -> Pandoc -> m Text writeTEI opts doc = do let Pandoc meta blocks = ensureValidXmlIdentifiers doc
src/Text/Pandoc/Writers/Typst.hs view
@@ -84,11 +84,13 @@ Right l -> resetField "lang" (langLanguage l) . maybe id (resetField "region") (langRegion l))+ $ defField "smart" (isEnabled Ext_smart options) $ defField "toc-depth" (tshow $ writerTOCDepth options) $ defField "figure-caption-position" (toPosition $ writerFigureCaptionPosition options) $ defField "table-caption-position" (toPosition $ writerTableCaptionPosition options)+ $ defField "page-numbering" ("1" :: Text) $ (if writerNumberSections options then defField "numbering" ("1.1.1.1.1" :: Text) else id)@@ -351,8 +353,9 @@ inlineToTypst inline = case inline of Str txt -> do+ opts <- gets stOptions context <- gets stEscapeContext- return $ escapeTypst context txt+ return $ escapeTypst (isEnabled Ext_smart opts) context txt Space -> return space SoftBreak -> do wrapText <- gets $ writerWrapText . stOptions@@ -396,11 +399,17 @@ contents <- inlinesToTypst inlines return $ toTypstTextElement typstTextAttrs contents <> lab Quoted quoteType inlines -> do- let q = case quoteType of- DoubleQuote -> literal "\""- SingleQuote -> literal "'"+ opts <- gets stOptions+ let smart = isEnabled Ext_smart opts contents <- inlinesToTypst inlines- return $ q <> contents <> q+ return $+ case quoteType of+ DoubleQuote+ | smart -> "\"" <> contents <> "\""+ | otherwise -> "“" <> contents <> "”"+ SingleQuote+ | smart -> "'" <> contents <> "'"+ | otherwise -> "‘" <> contents <> "’" Cite citations inlines -> do opts <- gets stOptions if isEnabled Ext_citations opts@@ -455,8 +464,8 @@ , needsEscapeAtLineStart c -> ("\\" <>) _ -> id -escapeTypst :: EscapeContext -> Text -> Doc Text-escapeTypst context t =+escapeTypst :: Bool -> EscapeContext -> Text -> Doc Text+escapeTypst smart context t = (case T.uncons t of Just (c, _) | needsEscapeAtLineStart c@@ -469,9 +478,17 @@ where escapeChar c | c == '\160' = "~"+ | c == '\8217', smart = "'" -- apostrophe+ | c == '\8212', smart = "---" -- em dash+ | c == '\8211', smart = "--" -- en dash | needsEscape c = "\\" <> T.singleton c | otherwise = T.singleton c needsEscape '\160' = True+ needsEscape '\8217' = smart+ needsEscape '\8212' = smart+ needsEscape '\8211' = smart+ needsEscape '\'' = smart+ needsEscape '"' = smart needsEscape '[' = True needsEscape ']' = True needsEscape '#' = True@@ -480,8 +497,6 @@ needsEscape '@' = True needsEscape '$' = True needsEscape '\\' = True- needsEscape '\'' = True- needsEscape '"' = True needsEscape '`' = True needsEscape '_' = True needsEscape '*' = True
+ test/Tests/Readers/Mdoc.hs view
@@ -0,0 +1,321 @@+{-# LANGUAGE OverloadedStrings #-}+{- |+ Module : Tests.Readers.Mdoc+ Copyright : © 2024 Evan Silberman+ License : GNU GPL, version 2 or above++ Maintainer : + Stability : alpha+ Portability : portable++Tests for the Mdoc reader.+-}++module Tests.Readers.Mdoc (tests) where++import Data.Text (Text)+import qualified Data.Text as T+import Test.Tasty+import Test.Tasty.HUnit (HasCallStack)+import Tests.Helpers+import Text.Pandoc+import Text.Pandoc.Arbitrary ()+import Text.Pandoc.Builder++mdoc :: Text -> Pandoc+mdoc = purely $ readMdoc def++infix 4 =:+(=:) :: (ToString c, HasCallStack)+ => String -> (Text, c) -> TestTree+(=:) = test mdoc++cls :: Text -> Attr+cls x = (mempty, [x], mempty)++tests :: [TestTree]+tests = [+ testGroup "one-line enclosures"+ [ "Dq" =:+ ".Dq hello world" =?>+ para (doubleQuoted "hello world")+ , "Sq" =:+ ".Sq hello world" =?>+ para (singleQuoted "hello world")+ , "empty" =:+ ".Dq" =?>+ para (doubleQuoted mempty)+ , "nested" =:+ ".Dq Pq hello world" =?>+ para (doubleQuoted "(hello world)")+ , "nested with closing delimiters" =:+ ".Dq Pq hi mom !" =?>+ para (doubleQuoted "(hi mom)" <> "!")+ , "nested multiline enclosure" =:+ ".Dq Po a \\&; b \\&; c Pc ." =?>+ para (doubleQuoted "(a ; b ; c)" <> ".")+ , "with inlines" =:+ ".Dq hello Sy world ." =?>+ para (doubleQuoted ("hello" <> space <> strong "world" <> "."))+ , "with text production" =:+ ".Dq I love the St -iso8601 standard!" =?>+ para (doubleQuoted "I love the ISO 8601 standard!")+ , "with Ns" =:+ ".Op , Ns Ar value ..." =?>+ para ("[," <> (codeWith (cls "variable") "value ...") <> "]")+ , "ending with open delimiter" =:+ ".Dq hi (" =?>+ para (doubleQuoted "hi (")+ ]+ , testGroup "multiline enclosures"+ [ "nested multiline" =:+ T.unlines [".Bo", ".Po", "hi", ".Pc", ".Bc"] =?>+ para ("[(hi)]")+ , "nested on one line" =:+ ".Bo Po hi Pc Bc" =?>+ para ("[(hi)]")+ , "with wacky delimiters" =:+ ".Bo ( | hi ! Bc ?" =?>+ para ("([| hi!]?")+ ]+ , testGroup "simple inlines"+ [ "Sy" =:+ ".Sy hello world" =?>+ para (strong "hello world")+ , "Em" =:+ ".Em hello world" =?>+ para (emph "hello world")+ , "Ev" =:+ ".Ev HELLO_WORLD ," =?>+ para (codeWith (cls "Ev") "HELLO_WORLD" <> ",")+ , "Ar" =:+ ".Ar ) z" =?>+ para (codeWith (cls "variable") "file ..." <> ") " <> codeWith (cls "variable") "z")+ , "In" =:+ ".In ( math.h ) b c" =?>+ para ("(" <> codeWith (cls "In") "<math.h>" <> ") b c")+ , "Mt" =:+ ".Mt a@example.org , b@example.org" =?>+ para ((link "mailto:a@example.org" "" "a@example.org") <>+ "," <> space <> (link "mailto:b@example.org" "" "b@example.org"))+ , "No" =:+ ".No ( hello , world ! )" =?>+ para "(hello, world!)"+ , "empty Pa with closing punctuation" =:+ ".Pa ) z" =?>+ para (spanWith (cls "Pa") "~" <> ")" <> space <> spanWith (mempty, ["Pa"], mempty) "z")+ , "delimiters" =:+ ".Sy ( hello world )" =?>+ para (mconcat ["(", strong "hello world", ")"])+ , "multiple" =:+ ".Sy hello Em world" =?>+ para (strong "hello" <> space <> emph "world")+ ]+ , testGroup "Fl"+ [ "simple" =:+ ".Fl w" =?>+ para (codeWith (cls "Fl") "-w")+ , "multiple" =:+ ".Fl W all" =?>+ para (codeWith (cls "Fl") "-W" <> space <> codeWith (cls "Fl") "-all")+ , "empty with following macro" =:+ ".Fl Cm x" =?>+ para (codeWith (cls "Fl") "-" <> codeWith (cls "Cm") "x")+ , "enclosed with following macro" =:+ ".Pq Fl Cm x" =?>+ para ("(" <> codeWith (cls "Fl") "-" <> codeWith (cls "Cm") "x" <> ")")+ -- XXX this is a mandoc delta, the period is meant to land outside+ -- the enclosure. parseInline has learned how to defer eols but not+ -- closing delimiters+ , "enclosed with following delimiters" =:+ ".Pq Fl x ." =?>+ para ("(" <> codeWith (cls "Fl") "-x" <> ".)")+ , "following Ns" =:+ ".Fl W Ns Cm all" =?>+ para (codeWith (cls "Fl") "-W" <> codeWith (cls "Cm") "all")+ , "GNU" =:+ ".Fl -help" =?>+ para (codeWith (cls "Fl") "--help")+ , "GNU escaped" =:+ ".Fl \\-help" =?>+ para (codeWith (cls "Fl") "--help")+ , "GNU Fl Fl" =:+ ".Fl Fl help" =?>+ para (codeWith (cls "Fl") "--help")+ , "punctuation" =:+ ".Op Fl a | b" =?>+ para ("[" <> codeWith (cls "Fl") "-a" <> " | " <> codeWith (cls "Fl") "-b" <> "]")+ , "middle close paren" =:+ ".Fl a ) z" =?>+ para (codeWith (cls "Fl") "-a" <> ") " <> codeWith (cls "Fl") "-z")+ , "empty with close paren" =:+ ".Fl ) z" =?>+ para (codeWith (cls "Fl") "-" <> ") " <> codeWith (cls "Fl") "-z")+ , "empty with pipe" =:+ ".Fl | z" =?>+ para (codeWith (cls "Fl") "-" <> " | " <> codeWith (cls "Fl") "-z")+ , "empty with parens" =:+ ".Fl ( )" =?>+ para ("(" <> codeWith (cls "Fl") "-" <> ")")+ ]+ , testGroup "links"+ [ "basic" =:+ ".Lk href name" =?>+ para (link "href" "" "name")+ , "complicated" =:+ ".Lk , ( href name )" =?>+ para ("," <> space <> "(" <> link "href" "" "name" <> ")")+ , "unnamed" =:+ ".Lk href" =?>+ para (link "href" "" "href")+ ]+ , testGroup "Ns macro"+ [ "at the beginning of a macro line (mandoc delta)" =:+ T.unlines [".Op before", ".Ns Op after"] =?>+ para "[before] [after]" -- mandoc: warning+ , "after a block closing macro" =:+ T.unlines [".Oo before", ".Oc Ns Op after"] =?>+ para "[before][after]"+ , "in the middle of a macro line" =:+ ".Oo before Oc Ns Op after" =?>+ para "[before][after]"+ , "before closing punctuation" =:+ ".Oo before Oc Ns : Op after" =?>+ para "[before]: [after]" -- mandoc: warning+ , "after closing punctuation" =:+ ".Oo before Oc : Ns Op after" =?>+ para "[before]:[after]"+ , "at the end of a macro line" =:+ T.unlines [".Oo before Oc Ns", ".Op after"] =?>+ para "[before][after]"+ , "at the end of a partial-implicit line" =:+ T.unlines [".Op before Ns", ".Op after"] =?>+ para "[before][after]"+ , "normal words" =:+ ".No no Ns ns No no" =?>+ para ("nons" <> space <> "no")+ , "opening punctuation" =:+ ".No no Ns \"(\" ns No no" =?>+ para ("no(ns" <> space <> "no")+ , "closing punctuation" =:+ ".No no \"Ns\" ns \")\" No no" =?>+ para ("nons)" <> space <> "no")+ ]+ , testGroup "spacing mode"+ [ "all text" =:+ T.unlines ["a", ".Sm off", "b c", "d", ".Sm on", "e"] =?>+ para ("a b c d e")+ , "text around macro" =:+ T.unlines ["a", ".Sm off", ".Sy b c", ".Sm on", "d"] =?>+ para ("a" <> space <> strong "bc" <> space <> "d")+ , "mulitple macros" =:+ T.unlines ["a", ".Sm off", ".Sy b Em c", ".Sm on", "d"] =?>+ para ("a" <> space <> strong "b" <> emph "c" <> space <> "d")+ , "mulitple control lines" =:+ T.unlines ["a", ".Sm off", ".Sy b", ".Em c", ".Sm on", "d"] =?>+ para ("a" <> space <> strong "b" <> emph "c" <> space <> "d")+ , "mixed control and text lines" =:+ T.unlines ["a", ".Sm off", ".Sy b", "c", ".Em d", ".Sm on", "d"] =?>+ para ("a" <> space <> strong "b" <> "c" <> emph "d" <> space <> "d")+ , "delimiters" =:+ T.unlines [".Sm off", ".Em a ", ".Em [ b | c ]", ".Sm on"] =?>+ para (emph "a" <> "[" <> emph "b" <> "|" <> emph "c" <> "]")+ ]+ , testGroup "Ap macro"+ [ "in the middle of a macro line" =:+ ".Xr mandoc 1 Ap s" =?>+ para (spanWith (cls "Xr") "mandoc(1)" <> "'s")+ -- mandoc difference: the edge case of "Ap (" tested in this mandoc regress+ -- isn't present in any actual OpenBSD base system manuals, where Ap is+ -- only ever followed by a letter. Furthermore, "Ap" is generally uncommon+ -- compared to "Ns '" (e.g. ".Xr mandoc 1 Ns 's"). I'm accepting a+ -- difference from mandoc here because correctly suppressing space after+ -- the "(" here would require more refactoring than I feel like doing at+ -- time of writing.+ -- per mandoc, should be: para (strong "bold" <> "'(" <> strong "bold")+ , "with punctuation and called macro" =:+ ".Sy bold Ap ( \"Sy\" bold" =?>+ para (strong "bold" <> "'( " <> strong "bold")+ ]+ , testGroup "Pf macro"+ [ "closing punctuation" =:+ ".Pf . right ." =?>+ para ".right."+ , "double punctuation" =:+ ".Pf . . double" =?>+ para ".. double"+ , "opening punctuation" =:+ ".Pf ( left ." =?>+ para "(left."+ , "unparsed argument" =:+ ".Pf Ar Sy gument " =?>+ para ("Ar" <> strong "gument")+ ]+ , testGroup "text production"+ [ "early NetBSD versions" =:+ T.unlines [".Nx 0.9a", "and", ".Nx 9.4b"] =?>+ para "NetBSD 0.9A and NetBSD 9.4b"+ , "with Ns" =:+ ".Ox Ns -specific" =?>+ para "OpenBSD-specific"+ , "with punctuation" =:+ ".Fx ," =?>+ para "FreeBSD,"+ , "with argument and punctuation" =:+ ".Fx 12.0 ." =?>+ para "FreeBSD 12.0."+ , "BSD alone" =:+ ".Bx ." =?>+ para "BSD."+ , "BSD with macro" =:+ ".Bx No rocks" =?>+ para "BSD rocks"+ , "BSD with version" =:+ ".Bx 4.4 ," =?>+ para "4.4BSD,"+ , "BSD with variant" =:+ ".Bx 4.3 tahoe !" =?>+ para "4.3BSD-Tahoe!"+ ]+ , testGroup "inline punctuation"+ [ testGroup "leading punctuation"+ [ "open paren" =: ".Em ( b" =?> para ("(" <> emph "b")+ , "open square bracket" =: ".Em \"[\" b" =?> para ("[" <> emph "b")+ , "pipe" =: ".Em | b" =?> para ("|" <> space <> emph "b")+ , "period" =: ".Em . b" =?> para ("." <> space <> emph "b")+ , "comma" =: ".Em , b" =?> para ("," <> space <> emph "b")+ , "semicolon" =: ".Em ; b" =?> para (";" <> space <> emph "b")+ , "colon" =: ".Em : b" =?> para (":" <> space <> emph "b")+ , "question mark" =: ".Em ? b" =?> para ("?" <> space <> emph "b")+ , "exclamation" =: ".Em ! b" =?> para ("!" <> space <> emph "b")+ , "close paren" =: ".Em ) b" =?> para (")" <> space <> emph "b")+ , "close square bracket" =: ".Em \"]\" b" =?> para ("]" <> space <> emph "b")+ ]+ , testGroup "trailing punctuation"+ [ "open paren" =: ".Em a (" =?> para (emph "a" <> space <> "(")+ , "open square bracket" =: ".Em a [" =?> para (emph "a" <> space <> "[")+ , "pipe" =: ".Em a |" =?> para (emph "a" <> space <> "|")+ , "period" =: ".Em a ." =?> para (emph "a" <> ".")+ , "comma" =: ".Em a ," =?> para (emph "a" <> ",")+ , "semicolon" =: ".Em a ;" =?> para (emph "a" <> ";")+ , "colon" =: ".Em a :" =?> para (emph "a" <> ":")+ , "question mark" =: ".Em a ?" =?> para (emph "a" <> "?")+ , "exclamation" =: ".Em a !" =?> para (emph "a" <> "!")+ , "close parens" =: ".Em a \")\"" =?> para (emph "a" <> ")")+ , "close square bracket" =: ".Em a ]" =?> para (emph "a" <> "]")+ ]+ , testGroup "middle punctuation"+ [ "open paren" =: ".Em a ( b" =?> para (mconcat [emph "a", space, "(", emph "b"])+ , "open square bracket" =: ".Em a [ b" =?> para (mconcat [emph "a", space, "[", emph "b"])+ , "pipe" =: ".Em a \"|\" b" =?> para (mconcat [emph "a", space, "|", space, emph "b"])+ , "period" =: ".Em a . b" =?> para (mconcat [emph "a", ".", space, emph "b"])+ , "comma" =: ".Em a , b" =?> para (mconcat [emph "a", ",", space, emph "b"])+ , "semicolon" =: ".Em a ; b" =?> para (mconcat [emph "a", ";", space, emph "b"])+ , "colon" =: ".Em a \":\" b" =?> para (mconcat [emph "a", ":", space, emph "b"])+ , "question mark" =: ".Em a ? b" =?> para (mconcat [emph "a", "?", space, emph "b"])+ , "exclamation" =: ".Em a ! b" =?> para (mconcat [emph "a", "!", space, emph "b"])+ , "close paren" =: ".Em a ) b" =?> para (mconcat [emph "a", ")", space, emph "b"])+ , "close square bracket" =: ".Em a ] b" =?> para (mconcat [emph "a", "]", space, emph "b"])+ ]+ ]+ ]
test/Tests/Readers/RST.hs view
@@ -50,13 +50,13 @@ , " on two lines" ] =?> doc (para "para" <>- definitionList [ (str "Hostname", [para "media08"])- , (text "IP address", [para "10.0.0.19"])- , (str "Size", [para "3ru"])- , (str "Version", [para "1"])- , (str "Indentation", [para "Since the field marker may be quite long, the second\nand subsequent lines of the field body do not have to line up\nwith the first line, but they must be indented relative to the\nfield name marker, and they must line up with each other."])- , (text "Parameter i", [para "integer"])- , (str "Final", [para "item\non two lines"])+ definitionList [ (str "Hostname", [plain "media08"])+ , (text "IP address", [plain "10.0.0.19"])+ , (str "Size", [plain "3ru"])+ , (str "Version", [plain "1"])+ , (str "Indentation", [plain "Since the field marker may be quite long, the second\nand subsequent lines of the field body do not have to line up\nwith the first line, but they must be indented relative to the\nfield name marker, and they must line up with each other."])+ , (text "Parameter i", [plain "integer"])+ , (str "Final", [plain "item\non two lines"]) ]) , "metadata" =: T.unlines [ "====="@@ -69,7 +69,7 @@ , ":Version: 1" ] =?>- setMeta "version" (para "1") (setMeta "title" ("Title" :: Inlines)+ setMeta "version" (str "1") (setMeta "title" ("Title" :: Inlines) $ setMeta "subtitle" ("Subtitle" :: Inlines) $ doc mempty) , "with inline markup" =: T.unlines@@ -87,10 +87,10 @@ ] =?> setMeta "date" (str "today") (doc- $ definitionList [ (emph "one", [para "emphasis"])- , (link "http://example.com" "" "two", [para "reference"])- , (link "http://example.org" "" "three", [para "another one"])- , (code "four", [para "literal"])+ $ definitionList [ (emph "one", [plain "emphasis"])+ , (link "http://example.com" "" "two", [plain "reference"])+ , (link "http://example.org" "" "three", [plain "another one"])+ , (code "four", [plain "literal"]) ]) ] , "URLs with following punctuation" =:
test/ansi-test.ansi view
@@ -113,6 +113,10 @@ We see a log₁₀ reduction in 2⁹ seconds. +Hello++Goodbye+ ──────────────────── 1. Here’s the note.[0m]8;;\
test/ansi-test.txt view
@@ -121,3 +121,7 @@ [small caps]{.smallcaps} We see a log~10~ reduction in 2^9^ seconds.++| Hello+|+| Goodbye
+ test/command/10271.md view
@@ -0,0 +1,29 @@+```+% pandoc -f markdown+smart -t typst+smart+"don't do it---"+^D+"don't do it---"++```+```+% pandoc -f markdown+smart -t typst-smart+"don't do it---"+^D+“don’t do it—”++```+```+% pandoc -f markdown-smart -t typst+smart+"don't do it---"+^D+\"don\'t do it---\"++```+```+% pandoc -f markdown-smart -t typst-smart+"don't do it---"+^D+"don't do it---"++```+
+ test/command/10279.md view
@@ -0,0 +1,9 @@+```+% pandoc -f rst+See `the full compatibility guidelines <https://+example.com>`_ for more information.+^D+<p>See <a href="https://example.com">the full compatibility+guidelines</a> for more information.</p>++```
+ test/command/10281.md view
@@ -0,0 +1,68 @@+```+% pandoc -f rst -t native+`Want Speed? Pass by Value`_++.. note::+ For more information about the pass-by-value idiom, read: `Want Speed? Pass by Value`_.++ .. _Want Speed? Pass by Value: https://web.archive.org/web/20140205194657/http://cpp-next.com/archive/2009/08/want-speed-pass-by-value/+^D+[ Para+ [ Link+ ( "" , [] , [] )+ [ Str "Want"+ , Space+ , Str "Speed?"+ , Space+ , Str "Pass"+ , Space+ , Str "by"+ , Space+ , Str "Value"+ ]+ ( "https://web.archive.org/web/20140205194657/http://cpp-next.com/archive/2009/08/want-speed-pass-by-value/"+ , ""+ )+ ]+, Div+ ( "" , [ "note" ] , [] )+ [ Div ( "" , [ "title" ] , [] ) [ Para [ Str "Note" ] ]+ , Para+ [ Str "For"+ , Space+ , Str "more"+ , Space+ , Str "information"+ , Space+ , Str "about"+ , Space+ , Str "the"+ , Space+ , Str "pass-by-value"+ , Space+ , Str "idiom,"+ , Space+ , Str "read:"+ , Space+ , Link+ ( "" , [] , [] )+ [ Str "Want"+ , Space+ , Str "Speed?"+ , Space+ , Str "Pass"+ , Space+ , Str "by"+ , Space+ , Str "Value"+ ]+ ( "https://web.archive.org/web/20140205194657/http://cpp-next.com/archive/2009/08/want-speed-pass-by-value/"+ , ""+ )+ , Str "."+ ]+ ]+]++```+
+ test/command/10318.md view
@@ -0,0 +1,84 @@+```+% pandoc -f rst -t html5+-a Output all.+-c arg Output just arg.+--long Output all day long.+/V A VMS/DOS-style option.++-p This option has two paragraphs in the description.+ This is the first.++ This is the second.+ Blank lines may be omitted between options+ (as above) or left in (as here and below).++--very-long-option A VMS-style option. Note the adjustment+ for the required two spaces.++--an-even-longer-option+ The description can also start on the next line.++-2, --two This option has two variants.++-f FILE, --file=FILE These two options are synonyms; both have+ arguments.++-f <[path]file> Option argument placeholders must start with+ a letter or be wrapped in angle brackets.++-d <src dest> Angle brackets are also required if an option+ expects more than one argument.+^D+<dl>+<dt><code>-a</code></dt>+<dd>+<p>Output all.</p>+</dd>+<dt><code>-c arg</code></dt>+<dd>+<p>Output just arg.</p>+</dd>+<dt><code>--long</code></dt>+<dd>+<p>Output all day long.</p>+</dd>+<dt><code>/V</code></dt>+<dd>+<p>A VMS/DOS-style option.</p>+</dd>+<dt><code>-p</code></dt>+<dd>+<p>This option has two paragraphs in the description. This is the+first.</p>+<p>This is the second. Blank lines may be omitted between options (as+above) or left in (as here and below).</p>+</dd>+<dt><code>--very-long-option</code></dt>+<dd>+<p>A VMS-style option. Note the adjustment for the required two+spaces.</p>+</dd>+<dt><code>--an-even-longer-option</code></dt>+<dd>+<p>The description can also start on the next line.</p>+</dd>+<dt><code>-2, --two</code></dt>+<dd>+<p>This option has two variants.</p>+</dd>+<dt><code>-f FILE, --file=FILE</code></dt>+<dd>+<p>These two options are synonyms; both have arguments.</p>+</dd>+<dt><code>-f <[path]file></code></dt>+<dd>+<p>Option argument placeholders must start with a letter or be wrapped+in angle brackets.</p>+</dd>+<dt><code>-d <src dest></code></dt>+<dd>+<p>Angle brackets are also required if an option expects more than one+argument.</p>+</dd>+</dl>+```
+ test/command/10328.md view
@@ -0,0 +1,37 @@+Unwrap divs if they only have the `nonincremental` or `incremental` classes.++```+% pandoc --incremental --from=markdown --to=revealjs+## First slide++::: nonincremental++1. Note 1+2. Note 2+3. Note 3++:::++## Second Slide++1. Note 1+2. Note 2+3. Note 3+^D+<section id="first-slide" class="slide level2">+<h2>First slide</h2>+<ol type="1">+<li>Note 1</li>+<li>Note 2</li>+<li>Note 3</li>+</ol>+</section>+<section id="second-slide" class="slide level2">+<h2>Second Slide</h2>+<ol type="1">+<li class="fragment">Note 1</li>+<li class="fragment">Note 2</li>+<li class="fragment">Note 3</li>+</ol>+</section>+```
+ test/command/10390.md view
@@ -0,0 +1,23 @@+```+% pandoc -f mediawiki+ {| class="wikitable"+ |+ Overview of basic table markup+ ! Key+ |-+ | Value+ |}+^D+<table>+<caption>Overview of basic table markup</caption>+<thead>+<tr>+<th><p>Key</p></th>+</tr>+</thead>+<tbody>+<tr>+<td><p>Value</p></td>+</tr>+</tbody>+</table>+```
test/command/2649.md view
@@ -17,7 +17,7 @@ % pandoc -f mediawiki -t html5 {| border="4" cellspacing="2" cellpadding="0" WIDTH="100%" |------| peildatum Simbase || november 2005 || colspan=2 | '''uitslagen Flohrgambiet''' ||+| peildatum Simbase || november 2005 || colspan=2 | '''uitslagen Flohrgambiet''' |----- | totaal aantal partijen Simbase || 7.316.773 | wit wint || 53%
+ test/command/5081.md view
@@ -0,0 +1,8 @@+```+% pandoc -f rst+Go to `g`_ `g <www.example.com>`_.+^D+<p>Go to <a href="www.example.com">g</a> <a+href="www.example.com">g</a>.</p>++```
test/command/512.md view
@@ -37,7 +37,7 @@ __ link1_ ^D-2> [WARNING] Circular reference 'link1' at line 1 column 15+2> [WARNING] Circular reference 'link1' at line 8 column 1 <p><a href="">click here</a></p> ```
+ test/command/6588.md view
@@ -0,0 +1,10 @@+```+% pandoc -f rst+I recommend you try |Python|_.++.. |Python| replace:: Python, *the* best language around+.. _Python: http://www.python.org/+^D+<p>I recommend you try <a href="http://www.python.org/"><span>Python,+<em>the</em> best language around</span></a>.</p>+```
+ test/command/6992.md view
@@ -0,0 +1,76 @@+```+% pandoc -f mediawiki -t native+{| class="wikitable"+!colspan=4 | template request+|-+| mode || No || String || MUST be "template" or omitted+|}+^D+[ Table+ ( "" , [] , [] )+ (Caption Nothing [])+ [ ( AlignDefault , ColWidthDefault )+ , ( AlignDefault , ColWidthDefault )+ , ( AlignDefault , ColWidthDefault )+ , ( AlignDefault , ColWidthDefault )+ ]+ (TableHead+ ( "" , [] , [] )+ [ Row+ ( "" , [] , [] )+ [ Cell+ ( "" , [] , [] )+ AlignDefault+ (RowSpan 1)+ (ColSpan 4)+ [ Para [ Str "template" , Space , Str "request" ] ]+ ]+ ])+ [ TableBody+ ( "" , [] , [] )+ (RowHeadColumns 0)+ []+ [ Row+ ( "" , [] , [] )+ [ Cell+ ( "" , [] , [] )+ AlignDefault+ (RowSpan 1)+ (ColSpan 1)+ [ Para [ Str "mode" ] ]+ , Cell+ ( "" , [] , [] )+ AlignDefault+ (RowSpan 1)+ (ColSpan 1)+ [ Para [ Str "No" ] ]+ , Cell+ ( "" , [] , [] )+ AlignDefault+ (RowSpan 1)+ (ColSpan 1)+ [ Para [ Str "String" ] ]+ , Cell+ ( "" , [] , [] )+ AlignDefault+ (RowSpan 1)+ (ColSpan 1)+ [ Para+ [ Str "MUST"+ , Space+ , Str "be"+ , Space+ , Str "\"template\""+ , Space+ , Str "or"+ , Space+ , Str "omitted"+ ]+ ]+ ]+ ]+ ]+ (TableFoot ( "" , [] , [] ) [])+]++```
test/command/8966.md view
@@ -4,6 +4,6 @@ ^D #strong[Samsonov T.E.] Shape-Adaptive Geometric Simplification of Heterogeneous Line Datasets / T. E. Samsonov, O. P. Yakimova \/\/-International Journal of Geographical Information Science. — 2017. —-Vol. 31. — № 8. — pp.~1485-1520.+International Journal of Geographical Information Science. --- 2017. ---+Vol. 31. --- № 8. --- pp.~1485-1520. ```
+ test/command/9196.md view
@@ -0,0 +1,6 @@+```+% pandoc -f rst --wrap=none+Define _`my target`, then reference `my target`_.+^D+<p>Define <span id="my-target">my target</span>, then reference <a href="#my-target">my target</a>.</p>+```
test/command/bits-title-supress.md view
@@ -45,7 +45,7 @@ ``` % pandoc -f jats -t native <sec>- <title supress="no">The European Parliament</title>+ <title suppress="no">The European Parliament</title> <p>Members of the European Parliament (MEPs) are directly elected by EU citizens.</p> </sec> ^D@@ -89,7 +89,7 @@ ``` % pandoc -f jats -t native <sec>- <title supress="yes">The European Parliament</title>+ <title suppress="yes">The European Parliament</title> <p>Members of the European Parliament (MEPs) are directly elected by EU citizens.</p> </sec> ^D
+ test/command/jabberwocky.md view
@@ -0,0 +1,22 @@+```+% pandoc -t plain -f markdown+| 'Twas brillig, and the slithy toves+| Did gyre and gimble in the wabe:+| All mimsy were the borogoves,+| And the mome raths outgrabe.+|+| "Beware the Jabberwock, my son!+| The jaws that bite, the claws that catch!+| Beware the Jubjub bird, and shun+| The frumious Bandersnatch!"+^D+’Twas brillig, and the slithy toves+ Did gyre and gimble in the wabe:+All mimsy were the borogoves,+ And the mome raths outgrabe.++“Beware the Jabberwock, my son!+ The jaws that bite, the claws that catch!+Beware the Jubjub bird, and shun+ The frumious Bandersnatch!”+```
test/command/latex-color.md view
@@ -169,9 +169,12 @@ ^D [ Div ( "" , [] , [ ( "style" , "background-color: orange" ) ] )- [ BulletList- [ [ Para [ Str "Item" , Space , Str "1" ] ]- , [ Para [ Str "Item" , Space , Str "2" ] ]+ [ Div+ ( "" , [ "minipage" ] , [] )+ [ BulletList+ [ [ Para [ Str "Item" , Space , Str "1" ] ]+ , [ Para [ Str "Item" , Space , Str "2" ] ]+ ] ] ] ]@@ -190,9 +193,12 @@ ^D [ Div ( "" , [] , [ ( "style" , "background-color: blue" ) ] )- [ BulletList- [ [ Para [ Str "Item" , Space , Str "1" ] ]- , [ Para [ Str "Item" , Space , Str "2" ] ]+ [ Div+ ( "" , [ "minipage" ] , [] )+ [ BulletList+ [ [ Para [ Str "Item" , Space , Str "1" ] ]+ , [ Para [ Str "Item" , Space , Str "2" ] ]+ ] ] ] , Para
+ test/command/man-defines.md view
@@ -0,0 +1,15 @@+```+% pandoc -f man -t plain+.de test+ok+..+.test+.br+\A'test'+.br+\A'xyz'+^D+ok+1+0+```
+ test/command/mdoc-An.md view
@@ -0,0 +1,17 @@+```+% pandoc -f mdoc -t plain+.Sh DESCRIPTION+.An Emily Brontë ,+.An Anne Brontë+.Pp+.An -split+.An Charlotte Brontë+.An Bramwell Brontë+^D+DESCRIPTION++Emily Brontë, Anne Brontë++Charlotte Brontë+Bramwell Brontë+```
+ test/command/mdoc-Bd-unfilled.md view
@@ -0,0 +1,24 @@+```+% pandoc -f mdoc -t plain+.Bd -unfilled+’Twas brillig, and the slithy toves+ Did gyre and gimble in the wabe:+All mimsy were the borogoves,+ And the mome raths outgrabe.++“Beware the Jabberwock, my son!+ The jaws that bite, the claws that catch!+Beware the Jubjub bird, and shun+ The frumious Bandersnatch!”+.Ed+^D+’Twas brillig, and the slithy toves+ Did gyre and gimble in the wabe:+All mimsy were the borogoves,+ And the mome raths outgrabe.++“Beware the Jabberwock, my son!+ The jaws that bite, the claws that catch!+Beware the Jubjub bird, and shun+ The frumious Bandersnatch!”+```
+ test/command/mdoc-Bl-column.md view
@@ -0,0 +1,22 @@+```+% pandoc -f mdoc -t djot+.Bl -column a b+.It a Ta+.Sy b+b+.Ta c+.Pp+.It c Ta d Ta \&+.It Em e+.Ta f+.Ta g+.It h Ta Ta j+.It Pq a Ta b Ta c+.El+^D+| a | *b* b | c |+| c | d | |+| _e_ | f | g |+| h | | j |+| (a) | b | c |+```
+ test/command/mdoc-Bl-tag.md view
@@ -0,0 +1,36 @@+```+% pandoc -f mdoc -t native+.Bl -tag+.It one+one+.It Xo+two+.Xc+two+.It Em three Xo+three+.Xc+three+.It Po one two+.Bq three+.Pc+four+.El+^D+[ DefinitionList+ [ ( [ Str "one" ] , [ [ Para [ Str "one" ] ] ] )+ , ( [ Str "two" ] , [ [ Para [ Str "two" ] ] ] )+ , ( [ Emph [ Str "three" ] , Space , Str "three" ]+ , [ [ Para [ Str "three" ] ] ]+ )+ , ( [ Str "(one"+ , Space+ , Str "two"+ , Space+ , Str "[three])"+ ]+ , [ [ Para [ Str "four" ] ] ]+ )+ ]+]+```
+ test/command/rst_block_subst.md view
@@ -0,0 +1,20 @@+```+% pandoc -f rst -t native+|figure05|++.. |figure05| figure:: img/dummy.png++ capt+^D+[ Figure+ ( "" , [] , [] )+ (Caption Nothing [ Plain [ Str "capt" ] ])+ [ Plain+ [ Image+ ( "" , [] , [] )+ [ Str "img/dummy.png" ]+ ( "img/dummy.png" , "" )+ ]+ ]+]+```
test/docx/empty_field.native view
@@ -1,5 +1,5 @@ [Para- [Str "\24076\26395\28145\20837\20102\35299\30340\35835\32773\21487\20197\21435\30475David",Space,Str "French",Space,Str "Belding\21644Kevin",Space,Str "J.",Space,Str "Mitchell\30340"+ [Str "\24076\26395\28145\20837\20102\35299\30340\35835\32773\21487\20197\21435\30475David",Space,Str "French",Span ( "" , [ "indexref" ] , [ ( "entry" , "French" ) ] ) [],Space,Str "Belding\21644Kevin",Space,Str "J.",Space,Str "Mitchell\30340" ,Link ("",[],[]) [Str "Foundations",Space,Str "of",Space,Str "Analysis,",Space,Str "2nd",Space,Str "Edition"] ("https://books.google.com/books?id=sp_Zcb9ot90C&lpg=PR4&hl=zh-CN&pg=PA19#v=onepage&q&f=true",""),Str ",\21487\20174\&19\39029\30475\36215\65292\25110D.C.",Space,Str "Goldrei\30340",Space ,Link ("",[],[]) [Str "Classic",Space,Str "Set",Space,Str "Theory:",Space,Str "For",Space,Str "Guided",Space,Str "Independent",Space,Str "Study"] ("https://books.google.ae/books?id=dlc0DwAAQBAJ&lpg=PT29&hl=zh-CN&pg=PT26#v=onepage&q&f=true","") ,Str "\65292\20174\31532\20108\31456\30475\36215\65292\38405\35835\26102\35201\27880\24847\26412\25991\19982\36825\20123\20070\25152\19981\21516\30340\26159\24182\27809\26377\25226\23454\25968\30475\20316\26159\26377\29702\25968\38598\30340\20998\21106\12290"]
test/jats-reader.native view
@@ -536,7 +536,7 @@ , Space , Str "the" , Space- , Str "pragraph,"+ , Str "paragraph," , Space , Str "but" , Space@@ -568,7 +568,12 @@ ] , Para [ Strong- [ Str "Outside" , Space , Str "a" , Space , Str "pararaph" ]+ [ Str "Outside"+ , Space+ , Str "a"+ , Space+ , Str "paragraph"+ ] , Str "," , Space , Str "should"@@ -613,7 +618,7 @@ ] , Para [ Strong- [ Str "Inside" , Space , Str "a" , Space , Str "pragraph" ]+ [ Str "Inside" , Space , Str "a" , Space , Str "paragraph" ] , Str "," , Space , Str "should"
test/jats-reader.xml view
@@ -163,12 +163,12 @@ </sec> <sec id="disp-formulae"> <title>Block formulae with <italic>disp-formula</italic></title>- <p><bold>Inside a paragraph</bold>, should be natively coded inside the pragraph, but show as block:</p>+ <p><bold>Inside a paragraph</bold>, should be natively coded inside the paragraph, but show as block:</p> <p>Einstein showed that <disp-formula><alternatives> <tex-math><![CDATA[e=mc^2]]></tex-math> <mml:math display="block" xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>e</mml:mi><mml:mo>=</mml:mo><mml:mi>m</mml:mi><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:math></alternatives></disp-formula>. This formula is important because…</p>- <p><bold>Outside a pararaph</bold>, should show as block:</p>+ <p><bold>Outside a paragraph</bold>, should show as block:</p> <p>Einstein showed that </p> <disp-formula><alternatives> <tex-math><![CDATA[e=mc^2]]></tex-math>@@ -177,7 +177,7 @@ </sec> <sec id="no-formula-wrapper"> <title>Without a formula wrapper</title>- <p><bold>Inside a pragraph</bold>, should show inline:</p>+ <p><bold>Inside a paragraph</bold>, should show inline:</p> <p>Einstein showed that <alternatives> <tex-math><![CDATA[e=mc^2]]></tex-math> <mml:math display="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>e</mml:mi><mml:mo>=</mml:mo><mml:mi>m</mml:mi><mml:msup><mml:mi>c</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:mrow></mml:math></alternatives>.
test/rst-reader.native view
@@ -12,7 +12,7 @@ , MetaInlines [ Str "July" , Space , Str "17," , Space , Str "2006" ] )- , ( "revision" , MetaBlocks [ Para [ Str "3" ] ] )+ , ( "revision" , MetaInlines [ Str "3" ] ) , ( "subtitle" , MetaInlines [ Str "Subtitle" ] ) , ( "title" , MetaInlines@@ -598,11 +598,11 @@ , BlockQuote [ DefinitionList [ ( [ Str "address" ]- , [ [ Para [ Str "61" , Space , Str "Main" , Space , Str "St." ] ]+ , [ [ Plain [ Str "61" , Space , Str "Main" , Space , Str "St." ] ] ] ) , ( [ Str "city" ]- , [ [ Para+ , [ [ Plain [ Emph [ Str "Nowhere" ] , Str "," , Space@@ -613,16 +613,16 @@ ] ] )- , ( [ Str "phone" ] , [ [ Para [ Str "123-4567" ] ] ] )+ , ( [ Str "phone" ] , [ [ Plain [ Str "123-4567" ] ] ] ) ] ] , DefinitionList [ ( [ Str "address" ]- , [ [ Para [ Str "61" , Space , Str "Main" , Space , Str "St." ] ]+ , [ [ Plain [ Str "61" , Space , Str "Main" , Space , Str "St." ] ] ] ) , ( [ Str "city" ]- , [ [ Para+ , [ [ Plain [ Emph [ Str "Nowhere" ] , Str "," , Space@@ -633,7 +633,7 @@ ] ] )- , ( [ Str "phone" ] , [ [ Para [ Str "123-4567" ] ] ] )+ , ( [ Str "phone" ] , [ [ Plain [ Str "123-4567" ] ] ] ) ] , Header 1 ( "html-blocks" , [] , [] ) [ Str "HTML" , Space , Str "Blocks" ]
test/tables.typst view
@@ -55,10 +55,10 @@ aligned],), table.hline(), [First], [row], [12.0], [Example of a row that spans multiple lines.],- [Second], [row], [5.0], [Here’s another one. Note the blank line between+ [Second], [row], [5.0], [Here's another one. Note the blank line between rows.], )]- , caption: [Here’s the caption. It may span multiple lines.]+ , caption: [Here's the caption. It may span multiple lines.] , kind: table ) @@ -72,7 +72,7 @@ aligned],), table.hline(), [First], [row], [12.0], [Example of a row that spans multiple lines.],- [Second], [row], [5.0], [Here’s another one. Note the blank line between+ [Second], [row], [5.0], [Here's another one. Note the blank line between rows.], )] , kind: table@@ -98,7 +98,7 @@ columns: (15%, 13.75%, 16.25%, 35%), align: (center,left,right,auto,), [First], [row], [12.0], [Example of a row that spans multiple lines.],- [Second], [row], [5.0], [Here’s another one. Note the blank line between+ [Second], [row], [5.0], [Here's another one. Note the blank line between rows.], )] , kind: table
test/test-pandoc.hs view
@@ -29,6 +29,7 @@ import qualified Tests.Readers.RTF import qualified Tests.Readers.Txt2Tags import qualified Tests.Readers.Man+import qualified Tests.Readers.Mdoc import qualified Tests.Shared import qualified Tests.Writers.AsciiDoc import qualified Tests.Writers.ConTeXt@@ -97,6 +98,7 @@ , testGroup "Muse" Tests.Readers.Muse.tests , testGroup "Creole" Tests.Readers.Creole.tests , testGroup "Man" Tests.Readers.Man.tests+ , testGroup "Mdoc" Tests.Readers.Mdoc.tests , testGroup "FB2" Tests.Readers.FB2.tests , testGroup "DokuWiki" Tests.Readers.DokuWiki.tests ]
test/typst-reader.native view
@@ -1,7 +1,6 @@ Pandoc Meta { unMeta = fromList [] }- [ Para []- , Header+ [ Header 1 ( "" , [] , [] ) [ Str "Fibonacci" , Space , Str "sequence" ]@@ -46,7 +45,6 @@ DisplayMath "F_{n} = \\left\\lfloor {\\frac{1}{\\sqrt{5}}\\phi^{n}} \\right\\rceil,\\quad\\phi = \\frac{1 + \\sqrt{5}}{2}" ]- , Para [] , Para [ Str "The" , Space@@ -189,12 +187,6 @@ ] (TableFoot ( "" , [] , [] ) []) ]- , Para []- , Para []- , Para []- , Para []- , Para []- , Para [] , Div ( "" , [ "columns-flow" ] , [ ( "count" , "2" ) ] ) [ Para@@ -534,7 +526,7 @@ AlignDefault (RowSpan 1) (ColSpan 1)- [ Para [] ]+ [] , Cell ( "" , [] , [] ) AlignDefault@@ -1683,7 +1675,6 @@ , Code ( "" , [] , [] ) "overline(A)" , Str ")." ]- , Para [] , DefinitionList [ ( [ Str "Remark" ] , [ [ Para@@ -1832,7 +1823,7 @@ [ Para [ Span ( "" , [ "box" ] , [] )- [ Math InlineMath "f\\prime"+ [ Math InlineMath "f'" , Str "\8192" , Code ( "" , [] , [] ) "f'" , Str ","@@ -3194,7 +3185,7 @@ [ Para [ Span ( "" , [ "box" ] , [] )- [ Math InlineMath "\\preccurlyeq"+ [ Math InlineMath "\\preceq" , Str "\8192" , Code ( "" , [] , [] ) "prec.eq" ]@@ -3210,7 +3201,7 @@ ( "" , [ "box" ] , [] ) [ Math InlineMath "\\ominus" , Str "\8192"- , Code ( "" , [] , [] ) "minus.cirle"+ , Code ( "" , [] , [] ) "minus.circle" ] ] ]@@ -3287,7 +3278,7 @@ [ Para [ Span ( "" , [ "box" ] , [] )- [ Math InlineMath "\\succcurlyeq"+ [ Math InlineMath "\\succeq" , Str "\8192" , Code ( "" , [] , [] ) "succ.eq" ]@@ -5309,7 +5300,7 @@ AlignDefault (RowSpan 1) (ColSpan 1)- [ Para [] ]+ [] , Cell ( "" , [] , [] ) AlignDefault@@ -5347,7 +5338,6 @@ ] , Str "." ]- , Para [] , HorizontalRule , Para [ Str "johanvx"
test/undergradmath.typ view
@@ -259,7 +259,7 @@ cell($\u{224D}$, [`\u{224D}` @tricky]), cell($tilde.eq$, `tilde.eq`), cell($n!$, `n!`), cell($equiv$, `equiv`), cell($tilde.not$, `tilde.not`), cell($diff$, `diff`), cell($prec$, `prec`), cell($plus.circle$, `plus.circle`), cell($nabla$, `nabla`),- cell($prec.eq$, `prec.eq`), cell($minus.circle$, `minus.cirle`), cell($planck.reduce$, `planck.reduce`),+ cell($prec.eq$, `prec.eq`), cell($minus.circle$, `minus.circle`), cell($planck.reduce$, `planck.reduce`), cell($succ$, `succ`), cell($dot.circle$, `dot.circle`), cell($circle.stroked.tiny$, `circle.stroked.tiny`), cell($succ.eq$, `succ.eq`), cell($times.circle$, `times.circle`), cell($star$, `star`), cell($prop$, `prop`), cell($\u{2298}$, [`\u{2298}` @tricky]), cell($sqrt("")$, `sqrt("")`),
test/writer.typst view
@@ -1,11 +1,5 @@-// Some definitions presupposed by pandoc's typst output.-#let horizontalrule = [- #line(start: (25%,0%), end: (75%,0%))-]+#let horizontalrule = line(start: (25%,0%), end: (75%,0%)) -#let endnote(num, contents) = [- #stack(dir: ltr, spacing: 3pt, super[#num], contents)-] #show terms: it => { it.children .map(child => [@@ -54,6 +48,7 @@ font: (), fontsize: 11pt, sectionnumbering: none,+ pagenumbering: "1", doc, ) = { set document(@@ -64,8 +59,9 @@ set page( paper: paper, margin: margin,- numbering: "1",- )+ numbering: pagenumbering,+ columns: cols,+ ) set par(justify: true) set text(lang: lang, region: region,@@ -73,7 +69,8 @@ size: fontsize) set heading(numbering: sectionnumbering) - if title != none {+ place(top, float: true, scope: "parent", clearance: 4mm)[+ #if title != none { align(center)[#block(inset: 2em)[ #text(weight: "bold", size: 1.5em)[#title] #(if subtitle != none {@@ -83,7 +80,7 @@ ]] } - if authors != none and authors != [] {+ #if authors != none and authors != [] { let count = authors.len() let ncols = calc.min(count, 3) grid(@@ -99,23 +96,20 @@ ) } - if date != none {+ #if date != none { align(center)[#block(inset: 1em)[ #date ]] } - if abstract != none {+ #if abstract != none { block(inset: 2em)[ #text(weight: "semibold")[Abstract] #h(1em) #abstract ] }+ ] - if cols == 1 {- doc- } else {- columns(cols, doc)- }+ doc } #show: doc => conf( title: [Pandoc Test Suite],@@ -128,12 +122,13 @@ email: "" ), ), date: [July 17, 2006],+ pagenumbering: "1", cols: 1, doc, ) -This is a set of tests for pandoc. Most of them are adapted from John Gruber’s+This is a set of tests for pandoc. Most of them are adapted from John Gruber's markdown test suite. #horizontalrule@@ -164,13 +159,13 @@ = Paragraphs <paragraphs>-Here’s a regular paragraph.+Here's a regular paragraph. In Markdown 1.0.0 and earlier. Version 8. This line turns into a list item. Because a hard-wrapped line in the middle of a paragraph looked like a list item. -Here’s one with a bullet. \* criminey.+Here's one with a bullet. \* criminey. There should be a hard line break \ here.@@ -320,7 +315,7 @@ + Item 1, graf one. - Item 1. graf two. The quick brown fox jumped over the lazy dog’s back.+ Item 1. graf two. The quick brown fox jumped over the lazy dog's back. + Item 2. @@ -332,7 +327,7 @@ - Tab - Tab -Here’s another:+Here's another: + First + Second:@@ -549,7 +544,7 @@ This is #emph[emphasized] And this is #strong[strong]-Here’s a simple block:+Here's a simple block: #block[ foo@@ -596,7 +591,7 @@ <hr /> ``` -Hr’s:+Hr's: #horizontalrule @@ -637,14 +632,14 @@ 'Oak,' 'elm,' and 'beech' are names of trees. So is 'pine.' -'He said, "I want to go."' Were you alive in the 70’s?+'He said, "I want to go."' Were you alive in the 70's? Here is some quoted '`code`' and a "#link("http://example.com/?foo=1&bar=2")[quoted link];". -Some dashes: one—two — three—four — five.+Some dashes: one---two --- three---four --- five. -Dashes between numbers: 5–7, 255–66, 1987–1999.+Dashes between numbers: 5--7, 255--66, 1987--1999. Ellipses…and…and…. @@ -658,11 +653,11 @@ - $alpha and omega$ - $223$ - $p$-Tree-- Here’s some display math:+- Here's some display math: $ frac(d, d x) f (x) = lim_(h arrow.r 0) frac(f (x + h) - f (x), h) $-- Here’s one that has a line break in it: $alpha + omega times x^2$.+- Here's one that has a line break in it: $alpha + omega times x^2$. -These shouldn’t be math:+These shouldn't be math: - To get the famous equation, write `$e = mc^2$`. - \$22,000 is a #emph[lot] of money. So is \$34,000. (It worked if "lot" is@@ -670,7 +665,7 @@ - Shoes (\$20) and socks (\$5). - Escaped `$`: \$73 #emph[this should be emphasized] 23\$. -Here’s a LaTeX table:+Here's a LaTeX table: #horizontalrule @@ -776,14 +771,14 @@ == With ampersands <with-ampersands>-Here’s a+Here's a #link("http://example.com/?foo=1&bar=2")[link with an ampersand in the URL];. -Here’s a link with an amersand in the link text: #link("http://att.com/")[AT&T];.+Here's a link with an amersand in the link text: #link("http://att.com/")[AT&T];. -Here’s an #link("/script?foo=1&bar=2")[inline link];.+Here's an #link("/script?foo=1&bar=2")[inline link];. -Here’s an #link("/script?foo=1&bar=2")[inline link in pointy braces];.+Here's an #link("/script?foo=1&bar=2")[inline link in pointy braces];. == Autolinks <autolinks>@@ -825,7 +820,7 @@ <footnotes> Here is a footnote reference,#footnote[Here is the footnote. It can go anywhere after the footnote reference. It need not be placed at the end of the document.]-and another.#footnote[Here’s the long note. This one contains multiple blocks.+and another.#footnote[Here's the long note. This one contains multiple blocks. Subsequent blocks are indented to show that they belong to the footnote (as with list items).