diff --git a/MANUAL.txt b/MANUAL.txt --- a/MANUAL.txt +++ b/MANUAL.txt @@ -1,7 +1,7 @@ --- title: Pandoc User's Guide author: John MacFarlane -date: October 20, 2020 +date: November 3, 2020 --- # Synopsis @@ -665,18 +665,18 @@ : Specifies what to do with insertions, deletions, and comments produced by the MS Word "Track Changes" feature. `accept` (the - default), inserts all insertions, and ignores all - deletions. `reject` inserts all deletions and ignores - insertions. Both `accept` and `reject` ignore comments. `all` puts - in insertions, deletions, and comments, wrapped in spans with - `insertion`, `deletion`, `comment-start`, and `comment-end` - classes, respectively. The author and time of change is - included. `all` is useful for scripting: only accepting changes - from a certain reviewer, say, or before a certain date. If a - paragraph is inserted or deleted, `track-changes=all` produces a - span with the class `paragraph-insertion`/`paragraph-deletion` - before the affected paragraph break. This option only affects the - docx reader. + default) processes all the insertions and deletions. + `reject` ignores them. Both `accept` and `reject` ignore comments. + `all` includes all insertions, deletions, and comments, wrapped + in spans with `insertion`, `deletion`, `comment-start`, and + `comment-end` classes, respectively. The author and time of + change is included. `all` is useful for scripting: only + accepting changes from a certain reviewer, say, or before a + certain date. If a paragraph is inserted or deleted, + `track-changes=all` produces a span with the class + `paragraph-insertion`/`paragraph-deletion` before the + affected paragraph break. This option only affects the docx + reader. `--extract-media=`*DIR* @@ -1533,6 +1533,7 @@ bibliography: - foobar.bib - barbaz.json +citation-abbreviations: abbrevs.json # Filters will be assumed to be Lua filters if they have # the .lua extension, and json filters otherwise. But @@ -2189,7 +2190,7 @@ ### Variables for HTML `document-css` -: Enables inclusion of most of the CSS in the `styles.html` +: Enables inclusion of most of the [CSS] in the `styles.html` [partial](#partials) (have a look with `pandoc --print-default-data-file=templates/styles.html`). Unless you use [`--css`](#option--css), this variable @@ -2213,6 +2214,10 @@ `monofont` : sets the CSS `font-family` property on `code` elements. +`monobackgroundcolor` +: sets the CSS `background-color` property on `code` elements + and adds extra padding. + `linestretch` : sets the CSS `line-height` property on the `html` element, which is preferred to be unitless. @@ -2222,6 +2227,28 @@ `margin-left`, `margin-right`, `margin-top`, `margin-bottom` : sets the corresponding CSS `padding` properties on the `body` element. + +To override or extend some [CSS] for just one document, include for example: + + --- + header-includes: | + + --- + +[CSS]: https://developer.mozilla.org/en-US/docs/Learn/CSS ### Variables for HTML math diff --git a/cabal.project b/cabal.project --- a/cabal.project +++ b/cabal.project @@ -4,8 +4,8 @@ flags: +embed_data_files -trypandoc ghc-options: -j +RTS -A64m -RTS --- source-repository-package --- type: git --- location: https://github.com/jgm/citeproc --- tag: 0.1.0.3 +source-repository-package + type: git + location: https://github.com/jgm/citeproc + tag: 1860f189e9995c1dc27a68893bedfbf8de1ee67f diff --git a/changelog.md b/changelog.md --- a/changelog.md +++ b/changelog.md @@ -1,5 +1,110 @@ # Revision history for pandoc +## pandoc 2.11.1 (2020-11-03) + + * DocBook Reader: fix duplicate bibliography bug (#6773, Nils Carlson). + + * HTML reader: + + + Parse contents of iframes (#6770). + + Parse inline svg as image unless `raw_html` is set in the reader (in + which case the svg is passed through as raw HTML) (#6770). + + * LaTeX reader: + + + Fix bug parsing macro arguments (#6796). If `\cL` is + defined as `\mathcal{L}`, and `\til` as `\tilde{#1}`, then + `\til\cL` should expand to `\tilde{\mathcal{L}}`, but + pandoc was expanding it to `\tilde\mathcal{L}`. This is + fixed by parsing the arguments in "verbatim mode" when the + macro expands arguments at the point of use. + + Properly support optional (cite) argument for + `\blockquote` from `csquotes` (#6802). + + * LaTeX writer: Improved calculation of table column widths. + We now have LaTeX do the calculation, using `\tabcolsep`. + So we should now have accurate relative column widths no + matter what the text width. The default template has been modified to load + the calc package if tables are used. + + * HTML writer: Fix duplicate "class" attribute for table + rows (Andy Morris). + + * Text.Pandoc.Filter: allow shorter YAML representation of Citeproc + (Albert Krewinkel). The map-based YAML representation of filters expects + `type` and `path` fields. The path field had to be present for all filter + types, but is not used for citeproc filters. The field can now be omitted + when type is "citeproc", as described in the MANUAL. + + * Text.Pandoc.Error: Add `PandocBibliographyError` constructor + for `PandocError` [API change]. This ensures that bibliography parsing + errors generate messages that include the bibliography file name -- + otherwise it can be quite mysterious where it is coming from. + + * Citeproc: properly handle `csl` field with `data:` URI (#6783). + This is used with the JATS writer, so this fixes a regression + in pandoc 2.11 with JATS output and citeproc. + + * Allow `citation-abbreviations` in defaults file. + + * JATS templates: ensure `jats_publishing` output is valid + (Albert Krewinkel). + + * LaTeX template: Fix `CSLRightInline`, so that it does not + run over the right margin. + + * HTML template: default CSS tweaks (Mauro Bieg and John + MacFarlane). + + - Fix margin before codeblock + - Add `monobackgroundcolor` variable, making the background color + and padding of code optional. + - Ensure that backgrounds from highlighting styles take precedence over + monobackgroundcolor + - Remove list markers from TOC + - Add margin-bottom where needed + - Remove italics from blockquote styling + - Change borders and spacing in tables to be more consistent with other + output formats + - Style h5, h6 + - Set font-size for print media to 12pt. + - Reduce interline space. + - Reduce interparagraph space. + - Reduce line width. + - Remove the special `line-height: 1` for table cells. + - Remove the special line-height for pre. + - Ensure that there is a bit more space before a heading + than after. + - Slightly reduced space after title header. + - Add CSS example to MANUAL + + * man template: Change comment that triggers `tbl` from + `.\"t` to `'\" t`, as specified in groff_man(7) (#6803). + + * Use latest commonmark, commonmark-extensions. + This fixes a bug with nested blocks in footnotes with the + `footnote` extension to `commonmark`. See jgm/commonmark-hs#63. + + * Citeproc: use comma for in-text citations inside footnotes. + When an author-in-text citation like `@foo` occurs in a footnote, + we now render it with: `AUTHOR NAME + COMMA + SPACE + REST`. + Previously we rendered: `AUTHOR NAME + SPACE + "(" + REST + ")"`. + This gives better results. Note that normal citations are still + rendered in parentheses. + + * Use latest citeproc: + + + citeproc no longer capitalizes notes, so we do it + in pandoc when appropriate. + + Closes #6783. + + * Clarify manual on `--track-changes` (#6801). + + * Add `doc/jats.md` to document pandoc's handling of JATS + (#6794, Albert Krewinkel). + + * Fix code example in lua-filters.md (#6795). + ## pandoc 2.11.0.4 (2020-10-21) * Commonmark writer: fix regression with fenced divs (#6768). diff --git a/data/templates/article.jats_publishing b/data/templates/article.jats_publishing --- a/data/templates/article.jats_publishing +++ b/data/templates/article.jats_publishing @@ -14,6 +14,13 @@ $if(journal.pmc)$ $journal.pmc$ $endif$ +$-- Fallback: an empty journal-id in case none is available. +$if(journal.publisher-id)$ +$elseif(journal.nlm-ta)$ +$elseif(journal.pmc)$ +$else$ + +$endif$ $if(journal.title)$ $journal.title$ @@ -27,6 +34,12 @@ $endif$ $if(journal.eissn)$ $journal.eissn$ +$endif$ +$-- At least one issn element is required; use empty issn as fallback +$if(journal.pissn)$ +$elseif(journal.eissn)$ +$else$ + $endif$ $journal.publisher-name$ diff --git a/data/templates/default.latex b/data/templates/default.latex --- a/data/templates/default.latex +++ b/data/templates/default.latex @@ -257,6 +257,7 @@ $endif$ $if(tables)$ \usepackage{longtable,booktabs} +\usepackage{calc} % for calculating minipage widths $if(beamer)$ \usepackage{caption} % Make caption package work with longtable @@ -383,10 +384,10 @@ \fi }% {} -\usepackage{calc} % for \widthof, \maxof +\usepackage{calc} \newcommand{\CSLBlock}[1]{#1\hfill\break} -\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\maxof{\widthof{#1}}{\csllabelwidth}}{#1}} -\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth}{#1}} +\newcommand{\CSLLeftMargin}[1]{\parbox[t]{\csllabelwidth}{#1}} +\newcommand{\CSLRightInline}[1]{\parbox[t]{\linewidth - \csllabelwidth}{#1}} \newcommand{\CSLIndent}[1]{\hspace{\cslhangindent}#1} $endif$ diff --git a/data/templates/default.man b/data/templates/default.man --- a/data/templates/default.man +++ b/data/templates/default.man @@ -1,5 +1,5 @@ $if(has-tables)$ -.\"t +'\" t $endif$ $if(pandoc-version)$ .\" Automatically generated by Pandoc $pandoc-version$ diff --git a/data/templates/styles.html b/data/templates/styles.html --- a/data/templates/styles.html +++ b/data/templates/styles.html @@ -1,6 +1,6 @@ $if(document-css)$ html { - line-height: $if(linestretch)$$linestretch$$else$1.7$endif$; + line-height: $if(linestretch)$$linestretch$$else$1.5$endif$; font-family: $if(mainfont)$$mainfont$$else$Georgia, serif$endif$; font-size: $if(fontsize)$$fontsize$$else$20px$endif$; color: $if(fontcolor)$$fontcolor$$else$#1a1a1a$endif$; @@ -8,7 +8,7 @@ } body { margin: 0 auto; - max-width: 40em; + max-width: 36em; padding-left: $if(margin-left)$$margin-left$$else$50px$endif$; padding-right: $if(margin-right)$$margin-right$$else$50px$endif$; padding-top: $if(margin-top)$$margin-top$$else$50px$endif$; @@ -28,6 +28,7 @@ body { background-color: transparent; color: black; + font-size: 12pt; } p, h2, h3 { orphans: 3; @@ -38,7 +39,7 @@ } } p { - margin-top: 1.7em; + margin: 1em 0; } a { color: $if(linkcolor)$$linkcolor$$else$#1a1a1a$endif$; @@ -50,59 +51,90 @@ max-width: 100%; } h1, h2, h3, h4, h5, h6 { - margin-top: 1.7em; + margin-top: 1.4em; } +h5, h6 { + font-size: 1em; + font-style: italic; +} +h6 { + font-weight: normal; +} ol, ul { padding-left: 1.7em; - margin-top: 1.7em; + margin-top: 1em; } li > ol, li > ul { margin-top: 0; } blockquote { - margin: 1.7em 0 1.7em 1.7em; + margin: 1em 0 1em 1.7em; padding-left: 1em; border-left: 2px solid #e6e6e6; - font-style: italic; + color: #606060; } code { font-family: $if(monofont)$$monofont$$else$Menlo, Monaco, 'Lucida Console', Consolas, monospace$endif$; - background-color: #f0f0f0; +$if(monobackgroundcolor)$ + background-color: $monobackgroundcolor$; + padding: .2em .4em; +$endif$ font-size: 85%; margin: 0; - padding: .2em .4em; } pre { - line-height: 1.5em; + margin: 1em 0; +$if(monobackgroundcolor)$ + background-color: $monobackgroundcolor$; padding: 1em; - background-color: #f0f0f0; +$endif$ overflow: auto; } pre code { padding: 0; overflow: visible; } +.sourceCode { + background-color: transparent; + overflow: visible; +} hr { background-color: #1a1a1a; border: none; height: 1px; - margin-top: 1.7em; + margin: 1em 0; } table { + margin: 1em 0; border-collapse: collapse; width: 100%; overflow-x: auto; display: block; + font-variant-numeric: lining-nums tabular-nums; } -th, td { - border-bottom: 1px solid lightgray; - padding: 1em 3em 1em 0; +table caption { + margin-bottom: 0.75em; } +tbody { + margin-top: 0.5em; + border-top: 1px solid $if(fontcolor)$$fontcolor$$else$#1a1a1a$endif$; + border-bottom: 1px solid $if(fontcolor)$$fontcolor$$else$#1a1a1a$endif$; +} +th { + border-top: 1px solid $if(fontcolor)$$fontcolor$$else$#1a1a1a$endif$; + padding: 0.25em 0.5em 0.25em 0.5em; +} +td { + padding: 0.125em 0.5em 0.25em 0.5em; +} header { - margin-bottom: 6em; + margin-bottom: 4em; text-align: center; } -nav a:not(:hover) { +#TOC li { + list-style: none; +} +#TOC a:not(:hover) { text-decoration: none; } $endif$ diff --git a/man/pandoc.1 b/man/pandoc.1 --- a/man/pandoc.1 +++ b/man/pandoc.1 @@ -1,7 +1,7 @@ -.\"t -.\" Automatically generated by Pandoc 2.11.0.4 +'\" t +.\" Automatically generated by Pandoc 2.11.1 .\" -.TH "Pandoc User\[cq]s Guide" "" "October 20, 2020" "pandoc 2.11.0.4" "" +.TH "Pandoc User\[cq]s Guide" "" "November 3, 2020" "pandoc 2.11.1" "" .hy .SH NAME pandoc - general markup converter @@ -704,12 +704,12 @@ \f[B]\f[CB]--track-changes=accept\f[B]\f[R]|\f[B]\f[CB]reject\f[B]\f[R]|\f[B]\f[CB]all\f[B]\f[R] Specifies what to do with insertions, deletions, and comments produced by the MS Word \[lq]Track Changes\[rq] feature. -\f[C]accept\f[R] (the default), inserts all insertions, and ignores all +\f[C]accept\f[R] (the default) processes all the insertions and deletions. -\f[C]reject\f[R] inserts all deletions and ignores insertions. +\f[C]reject\f[R] ignores them. Both \f[C]accept\f[R] and \f[C]reject\f[R] ignore comments. -\f[C]all\f[R] puts in insertions, deletions, and comments, wrapped in -spans with \f[C]insertion\f[R], \f[C]deletion\f[R], +\f[C]all\f[R] includes all insertions, deletions, and comments, wrapped +in spans with \f[C]insertion\f[R], \f[C]deletion\f[R], \f[C]comment-start\f[R], and \f[C]comment-end\f[R] classes, respectively. The author and time of change is included. @@ -1748,6 +1748,7 @@ bibliography: - foobar.bib - barbaz.json +citation-abbreviations: abbrevs.json # Filters will be assumed to be Lua filters if they have # the .lua extension, and json filters otherwise. But @@ -2489,6 +2490,10 @@ \f[B]\f[CB]monofont\f[B]\f[R] sets the CSS \f[C]font-family\f[R] property on \f[C]code\f[R] elements. .TP +\f[B]\f[CB]monobackgroundcolor\f[B]\f[R] +sets the CSS \f[C]background-color\f[R] property on \f[C]code\f[R] +elements and adds extra padding. +.TP \f[B]\f[CB]linestretch\f[B]\f[R] sets the CSS \f[C]line-height\f[R] property on the \f[C]html\f[R] element, which is preferred to be unitless. @@ -2500,6 +2505,31 @@ \f[B]\f[CB]margin-left\f[B]\f[R], \f[B]\f[CB]margin-right\f[B]\f[R], \f[B]\f[CB]margin-top\f[B]\f[R], \f[B]\f[CB]margin-bottom\f[B]\f[R] sets the corresponding CSS \f[C]padding\f[R] properties on the \f[C]body\f[R] element. +.PP +To override or extend some CSS for just one document, include for +example: +.IP +.nf +\f[C] +--- +header-includes: | + +--- +\f[R] +.fi .SS Variables for HTML math .TP \f[B]\f[CB]classoption\f[B]\f[R] diff --git a/pandoc.cabal b/pandoc.cabal --- a/pandoc.cabal +++ b/pandoc.cabal @@ -1,6 +1,6 @@ cabal-version: 2.2 name: pandoc -version: 2.11.0.4 +version: 2.11.1 build-type: Simple license: GPL-2.0-or-later license-file: COPYING.md @@ -405,8 +405,8 @@ bytestring >= 0.9 && < 0.12, case-insensitive >= 1.2 && < 1.3, citeproc >= 0.1.0.3 && < 0.2, - commonmark >= 0.1.0.2 && < 0.2, - commonmark-extensions >= 0.2 && < 0.3, + commonmark >= 0.1.1 && < 0.2, + commonmark-extensions >= 0.2.0.2 && < 0.3, commonmark-pandoc >= 0.2 && < 0.3, connection >= 0.3.1, containers >= 0.4.2.1 && < 0.7, diff --git a/src/Text/Pandoc/App/Opt.hs b/src/Text/Pandoc/App/Opt.hs --- a/src/Text/Pandoc/App/Opt.hs +++ b/src/Text/Pandoc/App/Opt.hs @@ -367,6 +367,11 @@ foldr addItem o xs) <|> (parseYAML v >>= \(x :: Text) -> return $ \o -> addItem x o) + "citation-abbreviations" -> + parseYAML v >>= \x -> + return (\o -> o{ optMetadata = + addMeta "citation-abbreviations" (T.unpack x) + (optMetadata o) }) "ipynb-output" -> parseYAML v >>= \x -> return (\o -> o{ optIpynbOutput = x }) "include-before-body" -> diff --git a/src/Text/Pandoc/Citeproc.hs b/src/Text/Pandoc/Citeproc.hs --- a/src/Text/Pandoc/Citeproc.hs +++ b/src/Text/Pandoc/Citeproc.hs @@ -28,7 +28,7 @@ import Text.Pandoc (PandocMonad(..), PandocError(..), readDataFile, ReaderOptions(..), pandocExtensions, report, LogMessage(..), fetchItem) -import Text.Pandoc.Shared (stringify, ordNub, blocksToInlines) +import Text.Pandoc.Shared (stringify, ordNub, blocksToInlines, tshow) import qualified Text.Pandoc.UTF8 as UTF8 import Data.Aeson (eitherDecode) import Data.Default @@ -60,7 +60,7 @@ setResourcePath $ oldRp ++ maybe [] (\u -> [u <> "/csl", u <> "/csl/dependent"]) mbUdd - let fp' = if T.any (=='.') fp + let fp' = if T.any (=='.') fp || "data:" `T.isPrefixOf` fp then fp else fp <> defaultExtension (result, _) <- fetchItem fp' @@ -90,7 +90,6 @@ UTF8.toText <$> catchError (getFile ".csl" basename) (\_ -> fst <$> fetchItem url) - -- TODO check .csl directory if not found styleRes <- Citeproc.parseStyle getParentStyle cslContents style <- case styleRes of @@ -164,8 +163,10 @@ _ -> id let Pandoc meta'' bs' = - maybe id (setMeta "nocite") metanocites $ - walk (fixQuotes . mvPunct moveNotes locale) $ walk deNote $ + maybe id (setMeta "nocite") metanocites . + walk (map capitalizeNoteCitation . + fixQuotes . mvPunct moveNotes locale) . + walk deNote . evalState (walkM insertResolvedCitations $ Pandoc meta' bs) $ cits return $ Pandoc meta'' @@ -206,16 +207,18 @@ -> Maybe FilePath -> ByteString -> m [Reference Inlines] -getRefs locale format idpred mbfp raw = +getRefs locale format idpred mbfp raw = do + let err' = throwError . + PandocBibliographyError (maybe mempty T.pack mbfp) case format of Format_bibtex -> - either (throwError . PandocAppError . T.pack . show) return . + either (err' . tshow) return . readBibtexString Bibtex locale idpred . UTF8.toText $ raw Format_biblatex -> - either (throwError . PandocAppError . T.pack . show) return . + either (err' . tshow) return . readBibtexString Biblatex locale idpred . UTF8.toText $ raw Format_json -> - either (throwError . PandocAppError . T.pack) + either (err' . T.pack) (return . filter (idpred . unItemId . referenceId)) . cslJsonToReferences $ raw Format_yaml -> do @@ -514,15 +517,33 @@ extractText (NumVal n) = T.pack (show n) extractText _ = mempty -deNote :: Inline -> Inline -deNote (Note bs) = Note $ walk go bs +capitalizeNoteCitation :: Inline -> Inline +capitalizeNoteCitation (Cite cs [Note [Para ils]]) = + Cite cs + [Note [Para $ B.toList $ addTextCase Nothing CapitalizeFirst + $ B.fromList ils]] +capitalizeNoteCitation x = x + +deNote :: [Inline] -> [Inline] +deNote [] = [] +deNote (Note bs:rest) = + Note (walk go bs) : deNote rest where - go (Note bs') - = Span ("",[],[]) (Space : Str "(" : - (removeFinalPeriod - (blocksToInlines bs')) ++ [Str ")"]) + go (Cite (c:cs) ils) + | citationMode c == AuthorInText + = Cite cs (concatMap noteAfterComma ils) + | otherwise + = Cite cs (concatMap noteInParens ils) go x = x -deNote x = x + noteInParens (Note bs') + = Space : Str "(" : + removeFinalPeriod (blocksToInlines bs') ++ [Str ")"] + noteInParens x = [x] + noteAfterComma (Note bs') + = Str "," : Space : + removeFinalPeriod (blocksToInlines bs') + noteAfterComma x = [x] +deNote (x:xs) = x : deNote xs -- Note: we can't use dropTextWhileEnd indiscriminately, -- because this would remove the final period on abbreviations like Ibid. diff --git a/src/Text/Pandoc/Error.hs b/src/Text/Pandoc/Error.hs --- a/src/Text/Pandoc/Error.hs +++ b/src/Text/Pandoc/Error.hs @@ -62,6 +62,7 @@ | PandocUnknownWriterError Text | PandocUnsupportedExtensionError Text Text | PandocCiteprocError CiteprocError + | PandocBibliographyError Text Text deriving (Show, Typeable, Generic) instance Exception PandocError @@ -143,6 +144,8 @@ "for " <> f PandocCiteprocError e' -> err 24 $ prettyCiteprocError e' + PandocBibliographyError fp msg -> err 25 $ + "Error reading bibliography file " <> fp <> ":\n" <> msg err :: Int -> Text -> IO a err exitCode msg = do diff --git a/src/Text/Pandoc/Filter.hs b/src/Text/Pandoc/Filter.hs --- a/src/Text/Pandoc/Filter.hs +++ b/src/Text/Pandoc/Filter.hs @@ -47,11 +47,13 @@ parseYAML node = (withMap "Filter" $ \m -> do ty <- m .: "type" - fp <- m .: "path" + fp <- m .:? "path" + let missingPath = fail $ "Expected 'path' for filter of type " ++ show ty + let filterWithPath constr = maybe missingPath (return . constr . T.unpack) case ty of "citeproc" -> return CiteprocFilter - "lua" -> return $ LuaFilter $ T.unpack fp - "json" -> return $ JSONFilter $ T.unpack fp + "lua" -> filterWithPath LuaFilter fp + "json" -> filterWithPath JSONFilter fp _ -> fail $ "Unknown filter type " ++ show (ty :: T.Text)) node <|> (withStr "Filter" $ \t -> do diff --git a/src/Text/Pandoc/Readers/DocBook.hs b/src/Text/Pandoc/Readers/DocBook.hs --- a/src/Text/Pandoc/Readers/DocBook.hs +++ b/src/Text/Pandoc/Readers/DocBook.hs @@ -794,11 +794,10 @@ "titleabbrev" -> skip "authorinitials" -> skip "bibliography" -> sect 0 - "bibliodiv" -> do - tit <- case filterChild (named "title") e of - Just _ -> sect 1 - Nothing -> return mempty - (tit <>) <$> parseMixed para (elContent e) + "bibliodiv" -> + case filterChild (named "title") e of + Just _ -> sect 1 + Nothing -> return mempty "biblioentry" -> parseMixed para (elContent e) "bibliomisc" -> parseMixed para (elContent e) "bibliomixed" -> parseMixed para (elContent e) diff --git a/src/Text/Pandoc/Readers/HTML.hs b/src/Text/Pandoc/Readers/HTML.hs --- a/src/Text/Pandoc/Readers/HTML.hs +++ b/src/Text/Pandoc/Readers/HTML.hs @@ -47,6 +47,7 @@ import qualified Text.Pandoc.Builder as B import Text.Pandoc.Class.PandocMonad (PandocMonad (..)) import Text.Pandoc.CSS (foldOrElse, pickStyleAttrProps) +import qualified Text.Pandoc.UTF8 as UTF8 import Text.Pandoc.Definition import Text.Pandoc.Readers.LaTeX (rawLaTeXInline) import Text.Pandoc.Readers.LaTeX.Types (Macro) @@ -65,6 +66,7 @@ import Text.Pandoc.Walk import Text.Parsec.Error import Text.TeXMath (readMathML, writeTeX) +import Data.ByteString.Base64 (encode) -- | Convert HTML-formatted string to 'Pandoc' document. readHtml :: PandocMonad m @@ -87,7 +89,7 @@ result <- flip runReaderT def $ runParserT parseDoc (HTMLState def{ stateOptions = opts } - [] Nothing Set.empty [] M.empty) + [] Nothing Set.empty [] M.empty opts) "source" tags case result of Right doc -> return doc @@ -110,7 +112,8 @@ baseHref :: Maybe URI, identifiers :: Set.Set Text, logMessages :: [LogMessage], - macros :: M.Map Text Macro + macros :: M.Map Text Macro, + readerOpts :: ReaderOptions } data HTMLLocal = HTMLLocal { quoteContext :: QuoteContext @@ -183,6 +186,7 @@ , pDiv , pPlain , pFigure + , pIframe , pRawHtmlBlock ] trace (T.take 60 $ tshow $ B.toList res) @@ -399,6 +403,18 @@ else kvs return $ B.divWith (ident, classes', kvs') contents +pIframe :: PandocMonad m => TagParser m Blocks +pIframe = try $ do + guardDisabled Ext_raw_html + tag <- pSatisfy (tagOpen (=="iframe") (isJust . lookup "src")) + pCloses "iframe" <|> eof + url <- canonicalizeUrl $ fromAttrib "src" tag + (bs, _) <- openURL url + let inp = UTF8.toText bs + opts <- readerOpts <$> getState + Pandoc _ contents <- readHtml opts inp + return $ B.divWith ("",["iframe"],[]) $ B.fromList contents + pRawHtmlBlock :: PandocMonad m => TagParser m Blocks pRawHtmlBlock = do raw <- pHtmlBlock "script" <|> pHtmlBlock "style" <|> pHtmlBlock "textarea" @@ -655,6 +671,7 @@ , pLineBreak , pLink , pImage + , pSvg , pBdo , pCode , pCodeWithClass [("samp","sample"),("var","variable")] @@ -792,6 +809,19 @@ v -> [(k, v)] let kvs = concatMap getAtt ["width", "height", "sizes", "srcset"] return $ B.imageWith (uid, cls, kvs) (escapeURI url) title (B.text alt) + +pSvg :: PandocMonad m => TagParser m Inlines +pSvg = do + guardDisabled Ext_raw_html + -- if raw_html enabled, parse svg tag as raw + opent@(TagOpen _ attr') <- pSatisfy (matchTagOpen "svg" []) + let (ident,cls,_) = toAttr attr' + contents <- many (notFollowedBy (pCloses "svg") >> pAny) + closet <- TagClose "svg" <$ (pCloses "svg" <|> eof) + let rawText = T.strip $ renderTags' (opent : contents ++ [closet]) + let svgData = "data:image/svg+xml;base64," <> + UTF8.toText (encode $ UTF8.fromText rawText) + return $ B.imageWith (ident,cls,[]) svgData mempty mempty pCodeWithClass :: PandocMonad m => [(T.Text,Text)] -> TagParser m Inlines pCodeWithClass elemToClass = try $ do diff --git a/src/Text/Pandoc/Readers/LaTeX.hs b/src/Text/Pandoc/Readers/LaTeX.hs --- a/src/Text/Pandoc/Readers/LaTeX.hs +++ b/src/Text/Pandoc/Readers/LaTeX.hs @@ -309,18 +309,19 @@ else doubleQuoted . langspan <$> withQuoteContext InDoubleQuote tok blockquote :: PandocMonad m => Bool -> Maybe Text -> LP m Blocks -blockquote citations mblang = do - citePar <- if citations - then do - cs <- cites NormalCitation False - return $ para (cite cs mempty) - else return mempty +blockquote cvariant mblang = do + citepar <- if cvariant + then (\xs -> para (cite xs mempty)) + <$> cites NormalCitation False + else option mempty $ para <$> bracketed inline let lang = mblang >>= babelLangToBCP47 let langdiv = case lang of Nothing -> id Just l -> divWith ("",[],[("lang", renderLang l)]) + _closingPunct <- option mempty $ bracketed inline -- currently ignored bs <- grouped block - return $ blockQuote . langdiv $ (bs <> citePar) + optional $ symbolIn (".:;?!" :: [Char]) -- currently ignored + return $ blockQuote . langdiv $ (bs <> citepar) doAcronym :: PandocMonad m => Text -> LP m Inlines doAcronym form = do diff --git a/src/Text/Pandoc/Readers/LaTeX/Parsing.hs b/src/Text/Pandoc/Readers/LaTeX/Parsing.hs --- a/src/Text/Pandoc/Readers/LaTeX/Parsing.hs +++ b/src/Text/Pandoc/Readers/LaTeX/Parsing.hs @@ -484,7 +484,11 @@ Nothing -> mzero Just (Macro expansionPoint argspecs optarg newtoks) -> do let getargs' = do - args <- case optarg of + args <- + (case expansionPoint of + ExpandWhenUsed -> withVerbatimMode + ExpandWhenDefined -> id) + $ case optarg of Nothing -> getargs M.empty argspecs Just o -> do x <- option o bracketedToks diff --git a/src/Text/Pandoc/Writers/HTML.hs b/src/Text/Pandoc/Writers/HTML.hs --- a/src/Text/Pandoc/Writers/HTML.hs +++ b/src/Text/Pandoc/Writers/HTML.hs @@ -1089,16 +1089,18 @@ -> TableRow -> StateT WriterState m Html tableRowToHtml opts (TableRow tblpart attr rownum rowhead rowbody) = do - let rowclass = A.class_ $ case rownum of + let rowclass = case rownum of Ann.RowNumber x | x `rem` 2 == 1 -> "odd" _ | tblpart /= Thead -> "even" _ -> "header" + let attr' = case attr of + (id', classes, rest) -> (id', rowclass:classes, rest) let celltype = case tblpart of Thead -> HeaderCell _ -> BodyCell headcells <- mapM (cellToHtml opts HeaderCell) rowhead bodycells <- mapM (cellToHtml opts celltype) rowbody - rowHtml <- addAttrs opts attr $ H.tr ! rowclass $ do + rowHtml <- addAttrs opts attr' $ H.tr $ do nl opts mconcat headcells mconcat bodycells diff --git a/src/Text/Pandoc/Writers/LaTeX.hs b/src/Text/Pandoc/Writers/LaTeX.hs --- a/src/Text/Pandoc/Writers/LaTeX.hs +++ b/src/Text/Pandoc/Writers/LaTeX.hs @@ -851,9 +851,6 @@ -> [[Block]] -> LW m (Doc Text) tableRowToLaTeX header aligns widths cols = do - -- scale factor compensates for extra space between columns - -- so the whole table isn't larger than columnwidth - let scaleFactor = 0.97 ** fromIntegral (length aligns) let isSimple [Plain _] = True isSimple [Para _] = True isSimple [] = True @@ -861,9 +858,10 @@ -- simple tables have to have simple cells: let widths' = if all (== 0) widths && not (all isSimple cols) then replicate (length aligns) - (scaleFactor / fromIntegral (length aligns)) - else map (scaleFactor *) widths - cells <- mapM (tableCellToLaTeX header) $ zip3 widths' aligns cols + (1 / fromIntegral (length aligns)) + else widths + let numcols = length widths' + cells <- mapM (tableCellToLaTeX header numcols) $ zip3 widths' aligns cols return $ hsep (intersperse "&" cells) <> "\\tabularnewline" -- For simple latex tables (without minipages or parboxes), @@ -890,11 +888,12 @@ displayMathToInline (Math DisplayMath x) = Math InlineMath x displayMathToInline x = x -tableCellToLaTeX :: PandocMonad m => Bool -> (Double, Alignment, [Block]) +tableCellToLaTeX :: PandocMonad m + => Bool -> Int -> (Double, Alignment, [Block]) -> LW m (Doc Text) -tableCellToLaTeX _ (0, _, blocks) = +tableCellToLaTeX _ _ (0, _, blocks) = blockListToLaTeX $ walk fixLineBreaks $ walk displayMathToInline blocks -tableCellToLaTeX header (width, align, blocks) = do +tableCellToLaTeX header numcols (width, align, blocks) = do beamer <- gets stBeamer externalNotes <- gets stExternalNotes inMinipage <- gets stInMinipage @@ -912,9 +911,12 @@ AlignCenter -> "\\centering" AlignDefault -> "\\raggedright" return $ "\\begin{minipage}" <> valign <> - braces (text (printf "%.2f\\columnwidth" width)) <> + braces (text (printf + "(\\columnwidth - %d\\tabcolsep) * \\real{%.2f}" + (numcols - 1) width)) <> halign <> cr <> cellContents <> "\\strut" <> cr <> "\\end{minipage}" +-- (\columnwidth - 8\tabcolsep) * \real{0.15} notesToLaTeX :: [Doc Text] -> Doc Text notesToLaTeX [] = empty diff --git a/stack.yaml b/stack.yaml --- a/stack.yaml +++ b/stack.yaml @@ -19,10 +19,11 @@ - HsYAML-0.2.1.0 - HsYAML-aeson-0.2.0.0 - doctemplates-0.8.2 -- commonmark-0.1.0.2 -- commonmark-extensions-0.2.0.1 +- commonmark-0.1.1 +- commonmark-extensions-0.2.0.2 - commonmark-pandoc-0.2.0.1 -- citeproc-0.1.0.3 +- git: https://github.com/jgm/citeproc + commit: 1860f189e9995c1dc27a68893bedfbf8de1ee67f ghc-options: "$locals": -fhide-source-paths -Wno-missing-home-modules diff --git a/test/command/5367.md b/test/command/5367.md --- a/test/command/5367.md +++ b/test/command/5367.md @@ -23,18 +23,18 @@ \begin{longtable}[]{@{}c@{}} \caption[Sample table.]{Sample table.\footnote{caption footnote}}\tabularnewline \toprule -\begin{minipage}[b]{0.16\columnwidth}\centering +\begin{minipage}[b]{(\columnwidth - 0\tabcolsep) * \real{0.17}}\centering Fruit\footnote{header footnote}\strut \end{minipage}\tabularnewline \midrule \endfirsthead \toprule -\begin{minipage}[b]{0.16\columnwidth}\centering +\begin{minipage}[b]{(\columnwidth - 0\tabcolsep) * \real{0.17}}\centering Fruit{}\strut \end{minipage}\tabularnewline \midrule \endhead -\begin{minipage}[t]{0.16\columnwidth}\centering +\begin{minipage}[t]{(\columnwidth - 0\tabcolsep) * \real{0.17}}\centering Bans\footnote{table cell footnote}\strut \end{minipage}\tabularnewline \bottomrule diff --git a/test/command/6783.md b/test/command/6783.md new file mode 100644 --- /dev/null +++ b/test/command/6783.md @@ -0,0 +1,26 @@ +``` +% pandoc -f markdown -t plain -s --citeproc +--- +references: +- author: + - family: Jupyter + given: Project + container-title: Proceedings of the 17th Python in Science Conference + id: 'ref-1' + issued: 2018 + title: 'Binder 2.0 - Reproducible, interactive, sharable environments for science at scale' + type: 'paper-conference' + volume: +... + +This is a test[@ref-1]. +^D + + +This is a test(Jupyter 2018). + +Jupyter, Project. 2018. “Binder 2.0 - Reproducible, Interactive, +Sharable Environments for Science at Scale.” In Proceedings of the 17th +Python in Science Conference. +``` + diff --git a/test/command/6796.md b/test/command/6796.md new file mode 100644 --- /dev/null +++ b/test/command/6796.md @@ -0,0 +1,16 @@ +``` +% pandoc -f latex -t markdown +\newcommand{\cL}{\mathcal{L}} +\newcommand{\til}[1]{\tilde{#1}} + +$$\til\cL$$ + +\newcommand{\mc}[1]{\mathcal{#1}} +\newcommand{\dL}{\mc{L}} + +$$\til\dL$$ +^D +$$\tilde{\mathcal{L}}$$ + +$$\tilde{\mathcal{L}}$$ +``` diff --git a/test/command/6802.md b/test/command/6802.md new file mode 100644 --- /dev/null +++ b/test/command/6802.md @@ -0,0 +1,9 @@ +``` +% pandoc -f latex -t native +\blockquote[test][]{quote} +^D +[BlockQuote + [Para [Str "quote"] + ,Para [Str "test"]]] + +``` diff --git a/test/command/docbook-bibliography.md b/test/command/docbook-bibliography.md new file mode 100644 --- /dev/null +++ b/test/command/docbook-bibliography.md @@ -0,0 +1,20 @@ +``` +% pandoc -f docbook -t native --quiet + +Document References + +[1] First reference + + +[2] Second reference + + +[3] Third reference + + +^D +[Header 1 ("",[],[]) [Str "Document",Space,Str "References"] +,Para [Span ("refTheFirst",[],[]) [],Str "[1]",Space,Str "First",Space,Str "reference"] +,Para [Span ("refTheSecond",[],[]) [],Str "[2]",Space,Str "Second",Space,Str "reference"] +,Para [Span ("refTheThird",[],[]) [],Str "[3]",Space,Str "Third",Space,Str "reference"]] +``` diff --git a/test/command/pandoc-citeproc-chicago-fullnote-bibliography.md b/test/command/pandoc-citeproc-chicago-fullnote-bibliography.md --- a/test/command/pandoc-citeproc-chicago-fullnote-bibliography.md +++ b/test/command/pandoc-citeproc-chicago-fullnote-bibliography.md @@ -114,7 +114,7 @@ Roe, "Why Water Is Wet," in *Third Book*, ed. Sam Smith (Oxford: Oxford University Press, 2007)](#ref-пункт3). -[^7]: [Doe and Roe](#ref-пункт3) (["Why Water Is Wet," 12](#ref-пункт3)) +[^7]: [Doe and Roe](#ref-пункт3), ["Why Water Is Wet," 12](#ref-пункт3) and a citation without locators ([Doe and Roe, "Why Water Is Wet"](#ref-пункт3)). @@ -123,7 +123,7 @@ [^9]: See [Doe, *First Book*, 2005, 34--35](#ref-item1). -[^10]: Some citations (See [Doe, chap. 3](#ref-item1); [Doe and Roe, +[^10]: Some citations (see [Doe, chap. 3](#ref-item1); [Doe and Roe, "Why Water Is Wet"](#ref-пункт3); [Doe, "Article," 2006](#ref-item2)). diff --git a/test/command/pandoc-citeproc-locators-integrated.md b/test/command/pandoc-citeproc-locators-integrated.md --- a/test/command/pandoc-citeproc-locators-integrated.md +++ b/test/command/pandoc-citeproc-locators-integrated.md @@ -29,7 +29,7 @@ [@citekey, p. \[89\]] -[@citekey and nothing else]. +[@citekey and nothing else] [@citekey, 123(4)\[5\]6, and suffix] @@ -79,7 +79,7 @@ [^9] -.[^10] +[^10] [^11] diff --git a/test/lhs-test.html b/test/lhs-test.html --- a/test/lhs-test.html +++ b/test/lhs-test.html @@ -7,7 +7,7 @@ lhs-test