diff --git a/content/article/md-workflow.md b/content/article/md-workflow.md index d17f21e..99e8691 100644 --- a/content/article/md-workflow.md +++ b/content/article/md-workflow.md @@ -11,7 +11,7 @@ abstract: | This article outlines a workflow for incorporating bibliographic citations/references in Markdown documents, and exporting new documents with compiled bibliographies according to specific citation styles. All tools involved in this pipeline are free, open source, and cross platform. --- -[Better Bibtex]: https://retorque.re/zotero-better-bibtex/ "an extension for Zotero for managing bibliographic data, especially text-based document authoring toolchains" +[Better Bibtex]: https://retorque.re/zotero-better-bibtex/ "an extension for Zotero for managing bibliographic data, especially for text-based document authoring toolchains" [Zotero]: https://www.zotero.org/ "free and open source, cross-platform citation management software." [Pandoc]: https://pandoc.org/ "universal document converter" @@ -23,14 +23,16 @@ bibliography: references.bib # Introduction -This guide outlines a workflow for incorporating bibliographic referencing (citations and references) in ![Markdown](article:md-formatting) documents. The general idea is incorporating citation keys in a Markdown document, and then rendering the bibliography by generating a new document against a bibliographic collection (`bib` file), and a citation style (`csl` file). -The following describes: -- creating and exporting a bibliography collection with [Zotero][]; -- using citation keys in a Markdown document; -- getting a citation style file; -- exporting new documents (e.g. PDF, DOCX, MD, etc.) with [Pandoc][], where citations and bibliographic references are formatted as per any specific citation style. +This guide outlines a workflow for incorporating bibliographic referencing (citations and references) in ![Markdown](article:md-formatting) documents. The general idea is incorporating citation keys from a bibliography in a Markdown document, and then rendering citations and references by generating a new document against a bibliographic collection (`bib` file), and a citation style (`csl` file). +The steps described in the following are: -All tools involved are free, open source, and cross-platform. +1. Installing required tools and software; +1. creating and exporting a bibliography collection with [Zotero][]; +2. using citation keys in a Markdown document; +3. getting a citation style file; +4. exporting new documents (e.g. PDF, DOCX, MD, etc.) with [Pandoc][], where citations and bibliographic references are formatted as per any specific citation style. + +All tools involved in this workflow are free, open source, and cross-platform. --- @@ -42,9 +44,8 @@ For example, for an 'author-year' citation style like *Chicago* in-text citation 1. Some Markdown editor for authoring Markdown documents (see ![Markdown guide](article:md-formatting)). 2. [Zotero][] -- free and open source cross platform citation management software. -3. [Better Bibtex] -- "an extension for Zotero ... that makes it easier to manage bibliographic data, especially for people authoring documents using text-based toolchains (e.g. based on [LaTeX](https://www.latex-project.org/) / [Markdown](https://www.markdownguide.org/)"). -4. [Pandoc][] -- "universal document converter"; a command line tool for converting between different document formats (e.g. Markdown to PDF or DOCX), and compiling bibliographic references. - +3. [Better Bibtex][] -- "an extension for Zotero ... that makes it easier to manage bibliographic data, especially for people authoring documents using text-based toolchains (e.g. based on [LaTeX](https://www.latex-project.org/) / [Markdown](https://www.markdownguide.org/)"). +4. [Pandoc][] -- "universal document converter"; command line tool for converting between different document formats (e.g. Markdown, PDF, DOCX, epub, etc.), and compiling bibliographic references. ## Pandoc installation & upgrade {#pandoc-install} @@ -74,8 +75,7 @@ Installing via a terminal requires a package manager (e.g. [Chocolatey][] for Wi ### Upgrade -To upgrade [Pandoc][], if it was installed via an installer, download and run the latest installer. - +To upgrade [Pandoc][], if it was installed via an installer, download and run the latest installer. If it was installed via terminal, it can be upgraded with a single command. - For Windows: ``` @@ -86,7 +86,7 @@ If it was installed via terminal, it can be upgraded with a single command. brew upgrade pandoc ``` -Verify the version of pandoc with the command: +Verify the version of your Pandoc installation with the command: ``` pandoc --version ``` @@ -102,7 +102,7 @@ This section outlines how to create a bibliography collection in Zotero and expo ![](/markdown-guide/zotero-citation-key.png) 4. To export your reference collection, select the collection, right-click > Export Collection, select **Better BibLaTeX** (uncheck all options; background export optional): ![](/markdown-guide/zotero-export-options.png) -3. Save your collection, e.g. as `mycollection.bib` . The resulting file is a set of citation keys, each with its own citation metadata (you can preview the file with a text editor). +3. Save your collection (e.g. `mycollection.bib`). The resulting file is a dictionary of citation keys, each with its own citation metadata (you can preview the file with a text editor). Notes: - In case you modified your Zotero collection (e.g. added/removed items or edited existing items), re-export the collection and overwrite the previous `bib` file. @@ -114,26 +114,24 @@ Notes: To add a citation in a Markdown file, just prefix a citation key (from Zotero) with the `@` sign, as in `@khaled2018`. +Eventually, as described in the following steps, when the document is compiled (against a bibliography and a citation style), citation keys will be rendered according to the specified citation style, and a bibliographic reference will be added to the bibliography section of the document, per each unique cited document. - -When the document will be compiled against a bibliography and a citation style, citation keys will be rendered according to the specified citation style *and* a reference will be added to the bibliography section for each citation. - - For author-year type citation styles: +For author-year type citation styles: - To add parentheses to a reference use square brackets, as in `[@khaled2018]` --> "(Khaled et al. 2018)") - For citing multiple sources at once separate keys with semicolons, as in `[@khaled2018;@graziano2019]` --> "(Khaled et al. 2018; Graziano et al. 2019)" - To supress the author name (in author-date citation styles) do `[-@khaled2018]` eg. "According to Khaled and colleagues [-@khaled2018]" will render as "According to Khaled and colleagues (2018)") - To add a page or a note before or after a citation do `[see @graziano2019, p.125]` --> (see Graziano et al. 2019, p.125) - - # Get a citation style file (CSL) Use a provided citation style file (CSL), or download one from [the official citation style repository](https://github.com/citation-style-language/styles) (GitHub repository), or from the [Zotero Style Repository](https://www.zotero.org/styles) (search interface). -Note: Zotero features a citation style editor (Tools > Style Editor), for modifying existing citation styles if needed. +Notes: +- [Zotero] includes a few CSL files for popular citation styles. These are located in the `styles` directory of the Zotero Data Directory (Zotero > Edit > Preferences > Advanced > Show Data Directory). +- Zotero features a citation style editor (Tools > Style Editor), for modifying existing citation styles if needed. # Gather files and paths -The next step is to make a note of the paths of your Markdown document, citation style, and bibliography files, which will be used to compile a new document. +The next step is to gather all required files that will be used to compile a new document (Markdown document, citation style, and bibliography files) and make a note of their file paths. In addition, make sure that the path of any image referenced in the Markdown document is correct. Good practices: @@ -157,45 +155,50 @@ Good practices: Export directory: "/path/to/mydocument/export/" - [Here you can also assemble the pandoc command for converting this document (discussed in the next sections), so that it can be easily reused.] + Pandoc export command: ... --> ``` Notes: - File paths can be either absolute (e.g. `c:/users/me/some-folder/some-file.md`), or relative (e.g. `/some-folder/some-file.md`). However, for relative paths pandoc commands will need to be executed from the base directory of these relative paths. - An easy way to get the absolute path of a file or directory, is to drag and drop it into a terminal, then select the path that will appear, right click, and copy. -- Enclosing file paths within "quotes" is optional. However, doing so can bypass the need for special characters in cases where file or directory names contain spaces. - -# Test Pandoc and the Markdown document +- Enclosing file paths within "quotes" is optional. However, doing so can bypass the need for escape characters in cases where file or directory names contain spaces. +# Test Pandoc and Markdown document Open a terminal: -- to test that [Pandoc][] works do (if the `pandoc` command is not recognized as valid go to [installation instructions](#pandoc-install): +- first, to check that [Pandoc][] is in order do: ``` pandoc --version ``` + If the `pandoc` command is not recognized as valid go to [installation instructions](#pandoc-install). - to test that your Markdown document is in order do (this will not compile the bibliography): ``` For docx export: - pandoc "/path/to/mydocument/my-article.md" -o /path/to/mydocument/export/my-article.docx" + pandoc "/path/to/mydocument/my-article.md" -o "/path/to/mydocument/export/my-article.docx" or for pdf export: pandoc "/path/to/mydocument/my-article.md" -o "/path/to/mydocument/export/my-article.pdf" or for Markdown export: - pandoc "path/to/my-article.md" -o -o "/path/to/export-document.md" --atx-headers --strip-comments --wrap=preserve -t markdown_mmd-citations --standalone + pandoc "path/to/my-article.md" -o "/path/to/export-document.md" --atx-headers --strip-comments --wrap=preserve -t markdown_mmd --standalone ``` - # Compile document with bibliographic references -To compile a new document file including citations and bibliography in Markdown do: +To compile a new document including citations and bibliography add the options: + +| Pandoc option | Description | +|-|-| +| `--citeproc` | option for rendering citations/references| +|`--csl` |option to specify citation style, followed by the path to the `CSL` file| +| `--bib` | option to specify bibliography followed by the path to the `bib` file| -``` -pandoc "path/to/myarticle.md" --citeproc --bibliography "/path/to/mycollection.bib" --csl "/path/to/reference-style.csl" -o "/path/to/export-document.md" --atx-headers --strip-comments --wrap=preserve -t markdown_mmd-citations --standalone -``` To compile to PDF do: ``` pandoc "path/to/myarticle.md" --citeproc --bibliography "/path/to/mycollection.bib" --csl "/path/to/reference-style.csl" -o "/path/to/export-document.pdf" ``` +*Note: Conversions to PDF format require a LaTeX installation.* + +--- To compile to DOCX do: ``` @@ -206,8 +209,67 @@ To compile to DOCX using a DOCX document as reference/template add the option: --reference-doc="path/to/reference-document.docx" ``` -*Note: Conversions to *DOCX* format will use Footnotes, not Endnotes.* +*Note: Conversions to DOCX format will use Footnotes, not Endnotes.* +--- + +To compile to a new Markdown document (with references) do: + +``` +pandoc "path/to/myarticle.md" --citeproc --bibliography "/path/to/mycollection.bib" --csl "/path/to/reference-style.csl" -o "/path/to/export-document.md" --atx-headers --strip-comments --wrap=preserve -t markdown_mmd-citations --standalone +``` + +The additional options + +| Pandoc option | Description | +|-|-| +| `--atx-headers` | use common (ATX) formatting for Markdown headings | +| `--strip-comments`| remove HTML comments(``) | +| `--wrap=preserve` | preserve line wrapping (by default pandoc will introduce line breaks)| +| `-t markdown_mmd-citations` | `-t` (to) for specifying output format; ` markdown_mmd-citations` for `markdown_mmd` format *with* citations (by default Markdown exports don't render citations/bibliography)| +| `--standalone`| export standalone (will also render frontmatter) | +# Common Pandoc compilation errors + +Pandoc will not compile in case of errors. Some common errors include: + +- missing/wrong citation keys: [Pandoc][] will list all citation keys that were not found in the supplied bibliography file. Correct any potential citation key misspellings, or recompile your bibliography collection to include the missing citations. +- file not found: check file paths +- cannot write/open file: Pandoc is not able to overwrite files (such as PDF or DOCX) while they are open by other software. + + +# Pandoc command reference + +[pandoc options manual]: https://pandoc.org/MANUAL.html#options + +See also: +- [Pandoc demos](https://pandoc.org/demos.html), +- [Getting started with pandoc](https://pandoc.org/getting-started.html), and +- [pandoc options manual][]. + +A basic pandoc command example is: +``` +pandoc "path/to/input.txt" -o "path/to/output.docx" +``` +Additional options can be added to this command pattern to specify export parameters. Here is a brief list of some commonly used pandoc options (enclosing file paths inside quotation marks is optional): + +|Pandoc option | Description | +|- |- | +| `-i "path/to/input/file.ext"` | input file, followed by path to file including file extension (usually `-i` is omitted when input file is specified at the beginning of the command)| +| `-o "/path/to/output/file.ext"` | output file, followed by path to file, including file extension| +| `-f` | (from) for specifying input format, followed by option, see [pandoc options manual][]| +| `-t` |(to) for specifying output format, followed by option, see [pandoc options manual][]| +| `-s`, `--standalone` | to produce standalone file, which will render title, abstract, keywords, using data from front matter of the Markdown document| +| `-C`, `--citeproc` | option to render citations; will expect bibliography and csl files | +| `--bibliography "path/to/bibliography.bib"`| specify bibliography file | +| `--csl "path/to/citation-style.csl"` |specify citation style file| +| `--atx-headers` | use common (ATX) formatting for Markdown headings| +| `--wrap=preserve` | option to preserve line wrapping for plain text output formats (other options: `auto`\|`none`)| +| `--strip-comments` | remove HTML comments (``)| +| `--no-highlight` | Disables syntax highlighting for code blocks and inlines| # Specify bibliography placement @@ -238,47 +300,3 @@ or Lorem ipsum. ``` - -# Common errors - -- missing/wrong citation keys: [Pandoc][] will list all citation keys that were not found in the supplied `bib` file. Correct any potential citation key misspellings, or recompile your bibliography collection to include the missing citations. -- file not found: check file paths -- cannot write/open file: Pandoc is not able to overwrite files (such as PDF or DOCX) while they are open by other software. - - - -# Pandoc command reference - -[pandoc options manual]: https://pandoc.org/MANUAL.html#options - -See also [Pandoc demos](https://pandoc.org/demos.html), [Getting started with pandoc](https://pandoc.org/getting-started.html), and [pandoc options manual][]. - -A basic pandoc command example is: -``` -pandoc "path/to/input.txt" -o "path/to/output.docx" -``` -Additional options can be added to this command pattern to specify export parameters. - -Here is a brief list of some commonly used pandoc options (enclosing file paths inside quotation marks is optional): -- `-i "path/to/input/file.ext"` (input file, followed by path to file including file extension) -- `-o "/path/to/output/file.ext"` (output file, followed by path to file, including file extension) -- `-f` (output format, followed by option, see [pandoc options manual][]) -- `-t` (output format, followed by option, see [pandoc options manual][]) -- `-s`, `--standalone` (will render title, abstract, keywords, using data from front matter of the Markdown document) -- `-C`, `--citeproc` (option to render citations; will expect bibliography and csl files) -- `--bibliography "path/to/bibliography.bib"` -- `--csl "path/to/citation-style.csl"` (citation style file) -- `--atx-headers` (for common Markdown heading format) -- `--wrap=auto`|`none`|`preserve` (line wrapping options for plain text formats) -- `--strip-comments[=true|false]` (remove HTML comments) -- `--no-highlight` (Disables syntax highlighting for code blocks and inlines) - -