What is the MyST stack?

This section provides a high-level overview of the main concepts and tools in the MyST ecosystem. See the guiding principles of the MyST ecosystem for some more high-level context.

A brief overview of the MyST Stack¶

Here is a brief description of the workflow that is involved in the MyST ecosystem and the technical pieces that are involved at each step. You can hover over (or click) the links in the list below to see more information about each one.

• An author writes content in MyST Markdown in a file.
• A MyST Document Engine parses one or more files and builds them into structured data called the MyST AST.
• The MyST AST is a version of the content with a specific structure and with extra metadata added to it (like defining a “type” for each chunk of content, resolved cross-references, etc). It conforms to the MyST Specification.
• A MyST Renderer can then transform the MyST AST into different kinds of outputs (most commonly HTML, or PDF via LaTeX/Typst). It uses a MyST Theme to do so.
• The official MyST CLI contains both a parser and a renderer so that you can go from a local file to an output with one application.

Here’s a diagram showing how these all relate to one another:

Are there other MyST engines?¶

The MyST Specification and MyST Markdown Syntax was designed so that others could implement their own parsers, document engines, and renderers using MyST. Although there are presently no independent examples of a true MyST Document Engine, the MyST Parser for Sphinx project fulfills a similar role in the Sphinx ecosystem.

The MyST Parser for Sphinx is an extension for Sphinx that can parse MyST Markdown into Sphinx’s internal document structure. It was created to allow users to parse MyST Markdown syntax into Sphinx for V1 of Jupyter Book. It now exists as an independent extension for the Sphinx community, as Jupyter Book now uses the MyST Document Engine.

How does Jupyter Book relate to MyST?¶

The MyST Markdown Syntax was originally created to allow Jupyter Book V1 to use the Sphinx Document engine with Markdown content^[1], rather than the less well-known ReStructuredText language.

Over time, the Jupyter Book team decided that the most sustainable path forward was to maintain its own document engine, the MyST Document Engine. An early version of the MyST Document Engine was created by Curvenote. To promote its long-term sustainability, the project was open sourced and further developed in collaboration with the Jupyter Book team. It serves as the engine behind Jupyter Book V2.

You can think of Jupyter Book as a distribution of the MyST Document Engine. In other words, Jupyter Book wraps the MyST Document Engine application, with out-of-the-box configuration that supports a multi-page community knowledge base or documentation site. MyST is created and maintained by the Jupyter Book team.

Currently, Jupyter Book and the MyST Document Engine share much of the same functionality. Over time, we imagine that the MyST Document Engine will be a more flexible tool that makes fewer assumptions about the end use-case that it’s being used for. Jupyter Book will then have “more opinions” above the base configuration of the MyST Document Engine. That said — we are still figuring this out! We aren’t sure the exact relationship the two will have, and will revisit this as we learn more. See next.jupyterbook.org for Jupyter Book V2 documentation, and see this section of the Contributing Guide for our approach to documenting the two projects.

How do MyST and Sphinx compare?¶

Sphinx is an open-source documentation system used in many software projects, especially in the Python ecosystem. It builds an internal representation of technical documents as a tree (see docutils) similar to the MyST AST. Whilst Sphinx partially defines a specification of sorts^[2], a MyST Document Engine explicitly publishes its MyST AST for other tools to consume.

The Sphinx ecosystem has excellent support for Python documentation, referencing content, as well as externally providing an inventory of references known as intersphinx. You can link to Sphinx documentation with a MyST Document Engine by using intersphinx references, and the official MyST Document Engine automatically exposes the information required to allow Sphinx documentation to reference your MyST project.

At this time, the official MyST Document Engine does not support the authoring of software API documentation^[3]. As such, if your project is documenting Python software we suggest that you use Sphinx. If your project is primarily tutorials, educational textbooks (including with Jupyter Notebooks), a presentation, or scientific paper, we hope that you find a better fit with the MyST stack!

Sphinx and MyST take very different approaches to publishing to the web. In Sphinx, themes are used to customize the generation of HTML and JavaScript. Sites built using these themes can be deployed to static web servers like ReadTheDocs. In the MyST world, the equivalent to a Sphinx theme is a MyST Renderer that consumes the MyST AST and outputs HTML and JavaScript for your browser. Formally separating the MyST Document Engine that generates MyST AST and the MyST Renderer that consumes it makes it possible to create rich new experiences with MyST, such as the SciPy Conference Proceedings website that renders MyST AST on the fly.

Although the official engine also supports static HTML outputs, implementing themes as applications makes it easier to build more powerful MyST viewing experiences.