Mastering Sphinx Extensions

Enhancing the Document Builder to Create Presentations

NOTE:

This presentation is applied some patches after talk.
It does not change about content.

Introduction

Speaker introduction

Kazuya Takei ( @attakei )
Software engineer using Python in Japan
Sphinx lover and Sphinx extensions writer

My birth is Feb, 27th (3days ago!)

Outline of this talk

About Sphinx with advanced topic than standard usage.
Approach to build presentations by Sphinx using sphinx-revealjs.
Show case for power of Sphinx and sphinx-revealjs.

Sphinx overview

Ask questions for you

Please raise a hand if Yes.

Do you know Sphinx?
Do you see document generated by Sphinx?
Do you write document by Sphinx?

What is Sphinx?

Sphinx is “documentation generator” written by Python.

Convert document sources into readable style output.
Provide features for documentation using reStructuredText.

What is Sphinx?

Sphinx supports multiple inputs and outputs.

Input: reStructuredText, Markdown, and more.
Output: HTML, PDF, EPUB, mandoc, and more.

What is Sphinx?

Appendix: Related products

Pelican (written by Python)
MkDocs (written by Python)
Pandoc
Hugo
Astro

Made in Sphinx

Python and third party projects:

Made in Sphinx

Not Python:

_images/screenshot-phpmyadmin-manual.png

RE: Ask questions for you

✅ Do you know Sphinx?
✅ Do you see document generated by Sphinx?
🔲 Do you write document by Sphinx?

Inside of Sphinx

How do Sphinx works to generate document.

4-step of working Sphinx

Init: Sphinx core application with extensions.
Read: Parse “reStructuredText” and convert into “doctree” objects.
Transform: Modify “doctree” objects.
Write: Create document files from “doctree” objects.

4-step of working Sphinx

reStructuredText

reStructuredText is plain text format of lightweight markup to write sttrucuted document. This is like for Markdown, (but it is more extendable than MD)

reStructuredText

Standard specs includes:

Directive is block synxtax. It can have some attributes and content that has nested direvctives.
Role is inline directive. It can have some parameters.
Comment .. only directive. Nested content is not used for output.

reStructuredText

Example with Markdown

Let’s compare reStructuredText and Markdown for text that has the same expression. Header text is expressed differently. In Markdown, a hash is placed at the beginning of the text, but in reStructuredText, a line is drawn below the text. Code blocks are also expressed differently. Markdown uses three backquotes, but reStructuredText declares two periods followed by the “code-block” declaration. — 同じ表現をしているテキストを、reStructuredTextとMarkdownを並べてみます。ヘッダーテキストの表現が違います。Markdownではテキストの先頭にハッシュが置かれていますが、reStructuredTextではテキストの下に線が引かれています。コードブロックの表現も違っています。Markdownでは3個のバッククオートが使われていますが、reStructuredTextではピリオド2個のあとにcode-blockという宣言が行われています。

reStructuredText

Title
=====

Hello world.

* List item 1
* List item 2

.. code-block:: python

   print("hello world")

Markdown

# Title

Hello world.

* List item 1
* List item 2

```python
print("Hello world")
```

Parse to doctree

Reader convert from reStructuredText into node tree model.

Parse to doctree

reStructuredText

Title
=====

Hello world.

Sub title
---------

* List item 1
* List item 2

.. code-block:: python

   print("hello world")

Doctree

Write content

Write phase generate from doctree to files rules of itself.

Doctree

reStructuredText

<section>
  <h1>Title</h1>
  <p>Hello world</p>
  <section>
    <h2>Sub title</h2>
    <ul>
      <li>List item1</li>
      <li>List item2</li>
    </ul>
    <code>
      <pre></pre>
    </code>
  </section>
</section>

Extend Sphinx

When you don’t work by basic features, what can you do?

Sphinx can extend by other Python project.

When you want to change design of document.
When you want to use Markdown as document source.
When you want to display graphs in your document.

We can install and use Sphinx extensions.

Popular extensions

MyST-parser

Enable to parse Markdown text with extended syntax.
sphinxcontrib-mermaid

Render mermaid.js graph

See https://github.com/topics/sphinx-extension to know more extensions.

Um, there are not extensions to realize that you want.

You can create extensions!!

Very simple Sphinx extension

Write my_extension.py.

from sphinx.application import Sphinx

def setup(app: Sphinx) -> dict:
    print("Working this extension!")
    # Call methods of app....
    return {}

Very simple Sphinx extension

Edit your conf.py of document.

extensions = [
     # Register this!
     "my_extension",
]

Very simple Sphinx extension

Running Sphinx v8.2.1
loading translations [en]... done
Working this extension!           <=== Inserted!!
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
writing output...
building [revealjs]: targets for 0 source files that are out of date
updating environment: [extensions changed ('2')] 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done

Adding behaivors

Sphinx core application provides many methods to extend behaviors of it. And we can call every method in your setup().

Adding behaivors

add_config_value()
add_directive()
add_builder()
connect()

ref: Sphinx documentation

Sphinx has many “events”

https://www.sphinx-doc.org/en/master/_images/graphviz-8f41e3505b1f58d16c8c77a9ed7d9562fac30e74.png

Sphinx has many “events”

MANY!!

Please read docs when you need.

sphinx-revealjs

Introduction of the one of my OSS.

What is sphinx-revealjs?

sphinx-revealjs is Sphinx extension to add new builder with modules.

You can:

generate html presentation from reStructuredText/Markdown.
use very easy (call make revealjs instead of make html)

What is sphinx-revealjs?

Sphinx x Reveal.js

Work on Sphinx ecosystem

= Supports many Sphinx extensions
Layout by Reveal.js and work on ecosystem.

Motivation

I want to use reStructuredText than Makdown.
I want to use Sphinx extensions that output good design contents

Demo

This presentation is also made by sphinx-revealjs!!

Website:

https://attakei.github.io/the-slide/
Repository:

https://github.com/attakei/the-slide/

This works on “Init” and “Write” phases

Writer create HTML that is for Reveal.js format instead of documentation format.

This works on “Init” and “Write” phases

Title
=====

Sub title
---------

This works on “Init” and “Write” phases

HTML builder

<section>
  <h1>Title</h1>
  <section>
    <h2>Sub title</h2>
  </section>
</section>

Revealjs builder

<section>
  <section>
    <h1>Title</h1>
  </section>
</section>
<section>
  <section>
    <h2>Sub title</h2>
  </section>
</section>

Architecture

Register new direvctives, builders, and configurations
Builder use custom writer to generate HTML for Reveal.js.
Custom writer handles added direvctives for good layout.

Architecture

Added directives.

revealjs-slide , revealjs-slide , revealjs-vertical
revealjs-break
revealjs-notes
revealjs-code-block, revealjs-fragments

`revealjs-break`

This is to split slides keeping section title.

Section 1
---------

.. revealjs-break::

`revealjs-notes`

This is to manage speaker note per slides. Contents of this is hidden from readers.

.. revealjs-notes::

   This text does not display on presentation.
   But, speaker can read from speaker-note.

`revealjs-code-block`

This is to extend Sphinx’s code-block using animation.

.. revealjs-code-block:: rst
   :data-line-numbers: 1,2|4-5|7|8|9|7-9

   Hello world
   ===========

   * List item 1
   * List item 2
   * List item 3

Benefits for users

you can write presentations by as same as documentation.
you can embed many contents from Python as Sphinx extensions.
you can manage content as plain-text that is easy to manager on repository.

Benefits for users

When content manage in GitHub…

Check content by GitHub Actions.
Deploy content to GitHub Pages.
It may search easily by GitHub Copilot.

Benefits (only for me)

Geven feedback that this is used by few enginieers but world wide.
Gain use caes by community: presentation about OSGeoLive by OSGeo.
Get a chance to talk on PyCon outside of Japan.

It’s a good loop of motivation for OSS writer!

Show cases

Examples of using other Sphinx extensions.

oEmbedPy

..
   This is URL of Opening Remarks of PyCon PH 2024

.. oembed:: https://www.youtube.com/watch?v=Cu9JIdlbnbc
   :maxwidth: 720
   :maxheight: 720

oEmbedPy

Plotly

.. plotly::
   :fig-vars: fig1, fig2
   :include-source: false

   x = np.arange(5)
   y = x ** 2

   title = "plotly version: {}".format(plotly.__version__)
   fig1 = go.Figure(go.Scatter(x=x, y=y), layout=dict(title=title))
   fig2 = px.scatter(x=x, y=y, title=title)

(Source code, html, html)

(html)

PyVista

.. pyvista-plot::

   >>> import pyvista
   >>> sphere = pyvista.Sphere()
   >>> out = sphere.plot()

>>> import pyvista
>>> sphere = pyvista.Sphere()
>>> out = sphere.plot()

Static Scene

Interactive Scene

asciinema

.. asciinema:: ./demo.cast
   :preload: 1
   :autoplay: 1
   :rows: 15
   :cols: 80
   :terminalfontsize: 16px

asciinema

sphinx-nekochan

.. container:: r-fit-text

   :nekochan:`clap-nya`
   :nekochan:`beer-nya`
   :nekochan:`isogu-nya`
   :nekochan:`jikan-nya`

sphinx-nekochan

clap-nya beer-nya isogu-nya jikan-nya

There are many cats in Philippines!!

Enjoy presentation by documentation!!

[Feedback at here!]