When I was working as developer advocate at Read the Docs, one of the major themes we observed after talking to many scientific users was friction when writing in reStructuredText (reST). reST is a powerful markup language commonly found in the Python ecosystem; it’s what you use to write docstrings and narrative documentation with Sphinx. However, nowadays Markdown is a much more pervasive language, and the tooling around it is more developed and user friendly. Read the Docs used to maintain an extension for writing CommonMark, the standardized version of Markdown, inside Sphinx, which served many users well. However, in 2021 we decided to deprecate it in favor of Markedly Structured Text or MyST, an extensible markup language created by the amazing Executable Books Project that combines the powerful features of reST with the universality of Markdown.
I wanted to promote MyST as much as possible and help folks that wanted to move away from reST make the transition, and to do that I created MySTyc, a small web application that converts reST to MyST on the fly. Literally a weekend hack, but quite effective!
The application used to work in this way: For every keystroke (with some throttling; keep reading), the front end sent the reST markup to the server, and the server converted it to MyST using rst-to-myst and returned the result to the front end to display it. This worked beautifully, but there was a catch.
MySTyc was hosted by Heroku, which was a very convenient option back then; it offered a Python runtime, automatic deployments from GitHub, and a generous free tier. However, it also imposed several limitations:
Heroku had a “go to sleep” functionality for applications using the free tier that turned off the back end after a period of inactivity. As a result, the application would consume fewer resources and easily stay within the limits of the free tier. However, for anyone having to “wake up” the application, the latency was notably high (in the order of 10 seconds).
Sending a request for every keystroke hammered the server unnecessarily and harmed performance. I solved this on the front-end side by throttling the conversion function with an extra JavaScript dependency.
Even with these guardrails in place, it was unclear if MySTyc could stay responsive or cost free in the event of a sudden traffic spike, or a significant increase in daily usage. But before I had the opportunity to find out, Heroku announced the end of their free product plans. It was a signal that I needed to look for alternatives.
My first idea was to move the application to some other hosting service that still offered a free tier; fortunately, there are many alternatives to Heroku and people were eager to share recommendations in the wake of the fateful announcement. However, after successfully migrating the application to a different service, I found that all of its main limitations persisted; I still needed to throttle the requests, and it was still unclear whether the architecture would stand a “Hacker News effect.”
After thinking a bit more, and inspired by the success of projects like JupyterLite, I considered moving all the logic to the front end. The idea was very promising; without a back end having to relay the requests, MySTyc could essentially be served as static HTML with some JavaScript, leveraging the user’s computer to perform all the processing and freeing me from having to maintain a hosted Python application. How hard could it be (said no one, ever)?
After some browsing, I sadly could not find any reStructuredText parsers written in JavaScript. Therefore, my only option was to essentially rewrite rst-to-myst in JavaScript and hope for the best—definitely too much effort for a weekend project. And then it dawned on me: What if I could run Python in the browser?
I have been following the different efforts to run Python in the browser for many years; almost a decade ago a close colleague poured a lot of effort into Brython, and later I became a fan of Pyodide (which was initially incubated at Mozilla then transitioned to an independent project). Hence my excitement when I saw Anaconda’s PyCon US 2022 PyScript announcement!
Since the project was still quite young, I wasn’t sure I’d be able to run all the necessary dependencies in the browser. But after some experimentation, it became apparent that rst-to-myst and all its dependencies successfully ran in Pyodide, and therefore could be used in PyScript!
The setup process was a breeze; first, I followed the installation instructions and added these two lines to my HTML:
Then, I declared the Python dependencies and paths to my custom scripts:
Finally, I combined the Document Object Model (DOM) capabilities of PyScript with my conversion code:
The result: No more back end, the Python logic was running successfully in the browser, and I could trivially deploy the result to GitHub Pages. Fantastic!
Even though pip has a quite sophisticated dependency resolution algorithm, micropip is much simpler and might fail to resolve some complex situations. For example, at the time of writing, micropip fails to install rst-to-myst on its own:
The solution in this case is easy; avoid installing the problematic dependency by pinning an older version of ruamel-yaml:
Finally, the current version incorrectly translates some reserved HTML characters like “<,” “>,” and so forth, which adds some extra backslashes to the converted text. This is a known issue that will hopefully be addressed in the next PyScript version.
The Python ecosystem is vast, but still largely confined to the back end. For some applications, having a back end can be too limiting—especially when no sophisticated functionalities like user authentication or background tasks are needed. Pyodide and PyScript are bringing the joy of working with Python to front-end programming, and even though the projects are still in their infancy, they already offer a perfect solution for small applications with few dependencies. Looking forward to running more Python in the browser!
Juan Luis (he/him/él) is an aerospace engineer with a passion for STEM, programming, outreach, and sustainability. He works as a data scientist advocate at Orchest, where he empowers data scientists by building an open-source, scalable, easy-to-use workflow orchestrator. He has worked as a developer advocate at Read the Docs, as a software engineer in the space, consulting, and banking industries, and as a Python trainer for several private and public entities.
Apart from being a long-time user and contributor to many projects in the scientific Python stack (NumPy, SciPy, Astropy), Juan Luis has published several open-source packages, the most important being poliastro, an open-source Python library for orbital mechanics used in academia and industry.
Finally, Juan Luis is the founder and former chair of the Python España association, the point of contact for the Spanish Python community, a former organizer of PyCon Spain, which attracted more than 800 attendees to its last in-person edition in 2022, and a current organizer of the PyData Madrid monthly meetups.
Anaconda is amplifying the voices of some of its most active and cherished community members in a monthly blog series. If you’re a Maker who has been looking for a chance to tell your story, elaborate on a favorite project, educate your peers, and build your personal brand, consider submitting an abstract. For more details and to access a wealth of educational data science resources and discussion threads, visit Anaconda Nucleus.
Talk to one of our financial services and banking industry experts to find solutions for your AI journey.
