Benchmarking Markdown in Crystal
I am working a bit (slowly) on Nicolino a static site generator written in Crystal. One of the things it does is, it renders markdown files.
Since the markdown usage is limited to, like, 3 lines, I thought "Why not try all the markdown libraries and see which one is faster?"
So, I did.
The benchmark is simple:
- An empty Nicolino site.
- 4000 simple markdown files (a few lorem ipsum paragraphs)
- Nicolino compiled in release mode
- Render the whole site 10 times, average the last 7 runs
- Default configs for everything
All this on this machine:
a8888b. Host - ralsina@mindy
d888888b. Machine - Micro Computer (HK) Tech Limited Default string UM560 XT
8P"YP"Y88 Kernel - 6.9.9-arch1-1
8|o||o|88 Distro - EndeavourOS
8' .88 DE - Qtile
8`._.' Y8. Packages - 1340 (pacman), 1 (cargo), 18446744073709551615 (Homebrew)
d/ `8b. Terminal - zellij
dP Y8b. Shell - fish
d8: ::88b. Uptime - 10d 18h 7m
d8" 'Y88b CPU - AMD Ryzen 5 5600H with Radeon Graphics (12)
:8P :888 Resolution - 4096x2160, 1920x1080
8a. _a88P CPU Load - 15%
._/"Yaa .| 88P| Memory - 9.2 GB/24.5 GB
\ YP" `| `.
/ \.___.d| .'
`--..__) `._.'
Of course Nicolino does some other things besides rendering markdown, so to isolate that part I ran a couple of dummy implementations:
- NOOP It does nothing. When asked to compile a string to markdown, it returns the same string.
- TOEMPTY It does less than nothing. When asked to compile a string to markdown, it returns an empty string.
So, the difference between empty and noop is probably what it takes to render some templates and store the output in disk.
Then I ran the benchmark for the 5 markdown libraries I could find:
Here is a chart showing the times in seconds for each library without the time that is used in other things (NOOP's time, which was 2.62 seconds).
So, luce is much, much, much slower, and crystal-cmark is the fastest. And cr-discount (MY OWN BINDING) is much slower than the others, which is a bit disappointing.
On the other hand, there are two sides to optimizing. One is choosing the fastest library, the other is not caring if the difference is small in absolute, even if it's large in relative.
What does that mean?
This is over 4000 documents.
So, while cr-discount is slower, it's still rendering 4000 documents in 0.66 seconds. That's 0.000165 seconds per document.
That's 1.65 tenths of a thousand of a second. That's over 6000 documents per second.
If the normal usecase was to render thousands of documents, then that would make a difference. But it's not. It's usually 3 documents.
So, as long as cr-discount has any feature I need and it's not in the other libraries, it's fine to use it, and the same goes for the others (except luce, I guess, but still: 757 documents per second is not bad).
Update: Compiling discount with -O3 brings it down to 0.48, which is bettern than 0.66
but makes no difference for the conclusions.