A while back I saw glow and I won­dered ... how hard could it be to write that?

I mean, it looks im­pres­sive! It ren­ders mark­down to a ter­mi­nal!

Well, it's not all that hard if you have a nice mark­down pars­er you can hack, so I spent a cou­ple of hours and wrote it.

It's called mark­term and it ... well, it's done?

You can use it as a li­brary from Crys­tal, or you can use the mark­term bi­na­ry.

It can syn­tax high­light code (as long as you have chro­ma in­stalled), it does the right thing when piped to an­oth­er pro­gram, and it tries to look good but not gaudy.

What will I use it for?

Show­ing CLI pro­gram's help. I am us­ing do­copt late­ly and the for­mat of the do­copt in­put is al­most mark­down. With a bit of flair it can be both, and us­ing mark­term I can show it in a ter­mi­nal in a much nicer way than usu­al.

Ex­tra: why am I start­ing so many projects late­ly?

• Be­cause it makes me hap­py
• Be­cause in most cas­es it's not code that will need main­te­nance. I ex­pect mark­term to just work for the next 10 years with­out me touch­ing it, if need­ed.
• Be­cause I can

I have fun in weird ways.

For ex­am­ple, I may read about a tool and then de­cide to trans­late a clone of the tool from one pro­gram­ming lan­guage to an­oth­er. And then af­ter it works but the code is aw­ful, re­write it, then re­lease it

In a week­end. It helps that the thing is small (un­der 200LOC) but it's still an in­di­ca­tion that I am hav­ing fun do­ing it.

So, what is it? I wrote Cryc­co, a Crys­tal tool sim­i­lar to Doc­co, which is a lit­er­ate pro­gram­ming tool.

A what?

A lit­er­ate pro­gram­ming tool. Mean­ing a tool that lets you:

• Gen­er­ate a pro­gram's code out of a doc­u­ment
• Gen­er­ate a doc­u­ment out of a pro­gram's code
• Gen­er­ate a doc­u­ment and a pro­gram's code out of a "lit­er­ate doc­u­men­t"

That may be do­ing the op­po­site of ex­plain­ing what it ac­tu­al­ly does, so here's an ex­am­ple.

I wrote Cryc­co which if you look at it looks like a nor­mal, al­though very heav­i­ly com­ment­ed crys­tal pro­gram.

But if you pass the source code for Cryc­co through cryc­co it pro­duces the cryc­co web­site ... go take a look!

As you can see it's a de­tailed ex­pla­na­tion of what the pro­gram does and how the code work­s. It's the same thing on­ly pre­sent­ed dif­fer­ent­ly.

It's much eas­i­er to fol­low the ex­pla­na­tions when they run along­side the code rather than in­ter­rupt­ing it, is­n't it?

Let's see another example. Crycco uses a YAML file for some aspects of its configuration. Usually I would just add some comments and tell you to go and read the file.

But that would present you with ... well, YAML and com­ments. Github does an ad­mirable job con­vey­ing that data, but com­pare it with Cryc­co's lan­guages.yml

What do you think is eas­i­er to read and more un­der­stand­able?

So, there it is, and it's pret­ty us­able, even if I still want to ex­tend it in some ways (like... let's gen­er­ate Mark­down! And use that to gen­er­ate a PDF! Like it's 1989 and we are us­ing nuwe­b!)

Yes, lit­er­ate pro­gram­ming has been dead for a cou­ple of decades, but it's far from a worth­less idea. It can be used, it can be used pur­pose­ful­ly and be valu­able, and I have ideas that can be achieved us­ing Cryc­co or a sim­i­lar tool.

Hope it's use­ful for some­one else too.

I am work­ing a bit (s­low­ly) on Nicol­i­no a stat­ic site gen­er­a­tor writ­ten in Crys­tal. One of the things it does is, it ren­ders mark­down files.

Since the mark­down us­age is lim­it­ed to, like, 3 lines, I thought "Why not try all the mark­down li­braries and see which one is faster?"

So, I did.

The bench­mark is sim­ple:

• An emp­ty Nicol­i­no site.
• 4000 sim­ple mark­down files (a few lorem ip­sum para­graph­s)
• Nicol­i­no com­piled in re­lease mode
• Ren­der the whole site 10 times, av­er­age the last 7 runs
• De­fault con­figs for ev­ery­thing

All this on this ma­chine:

        a8888b.           Host        -  ralsina@mindy
       d888888b.          Machine     -  Micro Computer (HK) Tech Limited Default string UM560 XT
       8P"YP"Y88          Kernel      -  6.9.9-arch1-1
       8|o||o|88          Distro      -  EndeavourOS
       8'    .88          DE          -  Qtile
       8`._.' Y8.         Packages    -  1340 (pacman), 1 (cargo), 18446744073709551615 (Homebrew)
      d/      `8b.        Terminal    -  zellij
     dP        Y8b.       Shell       -  fish
    d8:       ::88b.      Uptime      -  10d 18h 7m
   d8"         'Y88b      CPU         -  AMD Ryzen 5 5600H with Radeon Graphics (12)
  :8P           :888      Resolution  -  4096x2160, 1920x1080
   8a.         _a88P      CPU Load    -  15%
 ._/"Yaa     .| 88P|      Memory      -  9.2 GB/24.5 GB
 \    YP"    `|     `.
 /     \.___.d|    .'
 `--..__)     `._.'

Of course Nicol­i­no does some oth­er things be­sides ren­der­ing mark­down, so to iso­late that part I ran a cou­ple of dum­my im­ple­men­ta­tion­s:

• NOOP It does noth­ing. When asked to com­pile a string to mark­down, it re­turns the same string.
• TOEMP­TY It does less than noth­ing. When asked to com­pile a string to mark­down, it re­turns an emp­ty string.

So, the dif­fer­ence be­tween emp­ty and noop is prob­a­bly what it takes to ren­der some tem­plates and store the out­put in disk.

Then I ran the bench­mark for the 5 mark­down li­braries I could find:

• markd
• cr-dis­count
• crys­tal-c­mark
• cr-­mark-gfm
• luce

Here is a chart show­ing the times in sec­onds for each li­brary with­out the time that is used in oth­er things (NOOP's time, which was 2.62 sec­ond­s).

So, luce is much, much, much slow­er, and crys­tal-c­mark is the fastest. And cr-dis­count (MY OWN BIND­ING) is much slow­er than the oth­er­s, which is a bit dis­ap­point­ing.

On the oth­er hand, there are two sides to op­ti­miz­ing. One is choos­ing the fastest li­brary, the oth­er is not car­ing if the dif­fer­ence is small in ab­so­lute, even if it's large in rel­a­tive.

What does that mean?

This is over 4000 doc­u­ments.

So, while cr-dis­count is slow­er, it's still ren­der­ing 4000 doc­u­ments in 0.66 sec­ond­s. That's 0.000165 sec­onds per doc­u­men­t.

That's 1.65 tenths of a thou­sand of a sec­ond. That's over 6000 doc­u­ments per sec­ond.

If the nor­mal use­case was to ren­der thou­sands of doc­u­ments, then that would make a dif­fer­ence. But it's not. It's usu­al­ly 3 doc­u­ments.

So, as long as cr-dis­count has any fea­ture I need and it's not in the oth­er li­braries, it's fine to use it, and the same goes for the oth­ers (ex­cept luce, I guess, but stil­l: 757 doc­u­ments per sec­ond is not bad).

Update: Compiling discount with -O3 brings it down to 0.48, which is bettern than 0.66 but makes no difference for the conclusions.

I host a num­ber of stat­ic web­sites on my per­son­al serv­er. Over the years, they have been served by a va­ri­ety of web server­s, but I re­cent­ly de­cid­ed I want­ed to make it easy.

So, I looked for a mod­ern and min­i­mal­is­tic web serv­er that sup­port­ed a fea­ture of the ven­er­a­ble thttpd

That fea­ture is au­to­mat­ic vir­tu­al hosts. The main idea is that you have a fold­er and in there a bunch of sub­fold­er­s, each one is a site. And the name of the fold­er is the name of the site.

So, if I have /srv/ralsina.me then when a user asks for http://ralsina.me they get that. And if I have /srv/whatever.com and the user asks for whatever.com they get that.

Al­so, if a site is reach­able with more than one name, then just add sym­links and it should just work.

I found a way to do it us­ing Ng­inX but that's far from min­i­mal­is­tic.

So, I looked around and found that Cad­dy was pret­ty close ex­cept it did­n't do the au­to­mat­ic stuff.

So, I wrote the miss­ing bit, threw ev­ery­thing in a dock­er con­tain­er and now I have a 30-minute project that serves all my stat­ic sites.

Here's the dock­er com­pose I am us­ing:

version: "3.8"
services:
  web:
    logging:
      driver: journald
      options:
        tag: web
    container_name: web
    stdin_open: true
    tty: true
    ports:
      - 8080:8888
    volumes:
      - /data/websites/:/srv/
    image: ghcr.io/ralsina/caddy-static-arm64:latest
    restart: always
networks: {}

And here's the pro­jec­t: cad­dy-stat­ic

It probably needs some tweaking for anyone else specially in the Caddyserver config and maybe in the template it uses to generate sites which is hardcoded.

If any­one ac­tu­al­ly us­es this I can make those things con­fig­urable.

I have been us­ing a lot of Crys­tal for per­son­al projects for about a year. Think of it like "na­tive Ru­by with type in­fer­ence" or some­thing like that.

But this post is not about the lan­guage it­self, it's about one part of its ecosys­tem: the shard­s.

The Nice

Shards are like Ru­by gem­s, or like what­ev­er go calls them nowa­days, or rust crates, just "li­braries" that are writ­ten most­ly in crys­tal.

The thing is ... they are awe­some!

Suppose you want to use markdown in your app. You go to shards.info which crawls GitHub and finds things written in Crystal. You search for markdown and you find a few that look good. Suppose you want to use my cr-discount shard.

You just add the shard to your shard.yml and use it, like this:

dependencies:
  cr-discount:
    github: ralsina/cr-discount

Then in your code:

require "cr-discount"

markdown = "This *is* **markdown**"
html = Discount.compile(markdown)

And that's it, you just in­te­grat­ed it in­to your build, your code is us­ing it, and that's all there is to it.

But not on­ly that, sup­pose you are us­ing a shard like docr but you find a tiny bug or two. You fork it, fix the bugs and cre­ate a PR. If the au­thor is ac­tive, they will merge it, and you can go back to us­ing the shard, but even if they aren't ... you can just use your fork in the mean­time.

Just use ralsina/docr instead of marghidanu/docr in your shard.yml and you are using my fix.

And there's more! Sup­pose you find some code in your projects that you keep re­peat­ing. For ex­am­ple, I al­ways use the same log­ging set­up in CLI app­s:

• Col­or­ful if it makes sense
• Con­fig­urable ver­bosi­ty
• Er­rors and worse in stderr
• In­fo and bet­ter in std­out

So, why copy that ev­ery­where? Just make it a tiny shard.

There is no overhead in your code, you don't have to ask users to install anything, you just add it to your shard.yml and it's there for everyone that cares.

The Not So Nice

Of course noth­ing is free of cost.

• Be­cause it's easy to cre­ate shard­s, it's easy to aban­don shard­s.
• Be­cause it's easy to fork shard­s, it's easy to at­om­ize the ecosys­tem.
• Be­cause it's easy to find shards and it's de­cen­tral­ized, it's triv­ial to poi­son the sup­ply chain (although TBH it seems to be easy enough in any lan­guage).

So, you have to be care­ful. You have to check if the shard is main­tained, if it's the best one for the job, if you fork it you need to com­mit to keep­ing your fork work­ing, you al­ways need to push the PR even if the au­thor does­n't pick it up be­cause it's there for oth­er user­s.

So, con­clu­sion­s... they are what they are. For me they are the most prac­ti­cal thing ev­er and I of­ten wish Python had some­thing like them, but they are al­so quite ... scary? And see­ing the aban­doned shards makes me sad for a lan­guage that should be much more pop­u­lar than it is.