Skip to main content

Ralsina.Me — Roberto Alsina's website

New Nikola Thing: scripts

Niko­la is a stat­ic site gen­er­a­tor, and it knows its au­di­ence: Nerd­s, pro­gram­mer­s, sci­ence peo­ple, and the like. Oh, and me. I most­ly de­vel­op it for me.

One im­por­tant thing for this cat­e­go­ry of tools is that they should cater to what the users want to do, and al­so to how they want to do it.

So, faced with the need to do things like "set this spe­cif­ic meta­da­ta field in these 490 posts out of the 1450 you have" ... edit­ing them man­u­al­ly is not go­ing to hap­pen.

Sure, I could sed/python/what­ev­er my way to do it "au­to­mat­i­cal­ly". But that is go­ing to be aw­ful­ly er­ror prone.

So, I have start­ed a cam­paign to fix it. I want to make Niko­la be the API to its da­ta. This has two sides.

I need to be able to run one-off things

This needs to be easier than creating a Nikola command plugin but less annoying than typing them in nikola console

[ralsina@salma static]$ nikola console
Scanning posts..........done!
Python 3.8.2 (default, Feb 26 2020, 22:21:03) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.


Nikola v8.0.4 -- IPython Console (conf = configuration file, site, nikola_site = site engine, commands = nikola commands)

In [1]:  

The good news is: this is done in git mas­ter!

Now you can create a python script and run it as nikola console -s cool_script.py and the script runs in the same context as the console, so you magically have the site itself and the timeline and the configuration and all the good stuff ready to use.

I need the API to be useful

And this is where Niko­la has ... not been a good boy. Since it was meant to gen­er­ate stat­ic sites, it's pret­ty good about of­fer­ing you ways to know things about your da­ta.

Want to know what is the de­scrip­tion of the tags ap­plied to the post in slovene? it to­tal­ly can do that in two lines.

Want to add a tag to a post? Sor­ry dude, that's im­pos­si­ble.

So, I am adding these things, slow­ly.

And that's the cur­rent sta­tus.

Greenwich 0 Tribe

A week ago or so I changed jobs sud­den­ly. I was fired1 at noon, and it took me all the way un­til late af­ter­noon to find an­oth­er job2.

So, I lost a promised vis­it to Ljubl­jana of­fices, I gained a promised vis­it to Lon­don of­fices, mon­ey would come from a dif­fer­ent place, Async-flask-­clone was out, Djan­go was in, and so on.

Lit­tle did I know that a much more fun­da­men­tal change was in store, I would have to change tribes3. I left the tribe where I had lived my whole life, the usu­al­ly GMT-3 tribe of my birth, and joined Green­wich 0.

So, I now wake up at what these for­eign­ers around me claim is 4:45 AM, so I can get to my dai­ly standup at my 9:15. This is hav­ing in­ter­est­ing ef­fects in my ... ev­ery­thing.

My son did not wash the dish­es as he's sup­posed to, and promised to do it "later"? Well, I wake up, if dish­es are not done, I wake him up, and dish­es get done be­fore dawn.

I get ~4 hours of qui­et work, which in these quar­an­tined age is a lot.

I am done by my 5:30, which is my fam­i­ly's 13:30, so that means I can have lunch with ev­ery­one and then take a nap, like a farmer.

I can have two, maybe three brak­fasts a day if I try hard.

I go to bed at 3AM/11P­M, so I feel like I am a night owl, sleep 7 to 8 hours a day, but not all at once. I am slight­ly jet­lagged right now, or maybe all the time, but I did not move in space, on­ly in time.

I see dawn ev­ery day.

Days are long, like the long­est day of sum­mer is long, on­ly in fal­l. In the sum­mer they will be even longer, for­ev­er days, nor­we­gian days, 16 hours of light.

How do I feel about this? Well, I don't know. It's like a mod­est ad­ven­ture with­out leav­ing the house. It's like try­ing to live a week with­out us­ing the let­ter "M", like try­ing to eat rus­sian sal­ad and leave the car­rots un­touched, like go­ing to work in a strange neigh­bor­hood where ev­ery­one is asleep and cars don't run, and birds sing loud and it's day­time on ev­ery­one else's screen and...

Ok, not quite as Love­craftian as that sound­s, peo­ple.


  1. Well, not re­al­­ly, I was a con­­trac­­tor, but, you know. 

  2. Well, not re­al­­ly, I am a con­­trac­­tor here too, but, you know. 

  3. See http­s://en.wikipedi­a.org/wik­i/East­­ern_­S­­tan­­dard­­_Tribe  

Experimental Nikola Plugin: tagged_pages

There has been an is­sue open in Niko­la for a lit­tle over four years.

The is­sue ti­tle is "Sup­port tags in pages" ... and I had been in­ten­tion­al­ly not do­ing any­thing about it for a (bad) rea­son.

I did not un­der­stand what the is­sue was.

Sure, support tags in pages ... how? What's that supposed to mean? What happens when you tag a page? Should pages be listed along posts in each tag? How? How does that interact with TAG_PAGES_ARE_INDEXES which turns each tag into a mini-blog?

Well, it was a bad rea­son be­cause it could have been fixed by just talk­ing to user­s. So, I did. And I wrote a min­i­mal vi­able plug­in: tagged_­pages

It's far from per­fec­t, it's ba­si­cal­ly a copy of the tags plug­in with some stuff re­named, some delet­ed and some re­placed. But it's a start. And now it can be it­er­at­ed on.

So, talk to user­s, I guess.

Nikola's latest release back in the snap store

A few years back when I was work­ing at Canon­i­cal, I made Niko­la pack­ages for the Snap store ... which sad­ly got ne­glect­ed a fair amount af­ter I stopped work­ing there.

SO ... while I don't work there, there is no rea­son for ne­glec­t, so I just fixed the build, up­dat­ed some snapcraft.yml de­tails that had rust­ed from dis­use, checked the au­to­mat­ed build and pro­mot­ed cur­rent git mas­ter to the "stable" chan­nel.

What does this mean for you?

Well, prob­a­bly noth­ing, but if you want to try Niko­la, you can do this:

snap install nikola

At least in Ubuntu, Ubuntu derivatives and Arch derivatives if you have snapd installed, and end up with a working installation of Nikola.

Now, is this the rec­om­mend­ed way to get it? No. This has a num­ber of prob­lems be­cause of the na­ture of snap pack­ag­ing.

  • You can't launch bi­na­ries so some fea­tures will just not work (ex­am­ple: niko­la serve -b).

  • You can't in­­stall python pack­­ages in­­­to the snap, so if you in­­stall a plug­in with a python de­pen­­den­­cy, it will not work un­­less by chance that de­pen­­den­­cy is there al­ready (I added some!)

  • I am not test­ing it much, so there is no guar­an­­tee it works on any giv­en day (But it should!)

If you are us­ing Niko­la as a snap and run in­to any is­sues, ping me via an is­sue. Some­times I may take a while to get to it, but I will get to it.

https://pbs.twimg.com/media/EVhPfXKXsAEVKoc?format=png&name=900x900

Yes, this may take a while.

On how I accidentally may have made a feature 75% faster.

UP­DATE: Af­ter fix­ing bugs, it turns out the im­prove­ment is rough­ly 25%, not 75%, which is still nice :-)

Im­age gal­leries is one of my favourite fea­tures in Niko­la

I use them in my site and they are awe­some! Just dump a bunch of im­ages in a fold­er and you will have them nice­ly pre­sent­ed. Then you can add a file with a fold­er de­scrip­tion amd so on.

One thing I did not like was that they were pret­ty slow. My site has 3290 im­ages in the gal­leries sec­tion, and I dread­ed pro­cess­ing them be­cause they took for­ev­er, where for­ev­er means around 5 min­utes.

Un­re­lat­ed to per­for­mance, we had a fea­ture re­quest for sup­port­ing mul­ti­ple thumb­nail sizes, which is use­ful for things like pre­sent­ing the ide­al im­age size for your dis­play and such.

Well, to support that someday, a good first step is to have our generic image processor class support more than one thumbnail size. So, just take the scale_image implementation and have it support more than one size and more than one destination path, then loop over those, and that's that. Right?

OTO­H, it turns out we pro­cessed each im­age TWICE be­cause we would clean EXIF da­ta and/or re­size the "o­rig­i­nal" im­ages. Some­times the source im­ages have res­o­lu­tions that sim­ply make no sense on a web­site!

So, since our implementation of scale_image only accepted one destination, we would:

  1. read orig­i­nal im­age -> cleanup -> re­size to thumb size -> save
  2. read orig­i­nal im­age -> cleanup -> re­size to "large" size -> save

So, since I had now a ver­sion that could do the "re­size" and "save" parts on one cal­l, this be­came:

  1. read orig­i­nal im­age -> cleanup
  2. re­size to thumb size -> save
  3. re­size to "large" size -> save

And the re­al­ly slow part is ... read­ing the orig­i­nal im­age. So this sud­den­ly made the whole process take 25% of the time it took be­fore.

In my spe­cif­ic site, it went down from 356 sec­onds to 112 sec­ond­s. That's a 70% de­crease in ren­der­ing time. Which is big. And in code I first wrote around 2013, that is huge.

So, the les­son is ... I am a crap­py pro­gram­mer, may­be? Or used to be? Or maybe just that even old, ma­ture code could still have large op­por­tu­ni­ties for im­prove­men­t?

Who knows.


Contents © 2000-2020 Roberto Alsina