Ir al contenido principal

Ralsina.Me — El sitio web de Roberto Alsina

New Nikola Thing: scripts

Niko­la is a stat­ic site gen­er­a­tor, and it knows its au­di­ence: Nerd­s, pro­gram­mer­s, sci­ence peo­ple, and the like. Oh, and me. I most­ly de­vel­op it for me.

One im­por­tant thing for this cat­e­go­ry of tools is that they should cater to what the users want to do, and al­so to how they want to do it.

So, faced with the need to do things like "set this spe­cif­ic meta­da­ta field in these 490 posts out of the 1450 you have" ... edit­ing them man­u­al­ly is not go­ing to hap­pen.

Sure, I could sed/python/what­ev­er my way to do it "au­to­mat­i­cal­ly". But that is go­ing to be aw­ful­ly er­ror prone.

So, I have start­ed a cam­paign to fix it. I want to make Niko­la be the API to its da­ta. This has two sides.

I need to be able to run one-off things

This needs to be easier than creating a Nikola command plugin but less annoying than typing them in nikola console

[ralsina@salma static]$ nikola console
Scanning posts..........done!
Python 3.8.2 (default, Feb 26 2020, 22:21:03) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.


Nikola v8.0.4 -- IPython Console (conf = configuration file, site, nikola_site = site engine, commands = nikola commands)

In [1]:  

The good news is: this is done in git mas­ter!

Now you can create a python script and run it as nikola console -s cool_script.py and the script runs in the same context as the console, so you magically have the site itself and the timeline and the configuration and all the good stuff ready to use.

I need the API to be useful

And this is where Niko­la has ... not been a good boy. Since it was meant to gen­er­ate stat­ic sites, it's pret­ty good about of­fer­ing you ways to know things about your da­ta.

Want to know what is the de­scrip­tion of the tags ap­plied to the post in slovene? it to­tal­ly can do that in two lines.

Want to add a tag to a post? Sor­ry dude, that's im­pos­si­ble.

So, I am adding these things, slow­ly.

And that's the cur­rent sta­tus.

Tribu Greenwich 0

Ha­ce una se­ma­na mas o me­nos cam­bié de tra­ba­jo re­pen­ti­na­men­te. Me des­pi­die­ron1 al me­dio­día y me lle­vó has­ta tar­de esa mis­ma tar­de en­con­trar un nue­vo la­bu­ro.2

Per­dí una pro­me­sa de vi­si­ta a las ofi­ci­nas en Lju­bl­ja­na, ga­né una pro­me­sa de vi­si­ta a ofi­ci­nas en Lon­dres, el di­ne­ro ven­drá de un lu­gar di­fe­ren­te, Se va el clon as­ync de Fla­sk, en­tra Djan­go, co­sas así.

No pen­sé en ese mo­men­to que acon­te­ce­ría un cam­bio mu­cho más fun­da­men­ta­l, un cam­bio de tri­bus.3 Aban­do­né la tri­bu en que vi­ví mi vi­da, la ha­bi­tual­men­te GM­T-3 de mi na­ci­mien­to y me uní a Greenwi­ch 0.

Aho­ra me le­van­to a lo que es­tos ex­tran­je­ros a mi al­re­de­dor afir­man que son las 4:45 AM, así lle­go có­mo­do a mi stan­dup dia­rio, que es a mis 9:15. Es­to tie­ne efec­tos in­te­re­san­tes en mis ... to­do.

¿Mi hi­jo no la­vó los pla­tos co­mo se su­po­ne que ha­ce, y pro­me­tió ha­cer­lo "des­pué­s"? Bue­no, si me des­pier­to y los pla­tos no es­tán la­va­do­s, lo des­pier­to a él, y los pla­tos es­tán lim­pios an­tes del ama­ne­ce­r.

Ten­go ~4 ho­ras de tra­ba­jo tran­qui­lo con la ca­sa en si­len­cio, que en es­ta era en­cua­ren­te­na­da es un mon­tón.

Ter­mino de tra­ba­jar pa­ra las 5:30, cuan­do mi fa­mi­lia ju­ra que son las 13:30, así que pue­do al­mor­zar con ellos sin cor­tar el tra­ba­jo que ya ter­mi­né, y des­pués duer­mo la sies­ta, co­mo un gran­je­ro.

Pue­do des­ayu­nar dos ve­ces al día, tres si le pon­go ga­rra.

Me voy a la ca­ma a las 3A­M/11­PM, así que me sien­to un tras­no­cha­do­r, duer­mo 7 u 8 ho­ras al día pe­ro no to­das jun­ta­s. Es­toy con un li­ge­ro je­tlag en es­te mo­men­to, o tal vez en to­do mo­men­to, sin via­jar en el es­pa­cio, so­lo en el tiem­po.

Veo el ama­ne­cer to­dos los día­s.

Los días son lar­gos, co­mo es lar­go el día más lar­go del ve­ra­no, pe­ro en oto­ño. En ve­rano se­rán más lar­go­s, eter­no­s, días no­rue­go­s, 16 ho­ras de lu­z.

¿Có­mo me sien­to al res­pec­to? No sé. Es co­mo una mó­di­ca aven­tu­ra sin de­jar la ca­sa. Es co­mo vi­vir una se­ma­na sin usar la le­tra "M", co­mo co­mer en­sa­la­da ru­sa y no to­car las za­naho­ria­s, co­mo ir a tra­ba­jar a un ba­rrio dis­tin­to don­de to­do el mun­do es­tá dor­mi­do y no hay au­tos y los pá­ja­ros can­tan fuer­te y es de día en la pan­ta­lla de to­dos los otros y ...

Bue­no, no tan Lo­ve­cra­ft, che.


  1. No rea­l­­men­­te, soy co­n­­tra­­tis­­ta, pe­­ro bue­­­no, ya sa­­be­­n. 

  2. No rea­l­­men­­te, soy co­n­­tra­­tis­­ta, pe­­ro bue­­­no, ya sa­­be­­n. 

  3. Ver http­s://en.wiki­­pe­­dia.o­r­­g/wiki/Ea­s­­te­r­n_S­­tan­­da­r­­d_­­Tri­­be  

Experimental Nikola Plugin: tagged_pages

There has been an is­sue open in Niko­la for a lit­tle over four years.

The is­sue ti­tle is "Sup­port tags in pages" ... and I had been in­ten­tion­al­ly not do­ing any­thing about it for a (bad) rea­son.

I did not un­der­stand what the is­sue was.

Sure, support tags in pages ... how? What's that supposed to mean? What happens when you tag a page? Should pages be listed along posts in each tag? How? How does that interact with TAG_PAGES_ARE_INDEXES which turns each tag into a mini-blog?

Well, it was a bad rea­son be­cause it could have been fixed by just talk­ing to user­s. So, I did. And I wrote a min­i­mal vi­able plug­in: tagged_­pages

It's far from per­fec­t, it's ba­si­cal­ly a copy of the tags plug­in with some stuff re­named, some delet­ed and some re­placed. But it's a start. And now it can be it­er­at­ed on.

So, talk to user­s, I guess.

Nikola's latest release back in the snap store

A few years back when I was work­ing at Canon­i­cal, I made Niko­la pack­ages for the Snap store ... which sad­ly got ne­glect­ed a fair amount af­ter I stopped work­ing there.

SO ... while I don't work there, there is no rea­son for ne­glec­t, so I just fixed the build, up­dat­ed some snapcraft.yml de­tails that had rust­ed from dis­use, checked the au­to­mat­ed build and pro­mot­ed cur­rent git mas­ter to the "stable" chan­nel.

What does this mean for you?

Well, prob­a­bly noth­ing, but if you want to try Niko­la, you can do this:

snap install nikola

At least in Ubuntu, Ubuntu derivatives and Arch derivatives if you have snapd installed, and end up with a working installation of Nikola.

Now, is this the rec­om­mend­ed way to get it? No. This has a num­ber of prob­lems be­cause of the na­ture of snap pack­ag­ing.

  • You can't launch bi­na­ries so some fea­tures will just not work (ex­am­ple: niko­la serve -b).

  • You can't in­­stall python pack­­ages in­­­to the snap, so if you in­­stall a plug­in with a python de­pen­­den­­cy, it will not work un­­less by chance that de­pen­­den­­cy is there al­ready (I added some!)

  • I am not test­ing it much, so there is no guar­an­­tee it works on any giv­en day (But it should!)

If you are us­ing Niko­la as a snap and run in­to any is­sues, ping me via an is­sue. Some­times I may take a while to get to it, but I will get to it.

https://pbs.twimg.com/media/EVhPfXKXsAEVKoc?format=png&name=900x900

Yes, this may take a while.

On how I accidentally may have made a feature 75% faster.

UP­DATE: Af­ter fix­ing bugs, it turns out the im­prove­ment is rough­ly 25%, not 75%, which is still nice :-)

Im­age gal­leries is one of my favourite fea­tures in Niko­la

I use them in my site and they are awe­some! Just dump a bunch of im­ages in a fold­er and you will have them nice­ly pre­sent­ed. Then you can add a file with a fold­er de­scrip­tion amd so on.

One thing I did not like was that they were pret­ty slow. My site has 3290 im­ages in the gal­leries sec­tion, and I dread­ed pro­cess­ing them be­cause they took for­ev­er, where for­ev­er means around 5 min­utes.

Un­re­lat­ed to per­for­mance, we had a fea­ture re­quest for sup­port­ing mul­ti­ple thumb­nail sizes, which is use­ful for things like pre­sent­ing the ide­al im­age size for your dis­play and such.

Well, to support that someday, a good first step is to have our generic image processor class support more than one thumbnail size. So, just take the scale_image implementation and have it support more than one size and more than one destination path, then loop over those, and that's that. Right?

OTO­H, it turns out we pro­cessed each im­age TWICE be­cause we would clean EXIF da­ta and/or re­size the "o­rig­i­nal" im­ages. Some­times the source im­ages have res­o­lu­tions that sim­ply make no sense on a web­site!

So, since our implementation of scale_image only accepted one destination, we would:

  1. read orig­i­nal im­age -> cleanup -> re­size to thumb size -> save
  2. read orig­i­nal im­age -> cleanup -> re­size to "large" size -> save

So, since I had now a ver­sion that could do the "re­size" and "save" parts on one cal­l, this be­came:

  1. read orig­i­nal im­age -> cleanup
  2. re­size to thumb size -> save
  3. re­size to "large" size -> save

And the re­al­ly slow part is ... read­ing the orig­i­nal im­age. So this sud­den­ly made the whole process take 25% of the time it took be­fore.

In my spe­cif­ic site, it went down from 356 sec­onds to 112 sec­ond­s. That's a 70% de­crease in ren­der­ing time. Which is big. And in code I first wrote around 2013, that is huge.

So, the les­son is ... I am a crap­py pro­gram­mer, may­be? Or used to be? Or maybe just that even old, ma­ture code could still have large op­por­tu­ni­ties for im­prove­men­t?

Who knows.


Contents © 2000-2020 Roberto Alsina