Skip to main content

Ralsina.Me — Roberto Alsina's website

On how I accidentally may have made a feature 75% faster.

UP­DATE: Af­ter fix­ing bugs, it turns out the im­prove­ment is rough­ly 25%, not 75%, which is still nice :-)

Im­age gal­leries is one of my favourite fea­tures in Niko­la

I use them in my site and they are awe­some! Just dump a bunch of im­ages in a fold­er and you will have them nice­ly pre­sent­ed. Then you can add a file with a fold­er de­scrip­tion amd so on.

One thing I did not like was that they were pret­ty slow. My site has 3290 im­ages in the gal­leries sec­tion, and I dread­ed pro­cess­ing them be­cause they took for­ev­er, where for­ev­er means around 5 min­utes.

Un­re­lat­ed to per­for­mance, we had a fea­ture re­quest for sup­port­ing mul­ti­ple thumb­nail sizes, which is use­ful for things like pre­sent­ing the ide­al im­age size for your dis­play and such.

Well, to support that someday, a good first step is to have our generic image processor class support more than one thumbnail size. So, just take the scale_image implementation and have it support more than one size and more than one destination path, then loop over those, and that's that. Right?

OTO­H, it turns out we pro­cessed each im­age TWICE be­cause we would clean EXIF da­ta and/or re­size the "o­rig­i­nal" im­ages. Some­times the source im­ages have res­o­lu­tions that sim­ply make no sense on a web­site!

So, since our implementation of scale_image only accepted one destination, we would:

  1. read orig­i­nal im­age -> cleanup -> re­size to thumb size -> save
  2. read orig­i­nal im­age -> cleanup -> re­size to "large" size -> save

So, since I had now a ver­sion that could do the "re­size" and "save" parts on one cal­l, this be­came:

  1. read orig­i­nal im­age -> cleanup
  2. re­size to thumb size -> save
  3. re­size to "large" size -> save

And the re­al­ly slow part is ... read­ing the orig­i­nal im­age. So this sud­den­ly made the whole process take 25% of the time it took be­fore.

In my spe­cif­ic site, it went down from 356 sec­onds to 112 sec­ond­s. That's a 70% de­crease in ren­der­ing time. Which is big. And in code I first wrote around 2013, that is huge.

So, the les­son is ... I am a crap­py pro­gram­mer, may­be? Or used to be? Or maybe just that even old, ma­ture code could still have large op­por­tu­ni­ties for im­prove­men­t?

Who knows.

Python Faster than C [*]

[*] Terms and con­di­tions may ap­ply.

Evil tricks to make even your [my] crap­py code faster. How much faster? FASTER.

The eng­lish ver­sion of this talk I al­ready gave like 3 times, but, you know, the talk stays the same, you just need to change the au­di­ence!

Python May not be the "Language of the Future", but this is not why.

Google in its in­fi­nite evil put this ar­ti­cle in my phone's news feed to­day. 1

The ti­tle is "Why Python is not the pro­gram­ming lan­guage of the fu­ture" and ... well, I sort of agree with it. The fu­ture is a very long time, so I hope at some point Python will stop be­ing used for any­thing. I even sort of agree with the the­sis that Python will not be as rel­e­vant as it is now in 5 to 10 years, be­cause ... why not?

Al­though it's sur­pris­ing­ly hard to find a lan­guage re­plac­ing an­oth­er in the past. Python most­ly re­placed Perl as the gen­er­al pur­pose script­ing lan­guage, and maybe Python re­placed BA­SIC as the be­gin­ner's first lan­guage, but oth­er than that?

Usu­al­ly peo­ple start do­ing new things, and those things are done in a new­er lan­guage, and the old things are done in the old lan­guages ... FOR­EV­ER.

That is why there are still COBOL pro­gram­mers do­ing busi­ness log­ic in main­frames, there are still FOR­TRAN pro­gram­mers do­ing math code, and there are still C pro­gram­mers do­ing em­bed­ded, and so on, but there are JS pro­gram­mers do­ing we­bapp­s, and Kotlin or Swift pro­gram­mers do­ing mo­bile app­s, and go pro­gram­mers do­ing what­ev­er go pro­gram­mers do. 2

So, let's look at some of the things in the ar­ti­cle. The head­ings are copied from the orig­i­nal ar­ti­cle.

It's old

Well, yeah. So is JORF. What makes it pop­u­lar is not its age, but that the age has made it pos­si­ble for Python to have been pop­u­lar for a long time. Yes, it's cir­cu­lar and a bit un­in­ter­est­ing, but pop­u­lar­i­ty is self­-prop­a­gat­ing.

The dif­fer­ence be­tween Python and JORF is that Python has a user­base, not age. And how did Python get it's user­base? By be­ing good.

It’s beginner-friendly

Yep. Al­so, the au­thor men­tions that dy­nam­ic typ­ing is nice for be­gin­ner­s. In­deed! And be­ing nice for be­gin­ners is why Python is one of the two ex­am­ples I can think of for a pro­gram­ming lan­guage re­plac­ing an­oth­er.

It's versatile

Oh yeah! When you are a ca­su­al or be­gin­ning pro­gram­mer, a gen­er­al pur­pose pro­gram­mer is a gift from the goods of com­put­ing.

Me? One of my first pro­gram­ming lan­guages was POCO, an in­ter­pret­ed C em­bed­ded in Au­todesk An­i­ma­tor PRO for DOS. Ver­sa­tile that was­n't, and in­deed forced me to have to learn oth­er lan­guages lat­er on. I would much rather have learned Python in­stead.

It's al­so why Python is pop­u­lar in ML and sci­en­tif­ic com­put­ing. Learn one be­gin­ner-friend­ly lan­guage and do stuff ... it's ex­hil­a­rat­ing.

Downsides of Python - and whether they'll be fatal

Based on the pre­vi­ous elab­o­ra­tions, you could imag­ine that Python will stay on top of sh*t for ages to come.

Well, no. The fu­ture is long.

Speed

Python is slow. Like, re­al­ly slow. On av­er­age, you’ll need about 2–10 times longer to com­plete a task with Python than with any oth­er lan­guage.

Oh, have not tried Ruby, have you, you sweet sum­mer child?

In any case: yeah, Python is not par­tic­u­lar­ly fast. I should trans­late my "Python faster than C" light­ning talk to eng­lish some­time. Us­ing just tool­ing, with­out chang­ing the code it­self, Python can have a vari­a­tion in per­for­mance of 100x. 3

Here you can see it with sub­ti­tles if that's ok.

An­oth­er rea­son is that Python can on­ly ex­e­cute one task at a time. This is a con­se­quence of flex­i­ble datatypes — Python needs to make sure each vari­able has on­ly one datatype, and par­al­lel pro­cess­es could mess that up.

No. Nope. No no no no no no no. Not touch­ing that. But no.

In the end, it does­n't mat­ter be­cause ...

[...] none of the speed is­sues mat­ter. Com­put­ers and servers have got­ten so cheap that we’re talk­ing about frac­tions of sec­ond­s. And the end us­er does­n’t re­al­ly care whether their app loads in 0.001 or 0.01 sec­ond­s.

Scope

Yes, Python is dy­nam­i­cal­ly scope­d. How­ev­er ...

Python tried to tran­si­tion to stat­ic scop­ing, but messed it up. Usu­al­ly, in­ner scopes — for ex­am­ple func­tions with­in func­tions — would be able to see and change out­er scopes. In Python, in­ner scopes can on­ly see out­er scopes, but not change them. This leads to a lot of con­fu­sion.

If there is a lan­guage with dy­nam­ic scop­ing and first class func­tions where func­tions can change out­er scopes, I re­al­ly don't want to use that lan­guage. 4

Lambdas

De­spite all of the flex­i­bil­i­ty with­in Python, the us­age of Lamb­das is rather re­stric­tive. Lamb­das can on­ly be ex­pres­sions in Python, and not be state­ments.

Yes. If you want state­ments use a func­tion.

On the oth­er hand, vari­able dec­la­ra­tions and state­ments are al­ways state­ments. This means that Lamb­das can­not be used for them.

ENOPARSE. Sure, you can't as­sign to vari­ables in­side lamb­das. There is no point in as­sign­ing to vari­ables in­side lamb­das, since lamb­das are ex­pres­sion­s. Al­so, yes, "s­tate­ments are al­ways state­ments".

Whitespaces

Oth­er lan­guages, for ex­am­ple C++, re­ly more on braces and semi­colon­s. While this might not be vis­ual­ly ap­peal­ing and be­gin­ner-friend­ly, it makes the code a lot more main­tain­able. For big­ger pro­ject­s, this is a lot more use­ful.

Ci­ta­tion need­ed. Al­so, Haskell is not new­er than Python.

Mobile Development

In­deed mo­bile de­vel­op­ment in Python is not awe­some.

Runtime Errors

A Python script is­n’t com­piled first and then ex­e­cut­ed.

Yes it is.

In­stead, it com­piles ev­ery time you ex­e­cute it

No it does­n't.

so any cod­ing er­ror man­i­fests it­self at run­time.

No they don't.

This leads to poor per­for­mance, time con­sump­tion, and the need for a lot of test­s. Like, a lot of test­s.

This is great for be­gin­ners since test­ing teach­es them a lot. But for sea­soned de­vel­op­er­s, hav­ing to de­bug a com­plex pro­gram in Python makes them go awry. This lack of per­for­mance is the big­gest fac­tor that sets a time­stamp on Python.

I am lit­er­al­ly mak­ing ges­tures at my screen.

First of al­l, there was a whole oth­er sec­tion about speed, so why is speed not crit­i­cal there but it's the big­gest fac­tor here?

Al­so, is the au­thor say­ing test­ing is great for be­gin­ners but not for "sea­soned de­vel­op­er­s"? Well cov­er me in salt and pep­per be­cause I must lack sea­son­ing.

Af­ter this he pro­pos­es that Rust, Go and Ju­lia will re­place Python in the next five to ten years. Who knows? May­be?

I sus­pect Rust is a bit too com­pli­cat­ed and low lev­el and Ju­lia too speci­fic, and Go is nice but bor­ing to write. But hey, lan­guages change, things change, I am not a psy­chic.


  1. No, Google is prob­a­bly not evil, but come on, it's a good open­ing. 

  2. Just kid­d­ing go is cool! I like go! 

  3. Yes re­al­­ly. Us­ing PyPy, Mypy­c, Cython. And type hin­t­ing. No code mod­­i­­fi­­ca­­tion. 

  4. Yes, I know how to do that with Python. Let's keep it a se­cret and be­have like adult­s. 

New Plugins for Nikola

Since ev­ery­one is in lock­down and I am no ex­cep­tion, I have spent some of my time do­ing things in open source project­s. Let me in­tro­duce you two new plug­ins for Niko­la my favourite stat­ic site gen­er­a­tor.

  1. marko

Yet an­oth­er im­ple­men­ta­tion of Mark­down! And since adding sup­port for new im­ple­men­ta­tions of mark­down is easy, I added it.

Marko has some in­ter­est­ing ideas. If a ecosys­tem of ex­ten­sions de­vel­op­s, it will be easy to sup­port them on the Niko­la side.

  1. myst

Yes, an­oth­er mark­down but, again, one with orig­i­nal ideas. In this case, myst has a lot of re­Struc­tured Text flavour, with sup­port for di­rec­tives and roles.

Of course when us­ing marko or myst in Niko­la you al­so get sup­port for short­codes so they are ex­tra-ex­ten­si­ble here.

If you are look­ing for a stat­ic site gen­er­a­tor, I am fair­ly sure "it does­n't sup­port my favourite markup lan­guage" is not a rea­son not to choose Niko­la :-)


Contents © 2000-2023 Roberto Alsina