Skip to main content

Ralsina.Me — Roberto Alsina's website

Nikola: Filters & Bundles

Two up­com­ing fea­tures for the next re­lease of Niko­la, my stat­ic site gen­er­a­tor, due some­time in Au­gust.

Filters

Fil­ters let you post­pro­cess your out­put. Think of it like in­sta­gram for web­sites, but use­ful. You can con­fig­ure per file ex­ten­sion a se­ries of python func­tions or shell com­mand­s, which will be ap­plied in place to the out­put file.

For ex­am­ple, sup­pose you want to ap­ply yui-­com­pres­sor to your CSS and JS files:

FILTERS = {
    ".css": [filters.yui_compressor],
    ".js": [filters.yui_compressor],
}

There, filters.yui_compressor is a simple wrapper around the command so that it applies in-place to the output files.

If you use strings there (untest­ed), they are tak­en as com­mand­s. The "%s" will be re­placed by the file­name, the usu­al crazy shell quot­ing rules ap­ply:

FILTERS = {
    ".jpg": ["jpegoptim '%s'"],
    ".png": ["pngoptim '%s'"],
}

Keep in mind that the fil­ters mod­i­fy the out­put of Niko­la, not the in­put, so your im­ages, CSS, and JS files will not be touched in any way. And of course chang­ing the fil­ters ap­plied to a file will force a re­build, so you can ex­per­i­ment freely.

Bundles

Hav­ing many sep­a­rate CSS or JS files is usu­al­ly a nono for per­for­mance rea­sons be­cause each one may in­volve a sep­a­rate HTTP trans­ac­tion. The so­lu­tion is to "bundle" those files in a sin­gle, larg­er file.

The rea­son not to do that is that usu­al­ly it means hav­ing a huge, un­com­fort­able thing to han­dle. So Niko­la tries to give you the best of both world­s, by let­ting you have sep­a­rate files, and bundling them (or not) on build.

There is a new option, USE_BUNDLES that defaults to False, and there are some changes in the theme templates so that it uses the bundled version when needed.

This was on­ly pos­si­ble thanks to We­bas­sets. How­ev­er, if you don't have We­bas­sets in­stalled, or you don't en­able USE_BUNDLES, this should cause no changes in the out­put.

Conclusion

These new fea­tures will al­low Niko­la users to im­prove their site's per­for­mance with min­i­mal tweak­ing, which is al­ways a good thing.

The Minimal Server

I was a sysad­min for a long time. I did that for mon­ey, so I nev­er re­al­ly want­ed to spend time do­ing the same thing in my own time, which lead to a se­vere case of cob­bler's chil­dren walk­ing bare­foot in my pri­vate serv­er.

So, to­day at lunch, I de­cid­ed to clean up my garbage. So this is what I end­ed up with, which is the min­i­mal serv­er that is good enough to be gen­er­al­ly use­ful for me.

Hosting

This is a cheap VPS pro­vid­ed by the nice folks at burst.net who are not giv­ing me any­thing to speak nice things about their ser­vice. How­ev­er, I will do it any­way:

  • Crazy cheap ($5.50 but I have a 20% dis­­­count for life)

  • Good amount of mon­th­­ly band­width

  • Lots of disk space

  • Good up­­­time

  • Fast net­­work

  • Very cheap

  • De­­cent per­­for­­mance

Distribution

I had Cen­tOS 5 in­stalled, and it stays. If burst ev­er starts of­fer­ing Ubun­tu Pre­cise, I may switch. Or, since this work­s, I may not.

What's good about Cen­tOS? It's sta­ble and bor­ing.

What's bad about Cen­tOS? It's too bor­ing. Lots of cool stuff just is­n't pack­aged.

Web Server

I need to serve a bunch of do­main­s, but I have a pe­cu­liar­i­ty: they are all stat­ic sites. I wan­t:

  • Low re­­source us­age

  • De­­cent per­­for­­mance (that most­­ly in­­­volves sup­­port­ing ranges and con­­tent ne­­go­ti­a­­tion)

  • Sta­ble

  • Sup­­port di­rec­­to­ry in­­dex­es

  • Easy con­­fig­u­ra­­tion

  • Vir­­tu­al do­­mains by name

Al­most any serv­er works well for this. Even Apache, ex­cept for the easy con­fig­u­ra­tion bit. I end­ed up with gatling be­cause it fits those cri­te­ria fair­ly well.

  • It us­es about 1.4MB of RAM , which is al­ways nice in a VPS

  • It's pret­­ty fast

  • Has not crashed in 2 hours?

  • Sup­­ports in­­dex­es

  • Here's the con­­fig­u­ra­­tion: "-c /s­rv/www -P 2M -d -v -p 80 -F -S" (yes, there is no con­­fig file at al­l)

  • Vir­­tu­al do­­mains are just fold­ers and sym­links in­­­side /s­rv/www which is the eas­i­est pos­sil­ble way to do it.

  • It sup­­ports re­­verse prox­­y­ing for when I want to try a python web app I am work­ing on.

Mail Server

No, I don't want a mail serv­er. I have gmail and/or a re­al mail serv­er for that. I want to get the mails from cron. For this, I used ssmtp and an ex­tra gmail ac­coun­t. It work­s, and here's the whole con­fig:

root=roberto.alsina@gmail.com
mailhub=smtp.gmail.com:587
UseTLS=YES
UseSTARTTLS=YES
AuthMethod=LOGIN
AuthUser=roberto.alsina.3@gmail.com
AuthPass=notputtingthetrueoneheredude

The best I can say about this con­fig­u­ra­tion is that it work­s, and does­n't in­volve run­ning a dae­mon.

Misc

For when I need to be in two places at the same time: Open­VPN rules, and there is no ar­gu­men­t. I have a squid run­ning oc­ca­sion­al­ly, and there is a Quas­sel core for IRC stuff. I in­stalled mosh to make ssh less painful, rsync han­dles file de­ploy­ment and back­up stor­age, cron sched­ules stuff, and that's it.

Status

Plen­ty of free RAM and CPU (yes, that's the full process list):

[root@burst1 ~]# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1   2156   664 ?        Ss   22:01   0:00 init [3]
root      1135  0.0  0.1   2260   576 ?        S<s  22:01   0:00 /sbin/udevd -d
root      1518  0.0  0.1   1812   572 ?        Ss   22:01   0:00 syslogd -m 0
root      1594  0.0  0.1   7240  1032 ?        Ss   22:01   0:00 /usr/sbin/sshd
root      1602  0.0  0.2   4492  1112 ?        Ss   22:01   0:00 crond
root      1630  0.0  0.1   5684   716 ?        Ss   22:01   0:00 /usr/sbin/saslauthd -m /var/run/saslauthd -a pam -n 2
root      1631  0.0  0.0   5684   444 ?        S    22:01   0:00 /usr/sbin/saslauthd -m /var/run/saslauthd -a pam -n 2
root      1636  0.0  0.2   3852  1372 ?        S    22:01   0:01 /opt/diet/bin/gatling -c /srv/www -P 2M -d -v -p 80 -F -S
root      1677  0.0  0.2   4284  1232 ?        Ss   22:02   0:00 SCREEN /root/quasselcore-static-0.7.1
root      1678  0.0  2.1  36688 11148 pts/0    Ssl+ 22:02   0:03 /root/quasselcore-static-0.7.1
root      3228  1.0  0.7  12916  4196 ?        Ss   23:28   0:13 mosh-server new -s -c 8
root      3229  0.0  0.3   3848  1588 pts/2    Ss   23:28   0:00 -bash
root      3275  0.0  0.1   2532   908 pts/2    R+   23:48   0:00 ps aux
[root@burst1 ~]# w
 23:49:03 up  1:47,  1 user,  load average: 0.00, 0.01, 0.00
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/2    186.153.52.253   23:28    0.00s  0.01s  0.00s w
[root@burst1 ~]# free
             total       used       free     shared    buffers     cached
Mem:        524800      49100     475700          0          0          0
-/+ buffers/cache:      49100     475700
Swap:            0          0          0

All things con­sid­ered, fair­ly hap­py with the re­sult.

Wild Cards (Wild Cards, #1)

Review:

I liked this book enough, but the idea of it be­ing part of a 20+ book se­ries is daunt­ing enough that I may not con­tin­ue it.

Christians say the funniest things!

This is a re­sponse to a re­sponse to this we­b­com­ic ti­tled "How to suck at your re­li­gion". While Oat­meal's com­ic is crass and paints things in broad terms, it's a freak­ing we­b­com­ic. So it's sup­posed to do that. But­the re­sponse is so full of phal­la­cies (and lack­ing in we­b­comic-­ness) that it may de­serve a re­sponse.

I have promised not to be a troll (any­more) so I will try to an­swer in a sen­si­ble man­ner.

Here's the ar­ti­cle I am re­ply­ing to go read it if you wan­t. I will not re­ply to all of it, but will in­stead cher­ryp­ick a cou­ple of para­graph­s.

In re­sponse to the "forc­ing dog­ma" pan­el:

So... re­li­gion is fine, un­less you ac­tu­al­ly be­lieve in it? Should par­ents not pass their po­lit­i­cal, eth­i­cal or moral views on to their chil­dren as well? What parts of par­ent­ing would be left if par­ents were to avoid pass­ing their views on to their kid­s? The irony here is that si­lence is it­self a state­men­t. Avoid­ing any men­tion of God to your kids sends as clear a mes­sage as talk­ing about God: specif­i­cal­ly, it tells your kids that God's ex­is­tence is ei­ther un­true, un­known, or unim­por­tan­t. Be­cause if you knew Him to ex­ist, sure­ly you'd share that knowl­edge, right?

Let's start from the top: you don't know god ex­ist­s. You have faith that he ex­ist­s, but you don't know it for a fac­t. If you knew for a fact that he ex­ist­s, you could not pos­si­bly have faith be­cause faith ex­cludes cer­tain­ty. As your bible says, faith is "the sub­stance of things hoped for, the ev­i­dence of things not seen."

So, do I tell my son god does­n't ex­ist? Nope. I tell him I think he does­n't ex­ist, and that I have nev­er seen r heard of any re­li­able ev­i­dence or da­tum that points to­wards his ex­is­tence, but al­so that some peo­ple do be­lieve he does ex­ist. I told him that be­cause I feel that's a hon­est an­swer. If your hon­est an­swer is "god ex­ist­s", then bul­ly for you, but from the point of view of a non-­be­liev­er you are telling your son a lie, or at best a half-truth. And if you re­al­ly don't know he ex­ists for a fact then you are just ly­ing.

Now, are you sayin that you know god ex­ists fac­tu­al­ly? Based on what? That's the usu­al slip­pery slope for this ar­gu­men­t. The re­li­gious are the ones mak­ing state­ments of fact based on tra­di­tion. To the rest of us, they just seem to be play­ing loose with what "fac­t" mean­s, or what "god" means or what "know" mean­s.

So, no, don't avoid men­tions of god, just avoid ly­ing to your kids if you can.

This next sec­tion is prob­a­bly the worst, be­cause it's just an in­co­her­ent ar­gu­men­t. A kid asks, “Dad, what hap­pens to us af­ter we die?” The au­thor com­pares pro­vid­ing the Chris­tian an­swer to this ques­tion with cor­rect­ing your kid for hav­ing green as a fa­vorite col­or. What?? That just is­n’t a co­her­ent ar­gu­men­t. In what world are those two ideas par­al­lel, or even com­pa­ra­ble?

Ac­cord­ing to the we­b­comic, good par­ent­ing is to pre­tend to be ag­nos­tic, and say that “no one re­al­ly knows for sure.” Of course, if the Res­ur­rec­tion is true, that claim is false. So to be a good par­en­t, you ap­par­ent­ly have to de­ny the Res­ur­rec­tion and em­brace ag­nos­ti­cis­m, treat­ing be­liefs about the af­ter­life as mere mat­ters of per­son­al pref­er­ence like hav­ing a fa­vorite col­or. This is just… stupid. There’s just no oth­er way of de­scrib­ing it. Imag­ine if we treat­ed ev­ery­thing that way. “Dad, what’s 3 x 3?” “No one re­al­ly knows for sure. What do YOU think 3 x 3 is?”

So, com­par­ing life af­ter death with col­or pref­er­ence is stupid and in­co­her­en­t, but com­par­ing it the chris­tian be­lief of res­ur­rec­tion with ba­sic arith­me­thics is a-ok? That must have tak­en some ef­fort to write with a straight face, I'm sure.

So, let's go slow­ly on this one. Be­liefs about the af­ter­life are, like most oth­er be­lief­s, prob­a­bly not a per­son­al pref­er­ence, but just some­thing you have, be­cause of, in most cas­es, in­doc­tri­na­tion ear­ly in life, peer pres­sure, and just be­cause you live in a so­ci­ety where that be­lief is nor­mal and ap­proved of.

But what is it your be­lief in the af­ter­life is not?

  • It's not in­­her­ent to "y­ou". If you were born in an­oth­er place or time, you would prob­a­bly be­lieve some­thing else.

  • It's not undis­­put­ed. Be­­cause there ex­ists a ma­jor­i­­ty of peo­­ple who don't be­lieve the same thing, ei­ther by de­­tails or en­tire­­ly.

  • It's not unique. Be­­cause oth­­er re­li­­gions have had sim­i­lar res­ur­rec­­tion be­lief­s.

  • It's not re­li­able. Even if we were to ac­­cept ev­ery­thing the bible says as true that would not mean we know what will hap­pen to you or to me af­ter we die. We would have a tes­ti­­mo­ny about what hap­pened in a few days in the af­ter­life of a spe­­cif­ic per­­son, at a point in the past, as told to some­one by some­one. Is that the same as know­ing what will hap­pen? No it's not.

Let's com­pare that to 3x3 as the au­thor at­tempt­ed:

  • If I was a chi­­nese in the 12th cen­­tu­ry: 3x3 is 9.

  • There is no group of peo­­ple that be­lieves 3x3 is 8 or 10.

  • There has not been in the past any re­al dis­­a­gree­­ment about the val­ue of 3x3. We have not achieved that re­­sult via a grad­u­al im­prove­­men­t.

  • We re­­ly on 3x3 be­ing 9 ev­ery day in our lives. If you drive a car, use a phone, or zip your pants, you are agree­ing 3x3 is 9.

  • We don't ex­pect 3x3 not to be 9 in the fu­­ture.

No­tice any dif­fer­ences? Yes, me too.

Per­son­al­ly, I con­sid­er your faith in god more akin my lik­ing Queen (the band, not the ruler). I was ex­posed to Queen at the right time, it was ap­proved by my peer­s, and I like it. On the oth­er hand, I un­der­stand that Queen is not ev­ery­one's cup of tea, and I don't claim Queen to be the "right" band.

The whole "if the Res­ur­rec­tion is true, that claim is false" line of thought is not log­i­cal. If my cat had wings, then the claim that winged cats are awe­some is false. But my cat does­n't have wings. Does it make the winged cats less or more awe­some that he does­n't? It's not that it's not right, it's that it's not even wrong.

Al­so, Oat­meal, shame on you about Galileo, re­al­ly, look it up ;-)

Driving a Nail With a Shoe I: Do-Sheet

I had pro­posed a talk for Py­Con Ar­genti­na called "Driv­ing 3 Nails with a Shoe". I know, the ti­tle is sil­ly, but the idea was show­ing how to do things us­ing the wrong tool, in­ten­tion­al­ly. Why? Be­cause:

  1. It makes you think dif­fer­­ent

  2. It's fun

The bad side is, of course, that this talk's con­tents have to be a se­cret, or else the fun is spoiled for ev­ery­one. Since the re­view process for Py­ConAr talks is pub­lic, there was no way to ex­plain what this was about.

And since that means the re­view­ers ba­si­cal­ly have to take my word for this be­ing a good thing to have at a con­fer­ence, which is un­fair, I delet­ed the pro­pos­al. The good (may­be) news is that now ev­ery­one will see what those ideas I had were about. And here is nail num­ber 1: Writ­ing a spread­sheet us­ing doit.

This is not my first "spread­sheet". It all start­ed a long, long time ago with a fa­mous recipe by Ray­mond Het­tinger which I used again and again and again (I may even be miss­ing some post there).

Since I have been us­ing doit for Niko­la I am im­pressed by the pow­er it gives you. In short, doit lets you cre­ate tasks, and those tasks can de­pend on oth­er tasks, and op­er­ate on data, and pro­vide re­sults for oth­er tasks, etc.

See where this is go­ing?

So, here's the code, with ex­pla­na­tion­s:

cells is our spreadsheet. You can put anything there, just always use "cellname=formula" format, and the formula must be valid Python, ok?

from tokenize import generate_tokens

cells = ["A1=A3+A2", "A2=2", "A3=4"]
values = {}

task_calculate creates a task for each cell, called calculate:CELLNAME. The "action" to be performed by that task is evaluating the formula. But in order to do that successfully, we need to know what other cells have to be evaluated first!

This is im­ple­ment­ed us­ing doit's cal­cu­lat­ed de­pen­den­cies by ask­ing doit to run the task "get_de­p:­FOR­MU­LA" for this cel­l's for­mu­la.

def evaluate(name, formula):
    value = eval(formula, values)
    values[name] = value
    print "%s = %s" % (name, value)

def task_calculate():
    for cell in cells:
        name, formula = cell.split('=')
        yield {
            'name':name,
            'calc_dep': ['get_dep:%s' % formula],
            'actions': [(evaluate, (name, formula))],
            }

For example, in our test sheet, A1 depends on A3 and A2 but those depend on no other cells. To figure this out, I will use the tokenize module, and just remember what things are "names". More sophisticated approaches exist.

The task_get_dep function is a doit task that will create a task called "get_dep:CELLNAME" for every cell name in cells.

What get_dep returns is a list of doit tasks. For our A1 cell, that would be ["calculate:A2", "calculate:A3"] meaning that to calculate A1 you need to perform those tasks first.

def get_dep(formula):
    """Given a formula, return the names of the cells referenced."""
    deps = {}
    try:
        for token in generate_tokens([formula].pop):
            if token[0] == 1:  # A variable
                deps[token[1]] = None
    except IndexError:
        # It's ok
        pass
    return {
        'result_dep': ['calculate:%s' % key for key in deps.keys()]
        }

def task_get_dep():
    for cell in cells:
        name, formula = cell.split('=')
        yield {
            'name': formula,
            'actions': [(get_dep, (formula,))],
            }

And that's it. Let's see it in action. You can get your own copy here and try it out by installing doit, editing cells and then running it like this:

ralsina@perdido:~/dosheet$ doit -v2 calculate:A3
.  get_dep:4
{}
.  calculate:A3
A3 = 4
ralsina@perdido:~/dosheet$ doit -v2 calculate:A2
.  get_dep:2
{}
.  calculate:A2
A2 = 2
ralsina@perdido:~/dosheet$ doit -v2 calculate:A1
.  get_dep:A3+A2
{'A3': None, 'A2': None}
.  get_dep:4
{}
.  calculate:A3
A3 = 4
.  get_dep:2
{}
.  calculate:A2
A2 = 2
.  calculate:A1
A1 = 6

As you can see, it al­ways does the min­i­mum amount of ef­fort to cal­cu­late the de­sired re­sult. If you are so in­clined, there are some things that could be im­proved, and I am leav­ing as ex­er­cise for the read­er, for ex­am­ple:

  1. Use up­­­to­­date to avoid re­­cal­cu­lat­ing de­pen­­den­­cies.

  2. Get rid of the glob­al val­ues and use doit's com­put­ed val­ues in­stead.

Here is the full list­ing, en­joy!


Contents © 2000-2020 Roberto Alsina