Skip to main content

Ralsina.Me — Roberto Alsina's website

Nothing Ever Really Goes Away On The Internet: ra-plugins

I used to man­age a large num­ber of QMail in­stal­la­tion­s. And be­cause Qmail was ... weird­ly li­censed, I wrote a set of plug­ins that ran on top of a patch called Qmail-SP­P. I pret­ty much stopped do­ing that years ago be­cause life took me in oth­er di­rec­tion­s, and for­got all about it.

That col­lec­tion is called ra-­plu­g­ins and I had not touched it since late 2008.

And to­day... I got a patch with two whole plug­ins to add to it so that it makes Qmail han­dle email ad­dress­es more like Gmail does (alias­es us­ing user+­foo and mak­ing us­er.­foo the same as user­foo).

So, I got them, added them, fixed a few sim­ple build­ing is­sues, up­dat­ed the lib­smtp it us­es in­ter­nal­ly for one of the plug­ins to a lat­er ver­sion, and there it stays, per­haps not to be touched un­til 2018.

(Re)Introducing Alva, a Nikola Server

Over a year ago (time flies!) I post­ed some­thing about a project called Al­va. Let me quote my­self:

Al­va is al­most the op­po­site of Niko­la. If Niko­la is about mak­ing stat­ic sites, Al­va is a dy­nam­ic site. How­ev­er, as Hegel sug­gest­s, from the the­sis and the an­tithe­sis comes the syn­the­sis.

So, Al­va is about dy­nam­i­cal­ly cre­at­ing stat­ic sites. If you want to have Niko­la in your serv­er in­stead of in your own com­put­er, and have the con­ve­nience of an on­line tool, that's the niche Al­va tries to fil­l.

So, you would in­stall Al­va, and use it like any oth­er we­b-based blog­ging tool. Yet, be­hind the sce­nes, you would have Niko­la, and all the per­for­mance and se­cu­ri­ty ben­e­fits of stat­ic sites.

And maybe some­day, I (or some­one) will put up a mul­ti­-us­er ver­sion of Al­va, and you will be able to get host­ed blogs, know­ing all the da­ta is yours and you can leave any­time and do your own thing.

The ap­proach I was tak­ing at the time proved to be un­suc­cess­ful, and there were a few oth­er fail­ures along the way. Of course, the prob­lem was in how I was ap­proach­ing the task. So I did the right thing, and learned how to do it "right".

Still not us­able, still not host­ed any­where, but al­ready semi-­func­tion­al: Al­va lives now

There's a lot of work still to be done. But I now know how to do it. To pre­vent the usu­al ar­gu­ments, here is a lit­tle ex­pla­na­tion of mo­ti­va­tion, tool­ing, etc.

Motivation

I want a way to host blogs very cheap­ly. How cheap­ly? I want at least 1000 rea­son­ably ac­tive users in a $5 VP­S. That would make Al­va a rea­son­able al­ter­na­tive to host­ed mul­ti­-us­er word­press, which means it would be a rea­son­able so­lu­tion (if set­up is easy enough) for smal­l­-­to-medi­um or­ga­ni­za­tions which don't want to set­up ex­pen­sive in­fra­struc­ture yet want to own their da­ta (think school­s, small busi­ness­es, FLOSS pro­ject­s, etc.) I al­so want to pro­vide that ser­vice, for free. Which is why an­oth­er rea­son I want it to be su­per cheap.

How does Al­va help pro­vide this su­per-cheap blog host­ing?

  1. It needs to scale fol­low­ing the num­ber of ed­its not views.

  2. If it gets too busy with ed­it­s, changes take longer to ap­­pear, but the site it­­self does­n't get any slow­er.

  3. Ed­it­ing and serv­ing can be pro­vid­ed by sep­a­rate ser­vices, so I can use some su­per-­­fast stat­ic file serv­er and a su­per-­­con­­fig­urable WS­­GI de­­ploy­­men­t.

  4. In­­di­vid­u­al pages can be heav­i­­ly op­ti­mized so that they down­load fast

Tools

One of the guid­ing prin­ci­ples here is that to de­liv­er this sort of thing, in my spare time, the de­vel­op­ment process needs to be stingy with the most lim­it­ed re­source: me. I can't spend a lot of me here. I need to be care­ful and not over-promise.

So, when­ev­er there was a 3rd-­par­ty tool that saves a sig­nif­i­cant amount of time, that's what I am us­ing.

Django

Be­cause it has a much stronger 3rd-­par­ty toolset than Flask or any mi­cro-frame­work. For ex­am­ple, the Flask equiv­a­lent of djan­go-al­lauth broke my will to live. Be­cause the ad­min in­ter­face means I can start adding da­ta to see if it makes sense be­fore I write all the re­quired views.

Django-allauth

Be­cause I don't want you to have to cre­ate ac­counts here un­less you want to, this pro­vides (op­tion­al) so­cial lo­gin and reg­is­tra­tion. This was easy to set­up and works alm­sost out­-of-the-box

Bootstrap and Django-bootstrap-toolkit

Niko­la is al­ready heav­i­ly in­vest­ed in boot­strap, so it just made sense to go fur­ther down that road. I un­der­stand boot­strap, and djan­go-­boos­t­rap-­toolk­it is easy enough (although I can't make their datepick­er work)

Django-datetime-widget

Be­cause fight­ing is bor­ing.

Django-debug-toolbar

Be­cause Djan­go's mech­a­nisms to find tem­plates and stat­ic files are many and con­fuse me.

Redis + RQ + django-rq

It's crucial for the whole approach to use job queues in order to detach the rendering from the actual Django app. This combination makes job dispatching ridiculously easy, setup is trivial (install everything, start redis, a few lines of config, ./manage.py rqworker and off you go) and they provide a django admin page where I can see the failed jobs, which is awesome.

South

Be­cause it's easy enough, and al­lows me some free­dom ex­plor­ing da­ta or­ga­ni­za­tion in my mod­els with­out com­mit­ting to it for­ev­er or recre­at­ing data­bases with test da­ta all the time.

Gatling

I will prob­a­bly serve the gen­er­at­ed sites via gatling just like my cur­rent sites be­cause it has the sim­plest named do­main con­fig­u­ra­tion pos­si­ble, it's fast and very light in re­source us­age.

Markitup

A cool, sim­ple ed­i­tor with live pre­views that sup­ports al­most ev­ery markup. Not WYSI­WYG or even WYSI­WYM so pos­si­bly I will have to add an al­ter­na­tive. I start­ed us­ing djan­go-­mark­it­up but it's not a good idea (it us­es a old ver­sion of mark­it­up which re­quires JQuery < 1.9) and am in the process of just us­ing Mark­it­up man­u­al­ly.

So, feel free to give Al­va a try and/or give me a hand, com­ments wel­come.

Encuesta

Es­to es para ayu­darme en una char­la que voy a dar en el Py­Day Rosario por fa­vor re­spon­dan hon­es­ta­mente!

EN­CUES­TA FI­NAL­IZADA, GRA­CI­AS!

Endomondo Lied To Me

A week ago I re-s­tart­ed my di­et and ex­er­cise plan. Since I am gross­ly over­weight, the ex­er­cise plan is ba­si­cal­ly "walk around 5km ev­ery day, fat guy". Since I am a nerd, I want­ed da­ta so I could stop ly­ing to my­self about how much I was walk­ing.

I had seen En­domon­do men­tioned in my tweet­er time­line a bunch of times and the fea­ture­set looked pret­ty much ex­act­ly as I need­ed:

  • Track my walk­ing

  • Keep his­­to­ry

  • Show it in google maps (be­­cause it's nice)

It even did things like track­ing calo­ries burnt and so on.

The on­ly prob­lem was... it re­al­ly sucks at fig­ur­ing out how much you walked. It con­sis­tent­ly over­es­ti­mates by around 50% the dis­tances, and since it cal­cu­lates the av­er­age speed based on time and dis­tance (and the time mea­sure­ment is cor­rec­t) it over­es­ti­mates speed by 50%, which then means it over­es­ti­mates calo­ries burnt by (I am guess­ing) 125%.

How did I ver­i­fy that En­domon­do is wrong, and avoid the ob­vi­ous ex­pla­na­tion of "y­our GPS is bro­ken"?

  1. I tracked my­­self us­ing En­­domon­­do and Google Trails at the same time.

  2. I coun­t­ed steps ro­­man-mile style (count ev­ery "left­­-right", mul­ti­­ply by 1.6)

  3. I mea­­sured the path I walked in Google Maps and Bing Maps

All those mea­sure­ments tell me a walk of 1100m +/- 150m is mea­sured by En­domon­do as 1.68 km


Ver mapa más grande

Why does this hap­pen? I could as­sume En­domon­do is just crap, and prob­a­bly be right, but try­ing to come up with a "in­ter­est­ing" ex­pla­na­tion, I am lean­ing to­wards noisy mea­sure­ments. For ex­am­ple, if En­domon­do saw my po­si­tion shift­ing ran­dom­ly 10 or 15 me­ters left or right it would prob­a­bly add enough noise to make the path 50% longer (for a much more fun ex­am­ple of this, read this pa­per (by no oth­er than Benoit Man­del­brot!) but this does­n't ex­plain why Google Trails works so much bet­ter (un­less Trails does some­thing smart with an­tialias­ing and in­ter­po­la­tion).

If you use En­domon­do, care to share your ex­pe­ri­ence? I am re­luc­tant to 1-s­tar it in Google Play with­out in­de­pen­dent con­fir­ma­tion.


Contents © 2000-2020 Roberto Alsina