Skip to main content

Ralsina.Me — Roberto Alsina's website

Posts about programming (old posts, page 7)

Sometimes things just click

I have been writ­ing we­b-based in­ter­faces for ap­pli­ca­tions for about 5 years. Noth­ing pub­lic, noth­ing very in­ter­est­ing, just tiny front ends for cus­tom tools in clients' in­stal­la­tion­s.

And I have hat­ed ev­ery minute of it. PHP hurt­s, Twist­ed hurt­s, mod­_python hurt­s...

For a cou­ple of months I have been us­ing Cher­ryPy and I fi­nal­ly am hav­ing fun do­ing it.

And af­ter I fig­ured out how to do AJAX us­ing it, it's even more fun ( be­cause the apps in­ter­ac­tion mod­el is not to­tal­ly brain­dead :-)

I don't ex­pect it to be as fun as PyQt/PyKDE, but it's to­tal­ly not aw­ful. I sup­pose the same epiphany comes to peo­ple when they use rails or some oth­er de­cen­t, pro­duc­tive, fun frame­work.

All in al­l, I could get used to this.

Source-based distributions, the good side.

I am no fan of source-based dis­tri­bu­tion­s. I think that for most prac­ti­cal pur­pos­es, a dis­tri­bu­tion where in­stalling KDE takes over a day (I own a low­ly Duron as my fast com­put­er) is use­less.

How­ev­er, they are good for one par­tic­u­lar niche.

Cus­tom dis­tri­bu­tion­s. Weird dis­tri­bu­tion­s. Per­son­al dis­tri­bu­tion­s.

I have sev­er­al com­put­er­s. Most of them too slow and small.

As I was con­sid­er­ing re­in­stalling Lin­ux on my Li­bret­to for fun, I looked at what was in­stalled in it, and de­cid­ed that re­al­ly, 90% of it was of no use what­so­ev­er.

The prob­lem was, since it had De­bian in­stalled, it has a rather wide net­work of de­pen­den­cies that sim­ply could not be done with­out.

On a reg­u­lar com­put­er, that's not an is­sue, but in this tiny workhorse, with 16MB of RAM and a 800MB HD, it makes a lot of dif­fer­ence. The small­est I could get De­bian to be, and still have net­work, PCM­CIA and X11, was about 250M­B.

And the per­for­mance was not awe­some, ei­ther (but not ter­ri­ble).

So, what would hap­pen if in­stead of a reg­u­lar dis­tri­bu­tion it had some­thing like this:

  • uClibc in­­stead of glibc

  • runit in­­stead of SYSVinit

  • drop­bear in­­stead of OpenSSH

  • X built as kdrive (former­­ly tinyX)

  • python/qt (or maybe python/fltk) for ba­sic ad­min tools

And so on. Ba­si­cal­ly re­jig­ger the whole soft­ware se­lec­tion slant­ing it to­wards the small side, and drop­ping a bazil­lion non-­fun­da­men­tal de­pen­den­cies along the way.

Well, it can be done. It's just that it's a heck of a lot of work. But here, a source-based dis­tri­bu­tion gives you a head­start.

For ex­am­ple, I de­cid­ed to start from ucrux, a uClibc-based port of Crux. Since the na­tive toolchain in ucrux is uClibc, I don't have to wor­ry much about a whole class of mess that hap­pens when you build uClibc-based bi­na­ries on a glibc-based sys­tem (it's prac­ti­cal­ly cross-­com­pil­ing).

Qe­mu lets me in­stall ucrux and work on it some­what faster than on the tar­get P75 (if I could make KQE­mu work I'd be hap­pi­est).

Since crux's soft­ware pack­ag­ing mech­a­nism is sim­plic­i­ty it­self (a shell script that in­stalls to a fake root), al­though it's se­vere­ly un­der­pow­ered (no de­pen­den­cies), I can start from ucrux, hack the build of one pack­age at a time, then put ev­ery­thing on a CD very quick­ly.

So, if you want to hack your own dis­tri­bu­tion, Crux (or some oth­er sim­i­lar kit) is quite a nice thing.

For gen­er­al use... well, my re­quire­ments start at ap­t-get or equiv­a­lent ;-)

Now, if on­ly Tiny­CC worked for more pro­gram­s, that P75 would be a pock­et de­vel­op­ment pow­er­house!

Trac is cool. Cherrypy is cooler.

Trac is cool. Easy to set up, easy to run, low main­te­nance, and you get:

  • A tick­­et­ing sys­tem

  • Mile­­s­tones

  • A we­bcvs-­­like thing for sub­­ver­­sion

  • A wi­­ki (I mean,. what does­n't pro­­vide a wi­­ki nowa­­days?)

  • Bug re­­port­ing tool

  • The bug re­­port­ing tool and the sub­­ver­­sion change­sets can be linked us­ing Wi­­ki markup (now that's cool­er than it sounds ;-)

  • You don't need to be root to set it up, and you don't need apache or any­thing else, re­al­­ly.

Re­al­ly, re­al­ly nice stuff.

On the oth­er hand, Cher­ryPy is a tool that lets you "pub­lish your python ob­jects on the we­b", which does­n't re­al­ly mean much, but here's what I fig­ured out:

Cherrypy is the first way I have seen to write a useful web-based app in a reasonable amount of time and pain.

Ex­am­ple, I wrote a fron­tend to cla­mav (al­low­ing me to re­mote­ly trig­ger scans of in­di­vid­u­al nodes on a net­work) us­ing Cher­rypy and py­cla­mav in about 200 lines of code.

It works like a char­m, it's ro­bust, it even can be made to look nice us­ing some sort of tem­plat­ing en­gine (haven't both­ered yet).

And of course, I con­trol that ba­by us­ing a Trac project :-)

This is why dynamic languages are cool

I wrote a lit­tle spread­sheet thingie a few days ago. 1

Of course, it's a toy, not the re­al thing at al­l, but it was a nice hack, since it is a re­al, re­cal­cu­lat­ing, ex­ten­si­ble, de­pen­den­cy-check­ing, loop-avoid­ing spread­sheet en­gine in about 50 lines of code.

That's be­cause I was us­ing Python, which is a se­ri­ous­ly cool lan­guage to write that kind of thing in, since all you have to do to eval­u­ate an ex­pres­sion is call eval() on it.

Sure, that's nice, but the re­al core of the spread­sheet en­gine was that you could al­so cre­ate a dic­tio­nary-­like ob­ject that re­cal­cu­lat­ed on-de­mand its con­tents.

That way, when you ask for sheet['a1'], cus­tom code goes to see what a1 has in it (a for­mu­la), cal­cu­lates it if need­ed, and maybe trig­ger a few ex­tra re­cal­cu­la­tions if an­oth­er cell de­pends on a1. 2

But as any­one who us­es spread­sheets can tell you, weird things ex­ist in ssheet land.

For ex­am­ple, if you copy some­thing, then you paste it, it gets mod­i­fied in the process.

What oth­er app does that???

Here's an ex­am­ple you can check in any spread­sheet:

  • In A1, type "1".

  • In B1, type "A1+1" (should dis­­­play 2)

  • In A2, type 2

  • Copy B1 to B2, and it will dis­­­play 3

Fur­ther, if you look at the for­mu­la in B2, it says A2+1 now.

That's called rel­a­tive cell­names (I think).

In or­der to do that trick, you have to parse the for­mu­la in B1, and then, when you paste it in­to B2, take in­to ac­count the dis­place­ment and mod­i­fy ac­cord­ing­ly. Usu­al­ly, if you want ab­so­lute names, you use $ A1 in­stead, and that would stay un­mod­i­fied.

Now, that throws a nice mon­key­wrench in­to my neat lit­tle spread­sheet 3 be­cause now it sud­den­ly looks not like a spread­sheet at al­l!

So, I start­ed think­ing, how the hell could this be done? The whole ad­van­tage of a python sheet is us­ing eval(), so switch­ing to a pars­er (like if this were a C[++] sheet) would be sil­ly.

I delved in­to the python stan­dard lib. As ev­ery python pro­gram­mer knows, al­most ev­ery­ht­ing is there. If you write python, you read the li­brary ref­er­ence ev­ery day, and mem­o­rize chunks of it, be­cause it's one of the things that make python cool. It's just chock­full of use­ful stuff!

And here I was read­ing about the com­pil­er mod­ule, and the pars­er mod­ule, which can be used to do won­drous stuff with python code. But I could­n't un­der­stand jack­shit about them. I'm a sim­ple coder.

And just as I was go­ing to say, let's write in­stead about the con­nec­tion be­tween free soft­ware and the sex life of frogs 4 I found to­k­enize.

To­k­enize is a mod­ule that pars­es python and turns it in­to to­ken­s. Here's how a+2 looks af­ter you to­k­enize it:

1,0-1,1:        NAME    'a'
1,1-1,2:        OP      '+'
1,2-1,3:        NUMBER  '2'
2,0-2,0:        ENDMARKER       ''

The num­bers on the left side are po­si­tions in the text stream where the to­kens were.

It has just enough in­for­ma­tion that you can to­k­enize a piece of code, and then re­assem­ble it. There's code to do just that, it's called re­gur­gi­tate and it's writ­ten by Ka-Ping Yee.

So, the so­lu­tion is ob­vi­ous. When copy­ing a for­mu­la:

  • To­k­­enize the for­­mu­la to be copied

  • Look for to­kens of type NAME

  • See if it looks like a cel­l­­name, or _cel­l­­name

  • If it's _cel­l­­name, leave as is. That will be our no­­ta­­tion for ab­­so­­lute cells

  • If it's cel­l­­name, dis­­­place it nice­­ly

  • Re­gur­gi­­tate it

Lat­er, when eval­u­at­ing a for­mu­la, if some­one asks for cell _a1 give him cell a1.

And voilà, rel­a­tive cell­s.

This work­s, and it works well (ok, I had to in­tro­duce some ug­ly glob­al­s, I need to learn more stuff), and it is guar­an­teed to to­k­enize in the same way python does it. It's not even re­al­ly slow 5

I touched a bunch of oth­er things, in­clud­ing sup­port for all the func­tions in python's math mod­ule so you can use them in cell­s. Here's the code to do that:

for name in dir(math):
        if name[0]<>"_":

Freaky stuff, is­n't it?

What's the main is­sue? Per­for­mance. To put it sim­ply, I se­ri­ous­ly doubt a sheet writ­ten in python can be fast enough for gen­er­al use. But hey, it's ex­ten­si­ble, it's nice, and de­pend­ing on what you are try­ing to do, it may be good enough.

And here's to­day's ver­sion of Stupid­Sheet in­clud­ing rel­a­tive cell­s. Don't wor­ry, it's a small down­load ;-)


And al­most noone no­ticed ;-)


That trig­ger­ing is the on­ly part I wrote my­self, the rest is from ASP­N's cook­book.


I call it Stupid­Sheet.


I did write that any­way


I don't ad­vice you to copy a for­mu­la and paste it in­to a 10000x200 se­lec­tion. It will nev­er end. Op­ti­miza­tion for this is un­ex­is­tan­t. And un­like­ly.

Not a calculator

I have been play­ing with this code and it's been lots of fun.

I've hacked it in­to a func­tion­al spread­sheet in (ac­cord­ing to er­ic3) 508 lines of non-­doc code, of which 244 are gen­er­at­ed by pyuic.

Here's my code so far (re­quires PyQt). Give it a look, I think it's kin­da nice.

The on­ly hard part I wrote (at least hard for me) was the cell de­pen­den­cy and re­cal­cu­la­tion sup­port.

There's a test file you can use, too.

It is triv­ial to add func­tions you can use in the cell­s, just lookup python docs for eval() and check en­

To use it, unpack it, and from the directory it creates run python

I don't plan to make it a re­al spread­sheet, but it should be fun to hack on :-)