Posts about programming (old posts, page 7)

2007-02-12 23:31

Any regex wizard reading this?

If so, what is the C POSIX regex (you know regcomp & friends) equivalent of this python regular expresion:

re.compile(r'^([a-z][a-z0-9_\-\.]*)=', re.IGNORECASE)

Because it sure isn't this:


I have been playing with it for two hours and am bored :-)

2007-02-09 12:38

Playing with literate programming

I am using ra-plugins as a toy to do things I never bothered in other projects.

I am doing unit-testing. And now... some literate programming!

Ok, not much, and not very well, but at least I am playing with Lp4all which is a nice, simple tool to generate nice HTML from slightly wiki-marked sources.

You can see some little things in my code here. My veredict so far? A nice way to keep the code documented in a fashion that ocasinal browsers can follow.

The main thing missing is automatic cross-referencing.

In general, I am finding that this (and unit testing) helps me express explicitly to myself what the heck I am trying to do, and see if the code actually does it. Which is a really good thing.

2007-02-06 23:05


Ok, the SPF implementation situation is kinda pathetic.

There seems to be exactly one maintained C implementation. And it's windows-only.

  • libspf's website seems to have disappeared
  • libspf2's not RFC-compliant (verified for 1.2.5) and their issue reporting system bounces.

So, I have taken the most compliant one I found whose code I can actually follow (that would be the python one) and am reimplementing it in C (using bstrlib and libdjbdns).

It will probably not come to a good end, but hey, it may work ;-)

2006-11-17 11:04

To the other three guys (or gals)....

... who own a HP Jornada 720 and are using Opie on it and they have the spanish/latin-american keyboard... here is your keymap.

I will write something about how to get Linux going right on it soon, but here's the status report, 48 hours in.

This baby (unnamed yet) has:

  • 32MB of RAM
  • 1 GB of Flash
  • Wifi (802.11b pcmcia) + IRDA + Ethernet (pcmcia) + Anything once I find a 16-bit pcmcia-USB card (anyone has a spare and wants to recycle it? ;-)
  • Decent battery life (6 hours use with wifi, 9 without)
  • A keyboard
  • A decent screen (640x240)
  • A decent Linux-based GUI (Opie)
  • A somewhat erratic touchscreen

So, what can I do with it:

  • Email
  • Web browsing ( With Konqueror goodness )
  • Programming (Python, even PyQt2!). They keyboard and screen are surprisingly decent.
  • eBook reading. This is the most important one. In my work, I spend a lot of time waiting. Waiting for the train to arrive, for the trip to end, for someone to come to a meeting, for the waiter to bring my meal, for stuff to compile, for stuff to download... maybe I wait 3 hours a day. So I read. And this screen (long and somewhat thin) is quite spectacular for reading. Opie-reader is pretty good.
  • MP3 and Video player (haven't used it yet). I have streaming TV at home, courtesy of CherryTV (check the links at the left). This should work great when Rosario wants to see Montecristo and I'd rather see Penn & Teller's show.
  • General PIM stuff. Although I tend to keep that stuff in my head and my phone.

The bad side:

  • The bizarre screen aspect ratio confuses many configuration dialogs.
  • Almost no game works unless you rotate the screen.
  • The keyboard configuration took a while, and is not perfect yet ( I can't make dead_acute work for some reason)
  • The extra buttons don't work (external audio recorder, and alarm light-button)
  • I can't find a way to bind the function keys to apps in Opie
  • The reset button doesn't work (it's now a hang button)
  • Suspend is not really suspend on Linux (for unavoidable hardware reasons), so it spends battery when suspended (may last 12 hours or so, I think).
  • The only way to really turn it off is to take out the battery (not as bad as it sounds).
  • If you do that, it takes about one minute to boot.

So, I am using it more as a laptop (although a really, really small one, with very, very good battery life :-) than as a PDA.

The small memory and CPU means I can't run very demanding stuff, but I never seem to do that, anyway.

And of course, the really bad thing: it's so much fun to hack with, I have trouble working!

All in all, a great toy, lots of fun, and rather useful.

2006-11-07 12:15

rst2rst gets tables!

My Restructured Text to Restructured Text converter can now handle tables!

If you don't understand a word of what I just wrote... well, skip this. It's really quite esoteric.

I managed without linear programming by the usual route: make it ugly but functional. Basically, I overdimension things badly.

Here's the source table:

| f2,c1 | f2,c2 longer longer still |
|       +---------------------------+
| f3,c1 | 1. a                      |
| f3,c1 |                           |
| f3,c1 | 2. b                      |

Here's what rst2rst generates:

|       |                           |
| f2,c1 | f2,c2 longer longer still |
|       |                           |
| f3,c1 |                           |
| f3,c1 |                           |
| f3,c1 |                           |
|       |                           |
|       +---------------------------+
|       |                           |
|       | 1. a                      |
|       |                           |
|       | 2. b                      |
|       |                           |

Seriously non-optimal, but it is functional, until a braver soul improves the algorithm.

It can't handle table headers correctly, but other than that, it works just fine.

Update: It now supports all the RST table syntax, AFAIK.

2006-11-02 23:20

rst2rst works (80% or so)

What is it? A program that takes a docutils document tree ( parsed from a RST document or programatically generated) then dumps as close as I can guess to reasonable RST back.

This lets Restructured Text be a saveable data format, which is nice.

It's not done as a docutils writer. Sorry, I couldn't make that work.

What works? Most of it.

What doesn't? A dozen directives, custom interpreted text roles, and tables.

Yes, all of those are important. But the rest seems to work ok!

Look: a 804 line RST document containing almost every feature of the language, and the only difference in the generated HTML output between the original and rst2rst's is an invisible difference in continuation lines in line blocks.

[[email protected] wp]$ python t1.txt > t2.txt
[[email protected] wp]$ /usr/bin/ t1.txt t1.html ;  /usr/bin/ t2.txt t2.html
[[email protected] wp]$ diff t1.html t2.html
< <div class="line">But I'm expecting a postal order and I can pay you back
< as soon as it comes.</div>
> <div class="line">But I'm expecting a postal order and I can pay you back</div>
> <div class="line">as soon as it comes.</div>
[[email protected] wp]$ wc -l t1.txt
804 t1.txt

You can get and the testfile.

Anyone knows of a real docutils test suite I could borrow?

2006-10-17 21:06

No, I don't get a dime from them

For a few months I have been using an unmanaged virtual private server from Tektonic, and I love it.

What's that? Let's take it one word at a time, and then some more.

  1. It's a server: which means it's a full-ish linux installation. So it is capable of doing lots of things. I can run all sorts of weird python thingies in it if I want. IMAPS and SSMTP? No problemo.
  2. It's private: which means I am root on it. I have the shell. I choose what to install.
  3. It's virtual: it's a Virtuozzo partition in a real server. That means no custom kernel modules, and that since almost everything is shared with other instances, 5GB of disk and 128MB of RAM go a long way.
  4. It's unmanaged: which means I manage it. Which is just the way I prefer it, since that's my job.
  5. It's cheap. I started on a 8 dollars a month plan (which doesn't seem to be there anymore, the current cheapest is a 15 dollars plan).
  6. It's a throwaway. I want to host some client as a favour? I just put it there. I could even rent another of these servers for a while, use it, then close it. Backups? Clicking on a webpage saves the image! Other than that... I back it.
  7. Fixed IPs. All you want (for extra coins).
  8. A home away from home. All my stuff is there. I need it, I get it. Without bothering about having my own server at home via no-ip or somesuch (which of course I still have too ;-)
  9. It works. It hardly ever breaks. And having survived expensive, managed servers, this baby is working just as well.
  10. It's a nice gift. Suppose you have a connection to a free software project/LUG/family/whatever, and they need a place on the internet. Why not sponsor them with something like this? I offered one to PyAr (which didn't take it, but it's the thought that counts ;-)
  11. The ultimate learning experience: you can restore the system in 2 minutes. Want to play/learn sysadmining? Do it on the real virtual thing! Much cheaper than hosing your own box ;-)
  12. They offer a good service. So, people should know about it. And of course... if you know a similar, but even better deal... I'm all ears!

2006-10-05 13:38

A different UNIX Part II: A better shell language

One of the things people study when they "learn unix" is shell scripting and usage. Because every system has a shell, and if you learn to use it interactively, you are half way there to automating system tasks!

Let's consider that for a moment... what are the odds that the same language can be good for interactive use and for programming? I say slim.

Not to mention that learning shell as a way to learn unix is like going to a school that teaches TV production, and studying the remote. While useful, not really the important tool (ok, that analogy doesn't work at all. But it sounds neat, doesn't it?).

The first thing is that today's Linux domination of the unixsphere has caused a serious monoculture in shell scripting: everyone uses bash. The more enlightened ones may check that their scripts work on some other Bourne-style shell.

There are no important distributions (or proprietary unixes) that use a csh or anything like it. Debian has a policy that things should work without bashisms. That's about as good as it gets.

Writing a dozen pages on how shell sucks would be trivial. But uninteresting.

So, let's think it over, and start from the top.

What should a shell scripting language be like?

What doesn't matter?

Let's tackle these things. I invite anyone to add extra ideas in the comments section.

What should a shell scripting language be like?

  • Interpreted (obvious)

  • Dynamic typing (you will be switching ints to strs and viceversa all the time).

  • Easy incorporation of other programs as functions/methods/whatever.

    That pretty much is what makes it a shell. ls should be indistinguishable from something written using the shell itself.

  • Pipes. This is a must. Unix has a bazillion tools meant to be used in command pipelines. You can implement a RDBMS using that kind of thing (look for nosql). Leverage that.

    But even here, on its strength, the shell is not perfect. Why can't I easily pipe stderr and stdout to different processes? Why can't I pipe the same thing to two processes at the same time (yes, I know how to do it with a neat trick ;-)

  • Globbing. *.txt should give you a list of files. This is one of the obvious things where sh is broken. *.txt may be a string or a list, depending on context... and a list is just a series of strings with blanks. That is one of the bazillion things that makes writing shell scripts (at least good ones) hard:

    [[email protected] ralsina]\$ echo *out
    [[email protected] ralsina]\$ echo *outa
  • A list data type. No, writing strings separated with spaces is not ok. Maybe a python-style dictionary as well?

  • Functions (obvious)

  • Libraries (and ok, the shell source mechanism seems good enough)

  • Standalone. It shouldn't spawn sh for any reason ;-)

What doesn't matter?

  • Performance. Ok, it matters that a five-liner doesn't take 50 minutes unless it has to. But 1 seconds or two seconds? not that important.
  • Object orientation. I don't see it being too useful. Shell scripts are old-fashioned :-)
  • Compatibility to current shells. Come on. Why be like something that sucks? ;-)

Now, the example

Let's consider a typical piece of shell script and a rewrite in a more reasonable syntax.

This is bash (no it doesn't work on any other shell, I think):

DAEMONS=( syslog network cron )

# Start daemons
for daemon in "\${DAEMONS[@]}"; do
      if [ "\$daemon" = "\${daemon#!}" ]; then
              if [ "\$daemon" = "\${daemon#@}" ]; then
                      /etc/rc.d/\$daemon start
                      stat_bkgd "Starting \${daemon:1}"
                      (/etc/rc.d/\${daemon:1} start) &>/dev/null &

And since DAEMONS is something the admin writes, this script lets you shoot in the foot in half a dozen ways, too.

How about this:


# Start daemons
for daemon in DAEMONS {
      if ( daemon[0] != "!" ) {
              if ( daemon[0] == "@" ) {
                      stat_bkgd ("Starting "+daemon[1:])
                      /etc/rc.d/+daemon[1:] ("start") &> /dev/null &
              } else {
                      /etc/rc.d/+daemon ("start")

Of couse the syntax is something I just made up as I was writing, but isn't it nicer already?

2006-06-07 11:41

Eating like a Python, for Python, and other stuff

I am now almost recovered from the trip to Santa Fe for the Jornada Python and I have reached a few conclussions.

  1. I do kinda miss my old city.
  2. It does take slightly longer than 15 minutes to show it ;-)
  3. This kind of events is cool
  4. I am old, and can no longer take a 6-hour bus ride and then go to work.

On more detail:

The city is much prettier than it was when I left 5 years ago. Lots of construction going on, everything is cleaner, nicer, and more organized, it seems.

Man, I had forgotten how great you can eat there. And how cheaply. Remember I mentioned that fish-eating place? Well, it's only $22. That's pesos. So, it's about 7 dollars for all the delicious fish you can eat, and all the cold beer you can drink.

Which in my case is quite a lot. Although I am getting slow on the beer, too.

The only bad thing about it is that the guys that went on Sunday got Pacú and we didn't. Dammit!

Then there is the Sociedad Alemana. The ugliest place ever. The best picada [1] ever. It's a clubhouse founded by a former Graf Spee crewman. If they eat like that on the german navy, I would consider joining. And I get seasick.

Oh, and the lebanese restaurant. I was forgetting that.

Of course all this eating and imbibing also meant I missed half the event, but the morning conferences were introductory, so I got the good bits anyway.

Specially: I now understand metaclasses! and decorators! Yipee!

We had a good time with Rosario, she got to know better some of my friends, specially Javier and César [aka Ned Flanders] (and Cesar's family, Norma, Florencia and Julian).

We took a ride over the costanera (a riverwalk), peppered with comments like this (all dates are wrong):

Here was the Piedras Blancas beach, until the flood of 87.

That's where the old train bridge was until the flood of 69, then there was a liftchair, until the flood of 82.

That suspension bridge is not the original, which was washed off in 83 (and then stolen)

This piece of the costanera is new, the old one was gone in 78

And so on. You see, this place is... kinda floody. Architecture is not a permanent thing.

On yet another angle, I am going back to school next year. I don't intend to finish my Maths degree, but will go for System Engineering instead. We'll see what happens.

[1] We had the german style picada. That means plates of potato salad, smoked ham, three different sausages, leverwurst, pickles, three cheeses, and a sausage cazuela. The classic Santa Fe style picada may include any of those plus meatballs, milanesa (schnitzel-like thing), chips, olives, lupines (a kind of pickled bean), small sandwiches, and a hundred other things. All that in small pieces, ready to eat with fingers and/or toothpicks. All salty. So you get extra beer.

2006-05-23 19:28

A silly entry

Today's entry has no pieces of neat python code, no questions, no critic of anything, no interesting link, no hint of anything I wrote/(would/will)write, no funny piece, no unfunny piece, no nothing.

I am tired today.

So, have a nice day :-)

Contents © 2000-2019 Roberto Alsina