Skip to main content

Python is Not a Configuration File Format

There is a large thread in reddit about using Python as configuration file format. I want to be clear about that:


If you need to ask if it's a good idea, then you don't have a good reason. If you are sure that you have a good reason, then maybe you have a good reason.

There are many reasons for it, but I will explore just two, then offer a suggestion.

Python is read-only, and configuring is not programming.

Sure, it's easy to use python as a config file. You just import the thing, and there you go, all the data is there. But now your configuration syntax is a general purpose language that can do things like pop up dialogs when you parse it.

Your config can now depend on the whole internet, the hardware, the weather, and interactive input. Powerful? Sure. Good idea? Maybe, sometimes. But your application is now not able to configure itself.

If your application wants to store any kind of setting, it won't be able to. So most interactive, desktop apps, just should not use python for this, ever.

But what about non-interactive tools? Well, using python means that other tools can't write to the config file, either, which makes the tool less powerful. The power to have tools use tools is one of the cornerstones of modern computing, and you just cut your app off that ecosystem. Depending on what language the tool uses it may not even be able to parse your config file.

And what happens when someone is told "use this config fragment to achieve X"? Well, odds are, if the recipient has done anything that takes advantage of using python as a config format, then the fragment will not work. It would be like doing copy/paste from random code in github into your own program and expecting it to work.

So, you can't write to it from the app, you can't get configuration tips from the internet, you can't use other tools to modify config files, and other tools have a hard time parsing your files.

Also, it means that to handle the general case of configuring your app, you need a programmer. That is almost certainly overkill. Very few apps need that kind of thing. If your app can only be configured by programmers, you may have failed at making a good app (exceptions exist).

And what's the advice? Well, the advice is "don't do that" and the corollary is "configure using data, not code". use INIs, or XML, or YAML, or JSON, or plain text files, or whatever. But not code.

PS: My latest project, Nikola uses python as a configuration language. I thought I had a good reason. I didn't.

The Future of PyQt by Example

Three years ago, I started a series of long posts called "PyQt by Example". It reached five posts before I abandoned for a series of reasons that don't matter anymore. That series is coming back starting next week, rewritten, improved and extended.

It will do so in a new site, and the "old" posts will be retired to an archive page. Why? Well, the technologies used in some of them are obsolete or don't quite work nowadays. So, the new versions will be the preferred ones.

And while I am not promising anything, I have enough written to make this something quite longer, more nicely layouted, more interesting and make it cover more ground. BUT, while doing some checks on the traffic statistics for the old posts, some things popped out.

This was very popular

About 60% of my site's traffic goes to those five posts. Out of about 1200 posts over 12 years, 60% of the viewers go to the 0.4% of the pages. That is a lot.

It's a long tail

The traffic has not decreased in three years. If anything, it has increased

A long and tall tail.

So, all this means there is a desire for PyQt documentation that is not satisfied. I am not surprised: PyQt is great, and the recommended book is not free, so there is bound to be a lot of demand.

And, here's the not-so-rosy bit: I had unobtrusive, relevant, out-of-the-way-but-visible ads in those pages for more than two years. Of the 70000 unique visitors, not even one clicked on an ad. Don't worry, I was not expecting to get money out of them (although I would love to some day collect a $100 check instead of having google hold my money for me ad eternum).

But really? Not even one ad click? In more than two years, thousands of people? I have to wonder if I just attract cheap people ;-)

PyCamp Starts This Week

This friday marks the beginning of the 2012 PyCamp. What's PyCamp? It's a lot of python programmers (this year, about 50) gathering in an isolated place for a long weekend, with nothing to do except code. We will have our meals catered, there are no TVs, there is hardly anything within walking distance, and it's going to be very cold.

So, it's going to be awesome. It's a rare chance for me to spend a few days hacking at my own personal projects, uninterrupted by more important things like family, work, cooking, or socializing except between nerds.

Sure, there is the occasional monocycle riding, or juggling lesson, or shooting practice, but really, three or four solid days of hacking.

I intend to work on projects related to Nikola my static site generator, so if you are interested in that and going to pycamp, I want to talk with you.

And if you are interested but not going to pycamp, there is no reason not to join in a virtual sprint. We'll have internet. There is IRC. I will have time. It's a weekend! Please share any interesting ideas you have about static site generators in the nikola-discuss group and we'll see how much can get implemented or at least started.

A Simple Nikola Link Checker

One of the most important things when you are building a static site generator like Nikola is that your site should not be broken. So, I really should have done this earlier ;-)

This is a very simple link checker that ensures the pages Nikola generates have no broken links. I will make it part of Nikola proper once it's more polished and doit supports getting a list of targets

To try it, get it and run it from the same place where you have your, right after you run doit.

import os
import urllib
from urlparse import urlparse

import lxml.html

def analyze(filename):
        # Use LXML to parse the HTML
        d = lxml.html.fromstring(open(filename).read())
        for l in d.iterlinks():
            # Get the target link
            target = l[0].attrib[l[1]]
            if target == "#":  # These are always valid
            parsed = urlparse(target)
            # We only handle relative links.
            # TODO: check if the URL points to inside the generated
            # site and check it anyway
            if parsed.scheme:
            # Ignore the fragment, since the link will still work
            # TODO: check that the fragment is valid
            if parsed.fragment:
                target = target.split('#')[0]
            # Calculate what file or folder this points to
            target_filename = os.path.abspath(
                os.path.join(os.path.dirname(filename), urllib.unquote(target)))
            # Check if it exists, or report it
            if not os.path.exists(target_filename):
                print "In %s broken link: " % filename, target
    except Exception as exc:
        # Something bad happened, report
        print "Error with:", filename, exc

# This is hackish: we use doit to get a list of all
# generated files. Minor modifications would let you check
# the non-generated files as well.

for task in os.popen('doit list --all', 'r').readlines():
    task = task.strip()
    if task.split(':')[0] in (
        'render_site') and '.html' in task:
            # It looks like a generated HTML file

Confessions of a Troll

And then he told me "Roberto's a known troll" and I was like "really?" he seems like a nice guy!

—Not saying who said it

So, that happened a while ago in a dinner. And I am that troll. I must confess it, so here it is. I have a long, long, long history of trolling. Perhaps I could rationalize it into just being argumentative, or just enjoying a good argument, but no, really, I derive pleasure from seeing people react to what I say.

That could maybe be in some way constructive, I could just do it to make people see things from a different angle, and in my good moments I do that. But in others I just find someone I dislike, or someone who says something that rubs me the wrong way, and I ... well, it's not pretty.

What I usually do is up the ante. I escalate. I say outrageous things. Usually just slightly more outrageous than what the other guy says, so I can still rationalize it as "he started it", or "he deserves it" but honestly, deep inside, I know it's not true. I know I am being nasty because I like it. And that sucks. It sucks that I have this mean streak in me, and I can't get rid of it. It sucks that I have fun insulting the other poor bastard.

I could even say that most of the people I troll are trolls themselves. But that's not really an excuse. If they deserved contempt for their trolling, then I am just as deserving for mine. I am sure they, also, have their reasons and those reasons sound perfectly rational to themselves.

I could say that I often troll those who spread poisonous speech. That I troll those who are spreading misinformation and evil ideas. And yes, that is true, but again, I could reply without trolling. I could reply rationally, and just point out where they are saying nonsense, where they are spreading stupidity, and avoid the flair, avoid the gratuitous insult.

I could say that I troll those who bother me personally out of their own free will. So yes, telling a Jehovah's Witness that I will be at the door as soon as I wash my hands of the blood sacrifice I make to Satan is funny, but why is it any better than just telling them I am not interested.

Oh, sure, I could say that telling someone who insulted me "opinions are like assholes, and so are you" is somewhat clever, but I could just say "you are wrong, rude, and undeserving of my attention".

I sure could say that trolling someone who is rude, insulting, obnoxious and disruptive may teach him (it's always a he, isn't he?) a lesson, but it's just not true. They just see those responses as challenges.

I could say that feeding the other troll is just harmless fun, but it's not true because I know it hurts people, and creates an obnoxious environment. I could say that trolling those who deserve to be trolled is fair. But it's not true because I am noone's judge.

So, if I have trolled you, accept this post as an apology. Not a "sorry if I offended you" apology, but a straight apology. I am sorry I trolled you. I will try to stop.

I will not stop saying and writing what I think, and I will not abstain from replying to other people's opinions, but I will do a honest try to do it in a not-trolling manner, of offering my opinion as it is, instead of as a performace to make the other one feel humiliated or hurt or undeserving.

Because, and here is my only piece of excuse: I hate bullies. I was sent to school two years too early. So that you have an idea how that can affect someone, I am of average height, but until I was 13 I was convinced I was very short. I managed to become a decent swimmer, but until I was 12 or so, I was convinced I sucked at all sports and games. I was inceasingly bullied since I first ever met my "peers" until I was able to kick the crap out of them. I became a tough kid. I could take a beating like you can't imagine, because I was too small to hit back, and the only way not to be a total loser was to keep getting up and going to be beaten again, while making sarcastic comments.

And it turned out that those sarcastic comments worked. I got my ass kicked, but the next day that kid remembered how I was getting back on my feet and joking about his mother. And that, of course, lead to further bullying. When I was about 12, I was so extreme, I scared everyone so much, they stopped. I was scary. I talked like a freaking psycho. I probably was close to being one.

And the lesson I learned was that if you make fun of people, they got hurt enough to want to avoid it later. So, I was, in fact, a bully myself. And thus, I became that which I hated. I have had better times and worst times, I have been more in control and not so much. And it has taken me a long time to figure this out. I don't like what I became. So I will stop. And I am sorry.

Just gentle ribbing and self-deprecation from now on. Promise.

This is Not Math

My niece goes to school. She has homework. Every now and then, her mom and greatmom will not be able to help her. I am sort of the math helper of last resort in the family. So, the other day I got this question:

What is the next number on this series:

2, 12, 16, 17, 18, 19

I just could not come up with an answer at the time. I have been told the answer is 200. Why? Because it's a list of numbers starting with "d".

Oh, you may say two doesn't start with "d" but hey, this was in spanish, so it does. The problem is, this is just not mathematics, at all.

It's about a completely incidental feature of the representation of a number in spanish, which is a completely unmathematical thing. Makes me wonder why they are making students think about this. Anyone knows?

On Politeness, Street Signs, and Codes of Conduct

When I grow up, I want a pink car. Girls like pink.

—Tato (my son, age 4 at the time)

There has been a lot of talk lately about codes of conduct in conferences. I don't have answers to much, but I do have a lot of questions in my head, and some things seem to come to my mind because of that, so I will do a little head dumping, and let's see if clarity appears.

So. The main thing seems to be that the proposed codes of conduct aim at making events inclusive, and more diverse, and welcoming to people who may have felt unwelcome in the past. That these groups involve women should be a call of attention. Women? Women are half the world, and apparently we have been excluding them, whether intentionally or not.

So, in principle, if adopting a code of conduct helps that, I am all for it. Same about gays, lesbians, transexuals, etc. They are not 50% of the world, but they are about 10% of it, so it's a very large amount of people, and adding them to our groups is another easy optimization.

However, it concerns me a bit that these codes of conduct contain language like this:

Harassment includes offensive verbal comments related to gender, sexual orientation, disability, physical appearance, body size, race, religion, sexual images in public spaces, deliberate intimidation, stalking, following, unauthorized or inappropriate photography or recording, sustained disruption of talks or other events, inappropriate physical contact, and unwelcome sexual attention.

Some are completely obvious, people who disrupt the speakers at conferences are to be thrown out, intimidation is abominable, stalking is creepy, etc. But "offensive verbal comments related to [everything]"?

And here, I have to make a small detour and talk about street signs. When I visited London, one thing that called my attention was the language in street signs. Here's an example:

In other countries it would say "No dogs", wouldn't it?

So, why do they have signs like this? It's probably because they are very polite people. Also, it may be because offensive signs are punishable by jail.

I have a completely unfounded suspicion that the politeness came first, and then came the right not to be offended, which ended encoded into law, and now you just have to be polite, or else.

I am quite loudly atheist, yet I only mention it outside this blog if someone mentions his religion. So, for example, if someone says "god asks less and yet he forgives", I may say "that's because he doesn't exist", as a joke. I am now assuming that saying that in a conference with that kind of code of conduct is going to be considered offensive behaviour. And let's say I am ok with that, I can still go for a few hours without offending people. Have not tried it lately, but I am sure I can do it.

And I know that at a private event, like a conference, there is no inherent freedom of speech, because there is the right of admission, and I can just be kicked out without any explanation, and I am also fine with that, because I reserve the right to kick people out of my own home, too.

So, sure, let's keep religion out of it. It has no place in a technical conference one way or another, and in any case, I will wear my invisible pink unicorn shirt as an outward sign of my atheism (it looks just like a gray v-neck t-shirt).

And I am totally fine about not mocking or harassing people because of their gender or sexual preferences. I am old and provincial enough that when two men start kissing next to me, I feel awkward. Luckily, I am enough of an adult that I just think to myself, "dude, you are a provincial prude" and look the other way. After all, I have seen people take exception to me kissing my wife in public, so, live and let live, whatever. I like women, my wife likes men, so I can understand you liking either.

On the other hand, I understand that the mere existence and presence of some people can be offensive to others. I know people who would rather stand for 2 hours than sit next to a transexual. Or would rather get off the bus instead of being there. And I am enough of an old, provincial prude that I understand them. So, offending is not the thing here, because if offending is the thing, then the mere presence of someone can offend others, and that's the exact opposite of what we want. We want them to either not be offended, or be offended and get over it, or be offended and not care.

So, handing out invitations to threesomes to people in hallways is a bit too much (I never invite people to threesomes before the fourth date, it is gauche). Hitting on people in bars at night is probably not too bad, unless it's a constant thing that ruins the night for someone (what do I know, I have never hit on someone or been hit on in a bar. Except by other men. Just my luck!) in which case I expect a group of nice people to form a protective ring around the poor person who is just too attractive? (again, what do I know, I have never been atractive).

The thing we want is politeness. We want to be nice to each other. We want everyone to be as nice as they possibly can to as many people as they can. Specially, we want everyone to be extremely nice to the people they like the least. Because with people you get along with, you can do crazy stuff you can't do with others.

On the other hand, I suspect there is something else here I am missing. Because tolerance and respect is just not my thing. I am all for proselitizing and disrespect, for creative annoyance and pushing people outside their comfort zones. But I try not to do it personally, I try to throw things to the crowd and see what they do with them.

I mean, I have been photographed without my consent. I have even had my shirt scanned without asking permission (ok, I admit having a QR code in a shirt is sort of an implicit agreement), I have been called names, but I know that, in the words of a scifi writer, I live life in the low difficulty setting, because I am a rather healthy white heterosexual male born in middle class with a job, so again, I don't quite know what it's like to be insanely attractive, or gay, or insanely attractive to gays, or anything. I am not harassable. My face protects me. I know others don't have such powerful defenses.

So, while that kind of language does fill me with trepidation, and makes me wonder what kind of community I have been living in, oblivious to all these things I read about lately, I will accept those codes and try to follow them. I have never intentionally broken them, even before they existed (I did once take an inappropriate picture, it was a joke, I only showed it to one person, and I deleted it, and I really am sorry and would not do it again, ok?)

So, I hope to see a lot of people I don't know in the next free software events I attend. Hopefully I will not offend any of them in a bad way. I will not be too brash. I will try to be inclusive. I will try to be nice. But remember. If I am very, very nice to you, it may be because I can't stand you. You're welcome.

The Raspberry PI Sucks.

It sucks because it's expensive. Or at least it's expensive for how lame the hardware provided is.

The other day I got a Mele a1000 box for which I will do a full review soon, I hope. But I really need to get the word out: this thing kicks Raspberry PI in the ass so hard it's not even funny.

For starters, it costs twice as much. Which may seem bad, but trust me, even at twice the cost, it's cheaper. Because if you are the kind of person for whom the $35 makes a difference, then you are also probably someone who doesn't have a HDMI-capable TV or monitor. And the Mele works with HDMI, but also with RCA connectors and VGA, which means pretty much any TV or monitor manufactured in the last 20 years will work in a pinch. You can repurpose ancient monitors (1024x768 CRTs from 1996? WORKS FINE) that are available everywhere.

Also, it comes with Wifi, wihch the Pi doesn't. And of course Ethernet, too.

It comes with 4GB of internal storage and 512MB of RAM. The Pi? None and 256mb. See a pattern here? You pay twice as much and it comes with more than twice as much hardware in it.

Also, it has two USB connectors (1 in the Pi model A, 2 in the Pi model B).

One important feature: it comes in a box. A nice, sturdy, small box, that can be nicely placed so that it doesn't break apart.

The CPU is several times faster and several generations more modern. The GPU is slightly less powerful, but you won't notice, the only thing the Pi can do with that GPU is decode video really quick, for anything remotely interesting the CPU is a bottleneck.

And both are proprietary, but at least someone is working on open Mali400 drivers.

The Mele is useful out of the box, you can just install Ubuntu on it, or use Android, which is open source. It's fully rooted, you don't need to do anything strange to install other operating systems in it (just like the Pi!) except that since the CPU is so much better, it can just do stuff easier.

Did I mention that while the Pi has only a SD card for storage while the Mele has 4GB of flash, and a SD slot, and an eSata slot? And that using that it could be a nice small file server?

And the kicker: it's not a charity. It's $70 because you are paying a business to create the things, and doesn't depend on getting severely discounted chips from Broadcom, it just is a good product at a good price. And that's just awesome because it means in 2 years we will have something about as powerful as the Mele at the Pi's pricepoint, and something much more powerful at the Mele's price, and that's sustainable, because the people who are building it and designing it, and selling it, and shipping it, are making a living out of it, while kicking the ass of the charity. And that's good. And if you want 100000 of these, you order them, and you get them, instead of having to pre-buy, and wait months, and see them ship in tiny batches.

So, buy this, or something like it.

Also: $92.60 with free shipping

Fear the Oso

Since I have a small kid, I know stuff other people don't. Specifically, I know that a surveillance state is forthcoming and that noone will care. Noone that matters that is. Because they all will have learned about it when they are three. So, by 2035, the expectation of privacy will be: None.

There is a classic south american leftist book called "To Read Donald Duck" that explains, from a marxist-theorist point of view, how Donal Duck forges the public conscience. If Dorfman saw one episode of Special Agent Oso he would have an aneurysm. Let me summarise every episode:

  1. Adorable Kid in any country. The show specifically shows what country it is, and the characters have localized features, names and dresses.

  2. Adorable Kid has a problem.

  3. His predicament is filmed by a robot ladybug, which reports via a satellite to an unnamed organization.

  4. The misterious boss called "Mr. Dos" (By the way, the chief of the argentinian secret service is called "Señor Cinco". Just saying!) assigns the mission to one of his "agents", usually our main character, Oso.

  5. Oso uses advanced technology to find Adorable Kid, and with the guidance of his "Paw Pilot" (how dated is that?), teaches Kid the three easy steps to solve his predicament.

  6. Oso learns a valuable lesson for his apparently endless Special Agent training program.

This is wrong at so many levels it's hard to keep track of them, but let's try anyway.

  • There is an unnamed organization that has the resources to know when every kid can't tie his shoes, and send an agent to help.

  • Every ladybug may be a robotic satellite-capable surveillance device.

  • This organization will send agents to get in contact with kids in any country without any adult supervision.

  • They have unlimited resources, including space stations, artificially intelligent bird-robocopters.

  • Their agents are not only good, kind and helpful, they are adorable stuffed animals.

The message is so blunt that it's not even mildly hidden to require marxist analysis, this cartoon says, loud and clear, that unnamed organizations look at everything you do, but it's for your own good, and when those organizations enter your life, it's only to help you and protect you, and in the process, these virtuous groups become even more virtuous.

Having lived in latin america in the 70s and 80s, I can say: bugger, we wished for that to be so! In reality, these things usually hire very few stuffed animals, and quite a bunch of plain old animals.

The constant surveillance is not even thought about, it's just assumed to be there, there is no consideration that kids deserve, need or even have privacy or a privacy expectation, the ladybugs routinely film the kids in their homes or even bedrooms, and send the images to a satellite for automated monitoring. Jeremy Bentham lacked the imagination and technical resources to imagine this, so he had to put his prisoners in a circle, to be watched by mere human guards.

So, what can we do? Probably nothing. I fully expect my kid to grow up with no expectation of privacy, and no concept of doing things outside the purview of a government, officially or unofficially.

Is that evil? Maybe, but it will be their normal. Just like we don't expect to have silence, or private electronic communications unless we take specific measures (you all know that, right?), and we expect all our online actions to be tracked by someone (you do expect that, right?)

My hope is a world of hypocrites, who have a public facade and a secret life. I can only hope my son will become Batman.