What I do for a living

2009-12-22 09:38

So, what do you do for a living?

—Hardest question ever

Whenever I am speaking with people who don't know me [1] that's the question I dread.

If someone asks my wife what she does, all she has to do is say "I'm a lawyer". If someone asks my mother, she'd say "I am a retired teacher". Everyone understands what a lawyer does, or what a retired teacher did.

If someone asks me... oh, boy, that's hard. I usually weasel out by saying "I work with computers" but that has several problems:

They assume I repair PCs
They start telling me how their windows box was slow until they installed some kropotkina which supergarbled their frobnozzles [4], then ask me my opinion on frobnozzle garbling. For or against?

It's really hard to explain that yes, I work with computers every day, but I almost never open one (in fact, I have a policy of not touching my customers computers), and I have no idea what a frobnozzle is.

I have tried saying "I work on server side things, like mail servers and such. I install them, support them and also consulting work, explaining companies what the best ways to improve their services are.".

That one usually gets glassy eyes and a general "what?" look.

I could lie and say I program for a living, but that's not true. While I program a lot, it's usually not for money, and what little I do for money is just using programming as a sysadmin tool.

I could say "I'm a sysadmin" but most people have no idea what that is. It does tend to end conversations, though, so it has one thing going for it.

Nowadays I could say "I have a company", which is true (we are awesome, you should hire us to do whatever it is we do, more details at http://www.netmanagers.com.ar )

So, I usually manage to work around this question, but I have a problem: I'm not telling the truth, or if I am, I am not telling the truth in spirit because I am not conveying what my work is, but only what I do.

So, this post is about trying to explain what the hell I do for a living, in another way, which is more ... internally true, so to speak. This is really hard to do, so I am trying to just let the writing flow, maybe you can understand what I do even if it's not clearly explained.

I work with computers. I make them do what I want them to do. Whenever a regular user sits before his keyboard, he tries to make his computer follow his orders, which variable rates of success. I always succeed.

Sometimes, I am logged into a computer that manages data for thousands of people. They all are on my care. No, it's not their lives at stake, but a little part of their fun, or work is under my care. I help them. I care about them, and I want their fun, their work to be smooth and pleasant.

Often the computer will not do what they need. I will try with my craft to make it happen. I will write little programs, search for others on the Internet, carefully piece together a puzzle and make their needs be fulfilled.

I will write or install and configure those programs and do it well, because I am skilled, I have literally decades of training and experience, but I will mostly do it because I like order and function. I like when things flow unimpeded, I like when serendipitous accidents make things just click together.

I do those things for a living, yes, because I need to make a living. And later, when I'm off the clock and my boy is asleep and I have my own time, you know what I do? I do the same things because they are fun. And I will bother writing a 1300 word post about how I migrated my blog's comments from one site to another because it was fun.

Yes, I know, to most people that would not be fun at all, it would be a boring job, and they would hate doing it. And that's one of the many reasons I am a lucky man [5]: I have fun doing unusual things. That's really lucky, because if my idea of fun was watching "Gossip Girl" I would never have found anyone to pay me to do that!

But going back to what I do for a living, I create things. I don't create large, impressive things, I am not a bridge builder, an architect, I create small, useful things and try to do it with a certain taste or elegance. I am more like a silversmith doing cutlery. Sure, I'll try to make it nice to look at, but it must cut a chunk of beef first.

Yes, I work with computers, but how does that convey what I feel when after a solid day of work I can see that what was a lot of stupid computers and cables are now a working machine that can make 50000 phone calls a day?

How can I make anyone see the beauty in 3 hard lines of code that do nothing but print a bunch of numbers?

How can someone who makes a living any other way understand that I think things and they become real? No, not real as in a puff of smoke and there they are, but they become real through work and effort and thinking and cursing, which is what makes them really real.

I know most of this will sound like mysticism, but it's not, it's my honest truth, I really feel all these things as I work, all these things are my work. Sometimes when I crack a hard problem I want to fucking sing [7] that's how awesome it feels.

So, that's what I do for a living. I work with computers.

python-keyring is seriously nice

2009-12-21 14:39

Many programs require passwords from the user.

It's nice when a program can remember the password you give it.

It's nicer when it stores said password safely. However, it's not trivial to do that if you care for cross-platform support.

Or at least it wasn't until Kang Zhang wrote python keyring, a module that abstracts the password storage mechanisms for KDE, GNOME, OSX and windows (and adds a couple of file-based backends just in case).

So, how does it work?

Install it in the usual way. If it's not packaged for your distro/operating system, just use easy_install:

easy_install keyring

You could also get it from mercurial:

hg clone http://bitbucket.org/kang/python-keyring-lib/

The API is simplicity itself. This is how you save a secret:

import keyring
keyring.set_password('keyring_demo','username','thisisabadpassword')

You may get this dialog (or some analog on other platforms):

And here's the proof that it was saved correctly (this is KDE's password manager):

And how do you get the secret back?

import keyring
print keyring.get_password('keyring_demo','username')

This is how it runs:

$ python load.py
thisisabadpassword

As you can see, the API is as easy as it could possible get. It even chose the KWallet backend automatically because I am in KDE!

Python-keyring is a module that fixes a big problem, so a big thank you to Kang Zhang and Tarek Ziadé (who had the idea)

The smartest thing I ever wrote

2009-12-18 11:50

When I was migrating the comments I noticed a page looked wrong in the site, and started fixing it.

While I was reading, I noticed a couple of things:

It was published almost exactly 5 years ago
It may be the smartest thing I ever wrote

Sometimes, you get an idea, and you can give it shape. Since I really think it's not completely stupid, I translated it to spnish (there was only an english version) y "reprint" it today.

It's about programming, it's about evolution, and I hope you like Being a good Lamarckian froggie!.

Migrating from Haloscan to Disqus (if you can comment on it, it worked ;-)

2009-12-18 00:37

Introduction

If you are a Haloscan user, and are starting to wonder what can you do... this page will explain you a way to take your comments to Disqus, another free comment service.

A few days ago, Haloscan announced they were stopping their free comment service for blogs. Guess what service has in it the comments of the last 9 years of this blog? Yes, Haloscan.

They offered a simple migration to their Echo platform, which you have to pay for. While Echo looks like a perfectly nice comment platform, I am not going to spend any money on this blog if I can help it, since it already eats a lot of my time.

Luckily, the guys at Haloscan allow exporting the comments (that used to be only for their premium accounts), so thanks Haloscan, it has been nice!

So, I started researching where I could run to. There seems to be two large free comment systems:

Keep in mind that my main interest lays in not losing almost ten years of comments, not on how great the service is. That being said, they both seem to offer roughly the same features.

Let's consider how you can import comments to each service:

Disqus: It can import from blogger and some other hosted blog service. Not from Haloscan.
Intense Debate: Can import from some hosted services, and from some files. Not from the file Haloscan gave me.

So, what is a guy to do? Write a python program, of course! Here's where Disqus won: they have a public API for posting comments.

So, all I have to do then is:

Grok the Disqus API
Grok the Haloscan comments file (it's XML)
Create the necessary threads and whatever in Disqus
Post the comments from Haloscan to Disqus
Hack the blog so the links to Haloscan now work for Disqus

Piece of cake. It only took me half a day, which at my current rates is what 3 years of Echo would have costed me, but where's the fun in paying?

So, let's go step by step.

1. Grok the Disqus API

Luckily, there is a reasonable Disqus Python Client library and docs for the API so, this was not hard.

Just get the library and install it:

hg clone https://IanLewis@bitbucket.org/IanLewis/disqus-python-client/
cd disqus-python-client
python setup.py install

The API usage we need is really simple, so study the API docs for 15 minutes if you want. I got almost all the tips I needed from this pybloxsom import script

Basically:

Get your API Key
You login
You get the right "forum" (you can use a disqus account for more than one blog)
Post to the right thread

2. Grok the Haloscan comments file

Not only is it XML, it's pretty simple XML!

Here's a taste:

<?xml version="1.0" encoding="iso-8859-1" ?>
<comments>
    <thread id="BB546">
      <comment>
        <datetime>2007-04-07T10:21:54-05:00</datetime>
        <name>superstoned</name>
        <email>josje@aaaaaa.nl</email>
        <uri></uri>
        <ip>86.92.111.236</ip>
        <text><![CDATA[that is one hell of a cool website ;-)]]></text>
      </comment>
      <comment>
        <datetime>2007-04-07T16:14:53-05:00</datetime>
        <name>Remi Villatel</name>
        <email>maxilys@aaaaaa.fr</email>
        <uri></uri>
        <ip>77.216.206.65</ip>
        <text><![CDATA[Thank you for these rare minutes of sweetness in this rough world...]]></text>
      </comment>
    </thread>
</comments>

So, a comments tag that contains one or more thread tags, which contain one or more comment tags. Piece of cake to traverse using ElementTree!

There is an obvious match between comments and threads in Haloscan and Disqus. Good.

3. Create the necessary threads and whatever in Disqus

This is the tricky part, really, because it requires some things from your blog.

You must have a permalink for each post
Each permalink should be a separate page. You can't have permalinks with # in the URL
You need to know what haloscan id you used for each post's comments, and what the permalink for each post is.

For example, suppose you have a post at //ralsina.me/weblog/posts/ADV0.html and it has a Haloscan comments link like this:

<a href="javascript:HaloScan('ADV0');" target="_self"> <script type="text/javascript">postCount('ADV0');</script></a>

You know where else that 'ADV0' appears? In Haloscan's XML file, of course! It's the "id" attribute of a thread.

Also, the title of this post is "Advogato post for 2000-01-17 17:19:57" (hey, it's my blog ;-)

Got that?

Then we want to create a thread in Disqus with that exact same data:

URL
Thread ID
Title

The bad news is... you need to gather this information for your entire blog and store it somewhere. If you are lucky, you may be able to get it from a database, as I did. If not... well, it's going to be a lot of work :-(

For the purpose of this explanation, I will assume you got that data nicely in a dictionary indexed by thread id:

{
  id1: (url, title),
  id2: (url, title)
}

4. Post the comments from Haloscan to Disqus

Here's the code. It's not really tested, because I had to do several attempts and fixes, but it should be close to ok (download).

#!/usr/bin/python
# -*- coding: utf-8 -*-

# Read all comments from a CAIF file, the XML haloscan exports

from disqus import DisqusService
from xml.etree import ElementTree
from datetime import datetime
import time


# Obviously these should be YOUR comment threads ;-)
threads={
    'ADV0': ('//ralsina.me/weblog/posts/ADV0.html','My first post'),
    'ADV1': ('//ralsina.me/weblog/posts/ADV1.html','My second post'),
    }

key='USE YOUR API KEY HERE'
ds=DisqusService()
ds.login(key)
forum=ds.get_forum_list()[0]

def importThread(node):
    t_id=node.attrib['id']

    # Your haloscan thread data
    thr_data=threads[t_id]

    # A Disqus thread: it will be created if needed
    thread=ds.thread_by_identifier(forum,t_id,t_id)['thread']

    # Set the disqus thread data to match your blog
    ds.update_thread(forum, thread, url=thr_data[0], title=thr_data[1])


    # Now post all the comments in this thread
    for node in node.findall('comment'):
        dt=datetime.strptime(node.find('datetime').text[:19],'%Y-%m-%dT%H:%M:%S')
        name=node.find('name').text or 'Anonymous'
        email=node.find('email').text or ''
        uri=node.find('uri').text or ''
        text=node.find('text').text or 'No text'

        print '-'*80
        print 'Name:', name
        print 'Email:', email
        print 'Date:', dt
        print 'URL:', uri
        print
        print 'Text:'
        print text

        print ds.create_post(forum, thread, text, name, email,
                                   created_at=dt, author_url=uri)
        time.sleep(1)

def importComments(fname):
    tree=ElementTree.parse(fname)
    for node in tree.findall('thread'):
        importThread(node)


# Replace comments.xml with the file you downloaded from Haloscan
importComments('comments.xml')

Now, if we are lucky, you already have a nice and fully functioning collection of comments in your Disqus account, and you should be calm knowing you have not lost your data. Ready for the final step?

5. Hack the blog so the links to Haloscan now work for Disqus

You may not need to do anything beyond what the Disqus install guide suggests. If you have to do the custom generic install editing HTML manually, it's something like this:

First I changed my comment links. Here's the Haloscan version:

<a href="javascript:HaloScan('ADV0');" target="_self">
<script type="text/javascript">postCount('ADV0');</script></a>

And here's the Disqus version:

<a href="//ralsina.me/weblog/posts/ADV0.html#disqus_thread">Comments</a>

Add the monkeypatching javascript at the bottom, before </body> like the Disqus install guide says (here's mine, it's not the one you want!):

<script type="text/javascript">
//<![CDATA[
(function() {
        var links = document.getElementsByTagName('a');
        var query = '?';
        for(var i = 0; i < links.length; i++) {
        if(links[i].href.indexOf('#disqus_thread') >= 0) {
                query += 'url' + i + '=' + encodeURIComponent(links[i].href) + '&';
        }
        }
        document.write('<script charset="utf-8" type="text/javascript" src="http://disqus.com/forums/yourownblognamegoeshere/get_num_replies.js' + query + '"></' + 'script>');
})();
//]]>
</script>

Also, in the post's own webpage, add the "embed code" from the Disqus install guide wherever you want the comments to appear.

If you want more than one page to share a set of comments (for example, I use the same comments for the spanish and english versions of a post), you will have to use something like this before the embed code:

<script type="text/javascript">
    var disqus_url = "http://the_url_where_the_comments_really_belong.com";
</script>

And thatś about it, it should be enough to get you going. However:

If you found this guide useful, and it saved you money, why not be a nice guy and give it to me instead? There's a donate link at the left, you know ;-)
If you can't manage to fix it, ask me for help. I have very reasonable consulting rates!

Make sure to export your haloscan comments while you still can!

New 24-hour app coming (not so) soon: foley

2009-12-16 17:12

First a short explanation:

24-hour apps are small, self-contained projects where I intend to create a decent, useful application in 24 hours. The concept is that:

I will think about this app a lot for a while
I will design it in my head or in written notes
I will code, from scratch, for 24 hours.
That's not one day, really, but 24 hours of work. I can't work 24 hours straight anymore.

The last time around this didn't quite work as I intended, but it was fun and educational (for me at least ;-) and the resulting app is really not bad!

So, what's foley going to be? A note-taking app aimed at students and conference public.

In your last geeky conference, did you notice everyone is using a computer?

And what are they taking notes on? Vi? Kwrite? OpenOffice? Whatever it is they use, it's not meant to be used for this purpose.

So, what will foley do different? I don't quite know yet, but I have some ideas:

A strong timeline orientation. Every paragraph will be dated.
Twitter/Identica support. Want to liveblog your notes? Just click.
Multimedia incorporated in the timeline.
- Webcam/Audio recording synced to your notes?
- Images imported and added in the timeline?
- Attach files to the timeline? (Useful for slides?)
If provided with a PDF of slides, attach each slide to the right moment in the timeline
Easy web publishing: find a way to put this on a webpage easy and quick (single-click publishing is the goal)

I have only thought about this for about 10 minutes, but I see potential here.

The bad news is... I have a ton of paying work to do. So this will probably only happen in January. However, I wanted to post it so I can take input while in this planning phase.

So, any ideas?

Ralsina.Me — Roberto Alsina's website

Posts about programming (old posts, page 57)