Skip to main content

Ralsina.Me — Roberto Alsina's website

Posts about programming (old posts, page 57)

What I do for a living

So, what do you do for a liv­ing?

—Hard­est ques­tion ev­er

When­ev­er I am speak­ing with peo­ple who don't know me 1 that's the ques­tion I dread.

If some­one asks my wife what she does, all she has to do is say "I'm a lawyer". If some­one asks my moth­er, she'd say "I am a re­tired teacher". Ev­ery­one un­der­stands what a lawyer does, or what a re­tired teach­er did.

If some­one asks me... oh, boy, that's hard. I usu­al­ly weasel out by say­ing "I work with com­put­er­s" but that has sev­er­al prob­lem­s:

  • They as­­sume I re­­pair PCs

  • They start telling me how their win­­dows box was slow un­til they in­­stalled some kropotk­i­­na which su­per­­gar­bled their frob­noz­­zles 4, then ask me my opin­ion on frob­noz­­zle gar­bling. For or again­st?

It's re­al­ly hard to ex­plain that yes, I work with com­put­ers ev­ery day, but I al­most nev­er open one (in fac­t, I have a pol­i­cy of not touch­ing my cus­tomers com­put­er­s), and I have no idea what a frob­noz­zle is.

I have tried say­ing "I work on serv­er side things, like mail servers and such. I in­stall them, sup­port them and al­so con­sult­ing work, ex­plain­ing com­pa­nies what the best ways to im­prove their ser­vices are.".

That one usu­al­ly gets glassy eyes and a gen­er­al "what?" look.

I could lie and say I pro­gram for a liv­ing, but that's not true. While I pro­gram a lot, it's usu­al­ly not for mon­ey, and what lit­tle I do for mon­ey is just us­ing pro­gram­ming as a sysad­min tool.

I could say "I'm a sysad­min" but most peo­ple have no idea what that is. It does tend to end con­ver­sa­tion­s, though, so it has one thing go­ing for it.

Nowa­days I could say "I have a com­pa­ny", which is true (we are awe­some, you should hire us to do what­ev­er it is we do, more de­tails at http://www.net­man­ager­s.­com.ar )

So, I usu­al­ly man­age to work around this ques­tion, but I have a prob­lem: I'm not telling the truth, or if I am, I am not telling the truth in spir­it be­cause I am not con­vey­ing what my work is, but on­ly what I do.

So, this post is about try­ing to ex­plain what the hell I do for a liv­ing, in an­oth­er way, which is more ... in­ter­nal­ly true, so to speak. This is re­al­ly hard to do, so I am try­ing to just let the writ­ing flow, maybe you can un­der­stand what I do even if it's not clear­ly ex­plained.

I work with com­put­er­s. I make them do what I want them to do. When­ev­er a reg­u­lar us­er sits be­fore his key­board, he tries to make his com­put­er fol­low his or­der­s, which vari­able rates of suc­cess. I al­ways suc­ceed.

Some­times, I am logged in­to a com­put­er that man­ages da­ta for thou­sands of peo­ple. They all are on my care. No, it's not their lives at stake, but a lit­tle part of their fun, or work is un­der my care. I help them. I care about them, and I want their fun, their work to be smooth and pleas­an­t.

Of­ten the com­put­er will not do what they need. I will try with my craft to make it hap­pen. I will write lit­tle pro­gram­s, search for oth­ers on the In­ter­net, care­ful­ly piece to­geth­er a puz­zle and make their needs be ful­filled.

I will write or in­stall and con­fig­ure those pro­grams and do it well, be­cause I am skilled, I have lit­er­al­ly decades of train­ing and ex­pe­ri­ence, but I will most­ly do it be­cause I like or­der and func­tion. I like when things flow unim­ped­ed, I like when serendip­i­tous ac­ci­dents make things just click to­geth­er.

I do those things for a liv­ing, yes, be­cause I need to make a liv­ing. And lat­er, when I'm off the clock and my boy is asleep and I have my own time, you know what I do? I do the same things be­cause they are fun. And I will both­er writ­ing a 1300 word post about how I mi­grat­ed my blog's com­ments from one site to an­oth­er be­cause it was fun.

Yes, I know, to most peo­ple that would not be fun at al­l, it would be a bor­ing job, and they would hate do­ing it. And that's one of the many rea­sons I am a lucky man 5: I have fun do­ing un­usu­al things. That's re­al­ly luck­y, be­cause if my idea of fun was watch­ing "Gos­sip Girl" I would nev­er have found any­one to pay me to do that!

But go­ing back to what I do for a liv­ing, I cre­ate things. I don't cre­ate large, im­pres­sive things, I am not a bridge builder, an ar­chi­tec­t, I cre­ate smal­l, use­ful things and try to do it with a cer­tain taste or el­e­gance. I am more like a sil­ver­smith do­ing cut­lery. Sure, I'll try to make it nice to look at, but it must cut a chunk of beef first.

Yes, I work with com­put­er­s, but how does that con­vey what I feel when af­ter a sol­id day of work I can see that what was a lot of stupid com­put­ers and ca­bles are now a work­ing ma­chine that can make 50000 phone calls a day?

How can I make any­one see the beau­ty in 3 hard lines of code that do noth­ing but print a bunch of num­ber­s?

How can some­one who makes a liv­ing any oth­er way un­der­stand that I think things and they be­come re­al? No, not re­al as in a puff of smoke and there they are, but they be­come re­al through work and ef­fort and think­ing and curs­ing, which is what makes them re­al­ly re­al.

I know most of this will sound like mys­ti­cis­m, but it's not, it's my hon­est truth, I re­al­ly feel all these things as I work, all these things are my work. Some­times when I crack a hard prob­lem I want to fuck­ing sing 7 that's how awe­some it feel­s.

So, that's what I do for a liv­ing. I work with com­put­er­s.

1

Giv­en my shy­ness prob­lem, that's not too of­ten 2

2

To those who on­ly know me from pub­lic speak­ing 3: I am painful­ly shy.

3

Yes, it's per­fect­ly pos­si­ble to be a de­cent pub­lic speak­er and be shy.

4

At least that's how win­dows users sound to me half the time

5

The oth­ers are of course y wife and boy, and if they ev­er read this: kiss­es for both. 6

6

He's just 2.9 years old but you know, the In­ter­net keeps things for­ev­er.

7

If you know my voice, you know why I don't. My own son says "no, don't sing, dad­dy", ex­cept for his good night song, which is the on­ly one he lets me sing. Oh, and sor­ry for the curs­ing, but no oth­er word fit­s.

python-keyring is seriously nice

Many pro­grams re­quire pass­words from the us­er.

It's nice when a pro­gram can re­mem­ber the pass­word you give it.

It's nicer when it stores said pass­word safe­ly. How­ev­er, it's not triv­ial to do that if you care for cross-­plat­form sup­port.

Or at least it was­n't un­til Kang Zhang wrote python keyring, a mod­ule that ab­stracts the pass­word stor­age mech­a­nisms for KDE, GNOME, OSX and win­dows (and adds a cou­ple of file-based back­ends just in case).

So, how does it work?

In­stall it in the usu­al way. If it's not pack­aged for your dis­tro/­op­er­at­ing sys­tem, just use easy_in­stal­l:

easy_install keyring

You could al­so get it from mer­cu­ri­al:

hg clone http://bitbucket.org/kang/python-keyring-lib/

The API is sim­plic­i­ty it­self. This is how you save a se­cret:

import keyring
keyring.set_password('keyring_demo','username','thisisabadpassword')

You may get this di­a­log (or some ana­log on oth­er plat­form­s):

keyring1

And here's the proof that it was saved cor­rect­ly (this is KDE's pass­word man­ager):

keyring2

And how do you get the se­cret back?

import keyring
print keyring.get_password('keyring_demo','username')

This is how it run­s:

$ python load.py
thisisabadpassword

As you can see, the API is as easy as it could pos­si­ble get. It even chose the KWal­let back­end au­to­mat­i­cal­ly be­cause I am in KDE!

Python-keyring is a mod­ule that fix­es a big prob­lem, so a big thank you to Kang Zhang and Tarek Zi­adé (who had the idea)

The smartest thing I ever wrote

When I was mi­grat­ing the com­ments I no­ticed a page looked wrong in the site, and start­ed fix­ing it.

While I was read­ing, I no­ticed a cou­ple of things:

  • It was pub­­lished al­­most ex­ac­t­­ly 5 years ago

  • It may be the smartest thing I ev­er wrote

Some­times, you get an idea, and you can give it shape. Since I re­al­ly think it's not com­plete­ly stupid, I trans­lat­ed it to sp­nish (there was on­ly an eng­lish ver­sion) y "reprint" it to­day.

It's about pro­gram­ming, it's about evo­lu­tion, and I hope you like Be­ing a good Lamar­ck­ian frog­gie!.

Migrating from Haloscan to Disqus (if you can comment on it, it worked ;-)

Introduction

If you are a Haloscan user, and are start­ing to won­der what can you do... this page will ex­plain you a way to take your com­ments to Dis­qus, an­oth­er free com­ment ser­vice.

A few days ago, Haloscan an­nounced they were stop­ping their free com­ment ser­vice for blogs. Guess what ser­vice has in it the com­ments of the last 9 years of this blog? Yes, Haloscan.

They of­fered a sim­ple mi­gra­tion to their Echo plat­for­m, which you have to pay for. While Echo looks like a per­fect­ly nice com­ment plat­for­m, I am not go­ing to spend any mon­ey on this blog if I can help it, since it al­ready eats a lot of my time.

Luck­i­ly, the guys at Haloscan al­low ex­port­ing the com­ments (that used to be on­ly for their pre­mi­um ac­counts), so thanks Haloscan, it has been nice!

So, I start­ed re­search­ing where I could run to. There seems to be two large free com­ment sys­tem­s:

Keep in mind that my main in­ter­est lays in not los­ing al­most ten years of com­ments, not on how great the ser­vice is. That be­ing said, they both seem to of­fer rough­ly the same fea­tures.

Let's con­sid­er how you can im­port com­ments to each ser­vice:

  • Dis­­qus: It can im­­port from blog­ger and some oth­­er host­ed blog ser­vice. Not from Haloscan.

  • In­­tense De­­bate: Can im­­port from some host­ed ser­vices, and from some files. Not from the file Haloscan gave me.

So, what is a guy to do? Write a python pro­gram, of course! Here's where Dis­qus won: they have a pub­lic API for post­ing com­ments.

So, all I have to do then is:

  1. Grok the Dis­­qus API

  2. Grok the Haloscan com­­ments file (it's XM­L)

  3. Cre­ate the nec­es­sary threads and what­ev­er in Dis­­qus

  4. Post the com­­ments from Haloscan to Dis­­qus

  5. Hack the blog so the links to Haloscan now work for Dis­­qus

Piece of cake. It on­ly took me half a day, which at my cur­rent rates is what 3 years of Echo would have cost­ed me, but where's the fun in pay­ing?

So, let's go step by step.

1. Grok the Disqus API

Luck­i­ly, there is a rea­son­able Dis­qus Python Client li­brary and docs for the API so, this was not hard.

Just get the li­brary and in­stall it:

hg clone https://IanLewis@bitbucket.org/IanLewis/disqus-python-client/
cd disqus-python-client
python setup.py install

The API us­age we need is re­al­ly sim­ple, so study the API docs for 15 min­utes if you wan­t. I got al­most all the tips I need­ed from this py­blox­som im­port script

Ba­si­cal­ly:

  1. Get your API Key

  2. You lo­­gin

  3. You get the right "fo­rum" (y­ou can use a dis­­qus ac­­count for more than one blog)

  4. Post to the right thread

2. Grok the Haloscan comments file

Not on­ly is it XM­L, it's pret­ty sim­ple XM­L!

Here's a taste:

<?xml version="1.0" encoding="iso-8859-1" ?>
<comments>
    <thread id="BB546">
      <comment>
        <datetime>2007-04-07T10:21:54-05:00</datetime>
        <name>superstoned</name>
        <email>josje@aaaaaa.nl</email>
        <uri></uri>
        <ip>86.92.111.236</ip>
        <text><![CDATA[that is one hell of a cool website ;-)]]></text>
      </comment>
      <comment>
        <datetime>2007-04-07T16:14:53-05:00</datetime>
        <name>Remi Villatel</name>
        <email>maxilys@aaaaaa.fr</email>
        <uri></uri>
        <ip>77.216.206.65</ip>
        <text><![CDATA[Thank you for these rare minutes of sweetness in this rough world...]]></text>
      </comment>
    </thread>
</comments>

So, a com­ments tag that con­tains one or more thread tags, which con­tain one or more com­ment tags. Piece of cake to tra­verse us­ing El­e­ment­Tree!

There is an ob­vi­ous match be­tween com­ments and threads in Haloscan and Dis­qus. Good.

3. Create the necessary threads and whatever in Disqus

This is the tricky part, re­al­ly, be­cause it re­quires some things from your blog.

  • You must have a per­ma­link for each post

  • Each per­ma­link should be a sep­a­rate page. You can't have per­ma­links with # in the URL

  • You need to know what haloscan id you used for each post's com­­ments, and what the per­ma­link for each post is.

For ex­am­ple, sup­pose you have a post at //ralsi­na.me/we­blog/­post­s/AD­V0.html and it has a Haloscan com­ments link like this:

<a hre­f="javascrip­t:HaloScan('AD­V0');" tar­get="_­self"> <script type­="­tex­t/­javascrip­t">­post­Coun­t('AD­V0');</scrip­t></a>

You know where else that 'AD­V0' ap­pears? In Haloscan's XML file, of course! It's the "id" at­tribute of a thread.

Al­so, the ti­tle of this post is "Ad­voga­to post for 2000-01-17 17:19:57" (hey, it's my blog ;-)

Got that?

Then we want to cre­ate a thread in Dis­qus with that ex­act same da­ta:

  • URL

  • Thread ID

  • Ti­­tle

The bad news is... you need to gath­er this in­for­ma­tion for your en­tire blog and store it some­where. If you are luck­y, you may be able to get it from a database, as I did. If not... well, it's go­ing to be a lot of work :-(

For the pur­pose of this ex­pla­na­tion, I will as­sume you got that da­ta nice­ly in a dic­tio­nary in­dexed by thread id:

{
  id1: (url, title),
  id2: (url, title)
}

4. Post the comments from Haloscan to Disqus

Here's the code. It's not re­al­ly test­ed, be­cause I had to do sev­er­al at­tempts and fix­es, but it should be close to ok (down­load).

#!/usr/bin/python
# -*- coding: utf-8 -*-

# Read all comments from a CAIF file, the XML haloscan exports

from disqus import DisqusService
from xml.etree import ElementTree
from datetime import datetime
import time


# Obviously these should be YOUR comment threads ;-)
threads={
    'ADV0': ('//ralsina.me/weblog/posts/ADV0.html','My first post'),
    'ADV1': ('//ralsina.me/weblog/posts/ADV1.html','My second post'),
    }

key='USE YOUR API KEY HERE'
ds=DisqusService()
ds.login(key)
forum=ds.get_forum_list()[0]

def importThread(node):
    t_id=node.attrib['id']

    # Your haloscan thread data
    thr_data=threads[t_id]

    # A Disqus thread: it will be created if needed
    thread=ds.thread_by_identifier(forum,t_id,t_id)['thread']

    # Set the disqus thread data to match your blog
    ds.update_thread(forum, thread, url=thr_data[0], title=thr_data[1])


    # Now post all the comments in this thread
    for node in node.findall('comment'):
        dt=datetime.strptime(node.find('datetime').text[:19],'%Y-%m-%dT%H:%M:%S')
        name=node.find('name').text or 'Anonymous'
        email=node.find('email').text or ''
        uri=node.find('uri').text or ''
        text=node.find('text').text or 'No text'

        print '-'*80
        print 'Name:', name
        print 'Email:', email
        print 'Date:', dt
        print 'URL:', uri
        print
        print 'Text:'
        print text

        print ds.create_post(forum, thread, text, name, email,
                                   created_at=dt, author_url=uri)
        time.sleep(1)

def importComments(fname):
    tree=ElementTree.parse(fname)
    for node in tree.findall('thread'):
        importThread(node)


# Replace comments.xml with the file you downloaded from Haloscan
importComments('comments.xml')

Now, if we are luck­y, you al­ready have a nice and ful­ly func­tion­ing col­lec­tion of com­ments in your Dis­qus ac­coun­t, and you should be calm know­ing you have not lost your da­ta. Ready for the fi­nal step?

New 24-hour app coming (not so) soon: foley

First a short ex­pla­na­tion:

24-hour apps are smal­l, self­-­con­tained projects where I in­tend to cre­ate a de­cen­t, use­ful ap­pli­ca­tion in 24 hours. The con­cept is that:

  1. I will think about this app a lot for a while

  2. I will de­sign it in my head or in writ­ten notes

  3. I will code, from scratch, for 24 hours.

  4. That's not one day, re­al­­ly, but 24 hours of work. I can't work 24 hours straight any­­more.

The last time around this did­n't quite work as I in­tend­ed, but it was fun and ed­u­ca­tion­al (for me at least ;-) and the re­sult­ing app is re­al­ly not bad!

So, what's fo­ley go­ing to be? A note-­tak­ing app aimed at stu­dents and con­fer­ence pub­lic.

In your last geeky con­fer­ence, did you no­tice ev­ery­one is us­ing a com­put­er?

And what are they tak­ing notes on? Vi? Kwrite? OpenOf­fice? What­ev­er it is they use, it's not meant to be used for this pur­pose.

So, what will fo­ley do dif­fer­en­t? I don't quite know yet, but I have some ideas:

  1. A strong time­­line ori­en­­ta­­tion. Ev­ery para­­graph will be dat­ed.

  2. Twit­ter/I­­den­ti­­ca sup­­port. Want to live­blog your notes? Just click.

  3. Mul­ti­me­­dia in­­­cor­po­rat­ed in the time­­line.

    • We­b­­­cam/Au­­­dio record­ing synced to your notes?

    • Im­ages im­­­port­ed and added in the time­­­line?

    • At­­­tach files to the time­­­line? (Use­­­ful for slides?)

  4. If pro­vid­ed with a PDF of slides, at­­tach each slide to the right mo­­ment in the time­­line

  5. Easy web pub­­lish­ing: find a way to put this on a we­b­­page easy and quick (s­in­­gle-click pub­­lish­ing is the goal)

I have on­ly thought about this for about 10 min­utes, but I see po­ten­tial here.

The bad news is... I have a ton of pay­ing work to do. So this will prob­a­bly on­ly hap­pen in Jan­u­ary. How­ev­er, I want­ed to post it so I can take in­put while in this plan­ning phase.

So, any ideas?


Contents © 2000-2020 Roberto Alsina