This has been a pet peeve of mine for years: pro­gram­ming shell scripts suck. They are ug­ly and er­ror prone. The on­ly rea­son why we still do it? There is no re­al re­place­men­t.

Or at least that was the case, un­til to­day I met iter­pipes at python.red­dit.­com

Iter­pipes is "A li­brary for run­ning shell pipe­lines us­ing shel­l-­like syn­tax" and guess what? It's bril­liant.

Here's an ex­am­ple from its PYPI page:

# Total lines in *.py files under /path/to/dir,
# use safe shell parameters formatting:

>>> total = cmd(
...     'find {} -name {} -print0 | xargs -0 wc -l | tail -1 | awk {}',
...     '/path/to/dir', '\*.py', '{print $1}')
>>> run(total | strip() | join | int)
315

Here's how that would look in shel­l:

find /path/to/dir -name '*.py' -print0 | xargs -0 wc -l | tail -1 | awk '{print $1}'

You may say the shell ver­sion looks bet­ter. That's an il­lu­sion caused by the evil that is shell script­ing: the shell ver­sion is bug­gy.

Why is it buggy? Because if I control what's inside /path/to/dir I can make that neat little shell command fail [1], but at least in python I can handle errors!

Al­so, in most ver­sions you could at­tempt to write, this com­mand would be un­safe be­cause quot­ing and es­cap­ing in shell is in­sane!

The iter­pipes ver­sion us­es the equiv­a­lent of SQL pre­pared state­ments which are much safer.

It's near­ly im­pos­si­ble to do such a com­mand in pure shell and be sure it's safe.

Al­so, the shell ver­sion pro­duces a string in­stead of an in­te­ger, which sucks if you in­tend to do any­thing with it.

And the most im­por­tant ben­e­fit is, of course, not when you try to make python act like a shel­l, but when you can stop pre­tend­ing shell is a re­al pro­gram­ming lan­guage.

Consider this gem from Arch Linux's /etc/rc.shutdown script. Here, DAEMONS is a list of things that started on boot, and this script is trying to shut them down in reverse order, unless the daemon name starts with "!":

# Shutdown daemons in reverse order
let i=${#DAEMONS[@]}-1
while [ $i -ge 0 ]; do
        if [ "${DAEMONS[$i]:0:1}" != '!' ]; then
                ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@}
        fi
        let i=i-1
done

Nice uh?

Now, how would that look in python (I may have in­vert­ed the mean­ing of ck­_­dae­mon)?

# Shutdown daemons in reverse order
for daemon in reversed(DAEMONS):
    if daemon[0]=='!':
        continue
    if ck_daemon(daemon):
        stop_daemon(daemon)

Where stop_­dae­mon used to be this:

stop_daemon() {
    /etc/rc.d/$1 stop
}

And will now be this:

def stop_daemon(daemon):
    run(cmd('/etc/rc.d/{} stop',daemon))

So, come on, peo­ple, we are in the 21st cen­tu­ry, and shell script­ing sucked in the 20th al­ready.

So, what do you do for a liv­ing?

—Hard­est ques­tion ev­er

When­ev­er I am speak­ing with peo­ple who don't know me [1] that's the ques­tion I dread.

If some­one asks my wife what she does, all she has to do is say "I'm a lawyer". If some­one asks my moth­er, she'd say "I am a re­tired teacher". Ev­ery­one un­der­stands what a lawyer does, or what a re­tired teach­er did.

If some­one asks me... oh, boy, that's hard. I usu­al­ly weasel out by say­ing "I work with com­put­er­s" but that has sev­er­al prob­lem­s:

• They as­­sume I re­­pair PCs

• They start telling me how their win­­dows box was slow un­til they in­­stalled some kropotk­i­­na which su­per­­gar­bled their frob­noz­­zles [4], then ask me my opin­ion on frob­noz­­zle gar­bling. For or again­st?

It's re­al­ly hard to ex­plain that yes, I work with com­put­ers ev­ery day, but I al­most nev­er open one (in fac­t, I have a pol­i­cy of not touch­ing my cus­tomers com­put­er­s), and I have no idea what a frob­noz­zle is.

I have tried say­ing "I work on serv­er side things, like mail servers and such. I in­stall them, sup­port them and al­so con­sult­ing work, ex­plain­ing com­pa­nies what the best ways to im­prove their ser­vices are.".

That one usu­al­ly gets glassy eyes and a gen­er­al "what?" look.

I could lie and say I pro­gram for a liv­ing, but that's not true. While I pro­gram a lot, it's usu­al­ly not for mon­ey, and what lit­tle I do for mon­ey is just us­ing pro­gram­ming as a sysad­min tool.

I could say "I'm a sysad­min" but most peo­ple have no idea what that is. It does tend to end con­ver­sa­tion­s, though, so it has one thing go­ing for it.

Nowa­days I could say "I have a com­pa­ny", which is true (we are awe­some, you should hire us to do what­ev­er it is we do, more de­tails at http://www.net­man­ager­s.­com.ar )

So, I usu­al­ly man­age to work around this ques­tion, but I have a prob­lem: I'm not telling the truth, or if I am, I am not telling the truth in spir­it be­cause I am not con­vey­ing what my work is, but on­ly what I do.

So, this post is about try­ing to ex­plain what the hell I do for a liv­ing, in an­oth­er way, which is more ... in­ter­nal­ly true, so to speak. This is re­al­ly hard to do, so I am try­ing to just let the writ­ing flow, maybe you can un­der­stand what I do even if it's not clear­ly ex­plained.

I work with com­put­er­s. I make them do what I want them to do. When­ev­er a reg­u­lar us­er sits be­fore his key­board, he tries to make his com­put­er fol­low his or­der­s, which vari­able rates of suc­cess. I al­ways suc­ceed.

Some­times, I am logged in­to a com­put­er that man­ages da­ta for thou­sands of peo­ple. They all are on my care. No, it's not their lives at stake, but a lit­tle part of their fun, or work is un­der my care. I help them. I care about them, and I want their fun, their work to be smooth and pleas­an­t.

Of­ten the com­put­er will not do what they need. I will try with my craft to make it hap­pen. I will write lit­tle pro­gram­s, search for oth­ers on the In­ter­net, care­ful­ly piece to­geth­er a puz­zle and make their needs be ful­filled.

I will write or in­stall and con­fig­ure those pro­grams and do it well, be­cause I am skilled, I have lit­er­al­ly decades of train­ing and ex­pe­ri­ence, but I will most­ly do it be­cause I like or­der and func­tion. I like when things flow unim­ped­ed, I like when serendip­i­tous ac­ci­dents make things just click to­geth­er.

I do those things for a liv­ing, yes, be­cause I need to make a liv­ing. And lat­er, when I'm off the clock and my boy is asleep and I have my own time, you know what I do? I do the same things be­cause they are fun. And I will both­er writ­ing a 1300 word post about how I mi­grat­ed my blog's com­ments from one site to an­oth­er be­cause it was fun.

Yes, I know, to most peo­ple that would not be fun at al­l, it would be a bor­ing job, and they would hate do­ing it. And that's one of the many rea­sons I am a lucky man [5]: I have fun do­ing un­usu­al things. That's re­al­ly luck­y, be­cause if my idea of fun was watch­ing "Gos­sip Girl" I would nev­er have found any­one to pay me to do that!

But go­ing back to what I do for a liv­ing, I cre­ate things. I don't cre­ate large, im­pres­sive things, I am not a bridge builder, an ar­chi­tec­t, I cre­ate smal­l, use­ful things and try to do it with a cer­tain taste or el­e­gance. I am more like a sil­ver­smith do­ing cut­lery. Sure, I'll try to make it nice to look at, but it must cut a chunk of beef first.

Yes, I work with com­put­er­s, but how does that con­vey what I feel when af­ter a sol­id day of work I can see that what was a lot of stupid com­put­ers and ca­bles are now a work­ing ma­chine that can make 50000 phone calls a day?

How can I make any­one see the beau­ty in 3 hard lines of code that do noth­ing but print a bunch of num­ber­s?

How can some­one who makes a liv­ing any oth­er way un­der­stand that I think things and they be­come re­al? No, not re­al as in a puff of smoke and there they are, but they be­come re­al through work and ef­fort and think­ing and curs­ing, which is what makes them re­al­ly re­al.

I know most of this will sound like mys­ti­cis­m, but it's not, it's my hon­est truth, I re­al­ly feel all these things as I work, all these things are my work. Some­times when I crack a hard prob­lem I want to fuck­ing sing [7] that's how awe­some it feel­s.

So, that's what I do for a liv­ing. I work with com­put­er­s.

Many pro­grams re­quire pass­words from the us­er.

It's nice when a pro­gram can re­mem­ber the pass­word you give it.

It's nicer when it stores said pass­word safe­ly. How­ev­er, it's not triv­ial to do that if you care for cross-­plat­form sup­port.

Or at least it was­n't un­til Kang Zhang wrote python keyring, a mod­ule that ab­stracts the pass­word stor­age mech­a­nisms for KDE, GNOME, OSX and win­dows (and adds a cou­ple of file-based back­ends just in case).

So, how does it work?

In­stall it in the usu­al way. If it's not pack­aged for your dis­tro/­op­er­at­ing sys­tem, just use easy_in­stal­l:

easy_install keyring

You could al­so get it from mer­cu­ri­al:

hg clone http://bitbucket.org/kang/python-keyring-lib/

The API is sim­plic­i­ty it­self. This is how you save a se­cret:

import keyring
keyring.set_password('keyring_demo','username','thisisabadpassword')

You may get this di­a­log (or some ana­log on oth­er plat­form­s):

And here's the proof that it was saved cor­rect­ly (this is KDE's pass­word man­ager):

And how do you get the se­cret back?

import keyring
print keyring.get_password('keyring_demo','username')

This is how it run­s:

$ python load.py
thisisabadpassword

As you can see, the API is as easy as it could pos­si­ble get. It even chose the KWal­let back­end au­to­mat­i­cal­ly be­cause I am in KDE!

Python-keyring is a mod­ule that fix­es a big prob­lem, so a big thank you to Kang Zhang and Tarek Zi­adé (who had the idea)

When I was mi­grat­ing the com­ments I no­ticed a page looked wrong in the site, and start­ed fix­ing it.

While I was read­ing, I no­ticed a cou­ple of things:

• It was pub­­lished al­­most ex­ac­t­­ly 5 years ago

• It may be the smartest thing I ev­er wrote

Some­times, you get an idea, and you can give it shape. Since I re­al­ly think it's not com­plete­ly stupid, I trans­lat­ed it to sp­nish (there was on­ly an eng­lish ver­sion) y "reprint" it to­day.

It's about pro­gram­ming, it's about evo­lu­tion, and I hope you like Be­ing a good Lamar­ck­ian frog­gie!.