---
author: ''
category: ''
date: 2009/12/23 15:32
description: ''
link: ''
priority: ''
slug: BB860
tags: linux, programming, python, sysadmin
title: With iterpipes, python is ready to replace bash for scripting. Really.
type: text
updated: 2009/12/23 15:32
url_type: ''
---
This has been a pet peeve of mine for years: programming shell scripts suck. They are ugly and error prone. The only reason why we still do it? There is no real replacement.

Or at least that *was* the case, until today I met iterpipes at `python.reddit.com <http://www.reddit.com/r/Python/comments/ahpou/iterpipes_library_for_running_shell_pipelines/>`_

Iterpipes is "A library for running shell pipelines using shell-like syntax" and guess what? It's *brilliant*.

Here's an example from its `PYPI page <http://pypi.python.org/pypi/iterpipes>`_:

.. code-block:: pycon

   # Total lines in *.py files under /path/to/dir, 
   # use safe shell parameters formatting:

   >>> total = cmd(
   ...     'find {} -name {} -print0 | xargs -0 wc -l | tail -1 | awk {}',
   ...     '/path/to/dir', '\*.py', '{print $1}')
   >>> run(total | strip() | join | int)
   315

Here's how that would look in shell::

   find /path/to/dir -name '*.py' -print0 | xargs -0 wc -l | tail -1 | awk '{print $1}'

You may say the shell version looks better. That's an illusion caused by the evil that is shell scripting: the shell version is buggy.

Why is it buggy? Because if I control what's inside ``/path/to/dir`` I can make that neat little shell command fail [1]_, but at least in python I can *handle errors*! 

Also, in most versions you could attempt to write, this command would be unsafe because quoting and escaping in shell is insane!

The iterpipes version uses the equivalent of SQL `prepared statements <http://dev.mysql.com/tech-resources/articles/4.1/prepared-statements.html>`_ which are much safer.

It's nearly **impossible** to do such a command in pure shell and be sure it's safe.

Also, the shell version produces a string instead of an integer, which sucks if you intend to do anything with it.

And the most important benefit is, of course, not when you try to make python act like a shell, but when you can stop pretending shell is a real programming language.

Consider this gem from Arch Linux's ``/etc/rc.shutdown`` script. Here, DAEMONS is a list of things that started on boot, and this script is trying to shut them down in reverse order, unless the daemon name starts with "!"::

        # Shutdown daemons in reverse order
        let i=${#DAEMONS[@]}-1
        while [ $i -ge 0 ]; do
                if [ "${DAEMONS[$i]:0:1}" != '!' ]; then
                        ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@}
                fi
                let i=i-1
        done

Nice uh?

Now, how would that look in python (I may have inverted the meaning of ck_daemon)?

.. code-block:: python

        # Shutdown daemons in reverse order
        for daemon in reversed(DAEMONS):
            if daemon[0]=='!':
                continue
            if ck_daemon(daemon):
                stop_daemon(daemon)

Where stop_daemon used to be this::

    stop_daemon() {
        /etc/rc.d/$1 stop
    }


And will now be this:

.. code-block:: python

    def stop_daemon(daemon):
        run(cmd('/etc/rc.d/{} stop',daemon))


So, come on, people, we are in the 21st century, and shell scripting sucked in the 20th already.

.. [1] I leave that as exercise for the reader.