Posts about linux

The Minimal Server

I was a sysadmin for a long time. I did that for money, so I never really wanted to spend time doing the same thing in my own time, which lead to a severe case of cobbler's children walking barefoot in my private server.

So, today at lunch, I decided to clean up my garbage. So this is what I ended up with, which is the minimal server that is good enough to be generally useful for me.

Hosting

This is a cheap VPS provided by the nice folks at burst.net who are not giving me anything to speak nice things about their service. However, I will do it anyway:

  • Crazy cheap ($5.50 but I have a 20% discount for life)
  • Good amount of monthly bandwidth
  • Lots of disk space
  • Good uptime
  • Fast network
  • Very cheap
  • Decent performance

Distribution

I had CentOS 5 installed, and it stays. If burst ever starts offering Ubuntu Precise, I may switch. Or, since this works, I may not.

What's good about CentOS? It's stable and boring.

What's bad about CentOS? It's too boring. Lots of cool stuff just isn't packaged.

Web Server

I need to serve a bunch of domains, but I have a peculiarity: they are all static sites. I want:

  • Low resource usage
  • Decent performance (that mostly involves supporting ranges and content negotiation)
  • Stable
  • Support directory indexes
  • Easy configuration
  • Virtual domains by name

Almost any server works well for this. Even Apache, except for the easy configuration bit. I ended up with gatling because it fits those criteria fairly well.

  • It uses about 1.4MB of RAM , which is always nice in a VPS
  • It's pretty fast
  • Has not crashed in 2 hours?
  • Supports indexes
  • Here's the configuration: "-c /srv/www -P 2M -d -v -p 80 -F -S" (yes, there is no config file at all)
  • Virtual domains are just folders and symlinks inside /srv/www which is the easiest possilble way to do it.
  • It supports reverse proxying for when I want to try a python web app I am working on.

Mail Server

No, I don't want a mail server. I have gmail and/or a real mail server for that. I want to get the mails from cron. For this, I used ssmtp and an extra gmail account. It works, and here's the whole config:

[email protected]
mailhub=smtp.gmail.com:587
UseTLS=YES
UseSTARTTLS=YES
AuthMethod=LOGIN
[email protected]
AuthPass=notputtingthetrueoneheredude

The best I can say about this configuration is that it works, and doesn't involve running a daemon.

Misc

For when I need to be in two places at the same time: OpenVPN rules, and there is no argument. I have a squid running occasionally, and there is a Quassel core for IRC stuff. I installed mosh to make ssh less painful, rsync handles file deployment and backup storage, cron schedules stuff, and that's it.

Status

Plenty of free RAM and CPU (yes, that's the full process list):

[[email protected] ~]# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1   2156   664 ?        Ss   22:01   0:00 init [3]
root      1135  0.0  0.1   2260   576 ?        S<s  22:01   0:00 /sbin/udevd -d
root      1518  0.0  0.1   1812   572 ?        Ss   22:01   0:00 syslogd -m 0
root      1594  0.0  0.1   7240  1032 ?        Ss   22:01   0:00 /usr/sbin/sshd
root      1602  0.0  0.2   4492  1112 ?        Ss   22:01   0:00 crond
root      1630  0.0  0.1   5684   716 ?        Ss   22:01   0:00 /usr/sbin/saslauthd -m /var/run/saslauthd -a pam -n 2
root      1631  0.0  0.0   5684   444 ?        S    22:01   0:00 /usr/sbin/saslauthd -m /var/run/saslauthd -a pam -n 2
root      1636  0.0  0.2   3852  1372 ?        S    22:01   0:01 /opt/diet/bin/gatling -c /srv/www -P 2M -d -v -p 80 -F -S
root      1677  0.0  0.2   4284  1232 ?        Ss   22:02   0:00 SCREEN /root/quasselcore-static-0.7.1
root      1678  0.0  2.1  36688 11148 pts/0    Ssl+ 22:02   0:03 /root/quasselcore-static-0.7.1
root      3228  1.0  0.7  12916  4196 ?        Ss   23:28   0:13 mosh-server new -s -c 8
root      3229  0.0  0.3   3848  1588 pts/2    Ss   23:28   0:00 -bash
root      3275  0.0  0.1   2532   908 pts/2    R+   23:48   0:00 ps aux
[[email protected] ~]# w
 23:49:03 up  1:47,  1 user,  load average: 0.00, 0.01, 0.00
USER     TTY      FROM              [email protected]   IDLE   JCPU   PCPU WHAT
root     pts/2    186.153.52.253   23:28    0.00s  0.01s  0.00s w
[[email protected] ~]# free
             total       used       free     shared    buffers     cached
Mem:        524800      49100     475700          0          0          0
-/+ buffers/cache:      49100     475700
Swap:            0          0          0

All things considered, fairly happy with the result.

Ubuntu One APIs by Example (part 1)

One of the nice things about working at Canonical is that we produce open source software. I, specifically, work in the team that does the desktop clients for Ubuntu One which is a really cool job, and a really cool piece of software. However, one thing not enough people know, is that we offer damn nice APIs for developers. We have to, since all our client code is open source, so we need those APIs for ourselves.

So, here is a small tutorial about using some of those APIs. I did it using Python and PyQt for several reasons:

  • Both are great tools for prototyping
  • Both have good support for the required stuff (DBus, HTTP, OAuth)
  • It's what I know and enjoy. Since I did this code on a sunday, I am not going to use other things.

Having said that, there is nothing python-specific or Qt-specific in the code. Where I do a HTTP request using QtNetwork, you are free to use libsoup, or whatever.

So, on to the nuts and bolts. The main pieces of Ubuntu One, from a infrastructure perspective, are Ubuntu SSO Client, that handles user registration and login, and SyncDaemon, which handles file synchronization.

To interact with them, on Linux, they offer DBus interfaces. So, for example, this is a fragment of code showing a way to get the Ubuntu One credentials (this would normally be part of an object's __init__):

# Get the session bus
bus = dbus.SessionBus()

:
:
:

# Get the credentials proxy and interface
self.creds_proxy = bus.get_object("com.ubuntuone.Credentials",
                        "/credentials",
                        follow_name_owner_changes=True)

# Connect to signals so you get a call when something
# credential-related happens
self.creds_iface = dbus.Interface(self.creds_proxy,
    "com.ubuntuone.CredentialsManagement")
self.creds_proxy.connect_to_signal('CredentialsFound',
    self.creds_found)
self.creds_proxy.connect_to_signal('CredentialsNotFound',
    self.creds_not_found)
self.creds_proxy.connect_to_signal('CredentialsError',
    self.creds_error)

# Call for credentials
self._credentials = None
self.get_credentials()

You may have noticed that get_credentials doesn't actually return the credentials. What it does is, it tells SyncDaemon to fetch the credentials, and then, when/if they are there, one of the signals will be emitted, and one of the connected methods will be called. This is nice, because it means you don't have to worry about your app blocking while SyncDaemon is doing all this.

But what's in those methods we used? Not much, really!

def get_credentials(self):
    # Do we have them already? If not, get'em
    if not self._credentials:
        self.creds_proxy.find_credentials()
    # Return what we've got, could be None
    return self._credentials

def creds_found(self, data):
    # Received credentials, save them.
    print "creds_found", data
    self._credentials = data
    # Don't worry about get_quota yet ;-)
    if not self._quota_info:
        self.get_quota()

def creds_not_found(self, data):
    # No credentials, remove old ones.
    print "creds_not_found", data
    self._credentials = None

def creds_error(self, data):
    # No credentials, remove old ones.
    print "creds_error", data
    self._credentials = None

So, basically, self._credentials will hold a set of credentials, or None. Congratulations, we are now logged into Ubuntu One, so to speak.

So, let's do something useful! How about asking for how much free space there is in the account? For that, we can't use the local APIs, we have to connect to the servers, who are, after all, the ones who decide if you are over quota or not.

Access is controlled via OAuth. So, to access the API, we need to sign our requests. Here is how it's done. It's not particularly enlightening, and I did not write it, I just use it:

def sign_uri(self, uri, parameters=None):
    # Without credentials, return unsigned URL
    if not self._credentials:
        return uri
    if isinstance(uri, unicode):
        uri = bytes(iri2uri(uri))
    print "uri:", uri
    method = "GET"
    credentials = self._credentials
    consumer = oauth.OAuthConsumer(credentials["consumer_key"],
                                   credentials["consumer_secret"])
    token = oauth.OAuthToken(credentials["token"],
                             credentials["token_secret"])
    if not parameters:
        _, _, _, _, query, _ = urlparse(uri)
        parameters = dict(cgi.parse_qsl(query))
    request = oauth.OAuthRequest.from_consumer_and_token(
                                        http_url=uri,
                                        http_method=method,
                                        parameters=parameters,
                                        oauth_consumer=consumer,
                                        token=token)
    sig_method = oauth.OAuthSignatureMethod_HMAC_SHA1()
    request.sign_request(sig_method, consumer, token)
    print "SIGNED:", repr(request.to_url())
    return request.to_url()

And how do we ask for the quota usage? By accessing the https://one.ubuntu.com/api/quota/ entry point with the proper authorization, we would get a JSON dictionary with total and used space. So, here's a simple way to do it:

    # This is on __init__
    self.nam = QtNetwork.QNetworkAccessManager(self,
        finished=self.reply_finished)

:
:
:

def get_quota(self):
    """Launch quota info request."""
    uri = self.sign_uri(QUOTA_API)
    url = QtCore.QUrl()
    url.setEncodedUrl(uri)
    self.nam.get(QtNetwork.QNetworkRequest(url))

Again, see how get_quota doesn't return the quota? What happens is that get_quota will launch a HTTP request to the Ubuntu One servers, which will, eventually, reply with the data. You don't want your app to block while you do that. So, QNetworkAccessManager will call self.reply_finished when it gets the response:

def reply_finished(self, reply):
    if unicode(reply.url().path()) == u'/api/quota/':
        # Handle quota responses
        self._quota_info = json.loads(unicode(reply.readAll()))
        print "Got quota: ", self._quota_info
        # Again, don't worry about update_menu yet ;-)
        self.update_menu()

What else would be nice to have? How about getting a call whenever the status of syncdaemon changes? For example, when sync is up to date, or when you get disconnected? Again, those are DBus signals we are connecting in our __init__:

self.status_proxy = bus.get_object(
    'com.ubuntuone.SyncDaemon', '/status')
self.status_iface = dbus.Interface(self.status_proxy,
    dbus_interface='com.ubuntuone.SyncDaemon.Status')
self.status_iface.connect_to_signal(
    'StatusChanged', self.status_changed)

# Get the status as of right now
self._last_status = self.process_status(
    self.status_proxy.current_status())

And what's status_changed?

def status_changed(self, status):
    print "New status:", status
    self._last_status = self.process_status(status)
    self.update_menu()

The process_status function is boring code to convert the info from syncdaemon's status into a human-readable thing like "Sync is up-to-date". So we store that in self._last_status and update the menu.

What menu? Well, a QSystemTrayIcon's context menu! What you have read are the main pieces you need to create something useful: a Ubuntu One tray app you can use in KDE, XFCE or openbox. Or, if you are on unity and install sni-qt, a Ubuntu One app indicator!

http://ubuntuone.com/7iXTbysoMM9PIUS9Ai4TNn

My Ubuntu One indicator in action.

You can find the source code for the whole example app at my u1-toys project in launchpad and here is the full source code (missing some icon resources, just get the repo)

Coming soon(ish), more example apps, and cool things to do with our APIs!

$HOME is where .bashrc is

I have a confession to make. For the last year or so, my main operating system has been Windows 7. Yes, I know, it may come as a shock to some, specially if you have read this which is my post that had most hits in a day ever.

How did that happen? What happened to me? What was I thinking? It's a boring and uninteresting story.

I joined Canonical. My old notebook wouldn't cut it. My new one would not take Ubuntu without a fight. I said "hey, I will live in a VM!". The VM was deadly slow. I had to develop windows software (yes). Some stuff would not work right on the VM. And slowly, things just started piling up in the bare-metal OS, which was, yes Windows 7 Home Premium.

As a whole, Windows 7 is not horrible. Most things work well. What it is, is a desert for a developer. Sure, you can get a plant to grow there, but you have to put a lot of effort into it.

So, today I installed Kubuntu Oneiric (absolutely no problem now!), gathered all the data from the old notebook, the VM, the windows installation, deleted windows, and moved into Linux again, and made Windows the VM.

I missed it.

Android on x86: report

Since I expect Android on tablets to be a big thing in 2010, I am experimenting with the closest thing I can get: Android in my eee 701 Surf 4G:

SDC14690

I got the testing Android 2.0 image from http://android-x86.org. I had the 1.6 "stable" one but it was... well, it worked awful (half the key combos or menu options caused it to crash, reboot or otherwise autocombust).

So... how is it working? Slow, but it has potential!

The bad:

  • It boots quite fast... but my tricked full Arch Linux install boots faster.

  • It works sloooooow, you can see individual letters when you type in the search gadget. I read this is a temporary problem, though.

  • I am getting a "castrated" experience because the open android app stores are not as well stocked as the official android marketplace (and come on, why the heck can't I get free apps from there???)

    I see obvious holes in the app landscape that I suppose are well covered in the market (like, is there a RadioTray replacement?)

    No text editor?

    No semi-decent word processor? Not even one that generates HTML?

  • The web browser is pathetic. It may be nice for a phone, but for a "real" system? It's awful. You get the mobile versions of all sites (obviously) and many don't let you switch to the real ones! (even google does that, for Google Reader), and of course, no flash.

  • The email app is terrible. You can't not-top-post!!!! "In-Reply-To" is off-spec!

  • The WiFi settings are way too hidden. They should pop if you click on the wifi icon.

The good:

  • It shuts down incredibly fast.
  • Some apps are quite nice, specially the Aldiko book reader is awesome (and I can share the ePub books with fbReader on the arch linux side.
  • The included SSH client has great ideas.
  • I love the "all your data is in the SD" approach. I do the same thing with Linux. In fact, I have the same exact data organization now on both OSs :-)
  • The home screen with the sliding app drawer: nice
  • The "grabbable" system notifications on the top bar: very nice
  • The "use the menu key to get the menu" thing? genius ;-)
  • The "everything fullscreen all the time", thing? works on this screen.
  • App installation is a solved problem here.
  • I know I will be able to get Qt working native... can't wait!

I am not sold yet, Arch is just so much faster right now, and it can do so much more, but...

  • I am getting a touchscreen for it, so I can experience it more the way it's meant to be experienced.
  • I am using it a lot to read at night in bed (Just finished Makers, read it, it's cool!).
  • I am using it for casual mail reading (I refuse to reply with that broken app).
  • It's a pretty nice alarm clock, so it's becoming my bedside OS.

I'll write another report once I have the touch screen or a new (hopefully faster!) version running.

The day we saw the dinosaur (an Ada Lovelace Day story)

Today, March 24th is Ada Lovelace day, a day of blogging to celebrate the achievements of women in technology and science.. I am taking the liberty to tag this as python so it appears in the right planets, but that's just to promote Ada Lovelace day. Sorry 'bout that.

I will write about the only person who ever taught me programming, Claudia. I was young, so the earth was still lukewarm, the day we saw the dinosaur.

I was just a green sophomore in the School of Chemical Engineering where, paradoxically I would never take a chemistry class, being an applied math student and all that, and at the time "personal computers" were a novelty, a toy of the upper middle class.

We had spent the first two months of the semester learning how to program the obvious way: writing assembler for a fictional machine on paper by hand, when Claudia broke the news, we were going to see a real computer.

No, not a PC, not even an XT, but a real computer, the one real computer in all the university, and you could hear the type switching to bold as she spoke about it. Sadly it was not as real as the one at the research facility (A MiniVAX!) but it was a real enough PDP.

We would not be allowed to actually use it until the following year, but ... well, it was still something special.

I had been programming for years, even for a year before I saw my first (seriosuly not real) computer, I had followed BASIC programs in my head for days, imagining the space invaders float on the screen of my mind, and stepped into writing machine code inside REM statements in my Timex Sinclair 1000 onto the luxury of a C64, but never noone had taught me anything.

Our small class (maybe 10 students) spent endless hours doing things like traverse a matrix, first by rows, thn by columns, then in a spiral from the top-left, writing programs that followed our endless source of algorithms, the numerical solutions guide.

First assembler, then Fortran, we learned.

She was my Mr. Miyagi, I was a heterosexual Ralph Macchio, and I figured out the most important thing about programming: I was awful at it.

Over the next 20 years that situation has been slowly improving, but I never again had someone teach me programming. Claudia had already taught me everything I needed to know, that code can always improve, that there's more than one way to skin a cat.

That the dinosaur was real and that some day soon my computer would be faster and nicer than the dinosaur was then, and that programming was cool, and that if I could find a way to draw a polynomial graph horizontally on a printer without ever having the whole graph in memory (it didn't fit), those future computers would do awesome things, and that I was one of the many who would help bring that to reality.

That talking about code was fun in itself, that you could make a modest living and be happy about it, that you could in any case make jigsaw puzzles in your spare time and keep on teaching or whatever.

And later the dinosaur's bones were scavenged into a line of racks holding routers, and its glass terminals are destroyed, and the gold in its teeth was stolen and the rare bus cables sold, and its circuits scrapped, but I saw the dinosaur alive, and Claudia taught me how to make it jump, and for that, I will always be grateful.

Marave 0.5 is out!

Just uploaded Marave 0.5 to the usual place. Marave is a relaxed, fullscreen text editor that tries not to distract you.

It even includes a simple music player just so you don't have to think about switching to another application!

This release has several bugs fixed, and looks a bit nicer.

The main new feature is ... internationalizacion. It now includes a spanish translation only, but if you want to help translating it to any other language, please step forward!

There are only 60 phrases to translate, so it shouldn't be more than one hour of work.

Here's a screenshot of this release:

marave7

Marave is free software under the GPLv2, and should work on any platform where PyQt works, which means Windows, Mac, and Unix-like operating systems, at least.

Packaging and shipping is HARD

I have worked really hard on Marave, a full screen editor in the style of ommwriter, DarkRoom, WriteRoom, pyRoom, etc. I have worked very hard and I want users to use it.

Or not even that, I want them to have a chance of using it.

That means I want it to work on Windows (and maybe OSX some day, too, if someone helps me). Which means, I have to pakage it for windows.

Let's do a quick comparison here from the points of view of the user and the developer.

The User, In Linux

This is in Arch Linux, which is what I use, in other Linux variants it will be pretty much the same once Marave is a bit more well known.

yaourt -S marave-svn --noconfirm

That gets the code from SVN (right now it's the best way, later I will package releases, too), all required dependencies, builds and installs. It takes all of 15 seconds in my notebook.

After that, you have a fully working Marave.

In case it's not packaged for your distro, just install PyQt (which surely is) and run this command:

easy_install marave

The User, in Windows

You go to http://marave.googlecode.com, click on "Marave-0.5.win32.exe" (Not linked yet, it's not finished), then download a 10MB program. That is a 10MB program because windows doesn't believe in packages and dependencies. On Linux, a Marave package could be under 1MB (most of it images), and not be executable, just data.

Of course nowadays web browsers don't actually run programs on download, so... let's see it as a gallery!

110111105613-My-Desktop

Yes, save it.

11011111220-My-Desktop

Double click to open it

11011111417-My-Desktop

Yes, I agree

11011111514-My-Desktop

Sure, whatever

1101111167-My-Desktop

Nice...

11011111750-My-Desktop

Good to hear!

Now, this Marave that just got installed may or may not currently work because of a missing MSVCR90.DLL but that's for the next section...

The Developer, in Linux

First, here's the biggest problem a Linux packager can have:

Since Marave is a new app, and I develop it in the rather cutting-edge Arch Linux, it uses some newish features only available in recent versions of Qt. In fact, it doesn't work with PyQt < 4.6, which is not available in some slow distros, like Debian, or even in a not-latest Ubuntu.

Solution? Well, I could just ignore it, but what the heck, let's fix it instead!

Thanks to PyInstaller it's not even hard to do, here's the spec file:

a = Analysis([os.path.join(HOMEPATH,'support/_mountzlib.py'), os.path.join(HOMEPATH,'support/useUnicode.py'), 'marave/main.py'],
            pathex=['/home/ralsina/trunk/trunk'])

pyz = PYZ(a.pure)
exe = EXE(pyz,
        a.scripts,
        exclude_binaries=1,
        name=os.path.join('build/pyi.linux2/main', 'marave.exe'),
        debug=False,
        strip=False,
        upx=True,
        console=0 )

coll = COLLECT( exe,
            a.binaries,
            [('radios.txt','marave/radios.txt','DATA')],
            Tree('marave/icons','icons'),
            Tree('marave/backgrounds','backgrounds'),
            Tree('marave/clicks','clicks'),
            Tree('marave/stylesheets','stylesheets'),
            Tree('marave/themes','themes'),
            a.zipfiles,
            a.datas,
            strip=False,
            upx=True,
            name=os.path.join('dist', 'marave'))

Use this, and PyInstaller will produce a nice folder full of everything Marave needs to run on any Linux.

OTOH, if you can rely on a recent PyQt being available, it's also simple. Here's a packaging configuration for a similar package in Arch Linux (I must confess not having done one for Marave yet). For other distributions it should be about as simple, if more verbose, and someone else probably does it for you:

# Contributor: Roberto Alsina <[email protected]>
pkgname=python-rst2pdf
pkgver=0.12.1
pkgrel=4
pkgdesc="Create PDFs from simple text markup, no LaTeX required."
arch=('i686' 'x86_64')
url="http://rst2pdf.googlecode.com"
license=('custom')
depends=('python' 'setuptools' 'docutils' 'pygments' 'python-reportlab' 'python-simplejson' 'pil')
source=(http://rst2pdf.googlecode.com/files/rst2pdf-$pkgver.tar.gz LICENSE.txt)
optdepends=('uniconvertor: vector images support'
            'python-svglib: SVG support'
            'python-wordaxe: hyphenation'
            'pythonmagick: PDF images support')
build() {
cd $startdir/src/rst2pdf-$pkgver
python setup.py install --root=$startdir/pkg || return 1
install -D ../LICENSE.txt $startdir/pkg/usr/share/licenses/python-rst2pdf/COPYING
install -D doc/rst2pdf.1 $startdir/pkg/usr/share/man/man1/rst2pdf.1
}
md5sums=('ea6beda9a46f34ba42c4c94d48cc607a'
        '416f8046c66b9476cdbacda69a673afe')

And that's all you need to know about the process of packaging your app for Linux. It's easy to do, and most of the time, easy to do right!

Now, let's go to our final section...

Windows for the developer

First, remember that of relying on the system's version of Qt? Forget it, there is no system version available. And no python either. And noone is going to install it or your app, so it's "ship everything yourself" mode, or nothing.

But anyway, PyInstaller works for Windows too! So, using the same spec file, it works. Right?

Well, no beause of two problems.

Problem 1: You need an installer

Users are not going to open a zip somewhere, then do a shortcut to the binary on Windows, so you need to do some operations, and that means an installer.

Here's what I came up with to use NSIS, a free installer creator for Windows:

;--------------------------------
;Include Modern UI

!include "MUI2.nsh"

;--------------------------------
;General

;Name and file
Name "Marave"
OutFile "Marave-0.5.win32.exe"

;Default installation folder
InstallDir "$LOCALAPPDATA\Marave"

;Get installation folder from registry if available
InstallDirRegKey HKCU "Software\Marave" ""

;Request application privileges for Windows Vista
RequestExecutionLevel user

;--------------------------------
;Interface Settings

!define MUI_ABORTWARNING

;--------------------------------
;Pages

!insertmacro MUI_PAGE_LICENSE "LICENSE"
!insertmacro MUI_PAGE_DIRECTORY
!insertmacro MUI_PAGE_INSTFILES

!insertmacro MUI_UNPAGE_CONFIRM
!insertmacro MUI_UNPAGE_INSTFILES

;--------------------------------
;Languages

!insertmacro MUI_LANGUAGE "English"

;--------------------------------
;Installer Sections

Section "Install"

SetOutPath "$INSTDIR"
File /r "dist\marave"


;Store installation folder
WriteRegStr HKCU "Software\Marave" "" $INSTDIR

;Create uninstaller
WriteUninstaller "$INSTDIR\Uninstall.exe"

;Create shortcuts
CreateDirectory $SMPROGRAMS\Marave
CreateShortCut "$SMPROGRAMS\Marave\Marave.lnk" "$INSTDIR\marave\marave.exe" ; use defaults for parameters, icon, etc.
CreateShortCut "$SMPROGRAMS\Marave\Uninstall Marave.lnk" "$INSTDIR\Uninstall.exe" ; use defaults for parameters, icon, etc.

SectionEnd


;--------------------------------
;Uninstaller Section

Section "Uninstall"

Delete "$INSTDIR\Uninstall.exe"
RMDir /r "$INSTDIR"

DeleteRegKey /ifempty HKCU "Software\Marave"

SectionEnd

It's comparable to the effort of building a packaging file, really, except every time you want to test it... you install it. There is no way (AFAICS) to see what's inside the installer except running it!

When things fail, you get no error messages, at least not the kind that is useful for a developer, the guy that needs to know what went wrong.

And after it's finished, you may end with a non-working program because of...

Problem 2: system libraries don't exist

Python 2.6 binaries are built using Visual Studio. That means they require the Visual Studio Runtime, specifically MSVCR90.DLL. That contains what on Linux would be considered part of libc. (linux guy: imagine apps that depend on a specific libc... hard to do!)

On Linux that's part of the system. Further, if you wanted, you can redistribute it. On Windows... well, it's a bit different.

  1. It's part of the "Visual C++ redistributables"

  2. Installing that doesn't guarantee it will work (yes, I have tried)

  3. The license for those 'redistributables' says you can't make them available for download.

    I have been told that including that in your installer is fine and dandy, but how is that not making them available for download?

So what can you do when you need a library and can't ship it and the user won't install it?

Well, that's why there is no Windows binary of Marave yet. Of course if anyone can help, I'd be really, really happy!

Happy 10th blogiversary to me!

Since yesterday this blog is ten years old so, time for some history.

It all started in advogato where you could still read it today! (Please read it here instead ;-)

Then it moved to PyDS an early python desktop blog platform with a web interface, and was hosted in PyCS, a free service.

Then PyCS kinda died, and I started generating a static blog and hosting it in my ISP's free hosting. That sucked bad.

Then I started my own company, and I had my own servers, so I started hosting it there (even today this blog is completely static HTML!)

Then PyDS started acting weird, so I wrote my own blogging software, which is a real mess, perhaps 25% finished, but it does things exactly the way I like them.

Currently, this blog is syndicated in Planeta PyAr, Planet Python, Planet Qt, Planeta LUGLI, and a couple other places.

This year, I decided to make the blog completely bilingual (English and Spanish), but I hate translating it.

According to the stats I have available, the blog is in average more popular now than ever (but yes, my most popular posts were years ago ;-)

stats

These are the most popular pages in the last year:

Lessons:

  1. I need to write more about Qt and/or start flamewars with clueless IT writers
  2. I need to search for ancient material and deprecate it
  3. Having your own hosting and blogging software is neat
  4. 10 years is a lot of time: 860 posts (or 913, depending on how you count)

With iterpipes, python is ready to replace bash for scripting. Really.

This has been a pet peeve of mine for years: programming shell scripts suck. They are ugly and error prone. The only reason why we still do it? There is no real replacement.

Or at least that was the case, until today I met iterpipes at python.reddit.com

Iterpipes is "A library for running shell pipelines using shell-like syntax" and guess what? It's brilliant.

Here's an example from its PYPI page:

# Total lines in *.py files under /path/to/dir,
# use safe shell parameters formatting:

>>> total = cmd(
...     'find {} -name {} -print0 | xargs -0 wc -l | tail -1 | awk {}',
...     '/path/to/dir', '\*.py', '{print $1}')
>>> run(total | strip() | join | int)
315

Here's how that would look in shell:

find /path/to/dir -name '*.py' -print0 | xargs -0 wc -l | tail -1 | awk '{print $1}'

You may say the shell version looks better. That's an illusion caused by the evil that is shell scripting: the shell version is buggy.

Why is it buggy? Because if I control what's inside /path/to/dir I can make that neat little shell command fail [1], but at least in python I can handle errors!

Also, in most versions you could attempt to write, this command would be unsafe because quoting and escaping in shell is insane!

The iterpipes version uses the equivalent of SQL prepared statements which are much safer.

It's nearly impossible to do such a command in pure shell and be sure it's safe.

Also, the shell version produces a string instead of an integer, which sucks if you intend to do anything with it.

And the most important benefit is, of course, not when you try to make python act like a shell, but when you can stop pretending shell is a real programming language.

Consider this gem from Arch Linux's /etc/rc.shutdown script. Here, DAEMONS is a list of things that started on boot, and this script is trying to shut them down in reverse order, unless the daemon name starts with "!":

# Shutdown daemons in reverse order
let i=${#DAEMONS[@]}-1
while [ $i -ge 0 ]; do
        if [ "${DAEMONS[$i]:0:1}" != '!' ]; then
                ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@}
        fi
        let i=i-1
done

Nice uh?

Now, how would that look in python (I may have inverted the meaning of ck_daemon)?

# Shutdown daemons in reverse order
for daemon in reversed(DAEMONS):
    if daemon[0]=='!':
        continue
    if ck_daemon(daemon):
        stop_daemon(daemon)

Where stop_daemon used to be this:

stop_daemon() {
    /etc/rc.d/$1 stop
}

And will now be this:

def stop_daemon(daemon):
    run(cmd('/etc/rc.d/{} stop',daemon))

So, come on, people, we are in the 21st century, and shell scripting sucked in the 20th already.

[1] I leave that as exercise for the reader.