Posts about programming (old posts, page 35)

2012-01-08 23:21

Abandonment issues: rst2pdf

Of all the corpses of my projects, there is one thatI feel worse about, which is rst2pdf. I feel bad about abandnoning several, but rst2pdf was actually a useful tool, used by a bunch of people, and that it has never gathered enough momentum with other developers is sad.

So, I will pick it up. I will spend about 4 hours a week on it. The plan is to:

  1. Gather some patches that are lingering on the issue tracker
  2. Fix some simple-ish bugs
  3. Make another release with 1) and 2)

And of course:

  1. Not let it fall in disrepair again

In the meantime, here is a nice thing I just heard about. Dimitri Christodoulou has hacked rst2pdf so that it can handle the raw:: html directive.

This, dear friends is completely nuts, absolutely out of scope for any given docutils tool, and just too cool :-)

I will try to hijack his code (proper credit and so on), and incorporate it into rst2pdf.

And Dimitri, or anyone else who wants to do cool stuff with rst2pdf: let me know! I will give you commit rights immediately!

2012-01-07 23:06

Python context managers: they are easy!

This comes from this thread in the Python Argentina mailing list (which I strongly recommend if you read spanish).

I was the other day trying to do shell-script-like-things on python (as part of a monster setup.py) and I was annoyed that in shell it's easy to do this:

cd foo
bar -baz
cd -

Or this:

pushd foo
bar -baz
popd

Or this:

(cd foo && bar -baz)

And on Python I had to do this, which is verbose and ugly:

cwd = os.getcwd()
try:
    os.chdir('foo')
    os.system('bar -baz')
finally:
    os.chdir(cwd)

This is what I wanted to have:

with os.chdir('foo'):
    os.system('bar -baz')

And of course, you can't do that. So, I asked, how do you implement that context manager? I got several answers.

  1. That's available in Fabric:

    with cd("foo"):
        run("bar")
    
  2. It's not hard to do:

    class DirContextM(object):
        def __init__(self, new_dir):
            self.new_dir = new_dir
            self.old_dir = None
    
        def __enter__(self):
            self.old_dir = os.getcwd()
            os.chdir(self.new_dir)
    
        def __exit__(self, *_):
            os.chdir(self.old_dir)
    
  3. It's even easier to do:

    from contextlib import contextmanager
    
    @contextmanager
    def cd(path):
        old_dir = os.getcwd()
        os.chdir(path)
        yield
        os.chdir(old_dir)
    
  4. That's cool, so let's add it to path.py

  5. Maybe check for exceptions

    @contextmanager
    def cd(path):
        old_dir = os.getcwd()
        os.chdir(path)
        try:
            yield
        finally:
            os.chdir(old_dir)
    

All in all, I learned how to do context managers, about contextlib, about fabric and about path.py. Which is not bad for 15 minutes :-)

2011-07-24 21:20

Shipping your PyQt app for windows

I have written about this in the past, with the general conclusion being "it's a pain in the ass".

So, now, here is how it's done.

  1. Start with a working PyQt application. In this example, I will use devicenzo.py mostly because:
    1. It is a working PyQt application.
    2. It uses a big chunk of PyQt
    3. It's easy to test
  2. Now you need a setup.py. Here's one that works, with extensive commments.
# We will be using py2exe to build the binaries.
# You may use other tools, but I know this one.

from distutils.core import setup
import py2exe

# Now you need to pass arguments to setup
# windows is a list of scripts that have their own UI and
# thus don't need to run in a console.

setup(windows=['devicenzo.py'],
      options={

# And now, configure py2exe by passing more options;

          'py2exe': {

# This is magic: if you don't add these, your .exe may
# or may not work on older/newer versions of windows.

              "dll_excludes": [
                  "MSVCP90.dll",
                  "MSWSOCK.dll",
                  "mswsock.dll",
                  "powrprof.dll",
                  ],

# Py2exe will not figure out that you need these on its own.
# You may need one, the other, or both.

              'includes': [
                  'sip',
                  'PyQt4.QtNetwork',
                  ],

# Optional: make one big exe with everything in it, or
# a folder with many things in it. Your choice
#             'bundle_files': 1,
          }
      },

# Qt's dynamically loaded plugins and py2exe really don't
# get along.

data_files = [
            ('phonon_backend', [
                'C:\Python27\Lib\site-packages\PyQt4\plugins\phonon_backend\phonon_ds94.dll'
                ]),
            ('imageplugins', [
            'c:\Python27\lib\site-packages\PyQt4\plugins\imageformats\qgif4.dll',
            'c:\Python27\lib\site-packages\PyQt4\plugins\imageformats\qjpeg4.dll',
            'c:\Python27\lib\site-packages\PyQt4\plugins\imageformats\qsvg4.dll',
            ]),
],

# If you choose the bundle above, you may want to use this, too.
#     zipfile=None,
)
  1. Run python setup.py py2exe and get a dist folder full of binary goodness.

And that's it. Except of course, that's not it.

What this will do is create a binary set, either a folder full of things, or a single EXE file. And that's not enough. You have to consider at least the following:

  1. Put everything in resource files: images, qss files, icons, etc. Every file your app needs? Put it in a resource file and load it from there. That way you don't have to care about them if you go the "one exe" road.
  2. Compile .ui files to .py (same reason)
  3. Figure out if you use Qt's plugins, and make them work. This includes: using Phonon, using QtSQL, and using any image formats other than PNG.

After you have that, are you done? NO!

Your windows user will want an installer. I am not going to go into details, but I had a good time using BitRock's InstallBuilder for Qt. It's a nice tool, and it works. That's a lot in this field.

But is that all? NO!

You have to take care of the Visual Studio Runtime. My suggestion? Get a copy of the 1.1MB vcredist_x86.exe (not the larger one, the 1.1MB one), and either tell people to install it manually, or add it to your installer. You are legally allowed (AFAIK) to redistribute that thing as a whole. But not what's in it (unless you have a VS license).

And we are done? NO!

Once you run your app "installed", if it ever prints anything to stderr, you will get either a dialog telling you it did, or worse (if you are in aything newer than XP), a dialog telling you it can't write to a log file, and the app will never work again.

This is because py2exe catches stderr and tries to save it on a logfile. Which it tries to create in the same folder as the binary. Which is usually not allowed because of permissions.

Solution? Your app should never write to stderr. Write an excepthook and catch that. And then remove stderr or replace it with a log file, or something. Just don't let py2exe do it, because the way py2exe does it is broken.

And is that it?

Well, basically yes. Of course you should get 4 or 5 different versions of windows to test it on, but you are pretty much free to ship your app as you wish. Oh, mind you, don't upload it to downloads.com because they will wrap your installer in a larger one that installs bloatware and crap.

So, there you go.

2011-03-25 02:30

Creating a forum the easy way (32 LOC)

This is only the first part of a project to create the simplest (for me) software forum possible.

Here are the features I want:

  • Login using twitter / Facebook / Google / OpenID
  • Unlimited number of threads
  • Support for like / dislike both on threads and on posts
  • Avatars
  • HTML in comments
  • Mail the user on replies
  • RSS feeds for threads

You can see it in action at http://foro.netmanagers.com.ar (for a limited time only ;-)

And here is the code:

import bottle
import disqusapi as disqus
import json
shortname = 'magicmisteryforum'
api = disqus.DisqusAPI(open("key").read().strip())

@bottle.route('/', method='GET')
def index():
    msg = bottle.request.GET.get('msg', '')
    threads = api.forums.listThreads(forum=shortname, limit=100)
    print threads[0]
    return bottle.template('main.tpl', threads=threads, shortname=shortname, msg=msg)

@bottle.route('/new', method='POST')
def new():
    title = bottle.request.forms.get('title', None)
    if not title:
        bottle.redirect('/?msg=Missing%20Thread%20Name')
        return
    thread = api.threads.create(forum=shortname, title = title)
    thread_id = thread.__dict__['response']['id']
    # Redirecting to /thread/thread_id doesn't work
    # because threads take a few seconds to appear on the listing
    bottle.redirect('/')

@bottle.route('/thread/:id')
def thread(id):
    t = api.threads.details(thread=id)
    return bottle.template('thread.tpl', shortname=shortname, id=id, thread=t.__dict__['response'])

@bottle.route('/static/:path#.+#')
def server_static(path):
    return bottle.static_file(path, root='./static')

app = bottle.app()
app.catchall = False #Now most exceptions are re-raised within bottle.
bottle.run(host='184.82.108.14', port=80, app=app)

It requires Bottle and the Disqus python API

Of course, there is also a bit of templating involved, here is main.tpl and the thread.tpl. Since I suck at HTML, it uses Bluetrip CSS and it's more than simple enough to customize.

OF COURSE I AM CHEATING!

This thing is just a simple veneer around Disqus! More like a blog with comments and without posts than a forum! But ... what's missing to make this a real forum? It works, doesn't it? You could even use Disqus categories to create subforums...

All things considered, I think it's a cute hack.

And if you wait a few days, this will lead to something much more magical!

Full source code at http://magicforum.googlecode.com

2011-03-11 01:47

New golfing challenge: PatoCabrera

In the spirit of the De Vicenzo web browser, I am starting a new program, called Pato Cabrera. Here are the rules:

  • Twitter client (no identi.ca in the first version, but to be added later)
  • Has these features: http://pastebin.lugmen.org.ar/6464
  • Has to be implemented before April 4th
  • Smaller than 16384 bytes (of python code) but may be larger because of artwork.

Let's see how it works :-)

2011-03-08 21:45

OK, so THAT is how much browser I can put in 128 lines of code.

I have already posted a couple of times (1, 2) about De Vicenzo , an attempt to implement the rest of the browser, starting with PyQt's WebKit... limiting myself to 128 lines of code.

Of course I could do more, but I have my standards!

  • No using ;
  • No if whatever: f()

Other than that, I did a lot of dirty tricks, but right now, it's a fairly complete browser, and it has 127 lines of code (according to sloccount) so that's enough playing and it's time to go back to real work.

But first, let's consider how some features were implemented (I'll wrap the lines so they page stays reasonably narrow), and also look at the "normal" versions of the same (the "normal" code is not tested, please tell me if it's broken ;-).

This is not something you should learn how to do. In fact, this is almost a treatise on how not to do things. This is some of the least pythonic, less clear code you will see this week.

It is short, and it is expressive. But it is ugly.

I'll discuss this version.

Proxy Support

A browser is not much of a browser if you can't use it without a proxy, but luckily Qt's network stack has good proxy support. The trick was configuring it.

De Vicenzo supports HTTP and SOCKS proxies by parsing a http_proxy environment variable and setting Qt's application-wide proxy:

 proxy_url = QtCore.QUrl(os.environ.get('http_proxy', ''))
 QtNetwork.QNetworkProxy.setApplicationProxy(QtNetwork.QNetworkProxy(\
 QtNetwork.QNetworkProxy.HttpProxy if unicode(proxy_url.scheme()).startswith('http')\
 else QtNetwork.QNetworkProxy.Socks5Proxy, proxy_url.host(),\
 proxy_url.port(), proxy_url.userName(), proxy_url.password())) if\
'http_proxy' in os.environ else None

How would that look in normal code?

if 'http_proxy' in os.environ:
    proxy_url = QtCore.QUrl(os.environ['http_proxy'])
    if unicode(proxy_url.scheme()).starstswith('http'):
        protocol = QtNetwork.QNetworkProxy.HttpProxy
    else:
        protocol = QtNetwork.QNetworkProxy.Socks5Proxy
    QtNetwork.QNetworkProxy.setApplicationProxy(
        QtNetwork.QNetworkProxy(
            protocol,
            proxy_url.host(),
            proxy_url.port(),
            proxy_url.userName(),
            proxy_url.password()))

As you can see, the main abuses against python here are the use of the ternary operator as a one-line if (and nesting it), and line length.

Persistent Cookies

You really need this, since you want to stay logged into your sites between sessions. For this, first I needed to write some persistence mechanism, and then save/restore the cookies there.

Here's how the persistence is done (settings is a global QSettings instance):

def put(self, key, value):
    "Persist an object somewhere under a given key"
    settings.setValue(key, json.dumps(value))
    settings.sync()

def get(self, key, default=None):
    "Get the object stored under 'key' in persistent storage, or the default value"
    v = settings.value(key)
    return json.loads(unicode(v.toString())) if v.isValid() else default

It's not terribly weird code, except for the use of the ternary operator in the last line. The use of json ensures that as long as reasonable things are persisted, you will get them with the same type as you put them without needing to convert them or call special methods.

So, how do you save/restore the cookies? First, you need to access the cookie jar. I couldn't find whether there is a global one, or a per-webview one, so I created a QNetworkCookieJar in line 24 and assign it to each web page in line 107.

# Save the cookies, in the window's closeEvent
self.put("cookiejar", [str(c.toRawForm()) for c in self.cookies.allCookies()])

# Restore the cookies, in the window's __init__
self.cookies.setAllCookies([QtNetwork.QNetworkCookie.parseCookies(c)[0]\
for c in self.get("cookiejar", [])])

Here I confess I am guilty of using list comprehensions when a for loop would have been the correct thing.

I use the same trick when restoring the open tabs, with the added misfeature of using a list comprehension and throwing away the result:

# get("tabs") is a list of URLs
[self.addTab(QtCore.QUrl(u)) for u in self.get("tabs", [])]

Using Properties and Signals in Object Creation

This is a feature of recent PyQt versions: if you pass property names as keyword arguments when you create an object, they are assigned the value. If you pass a signal as a keyword argument, they are connected to the given value.

This is a really great feature that helps you create clear, local code, and it's a great thing to have. But if you are writing evil code... well, you can go to hell on a handbasket using it.

This is all over the place in De Vicenzo, and here's one example (yes, this is one line):

QtWebKit.QWebView.__init__(self, loadProgress=lambda v:\
(self.pbar.show(), self.pbar.setValue(v)) if self.amCurrent() else\
None, loadFinished=self.pbar.hide, loadStarted=lambda:\
self.pbar.show() if self.amCurrent() else None, titleChanged=lambda\
t: container.tabs.setTabText(container.tabs.indexOf(self), t) or\
(container.setWindowTitle(t) if self.amCurrent() else None))

Oh, boy, where do I start with this one.

There are lambda expressions used to define the callbacks in-place instead of just connecting to a real function or method.

There are lambdas that contain the ternary operator:

loadStarted=lambda:\
    self.pbar.show() if self.amCurrent() else None

There are lambdas that use or or a tuple to trick python into doing two things in a single lambda!

loadProgress=lambda v:\
(self.pbar.show(), self.pbar.setValue(v)) if self.amCurrent() else\
None

I won't even try to untangle this for educational purposes, but let's just say that line contains what should be replaced by 3 methods, and should be spread over 6 lines or more.

Download Manager

Ok, calling it a manager is overreaching, since you can't stop them once they start, but hey, it lets you download things and keep on browsing, and reports the progress!

First, on line 16 I created a bars dictionary for general bookkeeping of the downloads.

Then, I needed to delegate the unsupported content to the right method, and that's done in lines 108 and 109

What that does is basically that whenever you click on something WebKit can't handle, the method fetch will be called and passed the network request.

def fetch(self, reply):
    destination = QtGui.QFileDialog.getSaveFileName(self, \
        "Save File", os.path.expanduser(os.path.join('~',\
            unicode(reply.url().path()).split('/')[-1])))
    if destination:
        bar = QtGui.QProgressBar(format='%p% - ' +
            os.path.basename(unicode(destination)))
        self.statusBar().addPermanentWidget(bar)
        reply.downloadProgress.connect(self.progress)
        reply.finished.connect(self.finished)
        self.bars[unicode(reply.url().toString())] = [bar, reply,\
            unicode(destination)]

No real code golfing here, except for long lines, but once you break them reasonably, this is pretty much the obvious way to do it:

  • Ask for a filename
  • Create a progressbar, put it in the statusbar, and connect it to the download's progress signals.

Then, of course, we need ths progress slot, that updates the progressbar:

progress = lambda self, received, total:\
    self.bars[unicode(self.sender().url().toString())][0]\
    .setValue(100. * received / total)

Yes, I defined a method as a lambda to save 1 line. [facepalm]

And the finished slot for when the download is done:

def finished(self):
    reply = self.sender()
    url = unicode(reply.url().toString())
    bar, _, fname = self.bars[url]
    redirURL = unicode(reply.attribute(QtNetwork.QNetworkRequest.\
        RedirectionTargetAttribute).toString())
    del self.bars[url]
    bar.deleteLater()
    if redirURL and redirURL != url:
        return self.fetch(redirURL, fname)
    with open(fname, 'wb') as f:
        f.write(str(reply.readAll()))

Notice that it even handles redirections sanely! Beyond that, it just hides the progress bar, saves the data, end of story. The longest line is not even my fault!

There is a big inefficiency in that the whole file is kept in memory until the end. If you download a DVD image, that's gonna sting.

Also, using with saves a line and doesn't leak a file handle, compared to the alternatives.

Printing

Again Qt saved me, because doing this manually would have been a pain. However, it turns out that printing is just ... there? Qt, specially when used via PyQt is such an awesomely rich environment.

self.previewer = QtGui.QPrintPreviewDialog(\
    paintRequested=self.print_)
self.do_print = QtGui.QShortcut("Ctrl+p",\
    self, activated=self.previewer.exec_)

There's not even any need to golf here, that's exactly as much code as you need to hook Ctrl+p to make a QWebView print.

Other Tricks

There are no other tricks. All that's left is creating widgets, connecting things to one another, and enjoying the awesome experience of programming PyQt, where you can write a whole web browser (except the engine) in 127 lines of code.

2011-03-05 23:46

De Vicenzo: A much cooler mini web browser.

It seems it was only a few days ago that I started this project. Oh, wait, yes, it was just a few days ago!

If you don't want to read that again, the idea is to see just how much code is needed to turn Qt's WebKit engine into a fully-fledged browser.

To do that, I set myself a completely arbitrary limit: 128 lines of code.

So, as of now, I declare it feature-complete.

The new features are:

  • Tabbed browsing (you can add/remove tabs)
  • Bookmarks (you can add/remove them, and choose them from a drop-down menu)

This is what already worked:

  • Zoom in (Ctrl++)
  • Zoom out (Ctrl+-)
  • Reset Zoom (Ctrl+=)
  • Find (Ctrl+F)
  • Hide find (Esc)
  • Buttons for back/forward and reload
  • URL entry that matches the page + autocomplete from history + smart entry (adds http://, that kind of thing)
  • Plugins support (including flash)
  • The window title shows the page title (without browser advertising ;-)
  • Progress bar for page loading
  • Statusbar that shows hovered links URL
  • Takes a URL on the command line, or opens http://python.org
  • Multiplatform (works in any place QtWebKit works)

So... how much code was needed for this? 87 LINES OF CODE

Or if you want the PEP8-compliant version, 115 LINES OF CODE.

Before anyone says it: yes, I know the rendering engine and the toolkit are huge. What I wrote is just the chrome around them, just like Arora, Rekonq, Galeon, Epiphany and a bunch of others do.

It's simple, minimalistic chrome, but it works pretty good, IMVHO.

Here it is in (buggy) action:

It's more or less feature-complete for what I expected to be achievable, but it still needs some fixes.

You can see the code at it's own home page: http://devicenzo.googlecode.com

2011-02-28 21:10

How much web browser can you put in 128 lines of code?

UPDATE: If you read this and all you can say is "oh, he's just embedding WebKit", I have two things to tell you:

  1. Duh! Of course the 128 lines don't include the rendering engine, or the TCP implementation, or the GUI toolkit. This is about the rest of the browser, the part around the web rendering engine. You know, just like Arora, Rekonq, Epiphany, and everyone else that embeds webkit or mozilla does it? If you didn't get that before this explanation... facepalm.
  2. Get your favourite webkit fork and try to do this much with the same amount of code. I dare you! I double dog dare you!

Now back to the original article


Today, because of a IRC chat, I tried to find a 42-line web browser I had written a while ago. Sadly, the pastebin where I posted it was dead, so I learned a lesson: It's not a good idea to trust a pastebin as code repository

What I liked about that 42-line browser was that it was not the typical example, where someone dumps a Webkit view in a window, loads a page and tries to convince you he's cool. That one is only 7 lines of code:

import sys
from PyQt4 import QtGui,QtCore,QtWebKit
app=QtGui.QApplication(sys.argv)
wb=QtWebKit.QWebView()
wb.setUrl(QtCore.QUrl('http://www.python.org'))
wb.show()
sys.exit(app.exec_())

And if I wanted to make the code uglier, it could be done in 6.

But anyway, that 42-line browser actually looked useful!

This 42-line web browser, courtesy of #python and #qt -- http... on Twitpic

Those buttons you see actually worked correctly, enabling and disabling at the right moment, the URL entry changed when you clicked on links, and some other bits.

So, I have decided to start a small, intermittent project of code golf: put as much browser as I can in 128 lines of code (not counting comments or blanks), starting with PyQt4.

This has a useful purpose: I always suspected that if you assumed PyQt was part of the base system, most apps would fit in floppies again. This one fits on a 1.44MB floppy some 500 times (so you could use 360KB commodore floppies if you prefer!).

So far, I am at about 50 lines, and it has the following features:

  • Zoom in (Ctrl++)
  • Zoom out (Ctrl+-)
  • Reset Zoom (Ctrl+=)
  • Find (Ctrl+F)
  • Hide find (Esc)
  • Buttons for back/forward and reload
  • URL entry that matches the page + autocomplete from history + smart entry (adds http://, that kind of thing)
  • Plugins support (including flash)
  • The window title shows the page title (without browser advertising ;-)
  • Progress bar for page loading
  • Statusbar that shows hovered links URL
  • Takes a URL on the command line, or opens http://python.org
  • Multiplatform (works in any place QtWebKit works)

Missing are tabs and proxy support. I expect those will take another 40 lines or so, but I think it's probably the most featureful of these toy browsers.

The code... it's not all that hard. I am using lambda a lot, and I am using PyQt's keyword arguments for signal connection which makes lines long, but not hard. It could be made much smaller!

Here it is in action:

And here's the code:

#!/usr/bin/env python
"A web browser that will never exceed 128 lines of code. (not counting blanks)"

import sys
from PyQt4 import QtGui,QtCore,QtWebKit

class MainWindow(QtGui.QMainWindow):
    def __init__(self, url):
        QtGui.QMainWindow.__init__(self)
        self.sb=self.statusBar()

        self.pbar = QtGui.QProgressBar()
        self.pbar.setMaximumWidth(120)
        self.wb=QtWebKit.QWebView(loadProgress = self.pbar.setValue, loadFinished = self.pbar.hide, loadStarted = self.pbar.show, titleChanged = self.setWindowTitle)
        self.setCentralWidget(self.wb)

        self.tb=self.addToolBar("Main Toolbar")
        for a in (QtWebKit.QWebPage.Back, QtWebKit.QWebPage.Forward, QtWebKit.QWebPage.Reload):
            self.tb.addAction(self.wb.pageAction(a))

        self.url = QtGui.QLineEdit(returnPressed = lambda:self.wb.setUrl(QtCore.QUrl.fromUserInput(self.url.text())))
        self.tb.addWidget(self.url)

        self.wb.urlChanged.connect(lambda u: self.url.setText(u.toString()))
        self.wb.urlChanged.connect(lambda: self.url.setCompleter(QtGui.QCompleter(QtCore.QStringList([QtCore.QString(i.url().toString()) for i in self.wb.history().items()]), caseSensitivity = QtCore.Qt.CaseInsensitive)))

        self.wb.statusBarMessage.connect(self.sb.showMessage)
        self.wb.page().linkHovered.connect(lambda l: self.sb.showMessage(l, 3000))

        self.search = QtGui.QLineEdit(returnPressed = lambda: self.wb.findText(self.search.text()))
        self.search.hide()
        self.showSearch = QtGui.QShortcut("Ctrl+F", self, activated = lambda: (self.search.show() , self.search.setFocus()))
        self.hideSearch = QtGui.QShortcut("Esc", self, activated = lambda: (self.search.hide(), self.wb.setFocus()))

        self.quit = QtGui.QShortcut("Ctrl+Q", self, activated = self.close)
        self.zoomIn = QtGui.QShortcut("Ctrl++", self, activated = lambda: self.wb.setZoomFactor(self.wb.zoomFactor()+.2))
        self.zoomOut = QtGui.QShortcut("Ctrl+-", self, activated = lambda: self.wb.setZoomFactor(self.wb.zoomFactor()-.2))
        self.zoomOne = QtGui.QShortcut("Ctrl+=", self, activated = lambda: self.wb.setZoomFactor(1))
        self.wb.settings().setAttribute(QtWebKit.QWebSettings.PluginsEnabled, True)

        self.sb.addPermanentWidget(self.search)
        self.sb.addPermanentWidget(self.pbar)
        self.wb.load(url)


if __name__ == "__main__":
    app=QtGui.QApplication(sys.argv)
    if len(sys.argv) > 1:
        url = QtCore.QUrl.fromUserInput(sys.argv[1])
    else:
        url = QtCore.QUrl('http://www.python.org')
    wb=MainWindow(url)
    wb.show()
    sys.exit(app.exec_())

2011-02-22 22:45

The Outhouse and the Mall

Wearing the software engineer's hat: Code is the most trivial and the least important part of a feature.

—Michael Iatrou

Michael tweeted that, I replied, he replied, but what the heck, I think sometimes things can be better explained in more than 140 characters, thus this post [1].

So, why the mall and the outhouse? Because when we talk about software and code and features, we are not all talking about the same thing. Imagine if I told you that bricks are trivial. After all, they have existed in their current form for thousands of years, they are pretty simple to manufacture, and have no interesting features, really, except a certain resistence.

Now, suppose you are building an outhouse. Since you are a funny guy, you want to build an actual brick outhouse so you will use bricks to do it.

Now, since bricks are so boring, you may feel compelled to believe bricks are the least important part of your edifice, and that the overall design is more important. Should you carve a moon-shaped hole in the door? How deep should the latrine be?

However, that position is fatally flawed, since if you ignore those trivial, boring bricks, all you have is shit in a hole in the ground. That is because you are considering the bricks as just a mean to your end. You only care about the bricks insofar as they help you realize your grand outhouse vision. I am here to tell you that you are wrong.

The first way in which you are wrong is in that artificial separation between means and ends. Everyone is familiar with the ethical conundrum about whether the ends justify the means, but that's garbage. That'swhat you say when you try to convince yourself that doing things haphazardly is ok, because what you do is just the means to whatever other thing is the end. Life is not so easily divided into things that matter and things that don't.

Your work, your creation is not just some ideal isolated end towards which you travel across a sea of dirty means, trying to keep your silver armour clean. It's one whole thing. You are creating the means, you are creating your goal, you are responsible for both, and if you use shoddy bricks, your outhouse should shame you.

In the same way, if you do crappy code, your feature is demeaned. It may even work, but you will know it's built out of crap. You will know you will have to fix and maintain that crap for years to come, or, if you are lucky, ruin your karma by dumping it on the head of some poor sucker who follows your steps.

I am pretty much a materialist. If you remove the code, you don't have a feature, or software, you have a concept, maybe an idea, perhaps a design (or maybe not) but certainly not software, just like you don't have a brick outhouse without piling some damn bricks one on top of the other.

I always say, when I see someone calling himself a software engineer, that I am merely a software carpenter. I know my tools, I care about them, I use them as well as I can according to my lights [2] and I try to produce as good a piece of furniture as I can with what I am given.

This tends to produce humble software, but it's software that has one redeeming feature: it knows what it should do, and does it as well as I can make it. For example, I wrote rst2pdf. It's a program that takes some sort of text, and produces PDF files. It does that as well as I could manage. It does nothing else. It works well or not, but it is what it is, it has a purpose, a description and a goal, and I have tried to achieve that goal without embarrasing myself.

My programs are outhouses, made of carefully selected and considered bricks. They are not fancy, but they are what they are and you know it just by looking at them. And if you ever need an outhouse, well, an outhouse is what you should get.

Also, people tend to do weird stuff with them I never expected, but that's just the luck of the analogy.

But why did I mention malls in the title? Because malls are not outhouses. Malls are not done with a goal by themselves beyond making money for its builders. The actual function of a piece of mall is not even known when it's being built. Will this be a McDonalds, or will it be a comic book store? Who knows!

A mall is built quickly with whatever makes sense moneywise, and it should look bland and recognisable, to not scare the herd. It's a building made for pedestrians, but it's intended to confuse them and make the path form A to B as long and meandering as possible. The premises on which its design is based are all askew, corrupted and self-contradicting.

They also give builders a chance to make lots of money. Or to lose lots of money.

Nowadays, we live in an age of mall software. People build startups, get financing, build crappy software and sometimes they hit it big (Twitter, Facebook) or, more likely, fade into obscurity leaving behind nothing at all, except money lost and sad programmers who spent nights coding stuff noone will ever see or use, and not much else.

Far from me saying startups are not a noble or worthy endeavour. They are! It's just that people who work on them should realize that they are not building software. That's why code doesn't look important to them, because they are actually selling eyeballs to advertisers, or collected personal data from their users to whoever buys that, or captive public for game developers, or whatever your business model says (if you have one!).

They are building malls, where the value is not in the building, which is pretty ghastly and useless by itself, but on the people in it, those who rent space in the mall, those who will use the mall, the software, the social network, whatever it is you are building.

Twitter is not software, Facebook is not software. If they were, identi.ca and diaspora would be bigger! What they are is people in one place, like a mall is not a real building, but a collection of people under a roof.

So, there is nothing wrong with building malls. Just remember that your ends and your means are one and a whole, that code is important, that without code Facebook and Twitter don't work, and that without people they are a badland, and know what you are doing.

Because the only hard thing in life is knowing what you want to do. The rest is the easy part. And because malls without toilets suck.

[1] If you really want to see the whole conversation, it's here: http://bettween.com/ralsina/iatrou (if anyone knows a better conversation tracker please post it in a comment).
[2] Yet here I am, an Engineering Manager at Canonical. Sorry guys!

2010-12-06 18:56

Charla: docutils / rst y sus amigos

Again, spanish only because it's a video... in spanish.

Resulta que me olvidé que sí habían grabado mi charla de docutils y compañia. Gracias a Germán por hacerme acordar y mostrarme adonde estaba!

Y ... acá está:

Contents © 2000-2018 Roberto Alsina