Posts about python (old posts, page 35)

2012-01-19 18:14

PyQt Quickie: QTimer

QTimer is a fairly simple class: you use it when you want something to happen "in a while" or "every once in a while".

The first case is something like this:

# call f() in 3 seconds
QTimer.singleShot(3000, f)

The second is this:

# Create a QTimer
timer = QTimer()
# Connect it to f
# Call f() every 5 seconds

Simple, right? Well, yes, but it has some tricks.

  1. You have to keep a reference to``timer``

    If you don't, it willget garbage-collected, and f() will never be called.

  2. It may not call f() in 5 seconds.

    It will call f() more or less 5 seconds after you enter the event loop. That may not be quickly after you start the timer at all!

  3. You may get overlapping calls.

    If f() takes long to finish and re-enters the event loop (for example, by calling processEvents) maybe the timer will timeout and call it again before it's finished. That's almost never a good thing.

So, you can do this:

def f():
        # Do things
        QTimer.singleShot(5000, f)


What that snippet does, is, calls f() only once. But f itself schedules itself to run in 5 seconds. Since it does it in a finally, it will do so even if things break.

That means no overlapping calls. It also means it won't be called every 5 seconds, but 5 seconds plus whatever f takes to run. Also, no need to keep any reference to a QTimer.

Final tip: You can also use QTimer to do something "as soon as you are in the event loop"

QTimer.singleShot(0, f)

Hope it was useful!

2012-01-08 23:21

Abandonment issues: rst2pdf

Of all the corpses of my projects, there is one thatI feel worse about, which is rst2pdf. I feel bad about abandnoning several, but rst2pdf was actually a useful tool, used by a bunch of people, and that it has never gathered enough momentum with other developers is sad.

So, I will pick it up. I will spend about 4 hours a week on it. The plan is to:

  1. Gather some patches that are lingering on the issue tracker
  2. Fix some simple-ish bugs
  3. Make another release with 1) and 2)

And of course:

  1. Not let it fall in disrepair again

In the meantime, here is a nice thing I just heard about. Dimitri Christodoulou has hacked rst2pdf so that it can handle the raw:: html directive.

This, dear friends is completely nuts, absolutely out of scope for any given docutils tool, and just too cool :-)

I will try to hijack his code (proper credit and so on), and incorporate it into rst2pdf.

And Dimitri, or anyone else who wants to do cool stuff with rst2pdf: let me know! I will give you commit rights immediately!

2012-01-07 23:06

Python context managers: they are easy!

This comes from this thread in the Python Argentina mailing list (which I strongly recommend if you read spanish).

I was the other day trying to do shell-script-like-things on python (as part of a monster and I was annoyed that in shell it's easy to do this:

cd foo
bar -baz
cd -

Or this:

pushd foo
bar -baz

Or this:

(cd foo && bar -baz)

And on Python I had to do this, which is verbose and ugly:

cwd = os.getcwd()
    os.system('bar -baz')

This is what I wanted to have:

with os.chdir('foo'):
    os.system('bar -baz')

And of course, you can't do that. So, I asked, how do you implement that context manager? I got several answers.

  1. That's available in Fabric:

    with cd("foo"):
  2. It's not hard to do:

    class DirContextM(object):
        def __init__(self, new_dir):
            self.new_dir = new_dir
            self.old_dir = None
        def __enter__(self):
            self.old_dir = os.getcwd()
        def __exit__(self, *_):
  3. It's even easier to do:

    from contextlib import contextmanager
    def cd(path):
        old_dir = os.getcwd()
  4. That's cool, so let's add it to

  5. Maybe check for exceptions

    def cd(path):
        old_dir = os.getcwd()

All in all, I learned how to do context managers, about contextlib, about fabric and about Which is not bad for 15 minutes :-)

2011-07-24 21:20

Shipping your PyQt app for windows

I have written about this in the past, with the general conclusion being "it's a pain in the ass".

So, now, here is how it's done.

  1. Start with a working PyQt application. In this example, I will use mostly because:
    1. It is a working PyQt application.
    2. It uses a big chunk of PyQt
    3. It's easy to test
  2. Now you need a Here's one that works, with extensive commments.
# We will be using py2exe to build the binaries.
# You may use other tools, but I know this one.

from distutils.core import setup
import py2exe

# Now you need to pass arguments to setup
# windows is a list of scripts that have their own UI and
# thus don't need to run in a console.


# And now, configure py2exe by passing more options;

          'py2exe': {

# This is magic: if you don't add these, your .exe may
# or may not work on older/newer versions of windows.

              "dll_excludes": [

# Py2exe will not figure out that you need these on its own.
# You may need one, the other, or both.

              'includes': [

# Optional: make one big exe with everything in it, or
# a folder with many things in it. Your choice
#             'bundle_files': 1,

# Qt's dynamically loaded plugins and py2exe really don't
# get along.

data_files = [
            ('phonon_backend', [
            ('imageplugins', [

# If you choose the bundle above, you may want to use this, too.
#     zipfile=None,
  1. Run python py2exe and get a dist folder full of binary goodness.

And that's it. Except of course, that's not it.

What this will do is create a binary set, either a folder full of things, or a single EXE file. And that's not enough. You have to consider at least the following:

  1. Put everything in resource files: images, qss files, icons, etc. Every file your app needs? Put it in a resource file and load it from there. That way you don't have to care about them if you go the "one exe" road.
  2. Compile .ui files to .py (same reason)
  3. Figure out if you use Qt's plugins, and make them work. This includes: using Phonon, using QtSQL, and using any image formats other than PNG.

After you have that, are you done? NO!

Your windows user will want an installer. I am not going to go into details, but I had a good time using BitRock's InstallBuilder for Qt. It's a nice tool, and it works. That's a lot in this field.

But is that all? NO!

You have to take care of the Visual Studio Runtime. My suggestion? Get a copy of the 1.1MB vcredist_x86.exe (not the larger one, the 1.1MB one), and either tell people to install it manually, or add it to your installer. You are legally allowed (AFAIK) to redistribute that thing as a whole. But not what's in it (unless you have a VS license).

And we are done? NO!

Once you run your app "installed", if it ever prints anything to stderr, you will get either a dialog telling you it did, or worse (if you are in aything newer than XP), a dialog telling you it can't write to a log file, and the app will never work again.

This is because py2exe catches stderr and tries to save it on a logfile. Which it tries to create in the same folder as the binary. Which is usually not allowed because of permissions.

Solution? Your app should never write to stderr. Write an excepthook and catch that. And then remove stderr or replace it with a log file, or something. Just don't let py2exe do it, because the way py2exe does it is broken.

And is that it?

Well, basically yes. Of course you should get 4 or 5 different versions of windows to test it on, but you are pretty much free to ship your app as you wish. Oh, mind you, don't upload it to because they will wrap your installer in a larger one that installs bloatware and crap.

So, there you go.

2011-03-25 02:30

Creating a forum the easy way (32 LOC)

This is only the first part of a project to create the simplest (for me) software forum possible.

Here are the features I want:

  • Login using twitter / Facebook / Google / OpenID
  • Unlimited number of threads
  • Support for like / dislike both on threads and on posts
  • Avatars
  • HTML in comments
  • Mail the user on replies
  • RSS feeds for threads

You can see it in action at (for a limited time only ;-)

And here is the code:

import bottle
import disqusapi as disqus
import json
shortname = 'magicmisteryforum'
api = disqus.DisqusAPI(open("key").read().strip())

@bottle.route('/', method='GET')
def index():
    msg = bottle.request.GET.get('msg', '')
    threads = api.forums.listThreads(forum=shortname, limit=100)
    print threads[0]
    return bottle.template('main.tpl', threads=threads, shortname=shortname, msg=msg)

@bottle.route('/new', method='POST')
def new():
    title = bottle.request.forms.get('title', None)
    if not title:
    thread = api.threads.create(forum=shortname, title = title)
    thread_id = thread.__dict__['response']['id']
    # Redirecting to /thread/thread_id doesn't work
    # because threads take a few seconds to appear on the listing

def thread(id):
    t = api.threads.details(thread=id)
    return bottle.template('thread.tpl', shortname=shortname, id=id, thread=t.__dict__['response'])

def server_static(path):
    return bottle.static_file(path, root='./static')

app =
app.catchall = False #Now most exceptions are re-raised within bottle.'', port=80, app=app)

It requires Bottle and the Disqus python API

Of course, there is also a bit of templating involved, here is main.tpl and the thread.tpl. Since I suck at HTML, it uses Bluetrip CSS and it's more than simple enough to customize.


This thing is just a simple veneer around Disqus! More like a blog with comments and without posts than a forum! But ... what's missing to make this a real forum? It works, doesn't it? You could even use Disqus categories to create subforums...

All things considered, I think it's a cute hack.

And if you wait a few days, this will lead to something much more magical!

Full source code at

2011-03-11 01:47

New golfing challenge: PatoCabrera

In the spirit of the De Vicenzo web browser, I am starting a new program, called Pato Cabrera. Here are the rules:

  • Twitter client (no in the first version, but to be added later)
  • Has these features:
  • Has to be implemented before April 4th
  • Smaller than 16384 bytes (of python code) but may be larger because of artwork.

Let's see how it works :-)

2011-03-08 21:45

OK, so THAT is how much browser I can put in 128 lines of code.

I have already posted a couple of times (1, 2) about De Vicenzo , an attempt to implement the rest of the browser, starting with PyQt's WebKit... limiting myself to 128 lines of code.

Of course I could do more, but I have my standards!

  • No using ;
  • No if whatever: f()

Other than that, I did a lot of dirty tricks, but right now, it's a fairly complete browser, and it has 127 lines of code (according to sloccount) so that's enough playing and it's time to go back to real work.

But first, let's consider how some features were implemented (I'll wrap the lines so they page stays reasonably narrow), and also look at the "normal" versions of the same (the "normal" code is not tested, please tell me if it's broken ;-).

This is not something you should learn how to do. In fact, this is almost a treatise on how not to do things. This is some of the least pythonic, less clear code you will see this week.

It is short, and it is expressive. But it is ugly.

I'll discuss this version.

Proxy Support

A browser is not much of a browser if you can't use it without a proxy, but luckily Qt's network stack has good proxy support. The trick was configuring it.

De Vicenzo supports HTTP and SOCKS proxies by parsing a http_proxy environment variable and setting Qt's application-wide proxy:

 proxy_url = QtCore.QUrl(os.environ.get('http_proxy', ''))
 QtNetwork.QNetworkProxy.HttpProxy if unicode(proxy_url.scheme()).startswith('http')\
 else QtNetwork.QNetworkProxy.Socks5Proxy,,\
 proxy_url.port(), proxy_url.userName(), proxy_url.password())) if\
'http_proxy' in os.environ else None

How would that look in normal code?

if 'http_proxy' in os.environ:
    proxy_url = QtCore.QUrl(os.environ['http_proxy'])
    if unicode(proxy_url.scheme()).starstswith('http'):
        protocol = QtNetwork.QNetworkProxy.HttpProxy
        protocol = QtNetwork.QNetworkProxy.Socks5Proxy

As you can see, the main abuses against python here are the use of the ternary operator as a one-line if (and nesting it), and line length.

Persistent Cookies

You really need this, since you want to stay logged into your sites between sessions. For this, first I needed to write some persistence mechanism, and then save/restore the cookies there.

Here's how the persistence is done (settings is a global QSettings instance):

def put(self, key, value):
    "Persist an object somewhere under a given key"
    settings.setValue(key, json.dumps(value))

def get(self, key, default=None):
    "Get the object stored under 'key' in persistent storage, or the default value"
    v = settings.value(key)
    return json.loads(unicode(v.toString())) if v.isValid() else default

It's not terribly weird code, except for the use of the ternary operator in the last line. The use of json ensures that as long as reasonable things are persisted, you will get them with the same type as you put them without needing to convert them or call special methods.

So, how do you save/restore the cookies? First, you need to access the cookie jar. I couldn't find whether there is a global one, or a per-webview one, so I created a QNetworkCookieJar in line 24 and assign it to each web page in line 107.

# Save the cookies, in the window's closeEvent
self.put("cookiejar", [str(c.toRawForm()) for c in self.cookies.allCookies()])

# Restore the cookies, in the window's __init__
for c in self.get("cookiejar", [])])

Here I confess I am guilty of using list comprehensions when a for loop would have been the correct thing.

I use the same trick when restoring the open tabs, with the added misfeature of using a list comprehension and throwing away the result:

# get("tabs") is a list of URLs
[self.addTab(QtCore.QUrl(u)) for u in self.get("tabs", [])]

Using Properties and Signals in Object Creation

This is a feature of recent PyQt versions: if you pass property names as keyword arguments when you create an object, they are assigned the value. If you pass a signal as a keyword argument, they are connected to the given value.

This is a really great feature that helps you create clear, local code, and it's a great thing to have. But if you are writing evil code... well, you can go to hell on a handbasket using it.

This is all over the place in De Vicenzo, and here's one example (yes, this is one line):

QtWebKit.QWebView.__init__(self, loadProgress=lambda v:\
(, self.pbar.setValue(v)) if self.amCurrent() else\
None, loadFinished=self.pbar.hide, loadStarted=lambda:\ if self.amCurrent() else None, titleChanged=lambda\
t: container.tabs.setTabText(container.tabs.indexOf(self), t) or\
(container.setWindowTitle(t) if self.amCurrent() else None))

Oh, boy, where do I start with this one.

There are lambda expressions used to define the callbacks in-place instead of just connecting to a real function or method.

There are lambdas that contain the ternary operator:

loadStarted=lambda:\ if self.amCurrent() else None

There are lambdas that use or or a tuple to trick python into doing two things in a single lambda!

loadProgress=lambda v:\
(, self.pbar.setValue(v)) if self.amCurrent() else\

I won't even try to untangle this for educational purposes, but let's just say that line contains what should be replaced by 3 methods, and should be spread over 6 lines or more.

Download Manager

Ok, calling it a manager is overreaching, since you can't stop them once they start, but hey, it lets you download things and keep on browsing, and reports the progress!

First, on line 16 I created a bars dictionary for general bookkeeping of the downloads.

Then, I needed to delegate the unsupported content to the right method, and that's done in lines 108 and 109

What that does is basically that whenever you click on something WebKit can't handle, the method fetch will be called and passed the network request.

def fetch(self, reply):
    destination = QtGui.QFileDialog.getSaveFileName(self, \
        "Save File", os.path.expanduser(os.path.join('~',\
    if destination:
        bar = QtGui.QProgressBar(format='%p% - ' +
        self.bars[unicode(reply.url().toString())] = [bar, reply,\

No real code golfing here, except for long lines, but once you break them reasonably, this is pretty much the obvious way to do it:

  • Ask for a filename
  • Create a progressbar, put it in the statusbar, and connect it to the download's progress signals.

Then, of course, we need ths progress slot, that updates the progressbar:

progress = lambda self, received, total:\
    .setValue(100. * received / total)

Yes, I defined a method as a lambda to save 1 line. [facepalm]

And the finished slot for when the download is done:

def finished(self):
    reply = self.sender()
    url = unicode(reply.url().toString())
    bar, _, fname = self.bars[url]
    redirURL = unicode(reply.attribute(QtNetwork.QNetworkRequest.\
    del self.bars[url]
    if redirURL and redirURL != url:
        return self.fetch(redirURL, fname)
    with open(fname, 'wb') as f:

Notice that it even handles redirections sanely! Beyond that, it just hides the progress bar, saves the data, end of story. The longest line is not even my fault!

There is a big inefficiency in that the whole file is kept in memory until the end. If you download a DVD image, that's gonna sting.

Also, using with saves a line and doesn't leak a file handle, compared to the alternatives.


Again Qt saved me, because doing this manually would have been a pain. However, it turns out that printing is just ... there? Qt, specially when used via PyQt is such an awesomely rich environment.

self.previewer = QtGui.QPrintPreviewDialog(\
self.do_print = QtGui.QShortcut("Ctrl+p",\
    self, activated=self.previewer.exec_)

There's not even any need to golf here, that's exactly as much code as you need to hook Ctrl+p to make a QWebView print.

Other Tricks

There are no other tricks. All that's left is creating widgets, connecting things to one another, and enjoying the awesome experience of programming PyQt, where you can write a whole web browser (except the engine) in 127 lines of code.

2011-03-05 23:46

De Vicenzo: A much cooler mini web browser.

It seems it was only a few days ago that I started this project. Oh, wait, yes, it was just a few days ago!

If you don't want to read that again, the idea is to see just how much code is needed to turn Qt's WebKit engine into a fully-fledged browser.

To do that, I set myself a completely arbitrary limit: 128 lines of code.

So, as of now, I declare it feature-complete.

The new features are:

  • Tabbed browsing (you can add/remove tabs)
  • Bookmarks (you can add/remove them, and choose them from a drop-down menu)

This is what already worked:

  • Zoom in (Ctrl++)
  • Zoom out (Ctrl+-)
  • Reset Zoom (Ctrl+=)
  • Find (Ctrl+F)
  • Hide find (Esc)
  • Buttons for back/forward and reload
  • URL entry that matches the page + autocomplete from history + smart entry (adds http://, that kind of thing)
  • Plugins support (including flash)
  • The window title shows the page title (without browser advertising ;-)
  • Progress bar for page loading
  • Statusbar that shows hovered links URL
  • Takes a URL on the command line, or opens
  • Multiplatform (works in any place QtWebKit works)

So... how much code was needed for this? 87 LINES OF CODE

Or if you want the PEP8-compliant version, 115 LINES OF CODE.

Before anyone says it: yes, I know the rendering engine and the toolkit are huge. What I wrote is just the chrome around them, just like Arora, Rekonq, Galeon, Epiphany and a bunch of others do.

It's simple, minimalistic chrome, but it works pretty good, IMVHO.

Here it is in (buggy) action:

It's more or less feature-complete for what I expected to be achievable, but it still needs some fixes.

You can see the code at it's own home page:

2011-02-28 21:10

How much web browser can you put in 128 lines of code?

UPDATE: If you read this and all you can say is "oh, he's just embedding WebKit", I have two things to tell you:

  1. Duh! Of course the 128 lines don't include the rendering engine, or the TCP implementation, or the GUI toolkit. This is about the rest of the browser, the part around the web rendering engine. You know, just like Arora, Rekonq, Epiphany, and everyone else that embeds webkit or mozilla does it? If you didn't get that before this explanation... facepalm.
  2. Get your favourite webkit fork and try to do this much with the same amount of code. I dare you! I double dog dare you!

Now back to the original article

Today, because of a IRC chat, I tried to find a 42-line web browser I had written a while ago. Sadly, the pastebin where I posted it was dead, so I learned a lesson: It's not a good idea to trust a pastebin as code repository

What I liked about that 42-line browser was that it was not the typical example, where someone dumps a Webkit view in a window, loads a page and tries to convince you he's cool. That one is only 7 lines of code:

import sys
from PyQt4 import QtGui,QtCore,QtWebKit

And if I wanted to make the code uglier, it could be done in 6.

But anyway, that 42-line browser actually looked useful!

This 42-line web browser, courtesy of #python and #qt -- http... on Twitpic

Those buttons you see actually worked correctly, enabling and disabling at the right moment, the URL entry changed when you clicked on links, and some other bits.

So, I have decided to start a small, intermittent project of code golf: put as much browser as I can in 128 lines of code (not counting comments or blanks), starting with PyQt4.

This has a useful purpose: I always suspected that if you assumed PyQt was part of the base system, most apps would fit in floppies again. This one fits on a 1.44MB floppy some 500 times (so you could use 360KB commodore floppies if you prefer!).

So far, I am at about 50 lines, and it has the following features:

  • Zoom in (Ctrl++)
  • Zoom out (Ctrl+-)
  • Reset Zoom (Ctrl+=)
  • Find (Ctrl+F)
  • Hide find (Esc)
  • Buttons for back/forward and reload
  • URL entry that matches the page + autocomplete from history + smart entry (adds http://, that kind of thing)
  • Plugins support (including flash)
  • The window title shows the page title (without browser advertising ;-)
  • Progress bar for page loading
  • Statusbar that shows hovered links URL
  • Takes a URL on the command line, or opens
  • Multiplatform (works in any place QtWebKit works)

Missing are tabs and proxy support. I expect those will take another 40 lines or so, but I think it's probably the most featureful of these toy browsers.

The code... it's not all that hard. I am using lambda a lot, and I am using PyQt's keyword arguments for signal connection which makes lines long, but not hard. It could be made much smaller!

Here it is in action:

And here's the code:

#!/usr/bin/env python
"A web browser that will never exceed 128 lines of code. (not counting blanks)"

import sys
from PyQt4 import QtGui,QtCore,QtWebKit

class MainWindow(QtGui.QMainWindow):
    def __init__(self, url):

        self.pbar = QtGui.QProgressBar()
        self.wb=QtWebKit.QWebView(loadProgress = self.pbar.setValue, loadFinished = self.pbar.hide, loadStarted =, titleChanged = self.setWindowTitle)

        self.tb=self.addToolBar("Main Toolbar")
        for a in (QtWebKit.QWebPage.Back, QtWebKit.QWebPage.Forward, QtWebKit.QWebPage.Reload):

        self.url = QtGui.QLineEdit(returnPressed = lambda:self.wb.setUrl(QtCore.QUrl.fromUserInput(self.url.text())))

        self.wb.urlChanged.connect(lambda u: self.url.setText(u.toString()))
        self.wb.urlChanged.connect(lambda: self.url.setCompleter(QtGui.QCompleter(QtCore.QStringList([QtCore.QString(i.url().toString()) for i in self.wb.history().items()]), caseSensitivity = QtCore.Qt.CaseInsensitive)))

        self.wb.statusBarMessage.connect( l:, 3000)) = QtGui.QLineEdit(returnPressed = lambda: self.wb.findText(
        self.showSearch = QtGui.QShortcut("Ctrl+F", self, activated = lambda: ( ,
        self.hideSearch = QtGui.QShortcut("Esc", self, activated = lambda: (, self.wb.setFocus()))

        self.quit = QtGui.QShortcut("Ctrl+Q", self, activated = self.close)
        self.zoomIn = QtGui.QShortcut("Ctrl++", self, activated = lambda: self.wb.setZoomFactor(self.wb.zoomFactor()+.2))
        self.zoomOut = QtGui.QShortcut("Ctrl+-", self, activated = lambda: self.wb.setZoomFactor(self.wb.zoomFactor()-.2))
        self.zoomOne = QtGui.QShortcut("Ctrl+=", self, activated = lambda: self.wb.setZoomFactor(1))
        self.wb.settings().setAttribute(QtWebKit.QWebSettings.PluginsEnabled, True)

if __name__ == "__main__":
    if len(sys.argv) > 1:
        url = QtCore.QUrl.fromUserInput(sys.argv[1])
        url = QtCore.QUrl('')

2010-12-06 18:56

Charla: docutils / rst y sus amigos

Again, spanish only because it's a video... in spanish.

Resulta que me olvidé que sí habían grabado mi charla de docutils y compañia. Gracias a Germán por hacerme acordar y mostrarme adonde estaba!

Y ... acá está:

Contents © 2000-2018 Roberto Alsina