Posts about pyqtbyexample

Shipping your PyQt app for windows

I have written about this in the past, with the general conclusion being "it's a pain in the ass".

So, now, here is how it's done.

  1. Start with a working PyQt application. In this example, I will use devicenzo.py mostly because:
    1. It is a working PyQt application.
    2. It uses a big chunk of PyQt
    3. It's easy to test
  2. Now you need a setup.py. Here's one that works, with extensive commments.
# We will be using py2exe to build the binaries.
# You may use other tools, but I know this one.

from distutils.core import setup
import py2exe

# Now you need to pass arguments to setup
# windows is a list of scripts that have their own UI and
# thus don't need to run in a console.

setup(windows=['devicenzo.py'],
      options={

# And now, configure py2exe by passing more options;

          'py2exe': {

# This is magic: if you don't add these, your .exe may
# or may not work on older/newer versions of windows.

              "dll_excludes": [
                  "MSVCP90.dll",
                  "MSWSOCK.dll",
                  "mswsock.dll",
                  "powrprof.dll",
                  ],

# Py2exe will not figure out that you need these on its own.
# You may need one, the other, or both.

              'includes': [
                  'sip',
                  'PyQt4.QtNetwork',
                  ],

# Optional: make one big exe with everything in it, or
# a folder with many things in it. Your choice
#             'bundle_files': 1,
          }
      },

# Qt's dynamically loaded plugins and py2exe really don't
# get along.

data_files = [
            ('phonon_backend', [
                'C:\Python27\Lib\site-packages\PyQt4\plugins\phonon_backend\phonon_ds94.dll'
                ]),
            ('imageplugins', [
            'c:\Python27\lib\site-packages\PyQt4\plugins\imageformats\qgif4.dll',
            'c:\Python27\lib\site-packages\PyQt4\plugins\imageformats\qjpeg4.dll',
            'c:\Python27\lib\site-packages\PyQt4\plugins\imageformats\qsvg4.dll',
            ]),
],

# If you choose the bundle above, you may want to use this, too.
#     zipfile=None,
)
  1. Run python setup.py py2exe and get a dist folder full of binary goodness.

And that's it. Except of course, that's not it.

What this will do is create a binary set, either a folder full of things, or a single EXE file. And that's not enough. You have to consider at least the following:

  1. Put everything in resource files: images, qss files, icons, etc. Every file your app needs? Put it in a resource file and load it from there. That way you don't have to care about them if you go the "one exe" road.
  2. Compile .ui files to .py (same reason)
  3. Figure out if you use Qt's plugins, and make them work. This includes: using Phonon, using QtSQL, and using any image formats other than PNG.

After you have that, are you done? NO!

Your windows user will want an installer. I am not going to go into details, but I had a good time using BitRock's InstallBuilder for Qt. It's a nice tool, and it works. That's a lot in this field.

But is that all? NO!

You have to take care of the Visual Studio Runtime. My suggestion? Get a copy of the 1.1MB vcredist_x86.exe (not the larger one, the 1.1MB one), and either tell people to install it manually, or add it to your installer. You are legally allowed (AFAIK) to redistribute that thing as a whole. But not what's in it (unless you have a VS license).

And we are done? NO!

Once you run your app "installed", if it ever prints anything to stderr, you will get either a dialog telling you it did, or worse (if you are in aything newer than XP), a dialog telling you it can't write to a log file, and the app will never work again.

This is because py2exe catches stderr and tries to save it on a logfile. Which it tries to create in the same folder as the binary. Which is usually not allowed because of permissions.

Solution? Your app should never write to stderr. Write an excepthook and catch that. And then remove stderr or replace it with a log file, or something. Just don't let py2exe do it, because the way py2exe does it is broken.

And is that it?

Well, basically yes. Of course you should get 4 or 5 different versions of windows to test it on, but you are pretty much free to ship your app as you wish. Oh, mind you, don't upload it to downloads.com because they will wrap your installer in a larger one that installs bloatware and crap.

So, there you go.

Making your app modular: Yapsy

That a plugin architecture for a complex app is a good idea is one of those things that most people kinda agree on. One thing we don't quite agree is how the heck are we going to make out app modular?

One way to do it (if you are coding python) is using Yapsy.

Yapsy is awesome. Also, yapsy is a bit underdocumented. Let's see if this post fixes that a bit and leaves just the awesome.

Update: I had not seen the new Yapsy docs, released a few days ago. They are much better than what was there before :-)

Here's the general idea behind yapsy:

  • You create a Plugin Manager that can find and load plugins from a list of places (for example, from ["/usr/share/appname/plugins", "~/.appname/plugins"]).

  • A plugin category is a class.

  • There is a mapping between category names and category classes.

  • A plugin is a module and a metadata file. The module defines a class that inherits from a category class, and belongs to that category.

    The metadata file has stuff like the plugin's name, description, URL, version, etc.

One of the great things about Yapsy is that it doesn't specify too much. A plugin will be just a python object, you can put whatever you want there, or you can narrow it down by specifying the category class.

In fact, the way I have been doing the category classes is:

  • Start with an empty class
  • Implement two plugins of that category
  • If there is a chunk that's much alike in both, move it into the category class.

But trust me, this will all be clearer with an example :-)

I will be doing it with a graphical PyQt app, but Yapsy works just as well for headless of CLI apps.

Let's start with a simple app: an HTML editor with a preview widget.

http://lateral.netmanagers.com.ar/static/yapsy/editor1.jpeg

A simple editor with preview

Here's the code for the app, which is really simple (it doesn't save or do anything, really, it's just an example):

editor1.py

from PyQt4 import QtCore, QtGui, QtWebKit
import os, sys

class Main(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.layout = QtGui.QVBoxLayout()
        self.editor = QtGui.QPlainTextEdit()
        self.preview = QtWebKit.QWebView()
        self.layout.addWidget(self.editor)
        self.layout.addWidget(self.preview)
        self.editor.textChanged.connect(self.updatePreview)
        self.setLayout(self.layout)

    def updatePreview(self):
        self.preview.setHtml(self.editor.toPlainText())

def main():
    # Again, this is boilerplate, it's going to be the same on
    # almost every app you write
    app = QtGui.QApplication(sys.argv)
    window=Main()
    window.show()
    # It's exec_ because exec is a reserved word in Python
    sys.exit(app.exec_())

if __name__ == "__main__":
    main()

Note

From now on listings will not include the main function, because it never changes.

But this application has an obvious limit: you have to type HTML in it. Why not type python code in it and have it convert to HTML for display? Or Wiki markup, or restructured text?

You could, in principle, just implement all those modes, but then you are assuming the responsability of supporting every thing-that-can-be-turned-into-HTML. Your app would be a monolith. That's where yapsy enters the scene.

So, let's create a plugin category, called "Formatter" which takes plain text and returns HTML. Then we add stuff in the UI so the user can choose what formatter he wants, and implement two of those.

Here's our plugin category class:

categories.py

class Formatter(object):
    """Plugins of this class convert plain text to HTML"""

    name = "No Format"

    def formatText(self, text):
        """Takes plain text, returns HTML"""
        return text

Of course what good is a plugin architecture without any plugins for it? So, let's create two plugins.

First: a plugin that takes python code and returns HTML, thanks to pygments.

plugins/pygmentize.py

from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter

from categories import Formatter

class Pygmentizer(Formatter):
    name = "Python Code"

    def formatText(self, text):
        return highlight(text, PythonLexer(), HtmlFormatter(full=True))

See how it goes into a plugins folder? Later on we will tell yapsy to search there for plugins.

To be recognized as a plugin, it needs a metadata file, too:

plugins/pygmentize.yapsy-plugin

[Core]
Name = Python Code
Module = pygmentize

[Documentation]
Author = Roberto Alsina
Version = 0.1
Website = http://lateral.netmanagers.com.ar
Description = Highlights Python Code

And really, that's all there is to making a plugin. Here's another one for comparison, which uses docutils to format reStructured Text:

plugins/rest.py

from categories import Formatter
import docutils.core
import docutils.io


class Rest(Formatter):
    name = "Restructured Text"

    def formatText(self, text):
        output = docutils.core.publish_string(
            text, writer_name = 'html'
        )
        return output

plugins/rest.yapsy-plugin

[Core]
Name = Restructured Text
Module = rest

[Documentation]
Author = Roberto Alsina
Version = 0.1
Website = http://lateral.netmanagers.com.ar
Description = Formats restructured text

And here they are in action:

http://lateral.netmanagers.com.ar/static/yapsy/editor2.jpeg

reSt mode

http://lateral.netmanagers.com.ar/static/yapsy/editor3.jpeg

Python mode

Of course using categories you can do things like a "Tools" category, where the plugins get added to a Tools menu, too.

And here's the application code:

editor2.py

from categories import Formatter
from yapsy.PluginManager import PluginManager

class Main(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.layout = QtGui.QVBoxLayout()
        self.formattersCombo = QtGui.QComboBox()
        self.editor = QtGui.QPlainTextEdit()
        self.preview = QtWebKit.QWebView()

        self.layout.addWidget(self.formattersCombo)
        self.layout.addWidget(self.editor)
        self.layout.addWidget(self.preview)

        self.editor.textChanged.connect(self.updatePreview)
        self.setLayout(self.layout)

        # Create plugin manager
        self.manager = PluginManager(categories_filter={ "Formatters": Formatter})
        self.manager.setPluginPlaces(["plugins"])

        # Load plugins
        self.manager.locatePlugins()
        self.manager.loadPlugins()

        # A do-nothing formatter by default
        self.formattersCombo.addItem("None")
        self.formatters = {}
        print self.manager.getPluginsOfCategory("Formatters")
        for plugin in self.manager.getPluginsOfCategory("Formatters"):
            print  "XXX"
            # plugin.plugin_object is an instance of the plugin
            self.formattersCombo.addItem(plugin.plugin_object.name)
            self.formatters[plugin.plugin_object.name] = plugin.plugin_object

    def updatePreview(self):
        # Check what the current formatter is
        name =  unicode(self.formattersCombo.currentText())
        text = unicode(self.editor.toPlainText())
        if name in self.formatters:
            text = self.formatters[name].formatText(text)
        self.preview.setHtml(text)

In short: this is easy to do, and it leads to fixing your application's internal structure, so it helps you write better code.

Full source code for everything.

I am posting very little because I am writing a lot.

I am just not writing here. I am writing a book instead.

What book am I writing? A book about python programming, of course! It's called "Python No Muerde" (Python Doesn't Bite) and it's in spanish.

Now, I am the first to admin: I am not a great programmer. And I am not a great writer. But I have lots of things to say. If I can organize them correctly, they even make sense sometimes!

So, I am giving this write-long-stuff thing a try.

Of course since I am an open source nerd, I can't do things the usual way, therefore, the book is under Creative Commons. And because I am a programmer, I hacked together a (if I may say so myself) decent structure to handle book-writing.

  1. I write in restructured text
  2. I use rst2pdf to create PDFs both of individual chapters and the whole thing.
  3. I use rest2web to create a website
  4. I use mercurial (at googlecode) to handle revision control and history.
  5. I use make to control rebuilding of chapters when code changes, or images get updated, etc.

Of course it's more complicated than that, the PDFs are in the site, the site is uploaded via rsync, the uploads and rebuilds are triggered by hg push, and so on.

In any case, I may post a few times about how this whole thing works, here is the output of the machinery:

http://nomuerde.netmanagers.com.ar

Translations coming

The spanish translations of most sessions of "PyQt by Example" are almost done, thanks to several volunteers.

Tomorrow I will upload at least one.

If anyone wants to translate them to other languages, I will be happy to include those too.

Session 5 is slightly delayed

I planned to have it ready for today, but my current consulting job (migrating a firm from Exchange to FLOSS) is getting a bit gnarly (mostly because of Outlook 2000) (Yes, I know).

It may be done this saturday, maybe next tuesday, I just don't know yet.