Making your app modular: Yapsy

That a plugin architecture for a complex app is a good idea is one of those things that most people kinda agree on. One thing we don't quite agree is how the heck are we going to make out app modular?

One way to do it (if you are coding python) is using Yapsy.

Yapsy is awesome. Also, yapsy is a bit underdocumented. Let's see if this post fixes that a bit and leaves just the awesome.

Update: I had not seen the new Yapsy docs, released a few days ago. They are much better than what was there before :-)

Here's the general idea behind yapsy:

  • You create a Plugin Manager that can find and load plugins from a list of places (for example, from ["/usr/share/appname/plugins", "~/.appname/plugins"]).

  • A plugin category is a class.

  • There is a mapping between category names and category classes.

  • A plugin is a module and a metadata file. The module defines a class that inherits from a category class, and belongs to that category.

    The metadata file has stuff like the plugin's name, description, URL, version, etc.

One of the great things about Yapsy is that it doesn't specify too much. A plugin will be just a python object, you can put whatever you want there, or you can narrow it down by specifying the category class.

In fact, the way I have been doing the category classes is:

  • Start with an empty class
  • Implement two plugins of that category
  • If there is a chunk that's much alike in both, move it into the category class.

But trust me, this will all be clearer with an example :-)

I will be doing it with a graphical PyQt app, but Yapsy works just as well for headless of CLI apps.

Let's start with a simple app: an HTML editor with a preview widget.

http://lateral.netmanagers.com.ar/static/yapsy/editor1.jpeg

A simple editor with preview

Here's the code for the app, which is really simple (it doesn't save or do anything, really, it's just an example):

editor1.py

from PyQt4 import QtCore, QtGui, QtWebKit
import os, sys

class Main(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.layout = QtGui.QVBoxLayout()
        self.editor = QtGui.QPlainTextEdit()
        self.preview = QtWebKit.QWebView()
        self.layout.addWidget(self.editor)
        self.layout.addWidget(self.preview)
        self.editor.textChanged.connect(self.updatePreview)
        self.setLayout(self.layout)

    def updatePreview(self):
        self.preview.setHtml(self.editor.toPlainText())

def main():
    # Again, this is boilerplate, it's going to be the same on
    # almost every app you write
    app = QtGui.QApplication(sys.argv)
    window=Main()
    window.show()
    # It's exec_ because exec is a reserved word in Python
    sys.exit(app.exec_())

if __name__ == "__main__":
    main()

Note

From now on listings will not include the main function, because it never changes.

But this application has an obvious limit: you have to type HTML in it. Why not type python code in it and have it convert to HTML for display? Or Wiki markup, or restructured text?

You could, in principle, just implement all those modes, but then you are assuming the responsability of supporting every thing-that-can-be-turned-into-HTML. Your app would be a monolith. That's where yapsy enters the scene.

So, let's create a plugin category, called "Formatter" which takes plain text and returns HTML. Then we add stuff in the UI so the user can choose what formatter he wants, and implement two of those.

Here's our plugin category class:

categories.py

class Formatter(object):
    """Plugins of this class convert plain text to HTML"""

    name = "No Format"

    def formatText(self, text):
        """Takes plain text, returns HTML"""
        return text

Of course what good is a plugin architecture without any plugins for it? So, let's create two plugins.

First: a plugin that takes python code and returns HTML, thanks to pygments.

plugins/pygmentize.py

from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter

from categories import Formatter

class Pygmentizer(Formatter):
    name = "Python Code"

    def formatText(self, text):
        return highlight(text, PythonLexer(), HtmlFormatter(full=True))

See how it goes into a plugins folder? Later on we will tell yapsy to search there for plugins.

To be recognized as a plugin, it needs a metadata file, too:

plugins/pygmentize.yapsy-plugin

[Core]
Name = Python Code
Module = pygmentize

[Documentation]
Author = Roberto Alsina
Version = 0.1
Website = http://lateral.netmanagers.com.ar
Description = Highlights Python Code

And really, that's all there is to making a plugin. Here's another one for comparison, which uses docutils to format reStructured Text:

plugins/rest.py

from categories import Formatter
import docutils.core
import docutils.io


class Rest(Formatter):
    name = "Restructured Text"

    def formatText(self, text):
        output = docutils.core.publish_string(
            text, writer_name = 'html'
        )
        return output

plugins/rest.yapsy-plugin

[Core]
Name = Restructured Text
Module = rest

[Documentation]
Author = Roberto Alsina
Version = 0.1
Website = http://lateral.netmanagers.com.ar
Description = Formats restructured text

And here they are in action:

http://lateral.netmanagers.com.ar/static/yapsy/editor2.jpeg

reSt mode

http://lateral.netmanagers.com.ar/static/yapsy/editor3.jpeg

Python mode

Of course using categories you can do things like a "Tools" category, where the plugins get added to a Tools menu, too.

And here's the application code:

editor2.py

from categories import Formatter
from yapsy.PluginManager import PluginManager

class Main(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.layout = QtGui.QVBoxLayout()
        self.formattersCombo = QtGui.QComboBox()
        self.editor = QtGui.QPlainTextEdit()
        self.preview = QtWebKit.QWebView()

        self.layout.addWidget(self.formattersCombo)
        self.layout.addWidget(self.editor)
        self.layout.addWidget(self.preview)

        self.editor.textChanged.connect(self.updatePreview)
        self.setLayout(self.layout)

        # Create plugin manager
        self.manager = PluginManager(categories_filter={ "Formatters": Formatter})
        self.manager.setPluginPlaces(["plugins"])

        # Load plugins
        self.manager.locatePlugins()
        self.manager.loadPlugins()

        # A do-nothing formatter by default
        self.formattersCombo.addItem("None")
        self.formatters = {}
        print self.manager.getPluginsOfCategory("Formatters")
        for plugin in self.manager.getPluginsOfCategory("Formatters"):
            print  "XXX"
            # plugin.plugin_object is an instance of the plugin
            self.formattersCombo.addItem(plugin.plugin_object.name)
            self.formatters[plugin.plugin_object.name] = plugin.plugin_object

    def updatePreview(self):
        # Check what the current formatter is
        name =  unicode(self.formattersCombo.currentText())
        text = unicode(self.editor.toPlainText())
        if name in self.formatters:
            text = self.formatters[name].formatText(text)
        self.preview.setHtml(text)

In short: this is easy to do, and it leads to fixing your application's internal structure, so it helps you write better code.

Full source code for everything.

Comments

Comments powered by Disqus