Skip to main content

Ralsina.Me — Roberto Alsina's website

Making your app modular: Yapsy

That a plug­in ar­chi­tec­ture for a com­plex app is a good idea is one of those things that most peo­ple kin­da agree on. One thing we don't quite agree is how the heck are we go­ing to make out app mod­u­lar?

One way to do it (if you are cod­ing python) is us­ing Yap­sy.

Yap­sy is awe­some. Al­so, yap­sy is a bit un­der­doc­u­ment­ed. Let's see if this post fix­es that a bit and leaves just the awe­some.

Up­date: I had not seen the new Yap­sy doc­s, re­leased a few days ago. They are much bet­ter than what was there be­fore :-)

Here's the gen­er­al idea be­hind yap­sy:

  • You cre­ate a Plug­in Man­ag­er that can find and load plug­ins from a list of places (for ex­am­­ple, from ["/us­r/share/ap­p­­name/­­plu­g­in­s", "~/.ap­p­­name/­­plu­g­in­s"]).

  • A plug­in cat­e­­go­ry is a class.

  • There is a map­ping be­tween cat­e­­go­ry names and cat­e­­go­ry class­es.

  • A plug­in is a mod­­ule and a meta­­da­­ta file. The mod­­ule de­fines a class that in­­her­its from a cat­e­­go­ry class, and be­­longs to that cat­e­­go­ry.

    The meta­­da­­ta file has stuff like the plug­in's name, de­scrip­­tion, URL, ver­­sion, etc.

One of the great things about Yap­sy is that it does­n't spec­i­fy too much. A plug­in will be just a python ob­jec­t, you can put what­ev­er you want there, or you can nar­row it down by spec­i­fy­ing the cat­e­go­ry class.

In fac­t, the way I have been do­ing the cat­e­go­ry class­es is:

  • Start with an em­p­­ty class

  • Im­­ple­­ment two plug­ins of that cat­e­­go­ry

  • If there is a chunk that's much alike in both, move it in­­­to the cat­e­­go­ry class.

But trust me, this will all be clear­er with an ex­am­ple :-)

I will be do­ing it with a graph­i­cal PyQt ap­p, but Yap­sy works just as well for head­less of CLI app­s.

Let's start with a sim­ple ap­p: an HTML ed­i­tor with a pre­view wid­get.

//ralsina.me/static/yapsy/editor1.jpeg

A sim­ple ed­i­tor with pre­view

Here's the code for the ap­p, which is re­al­ly sim­ple (it does­n't save or do any­thing, re­al­ly, it's just an ex­am­ple):

ed­i­tor1.py

from PyQt4 import QtCore, QtGui, QtWebKit
import os, sys

class Main(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.layout = QtGui.QVBoxLayout()
        self.editor = QtGui.QPlainTextEdit()
        self.preview = QtWebKit.QWebView()
        self.layout.addWidget(self.editor)
        self.layout.addWidget(self.preview)
        self.editor.textChanged.connect(self.updatePreview)
        self.setLayout(self.layout)

    def updatePreview(self):
        self.preview.setHtml(self.editor.toPlainText())

def main():
    # Again, this is boilerplate, it's going to be the same on
    # almost every app you write
    app = QtGui.QApplication(sys.argv)
    window=Main()
    window.show()
    # It's exec_ because exec is a reserved word in Python
    sys.exit(app.exec_())

if __name__ == "__main__":
    main()

Note

From now on list­ings will not in­clude the main func­tion, be­cause it nev­er changes.

But this ap­pli­ca­tion has an ob­vi­ous lim­it: you have to type HTML in it. Why not type python code in it and have it con­vert to HTML for dis­play? Or Wi­ki markup, or re­struc­tured tex­t?

You could, in prin­ci­ple, just im­ple­ment all those mod­es, but then you are as­sum­ing the re­spon­s­abil­i­ty of sup­port­ing ev­ery thing-that-­can-be-­turned-in­to-HTM­L. Your app would be a mono­lith. That's where yap­sy en­ters the scene.

So, let's cre­ate a plug­in cat­e­go­ry, called "For­mat­ter" which takes plain text and re­turns HTM­L. Then we add stuff in the UI so the us­er can choose what for­mat­ter he wants, and im­ple­ment two of those.

Here's our plug­in cat­e­go­ry class:

cat­e­gories.py

class Formatter(object):
    """Plugins of this class convert plain text to HTML"""

    name = "No Format"

    def formatText(self, text):
        """Takes plain text, returns HTML"""
        return text

Of course what good is a plug­in ar­chi­tec­ture with­out any plug­ins for it? So, let's cre­ate two plug­ins.

First: a plug­in that takes python code and re­turns HTM­L, thanks to pyg­ments.

plu­g­in­s/pyg­men­tize.py

from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter

from categories import Formatter

class Pygmentizer(Formatter):
    name = "Python Code"

    def formatText(self, text):
        return highlight(text, PythonLexer(), HtmlFormatter(full=True))

See how it goes in­to a plug­ins fold­er? Lat­er on we will tell yap­sy to search there for plug­ins.

To be rec­og­nized as a plug­in, it needs a meta­da­ta file, too:

plu­g­in­s/pyg­men­tize.yap­sy-­plu­g­in

[Core]
Name = Python Code
Module = pygmentize

[Documentation]
Author = Roberto Alsina
Version = 0.1
Website = //ralsina.me
Description = Highlights Python Code

And re­al­ly, that's all there is to mak­ing a plug­in. Here's an­oth­er one for com­par­ison, which us­es do­cu­tils to for­mat re­Struc­tured Tex­t:

plu­g­in­s/rest.py

from categories import Formatter
import docutils.core
import docutils.io


class Rest(Formatter):
    name = "Restructured Text"

    def formatText(self, text):
        output = docutils.core.publish_string(
            text, writer_name = 'html'
        )
        return output

plu­g­in­s/rest.yap­sy-­plu­g­in

[Core]
Name = Restructured Text
Module = rest

[Documentation]
Author = Roberto Alsina
Version = 0.1
Website = //ralsina.me
Description = Formats restructured text

And here they are in ac­tion:

//ralsina.me/static/yapsy/editor2.jpeg

reSt mode

//ralsina.me/static/yapsy/editor3.jpeg

Python mode

Of course us­ing cat­e­gories you can do things like a "Tool­s" cat­e­go­ry, where the plug­ins get added to a Tools menu, too.

And here's the ap­pli­ca­tion code:

ed­i­tor2.py

from categories import Formatter
from yapsy.PluginManager import PluginManager

class Main(QtGui.QWidget):
    def __init__(self):
        QtGui.QWidget.__init__(self)
        self.layout = QtGui.QVBoxLayout()
        self.formattersCombo = QtGui.QComboBox()
        self.editor = QtGui.QPlainTextEdit()
        self.preview = QtWebKit.QWebView()

        self.layout.addWidget(self.formattersCombo)
        self.layout.addWidget(self.editor)
        self.layout.addWidget(self.preview)

        self.editor.textChanged.connect(self.updatePreview)
        self.setLayout(self.layout)

        # Create plugin manager
        self.manager = PluginManager(categories_filter={ "Formatters": Formatter})
        self.manager.setPluginPlaces(["plugins"])

        # Load plugins
        self.manager.locatePlugins()
        self.manager.loadPlugins()

        # A do-nothing formatter by default
        self.formattersCombo.addItem("None")
        self.formatters = {}
        print self.manager.getPluginsOfCategory("Formatters")
        for plugin in self.manager.getPluginsOfCategory("Formatters"):
            print  "XXX"
            # plugin.plugin_object is an instance of the plugin
            self.formattersCombo.addItem(plugin.plugin_object.name)
            self.formatters[plugin.plugin_object.name] = plugin.plugin_object

    def updatePreview(self):
        # Check what the current formatter is
        name =  unicode(self.formattersCombo.currentText())
        text = unicode(self.editor.toPlainText())
        if name in self.formatters:
            text = self.formatters[name].formatText(text)
        self.preview.setHtml(text)

In short: this is easy to do, and it leads to fix­ing your ap­pli­ca­tion's in­ter­nal struc­ture, so it helps you write bet­ter code.

Full source code for ev­ery­thing.

srid / 2010-10-01 23:25:

Perhaps worth reusing http://github.com/ActiveSta... for coming up with plugin dir in cross-platform way?

Roberto Alsina / 2010-10-03 20:13:

Yes, usually ~/.appname/plugins and "whereverthe.py_is_installed/plugins" are good enough, but that's probably a better idea.

Thibauld Nion / 2010-10-02 10:38:

Thanks for this great walk-through and presentation of yapsy!

I'm yapsy's maintainer and though I tried to improve the doc recently I reckon this kind of tutorial is really the best explanation of how and why yapsy can be used.

Thanks again and all the best for your own projects !

Roberto Alsina / 2010-10-03 20:14:

Thanks to you for yapsy, it has been a great help in the project where I'm using it :-)

Ernesto Savoretti / 2010-10-02 13:41:

Great!
Just a little bit:
Shouldn't editor2.py contain a line to import formatter, something like "from categories import Formatter"
Without that, line 22 throws, otherwise, it works great.

Roberto Alsina / 2010-10-03 20:15:

Yup, real bug. I'll fix it ASAP.

dan / 2010-11-19 20:18:

I can't seem to get the example to run. When I execute:

python editor2.py

The interface comes up, but the only option in the Formatters widget is None. Also, the line:

print self.manager.getPluginsOfCategory("Formatters")

just prints out "[]"

Could I be doing something wrong? I'm on Ubuntu 10.10 with python version 2.6.6 and yapsy version 1.8

Thanks,
dan

dan / 2010-11-19 20:29:

Nevermind. It turns out that I just didn't have the dependencies installed for the plugins (e.g., pygments), and so I guess the plugin loader was just faiing silently.

Roberto Alsina / 2010-11-19 20:55:

If you change the logging level to DEBUG you will get the errors for the plugins.

Something like

import logging
logging.basicConfig(level=logging.DEBUG)

oscherler / 2011-05-15 10:17:

Thanks a lot, that’s very helpful. I can’t imagine how it was before the “new, much better Yapsy docs” were released. It’s ridiculous, they don’t even have an example that shows what to do twith the plugins you write, just how to load them.

phone number lookup / 2011-12-03 22:27:

this is really interesting viewpoint on the subject i might add

employment background check / 2011-12-27 23:28:

Man ... Beautiful . Amazing ... I will bookmark your website and use the your RSS feed also

n00pX90 / 2012-06-25 07:01:

Hello, Thanks for this tut.

I'm searching for a complete example of using YAPSY for extending component GUI (menu, windows,..) of a PySide/pyQT app.

Do you have any sample explicit for me ?

Regards,