Skip to main content

Ralsina.Me — Roberto Alsina's website

Posts about open source (old posts, page 6)

The aha! moment

I had a small task to­day in Mar­ave. The goal was:

  1. Fade in a wid­get

  2. Set a var­i­able

  3. Fade in an­oth­er one

It's im­por­tant that things are done in that or­der and it's al­so im­por­tant that the app re­mains in­ter­ac­tive.

And here's the code to do that (sim­pli­fied):

def fadein(thing, target=1., thendo=None):
    """
    * thing is a QWidget
    * thing.proxy is a QGraphicsWidget
    * thendo is callable
    * target is the desired opacity
    """

    thing.anim=QtCore.QPropertyAnimation(thing.proxy, "opacity")
    thing.anim.setDuration(200)
    thing.anim.setStartValue(thing.proxy.opacity())
    thing.anim.setEndValue(target)
    thing.anim.start()
    thing.anim.finished.connect(thing.anim.deleteLater)
    if thendo:
        thing.anim.finished.connect(thendo)

And this is how you use it:

def later():
    avar=avalue
    fadein(widget2)

fadein(widget1, thendo=later)

Isn't that lovely? Having functions as first class objects means I can just take later as a closure, along with widget2 and avar, which need only be defined in the local scope, and the call chain will work just as I wanted!

Yes, many oth­er lan­guages can do this, and in Javascript it's one of the most com­mon trick­s, but con­sid­er that PyQt is a wrap­per for a C++ li­brary!

I think this kind of us­age shows the re­al added val­ue PyQt brings to the table, it's not just that python avoids the bor­ing com­piles, or that you have the awe­some stan­dard li­brary to use, but that the lan­guage it­self en­ables you to do things that are not prac­ti­cal in C++.

In C++ the only way I can think of is creating a slot that's the equivalent of later, then chaining the signals... which means that this throwaway later becomes part of the interface of a class!

I would have to define later somewhere else on the file, separate from its only usage (maybe even inlined in the header file).

Even then, that's not equivalent: avalue may be something that was only avalable before the first call to fadein, (for example, the time of the first fadein): I would have to create a place to store it, make it reachable by later... and wht happens if you try to do this again while the first fadein is in progress?... it gets hairy.

Pro­gram­ming is like a slap in the face some­times... you re­al­ize that things you use with­out even notic­ing are far from triv­ial.

So, re­mem­ber young padawan: you can choose you tool­s. Choose wise­ly.

Extending Marave

Mar­ave is a text ed­i­tor. If there's one thing that's true of most text ed­i­tors, it's this: they lack the ex­act fea­tures you need.

So, the so­lu­tion, in the an­cient tra­di­tion of Emacs and Vim is... make it ex­ten­si­ble.

I am a big fan of pro­grams that can be ex­tend­ed by users.

So... here's the anato­my of a Mar­ave plug­in as it stands right now on SVN trunk, which of course can change any minute.

Creating a plugin

You just need to cre­ate a .py file in the plug­ins fold­er.

Here's the most ba­sic plug­in, which does noth­ing:

# -*- coding: utf-8 -*-

from plugins import Plugin
class Smarty(Plugin):
    name='smarty'
    shortcut='Ctrl+.'
    description='Smart quote and dash replacement'
    mode="qBde"

De­fault val­ues for any­thing con­fig­urable (in this case, "mod­e") is just added to the class.

The manda­to­ry field­s:

  • short­­­cut: a key­board short­­­cut that trig­gers this plug­in

  • name: a short name

  • de­scrip­­tion: a one-­­line de­scrip­­tion of what it does

What does it do? It adds the plug­in to the plug­in list in the prefs di­alog, and you can open its con­fig­u­ra­tion di­alog, where you can change the short­cut:

maraveplugin1

If you en­able this plug­in, when­ev­er the short­cut is used the "run" method of the plug­in is called.

Making the Plugin Configurable

This plug­in sup­ports dif­fer­ent modes of op­er­a­tion. To make this reach­able to the user, you need to im­ple­ment a few ex­tra meth­od­s.

The ad­d­Con­fig­Wid­gets method takes a di­a­log ar­gu­ment and adds what­ev­er you want there:

@classmethod
def addConfigWidgets(self, dialog):
    print 'Adding widgets to smarty config'
    l=dialog.ui.layout
    self.q=QtGui.QCheckBox(dialog.tr('Replace normal quotes'))
    if 'q' in self.mode:
        self.q.setChecked(True)
    self.b=QtGui.QCheckBox(dialog.tr('Replace backtick-style quotes (` and ``)'))
    if 'B' in self.mode:
        self.b.setChecked(True)
    self.d=QtGui.QCheckBox(dialog.tr('Replace -- by en-dash, --- by em-dash'))
    if 'd' in self.mode:
        self.d.setChecked(True)
    self.e=QtGui.QCheckBox(dialog.tr('Replace ellipses'))
    if 'e' in self.mode:
        self.e.setChecked(True)
    l.addWidget(self.q)
    l.addWidget(self.b)
    l.addWidget(self.d)
    l.addWidget(self.e)

And then the con­fig di­a­log will look like this:

maraveplugin2

But then you need to save those op­tions some­where, which you do reim­ple­ment­ing save­Con­fig:

@classmethod
def saveConfig(self, dialog):

    self.shortcut=unicode(dialog.ui.shortcut.text())
    self.settings.setValue('plugin-'+self.name+'-shortcut', self.shortcut)

    newmode=""
    if self.q.isChecked():
        newmode+='q'
    if self.b.isChecked():
        newmode+='B'
    if self.d.isChecked():
        newmode+='d'
    if self.e.isChecked():
        newmode+='e'
    self.mode=newmode

    self.settings.setValue('plugin-smarty-mode',self.mode)
    self.settings.sync()

And you need to load those set­tings and put them in your class, too:

@classmethod
def loadConfig(self):
    print 'SMARTY loadconfig', self.settings
    if self.settings:
        sc=self.settings.value('plugin-'+self.name+'-shortcut')
        if sc.isValid():
            self.shortcut=unicode(sc.toString())
        mode=self.settings.value('plugin-smarty-mode')
        if mode.isValid():
            self.mode=unicode(mode.toString())

Making it Work

And yes, you need to make it do some­thing use­ful. The plug­in has ac­cess to a "clien­t" which is Mar­ave's main win­dow. Ev­ery­thing is avail­able there, some­where ;-)

def run(self):
    print 'running smarty plugin'
    text=unicode(self.client.editor.toPlainText()).splitlines()
    prog=QtGui.QProgressDialog(self.client.tr("Applying smarty"),
                               self.client.tr("Cancel"),
                               0,len(text),
                               self.client)
    prog.show()
    output=[]
    for i,l in enumerate(text):
        output.append(unescape(smartyPants(l,self.mode)))
        prog.setValue(i)
        QtGui.QApplication.instance().processEvents()
    prog.hide()
    self.client.editor.setPlainText('\n'.join(output))

And there it is, if you en­able the smar­ty plug­in, you can "fix" your quotes, dash­es and el­lip­sis with a key com­bi­na­tion :-)

Full source code here: http://­code.­google.­com/p/­mar­ave/­source/browse/trunk­/­mar­ave/­plu­g­in­s/s­mar­ty.py

Still to be done: oth­er ways to in­te­grate plug­ins in­to the UI, but­ton­s, pan­el­s, etc.

Yak Shavings for February 16, 2010

yak shaving

(id­iomat­ic) Any ap­par­ent­ly use­less ac­tiv­i­ty which, by al­low­ing you to over­come in­ter­me­di­ate dif­fi­cul­ties, al­lows you to solve a larg­er prob­lem.

A while ago, I wrote how I im­ple­ment­ed a gener­ic syn­tax high­lighter for PyQt us­ing Pyg­ments.

I got a re­quest for such a fea­ture in Mar­ave, so I digged that code and... it's freak­ing use­less. It's just too slow for rea­son­able use.

So, that yak's hair is all grown up again, and I just got this new pair of scis­sors!

The goal is a way to high­light syn­tax in a QPlain­TextE­d­it that:

  • Does­n't re­quire pro­­gram­ming to add a new high­­­lighter

  • Does­n't re­quire pro­­gram­ming to add a new col­or scheme

  • Does­n't re­quire me to spend a year writ­ing high­­­lighters for ex­ist­ing lan­guages

  • Is fast enough

A quick google shows that for C++ you can use Source high­light qt which is based on GNU source high­light.

Alas, no python bind­ing that I could find. So, let's write one!

Here it is: http://­mar­ave.­google­code.­com/svn/trunk­/­mar­ave/high­light/

And here's a screen­shot of the de­mo pro­gram run­ning, show­ing it­self in its en­tire­ty:

You can cre­ate a col­or scheme us­ing CSS, a lan­guage def­i­ni­tion is a text file, there are a bazil­lion al­ready writ­ten, it seems to be fast enough.

So, an­oth­er yak shaved, an­oth­er fea­ture (not fin­ished!) for Mar­ave

Marave 0.5 is out!

Just up­load­ed Mar­ave 0.5 to the usu­al place. Mar­ave is a re­laxed, fullscreen text ed­i­tor that tries not to dis­tract you.

It even in­cludes a sim­ple mu­sic play­er just so you don't have to think about switch­ing to an­oth­er ap­pli­ca­tion!

This re­lease has sev­er­al bugs fixed, and looks a bit nicer.

The main new fea­ture is ... in­ter­na­tion­al­iza­cion. It now in­cludes a span­ish trans­la­tion on­ly, but if you want to help trans­lat­ing it to any oth­er lan­guage, please step for­ward!

There are on­ly 60 phras­es to trans­late, so it should­n't be more than one hour of work.

Here's a screen­shot of this re­lease:

marave7

Mar­ave is free soft­ware un­der the GPLv2, and should work on any plat­form where PyQt work­s, which means Win­dows, Mac, and Unix-­like op­er­at­ing sys­tem­s, at least.

Packaging and shipping is HARD

I have worked re­al­ly hard on Mar­ave, a full screen ed­i­tor in the style of ommwriter, Dark­Room, Write­Room, py­Room, etc. I have worked very hard and I want users to use it.

Or not even that, I want them to have a chance of us­ing it.

That means I want it to work on Win­dows (and maybe OSX some day, too, if some­one helps me). Which mean­s, I have to pak­age it for win­dows.

Let's do a quick com­par­i­son here from the points of view of the us­er and the de­vel­op­er.

The User, In Linux

This is in Arch Lin­ux, which is what I use, in oth­er Lin­ux vari­ants it will be pret­ty much the same once Mar­ave is a bit more well known.

yaourt -S marave-svn --noconfirm

That gets the code from SVN (right now it's the best way, lat­er I will pack­age re­leas­es, too), all re­quired de­pen­den­cies, builds and in­stall­s. It takes all of 15 sec­onds in my note­book.

Af­ter that, you have a ful­ly work­ing Mar­ave.

In case it's not pack­aged for your dis­tro, just in­stall PyQt (which sure­ly is) and run this com­mand:

easy_install marave

The User, in Windows

You go to http://­mar­ave.­google­code.­com, click on "Mar­ave-0.5.win32.ex­e" (Not linked yet, it's not fin­ished), then down­load a 10MB pro­gram. That is a 10MB pro­gram be­cause win­dows does­n't be­lieve in pack­ages and de­pen­den­cies. On Lin­ux, a Mar­ave pack­age could be un­der 1MB (most of it im­ages), and not be ex­e­cutable, just da­ta.

Of course nowa­days web browsers don't ac­tu­al­ly run pro­grams on down­load, so... let's see it as a gallery!

110111105613-My-Desktop

Yes, save it.

11011111220-My-Desktop

Dou­ble click to open it

11011111417-My-Desktop

Yes, I agree

11011111514-My-Desktop

Sure, what­ev­er

1101111167-My-Desktop

Nice...

11011111750-My-Desktop

Good to hear!

Now, this Mar­ave that just got in­stalled may or may not cur­rent­ly work be­cause of a miss­ing MSVCR90.DLL but that's for the next sec­tion...

The Developer, in Linux

First, here's the big­gest prob­lem a Lin­ux pack­ager can have:

Since Mar­ave is a new ap­p, and I de­vel­op it in the rather cut­ting-edge Arch Lin­ux, it us­es some newish fea­tures on­ly avail­able in re­cent ver­sions of Qt. In fac­t, it does­n't work with PyQt < 4.6, which is not avail­able in some slow dis­tros, like De­bian, or even in a not-lat­est Ubun­tu.

So­lu­tion? Well, I could just ig­nore it, but what the heck, let's fix it in­stead!

Thanks to PyIn­staller it's not even hard to do, here's the spec file:

a = Analysis([os.path.join(HOMEPATH,'support/_mountzlib.py'), os.path.join(HOMEPATH,'support/useUnicode.py'), 'marave/main.py'],
            pathex=['/home/ralsina/trunk/trunk'])

pyz = PYZ(a.pure)
exe = EXE(pyz,
        a.scripts,
        exclude_binaries=1,
        name=os.path.join('build/pyi.linux2/main', 'marave.exe'),
        debug=False,
        strip=False,
        upx=True,
        console=0 )

coll = COLLECT( exe,
            a.binaries,
            [('radios.txt','marave/radios.txt','DATA')],
            Tree('marave/icons','icons'),
            Tree('marave/backgrounds','backgrounds'),
            Tree('marave/clicks','clicks'),
            Tree('marave/stylesheets','stylesheets'),
            Tree('marave/themes','themes'),
            a.zipfiles,
            a.datas,
            strip=False,
            upx=True,
            name=os.path.join('dist', 'marave'))

Use this, and PyIn­staller will pro­duce a nice fold­er full of ev­ery­thing Mar­ave needs to run on any Lin­ux.

OTO­H, if you can re­ly on a re­cent PyQt be­ing avail­able, it's al­so sim­ple. Here's a pack­ag­ing con­fig­u­ra­tion for a sim­i­lar pack­age in Arch Lin­ux (I must con­fess not hav­ing done one for Mar­ave yet). For oth­er dis­tri­bu­tions it should be about as sim­ple, if more ver­bose, and some­one else prob­a­bly does it for you:

# Contributor: Roberto Alsina <ralsina@kde.org>
pkgname=python-rst2pdf
pkgver=0.12.1
pkgrel=4
pkgdesc="Create PDFs from simple text markup, no LaTeX required."
arch=('i686' 'x86_64')
url="http://rst2pdf.googlecode.com"
license=('custom')
depends=('python' 'setuptools' 'docutils' 'pygments' 'python-reportlab' 'python-simplejson' 'pil')
source=(http://rst2pdf.googlecode.com/files/rst2pdf-$pkgver.tar.gz LICENSE.txt)
optdepends=('uniconvertor: vector images support'
            'python-svglib: SVG support'
            'python-wordaxe: hyphenation'
            'pythonmagick: PDF images support')
build() {
cd $startdir/src/rst2pdf-$pkgver
python setup.py install --root=$startdir/pkg || return 1
install -D ../LICENSE.txt $startdir/pkg/usr/share/licenses/python-rst2pdf/COPYING
install -D doc/rst2pdf.1 $startdir/pkg/usr/share/man/man1/rst2pdf.1
}
md5sums=('ea6beda9a46f34ba42c4c94d48cc607a'
        '416f8046c66b9476cdbacda69a673afe')

And that's all you need to know about the process of pack­ag­ing your app for Lin­ux. It's easy to do, and most of the time, easy to do right!

Now, let's go to our fi­nal sec­tion...

Windows for the developer

First, re­mem­ber that of re­ly­ing on the sys­tem's ver­sion of Qt? For­get it, there is no sys­tem ver­sion avail­able. And no python ei­ther. And noone is go­ing to in­stall it or your ap­p, so it's "ship ev­ery­thing your­self" mod­e, or noth­ing.

But any­way, PyIn­staller works for Win­dows too! So, us­ing the same spec file, it work­s. Right?

Well, no beause of two prob­lem­s.

Problem 1: You need an installer

Users are not go­ing to open a zip some­where, then do a short­cut to the bi­na­ry on Win­dows, so you need to do some op­er­a­tions, and that means an in­stall­er.

Here's what I came up with to use NSIS, a free in­stall­er cre­ator for Win­dows:

;--------------------------------
;Include Modern UI

!include "MUI2.nsh"

;--------------------------------
;General

;Name and file
Name "Marave"
OutFile "Marave-0.5.win32.exe"

;Default installation folder
InstallDir "$LOCALAPPDATA\Marave"

;Get installation folder from registry if available
InstallDirRegKey HKCU "Software\Marave" ""

;Request application privileges for Windows Vista
RequestExecutionLevel user

;--------------------------------
;Interface Settings

!define MUI_ABORTWARNING

;--------------------------------
;Pages

!insertmacro MUI_PAGE_LICENSE "LICENSE"
!insertmacro MUI_PAGE_DIRECTORY
!insertmacro MUI_PAGE_INSTFILES

!insertmacro MUI_UNPAGE_CONFIRM
!insertmacro MUI_UNPAGE_INSTFILES

;--------------------------------
;Languages

!insertmacro MUI_LANGUAGE "English"

;--------------------------------
;Installer Sections

Section "Install"

SetOutPath "$INSTDIR"
File /r "dist\marave"


;Store installation folder
WriteRegStr HKCU "Software\Marave" "" $INSTDIR

;Create uninstaller
WriteUninstaller "$INSTDIR\Uninstall.exe"

;Create shortcuts
CreateDirectory $SMPROGRAMS\Marave
CreateShortCut "$SMPROGRAMS\Marave\Marave.lnk" "$INSTDIR\marave\marave.exe" ; use defaults for parameters, icon, etc.
CreateShortCut "$SMPROGRAMS\Marave\Uninstall Marave.lnk" "$INSTDIR\Uninstall.exe" ; use defaults for parameters, icon, etc.

SectionEnd


;--------------------------------
;Uninstaller Section

Section "Uninstall"

Delete "$INSTDIR\Uninstall.exe"
RMDir /r "$INSTDIR"

DeleteRegKey /ifempty HKCU "Software\Marave"

SectionEnd

It's com­pa­ra­ble to the ef­fort of build­ing a pack­ag­ing file, re­al­ly, ex­cept ev­ery time you want to test it... you in­stall it. There is no way (AFAIC­S) to see what's in­side the in­stall­er ex­cept run­ning it!

When things fail, you get no er­ror mes­sages, at least not the kind that is use­ful for a de­vel­op­er, the guy that needs to know what went wrong.

And af­ter it's fin­ished, you may end with a non-­work­ing pro­gram be­cause of...

Problem 2: system libraries don't exist

Python 2.6 bi­na­ries are built us­ing Vis­ual Stu­dio. That means they re­quire the Vis­ual Stu­dio Run­time, specif­i­cal­ly MSVCR90.DL­L. That con­tains what on Lin­ux would be con­sid­ered part of libc. (lin­ux guy: imag­ine apps that de­pend on a spe­cif­ic libc... hard to do!)

On Lin­ux that's part of the sys­tem. Fur­ther, if you want­ed, you can re­dis­tribute it. On Win­dows... well, it's a bit dif­fer­en­t.

  1. It's part of the "Vi­­su­al C++ re­dis­­tribu­ta­bles"

  2. In­­stalling that does­n't guar­an­­tee it will work (yes, I have tried)

  3. The li­­cense for those 're­dis­­tribu­ta­bles' says you can't make them avail­able for down­load.

    I have been told that in­­­clud­ing that in your in­­stal­l­er is fine and dandy, but how is that not mak­ing them avail­able for down­load?

So what can you do when you need a li­brary and can't ship it and the us­er won't in­stall it?

Well, that's why there is no Win­dows bi­na­ry of Mar­ave yet. Of course if any­one can help, I'd be re­al­ly, re­al­ly hap­py!