Posts about programming (old posts, page 4)

2005-09-09 19:34

Authenticated Pages in CherryPy

CherryPy is a cool, pythonic, simple, quick, fun way to write web applications.

I often use CherryPy to write custom web admin tools for customers. Suppose you want to provide them with a simple way for password management. Usually I have the following requirements:

  • It must be simple ( not webmin )
  • It must not be a terminal session (bye ssh :-( )
  • It must not be a graphical session (sadly, that leaves out PyQt :-( )
  • It needs to do custom stuff: set the samba password at the same time, send a mail warning about the next forced change, whatever.

Someday I may be able to use a single-app freeNX session, but right now that's a bit too much problem for different reasons.

So, I wrote a CherryPy page. Over time, I have become quite fond of it, and wrote a bunch of small tools around it. One of them was a way to login the user into the site using the system's users and passwords. Now I got to throw it away :-)

The new CherryPy 2.1 has a mechanism for implementing password-protected pages, called the Session authenticate filter which is sadly not documented yet anywhere I can find.

So, here is my attempt, so people googling it up can use it. Excuse me:

cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter cherrypy sessionauthenticatefilter

That should do it :-)

What you need first is a function that takes a username and password, and returns None on success, or an error message for the failure.

For example, I can adapt something I wrote earlier using checkpassword-pam

def validPass(name,password):
                cmd='/usr/bin/checkpassword-pam -s xdm -- /bin/true 3<&0'
                p=os.popen(cmd,'w')
                s='%s\000%s\000xxx\000'%(name,password)
                print cmd,s
                p.write(s)
                r=p.close()
                if r==None: #Success
                        return None
                else:
        return "Login Incorrect"

Also, you may want a function that returns the login screen. If you do, remember the following:

  1. It must set the form action to doLogin
  2. The user field should be called login
  3. The password field should be called password
  4. You will take a fromPage argument that you should pass through, so the user will end on the page he wants.
  5. You will take a errorMsg argument which is probably the result of a failed previous login. Display it red or something like it. Unless it's empty, in which case it should not be visible.

Here's mine.

def loginScreen(fromPage, login = '', errorMsg = ''):
content="""
<form method="post" action="doLogin">
<div align=center>
        <span class=errormsg>%s</span><p>
        <table >
        <tr>
        <td>
                Login:
        <td>
                <input type="text" name="login" value="%s" size="40"/>
        <tr>
        <td>
                Password:
        <td>
        <input type="password" name="password" size="40"/>
        <input type="hidden" name="fromPage" value="%s"/>
        <tr>
        <td colspan=2 align=right>
        <input type="submit" value="Login" />
        </table>
        </div>
</form>
""" % (errorMsg, login, fromPage)
title='Login'
return renderTemplate(file='logintemplate.html')

Although I am using a template to display it nicely and with the right style, it should be pretty obvious how it works.

You could use the default login screen provided by the filter. While it works, it's just ugly.

Then you need to apply the filter to the set of your pass-protected pages. Suppose you want the whole site to be protected, except for your /static directory, which contains the stylesheet, images and such. Then you put this in your configuration file:

[/]
sessionAuthenticateFilter.on=True

[/static]
sessionAuthenticateFilter.on=False

Next thing is to hook the session authenticate filter to your custom auth code. In your app, do the following. It seems that you can't do this in the config file, though, so do it in code.

    settings={
            '/': {
                                    'sessionAuthenticateFilter.checkLoginAndPassword': validPass,
                'sessionAuthenticateFilter.loginScreen':loginScreen
        }
}
cherrypy.config.update(settings)

And that's it. Now your site is password protected. You can even have different authentication schemes for different pieces of the site, by resetting the hooks for the folder you prefer to a different function.

Also, I think this is a good example of why I like CherryPy. This mechanism is both flexible, powerful and simple.

2005-09-02 23:16

Small Linux Revisited

A Little History

Many moons (almost two years!) ago, I wrote an article called Small Linux detailing what I had done to make a reasonable Linux fit in a Toshiba Libretto 50.

That's a very limited notebook, with under 800MB of disk, a 75Mhz Pentium CPU and 16MB of RAM, and it has served me well for a long time.

Now, however, I am using the almost exact opposite, a second-hand Toshiba Satellite 1955-S805.

Where Salma (the Libretto) had a 640x480 screen, Monty (Toshiba), has a 16" 1280x1024. The RAM has increased 32 times. But they have one thing in common....

The 800MB HD

You see, Monty is second hand. My future mother-in-law and brother-in-law brought it from New York when they visited, for Rosario (My future wife, until February 18 2006 ;-).

And it had a broken HD. And I wanted to use it while I got a nice new 60GB one.

So, overcoming my fear of destroying expensive equipment, I got the HD out of Salma and into Monty, and started thinking....

The Tallest Guy In The World

He had a problem: really bad feet. He died of it, too. In the same way, Monty now booted, but the app selection was outdated, and really, lots of things Salma couldn't do, Monty could.

What on earth can one install on that disk when you don't have any other real hardware limitations?

The choice of distribution was tricky.

I am a CentOS guy lately, but the package selection is entangled enough that you can hardly get X installed without crossing the 800MB. The minimal install is about 450MB.

Debian again? Well... no.

Knoppix?

Now, that has some serious potential, since I could run the OS from DVD/CD, and then use the whole 800MB for data. But I wanted something where I could choose what to install.

I could have gone the path of one of the modular Knoppix derivatives, but it was yet another task on the pile.

So, I went with.... ARCH.

ARCH Is Arch

Yes, ARCH is saucy. It installs in roughly 200MB, including kernel sources and GCC. That's quite small.

I managed to create a reasonable desktop in about 550MB, including:

Scite:
A nice text editor
Python:
Needed it to work in a few proggies.
Firefox:
I have the RAM. The CPU is a 2.56 P4.
Xorg 8.2 with Nvidia drivers:
And just for kicks, 3ddesktop to see if they work ;-)
fluxbox:
I know it, and it's nice enough.
putty:
Nicer than xterm, and has a handy SSH builtin.
ROX Filer:
Not really all that useful, but what the hell, for 3MB you get sidebars, a file manager and a few extra tricks.
Slim:
A very nice xdm replacement. Specially with the mindlock theme :-)
upx:
Compress the exes. Save 30MB. Good.
CherryPy:
It's what I am working with. And it's only 74KB.
habak:
Simple, small tool to set the X root decoration.
rsync:
Stay synced with my older desktop.

All included, I am at 120 packages, using 557MB of disk (with extensive trimming, see the original article for some examples).

So, what's the difference between this set of apps and my previous choice....

Well, look at the result of free:

             total       used       free     shared    buffers     cached
Mem:        512544     408924     103620          0      33468     162916
-/+ buffers/cache:     212540     300004
Swap:            0          0          0

Just for laughs: here's the old one, when doing roughly the same things: editing an article, browsing the web, a few terminals:

                total    used    free    shared   buffers        cached
Mem:            14708   14144     564      4644       816          5304
-/+ buffers/cache:       8024    6684
Swap:           47992   18880   29112

Scary isn't it? I am using roughly 25 times the amount of memory I used on the libretto. It's easy to see why, tho.

Consider the desktop. It shows a pretty picture. It is 1280x1024. It is in millions of colors. That is in RAM. That is either 3932160 or 5242880 bytes. On the libretto, I was intentionally not using anything there :-)

So, it really is not comparable anyway, and Monty's life as a malformed box will be short. But it was quite a bit of fun :-)

2005-08-04 21:27

Not letting stuff fall off the ' net

For a bunch of apps I write, I often want to be able to add a systray icon.

But... I write them using PyQt, and the systray stuff is in PyKDE.

But... Torsten Marek did write a module to do that. The only problem for me is the python-ctypes requirement, but it's no big deal for my apps that are not massively deployed.

You can find his code, in a somewhat mangled form, here

And since building extension modules is not completely trivial, here's a simple setup.py that will do it:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from distutils.core import setup
from distutils.extension import Extension

setup(
  name = "systray",
  ext_modules=[
    Extension('traywin', ['traywin.c'],  libraries=["X11"],library_dirs=['/usr/X11R6/lib'])],
)

Put it in the same folder with his code, then you can do python setup.py install or somesuch, then you can use it in your app like in Torsten's systray2.py example, and it will work.

2005-05-14 16:22

Data aware widgets in KDE

Well, reading in planetkde about how nice data aware widgets would be, I have to say this:

  • Data aware widgets are great
  • Data aware widgets in C++ are not the best possible solution

Using a higher level language, and specifically a more dynamic language makes lots of things much simpler.

As a tiny, lame example, please check the little thing I wrote about data aware widgets in python here.

I am sure that some of our better programmers (or more creative thinkers) can come up with awesome stuff if they divorce from C++ in this subject :-)

2005-05-10 21:29

Extreme code reuse

I am, as always, playing with stuff. And I was faced with a problem I must have solved a dozen times before:

Given a list of items with obvious hierarchical names (say, a list of folders), turn it into a reasonable data structure.

Since there are not all that many names, there is no need to do it on-demand, or anything like that.

I must confess I suck at this kind of things because I hve to actually think them through. I don't know how to do this kind of things.

Mind you, I have done it before, but I said, hey, maybe google can help me...

And yeah! Python code from 1994 that does exactly what I needed. I had to touch a single line ( string.splitfields(a,'.') to a.split('/') ) and it worked.

The Internet never forgets!

2005-04-08 23:59

Sometimes things just click

I have been writing web-based interfaces for applications for about 5 years. Nothing public, nothing very interesting, just tiny front ends for custom tools in clients' installations.

And I have hated every minute of it. PHP hurts, Twisted hurts, mod_python hurts...

For a couple of months I have been using CherryPy and I finally am having fun doing it.

And after I figured out how to do AJAX using it, it's even more fun ( because the apps interaction model is not totally braindead :-)

I don't expect it to be as fun as PyQt/PyKDE, but it's totally not awful. I suppose the same epiphany comes to people when they use rails or some other decent, productive, fun framework.

All in all, I could get used to this.

2005-03-15 23:59

Source-based distributions, the good side.

I am no fan of source-based distributions. I think that for most practical purposes, a distribution where installing KDE takes over a day (I own a lowly Duron as my fast computer) is useless.

However, they are good for one particular niche.

Custom distributions. Weird distributions. Personal distributions.

I have several computers. Most of them too slow and small.

As I was considering reinstalling Linux on my Libretto for fun, I looked at what was installed in it, and decided that really, 90% of it was of no use whatsoever.

The problem was, since it had Debian installed, it has a rather wide network of dependencies that simply could not be done without.

On a regular computer, that's not an issue, but in this tiny workhorse, with 16MB of RAM and a 800MB HD, it makes a lot of difference. The smallest I could get Debian to be, and still have network, PCMCIA and X11, was about 250MB.

And the performance was not awesome, either (but not terrible).

So, what would happen if instead of a regular distribution it had something like this:

  • uClibc instead of glibc
  • runit instead of SYSVinit
  • dropbear instead of OpenSSH
  • X built as kdrive (formerly tinyX)
  • python/qt (or maybe python/fltk) for basic admin tools

And so on. Basically rejigger the whole software selection slanting it towards the small side, and dropping a bazillion non-fundamental dependencies along the way.

Well, it can be done. It's just that it's a heck of a lot of work. But here, a source-based distribution gives you a headstart.

For example, I decided to start from ucrux, a uClibc-based port of Crux. Since the native toolchain in ucrux is uClibc, I don't have to worry much about a whole class of mess that happens when you build uClibc-based binaries on a glibc-based system (it's practically cross-compiling).

Qemu lets me install ucrux and work on it somewhat faster than on the target P75 (if I could make KQEmu work I'd be happiest).

Since crux's software packaging mechanism is simplicity itself (a shell script that installs to a fake root), although it's severely underpowered (no dependencies), I can start from ucrux, hack the build of one package at a time, then put everything on a CD very quickly.

So, if you want to hack your own distribution, Crux (or some other similar kit) is quite a nice thing.

For general use... well, my requirements start at apt-get or equivalent ;-)

Now, if only TinyCC worked for more programs, that P75 would be a pocket development powerhouse!

2005-03-08 23:59

Trac is cool. Cherrypy is cooler.

Trac is cool. Easy to set up, easy to run, low maintenance, and you get:

  • A ticketing system
  • Milestones
  • A webcvs-like thing for subversion
  • A wiki (I mean,. what doesn't provide a wiki nowadays?)
  • Bug reporting tool
  • The bug reporting tool and the subversion changesets can be linked using Wiki markup (now that's cooler than it sounds ;-)
  • You don't need to be root to set it up, and you don't need apache or anything else, really.

Really, really nice stuff.

On the other hand, CherryPy is a tool that lets you "publish your python objects on the web", which doesn't really mean much, but here's what I figured out:

Cherrypy is the first way I have seen to write a useful web-based app in a reasonable amount of time and pain.

Example, I wrote a frontend to clamav (allowing me to remotely trigger scans of individual nodes on a network) using Cherrypy and pyclamav in about 200 lines of code.

It works like a charm, it's robust, it even can be made to look nice using some sort of templating engine (haven't bothered yet).

And of course, I control that baby using a Trac project :-)

2004-12-14 19:00

This is why dynamic languages are cool

I wrote a little spreadsheet thingie a few days ago. [1]

Of course, it's a toy, not the real thing at all, but it was a nice hack, since it is a real, recalculating, extensible, dependency-checking, loop-avoiding spreadsheet engine in about 50 lines of code.

That's because I was using Python, which is a seriously cool language to write that kind of thing in, since all you have to do to evaluate an expression is call eval() on it.

Sure, that's nice, but the real core of the spreadsheet engine was that you could also create a dictionary-like object that recalculated on-demand its contents.

That way, when you ask for sheet['a1'], custom code goes to see what a1 has in it (a formula), calculates it if needed, and maybe trigger a few extra recalculations if another cell depends on a1. [2]

But as anyone who uses spreadsheets can tell you, weird things exist in ssheet land.

For example, if you copy something, then you paste it, it gets modified in the process.

What other app does that???

Here's an example you can check in any spreadsheet:

  • In A1, type "1".
  • In B1, type "A1+1" (should display 2)
  • In A2, type 2
  • Copy B1 to B2, and it will display 3

Further, if you look at the formula in B2, it says A2+1 now.

That's called relative cellnames (I think).

In order to do that trick, you have to parse the formula in B1, and then, when you paste it into B2, take into account the displacement and modify accordingly. Usually, if you want absolute names, you use $ A1 instead, and that would stay unmodified.

Now, that throws a nice monkeywrench into my neat little spreadsheet [3] because now it suddenly looks not like a spreadsheet at all!

So, I started thinking, how the hell could this be done? The whole advantage of a python sheet is using eval(), so switching to a parser (like if this were a C[++] sheet) would be silly.

I delved into the python standard lib. As every python programmer knows, almost everyhting is there. If you write python, you read the library reference every day, and memorize chunks of it, because it's one of the things that make python cool. It's just chockfull of useful stuff!

And here I was reading about the compiler module, and the parser module, which can be used to do wondrous stuff with python code. But I couldn't understand jackshit about them. I'm a simple coder.

And just as I was going to say, let's write instead about the connection between free software and the sex life of frogs [4] I found tokenize.

Tokenize is a module that parses python and turns it into tokens. Here's how a+2 looks after you tokenize it:

1,0-1,1:        NAME    'a'
1,1-1,2:        OP      '+'
1,2-1,3:        NUMBER  '2'
2,0-2,0:        ENDMARKER       ''

The numbers on the left side are positions in the text stream where the tokens were.

It has just enough information that you can tokenize a piece of code, and then reassemble it. There's code to do just that, it's called regurgitate and it's written by Ka-Ping Yee.

So, the solution is obvious. When copying a formula:

  • Tokenize the formula to be copied
  • Look for tokens of type NAME
  • See if it looks like a cellname, or _cellname
  • If it's _cellname, leave as is. That will be our notation for absolute cells
  • If it's cellname, displace it nicely
  • Regurgitate it

Later, when evaluating a formula, if someone asks for cell _a1 give him cell a1.

And voilà, relative cells.

This works, and it works well (ok, I had to introduce some ugly globals, I need to learn more stuff), and it is guaranteed to tokenize in the same way python does it. It's not even really slow [5]

I touched a bunch of other things, including support for all the functions in python's math module so you can use them in cells. Here's the code to do that:

for name in dir(math):
        if name[0]<>"_":
                self.tools[name]=eval('math.'+name)

Freaky stuff, isn't it?

What's the main issue? Performance. To put it simply, I seriously doubt a sheet written in python can be fast enough for general use. But hey, it's extensible, it's nice, and depending on what you are trying to do, it may be good enough.

And here's today's version of StupidSheet including relative cells. Don't worry, it's a small download ;-)

[1] And almost noone noticed ;-)
[2] That triggering is the only part I wrote myself, the rest is from ASPN's cookbook.
[3] I call it StupidSheet.
[4] I did write that anyway
[5] I don't advice you to copy a formula and paste it into a 10000x200 selection. It will never end. Optimization for this is unexistant. And unlikely.

2004-12-07 19:02

Not a calculator

I have been playing with this code and it's been lots of fun.

I've hacked it into a functional spreadsheet in (according to eric3) 508 lines of non-doc code, of which 244 are generated by pyuic.

Here's my code so far (requires PyQt). Give it a look, I think it's kinda nice.

The only hard part I wrote (at least hard for me) was the cell dependency and recalculation support.

There's a test file you can use, too.

It is trivial to add functions you can use in the cells, just lookup python docs for eval() and check engine.py.

To use it, unpack it, and from the directory it creates run python ssheet.py

I don't plan to make it a real spreadsheet, but it should be fun to hack on :-)

Contents © 2000-2018 Roberto Alsina