Python Trick: Save anything in config files

2007-05-11 08:59

The Python config objects are convenient and simple, but they have a problem: you can only save strings. That means you need to store numbers as strings and remember to use the getint()/getfloat() methods (or coerce by hand!), which is error prone and anti-pythonic. Storing a list is even uglier.

You could store ascii pickles, but those are pretty unpleasant to read in some cases.

Here's my solution: Encode it using a JSON encoder first! (I am using demjson)

Silly obvious code fragment:

def getValue(section,key,default=None):
    try:
        return JSON().decode(conf.get (section,key))
    except:
        return default

def setValue(section,key,value):
    value=JSON().encode(value)
    try:
        r=conf.set(section,key,value)
    except ConfigParser.NoSectionError:
        conf.add_section(section)
        r=conf.set(section,key,value)
    f=open(os.path.expanduser('~/.bartleblog/config'),'w')
    conf.write(f)
    return r

With just a little effort you can have a readable ascii typed python config file.

Paul Giannaros / 2007-05-11 18:33:

I tend to make my configuration files Python code so they can just be imported. You need to write a little code to write out the file when config changes, but it is the height of pythonic :P. It can handle arbitary Python data types too which is very cool

Ralesk / 2007-05-11 18:37:

Why not Pickle it?

Oh, and another problem with ConfigParser I found is that it wouldn’t keep the order of items (uses disctionaries inside, so no wonder), and for a while I wanted to write my own INI file parser, but I just gave in to Pickle for now.

Roberto Alsina / 2007-05-11 19:09:

Ralesk:

I didn't pickle because I wanted it to be hand-editable.

And a pickle of [1,2,3] looks like this:

'(lp0nI1naI2naI3na.'

The same as JSON looks like this:

[1,2,3]

Or you mean you just pickle a dictionary as config file? Same problem, only larger.

Daniel "Suslik" D. / 2007-05-11 21:06:

I changed 2 or 3 INI-style config parsers while coding on A-Foto (a-foto.googlecode.com). As you will see later, the built-in ConfigParser has numerous problems:

- it tunrs ALL capital letter in section (and key?) names into lower case, thus destroying all KDE .rc files.
- it rearranges the contents of the ini/rc file.
- does not support nesting (or lists, ints etc)

This was discussed a lot, and there is a very very good wiki page somewhere on the python.org about replacements.

I tried 3, and the best one in my opinion is this one:
http://www.voidspace.org.uk...

- the whole content of the config file becomes available as a dictionary (Yey!!!)
- it supports round-tripping (read-change-save - no rearrangement of original file layout.)
- Respects the capitalization of the sections and keys.
- BSD license.
- Allows nested sections, lists.
- The resulting INI file is perfectly readable, and completely resembles the original INI style.

I wrote a small wrapper for it, to ensure that I talk to the new parser, whenever i change it in the same language as I would talk to the original ConfigParser.

see configobj.py and localconf.py files here:
a-foto.googlecode.com/svn/t...

Roberto Alsina / 2007-05-11 22:01:

Daniel: this is not supposed to be compatible with KDE's config files.

Further, I would consider trying to access the same config file using two different implementations crazy-risky but that's just me :-)

Using JSON you can save most structures, and resorting of the file doesn't really worry me (why is it important at all?)

Ralsina.Me — Roberto Alsina's website