Skip to main content

Ralsina.Me — Roberto Alsina's website

Posts about nikola (old posts, page 2)

Jinja support in Nikola ready for testing

Since some peo­ple re­al­ly don't like Mako, and pre­fer Jin­ja2, I de­cid­ed to make the tem­plate en­gine mod­u­lar in Niko­la my stat­ic site/blog gen­er­a­tor.

All things con­sid­ered, there is very lit­tle dif­fer­ence on the API side:

//ralsina.me/galleries/random/jinjavsmako.thumbnail.png

I even port­ed the de­fault theme to Jin­ja! And not all that much is dif­fer­ent on the tem­plate side, ei­ther:

//ralsina.me/galleries/random/jinjavsmako1.thumbnail.png

But hey, to each his own. Oth­er tem­plate en­gines are prob­a­bly easy to plug, too.

Branch up for re­view here in github and yes, any­one read­ing this who knows Jin­ja2 or python is wel­come to re­view it.

A port of blog.txt to Nikola

If you want a very min­i­mal­is­tic theme for a Niko­la-based site, I just did a quick and dirty port of Scott Wal­lick­'s blog.txt theme

If there is any­thing nice here, he did it. If there is some­thing wrong or bro­ken, I did it in­stead.

I did it ba­si­cal­ly to see if it was pos­si­ble to port word­press themes to Niko­la. And it is, but it in­volves read­ing php files and loose­ly rein­ter­pret­ing them in­to Mako tem­plates.

While the port is far from per­fec­t, it's a rea­son­able start­ing point for some­one who is re­al­ly in­ter­est­ed in it.

Here is how it look­s:

//ralsina.me/galleries/random/blogtxt.png

And here is the down­load. To use it, just un­zip it in your themes fold­er, set THEME to "blogtx­t" in your do­do.py file, and re­build the site.

At least the CSS files are eas­i­ly adapt­ed.

This theme is un­der a LGPL li­cense (see in­clud­ed li­cense.tx­t), en­joy!

Smiljan, a Small Planet Generator

I main­tain a cou­ple of small "plan­et" sites. If you are not fa­mil­iar with plan­et­s, they are sites that ag­gre­gate RSS/Atom feeds for a group of peo­ple re­lat­ed some­how. It makes for a nice, sin­gle, the­mat­ic feed.

Re­cent­ly, when chang­ing them from one serv­er to an­oth­er, ev­ery­thing broke. Old posts were new, feeds that had not been up­dat­ed in 2 years were al­ways with all its posts on top... a dis­as­ter.

I could have gone to the old server, and start­ed de­bug­ging why raw­dog was do­ing that, or switch to plan­et, or look for oth­er soft­ware, or use an on­line ag­gre­ga­tor.

In­stead, I start­ed think­ing... I had writ­ten a few RSS ag­gre­ga­tors in the past... Feed­pars­er is again un­der ac­tive de­vel­op­men­t... raw­dog and plan­et seem to be pret­ty much aban­doned... how hard could it be to im­ple­ment the min­i­mal plan­et soft­ware?

Well, not all that hard, that's how hard it was. Like it took me 4 hours, and was not even dif­fi­cult.

One rea­son why this was eas­i­er than what plan­et and raw­dog achieved is that I am not do­ing a stat­ic site gen­er­a­tor, be­cause I al­ready have one so all I need this pro­gram (I called it Smil­jan) to do is:

  • Parse a list of feeds and store it in a data­base if need­ed.

  • Down­load those feeds (re­spec­t­ing etag and mod­­i­­fied-s­ince).

  • Parse those feeds look­ing for en­tries (feed­­pars­er does that).

  • Load those en­tries (or rather, a tiny sub­­set of their data) in the data­base.

  • Use the en­tries to gen­er­ate a set of files to feed Niko­la

  • Use niko­la to gen­er­ate and de­­ploy the site.

So, here is the fi­nal re­sult: http://­plan­e­ta.python.org.ar which still needs them­ing and a lot of oth­er stuff, but work­s.

I im­ple­ment­ed Smil­jan as 3 doit tasks, which makes it very easy to in­te­grate with Niko­la (if you know Niko­la: add "from smil­jan im­port *" in your do­do.py and a feeds file with the feed list in raw­dog for­mat) and voilá, run­ning this up­dates the plan­et:

doit load_feeds update_feeds generate_posts render_site deploy

Here is the code for smil­jan.py, cur­rent­ly at the "gross hack that kin­da work­s" stage. En­joy!

# -*- coding: utf-8 -*-
import codecs
import datetime
import glob
import os
import sys

from doit.tools import timeout
import feedparser
import peewee


class Feed(peewee.Model):
    name = peewee.CharField()
    url = peewee.CharField(max_length = 200)
    last_status = peewee.CharField()
    etag = peewee.CharField(max_length = 200)
    last_modified = peewee.DateTimeField()

class Entry(peewee.Model):
    date = peewee.DateTimeField()
    feed = peewee.ForeignKeyField(Feed)
    content = peewee.TextField(max_length = 20000)
    link = peewee.CharField(max_length = 200)
    title = peewee.CharField(max_length = 200)
    guid = peewee.CharField(max_length = 200)

Feed.create_table(fail_silently=True)
Entry.create_table(fail_silently=True)

def task_load_feeds():
    feeds = []
    feed = name = None
    for line in open('feeds'):
        line = line.strip()
        if line.startswith('feed'):
            feed = line.split(' ')[2]
        if line.startswith('define_name'):
            name = ' '.join(line.split(' ')[1:])
        if feed and name:
            feeds.append([feed, name])
            feed = name = None

    def add_feed(name, url):
        f = Feed.create(
            name=name,
            url=url,
            etag='caca',
            last_modified=datetime.datetime(1970,1,1),
            )
        f.save()

    def update_feed_url(feed, url):
        feed.url = url
        feed.save()

    for feed, name in feeds:
        f = Feed.select().where(name=name)
        if not list(f):
            yield {
                'name': name,
                'actions': ((add_feed,(name, feed)),),
                'file_dep': ['feeds'],
                }
        elif list(f)[0].url != feed:
            yield {
                'name': 'updating:'+name,
                'actions': ((update_feed_url,(list(f)[0], feed)),),
                }


def task_update_feeds():
    def update_feed(feed):
        modified = feed.last_modified.timetuple()
        etag = feed.etag
        parsed = feedparser.parse(feed.url,
            etag=etag,
            modified=modified
        )
        try:
            feed.last_status = str(parsed.status)
        except:  # Probably a timeout
            # TODO: log failure
            return
        if parsed.feed.get('title'):
            print parsed.feed.title
        else:
            print feed.url
        feed.etag = parsed.get('etag', 'caca')
        modified = tuple(parsed.get('date_parsed', (1970,1,1)))[:6]
        print "==========>", modified
        modified = datetime.datetime(*modified)
        feed.last_modified = modified
        feed.save()
        # No point in adding items from missinfg feeds
        if parsed.status > 400:
            # TODO log failure
            return
        for entry_data in parsed.entries:
            print "========================================="
            date = entry_data.get('updated_parsed', None)
            if date is None:
                date = entry_data.get('published_parsed', None)
            if date is None:
                print "Can't parse date from:"
                print entry_data
                return False
            date = datetime.datetime(*(date[:6]))
            title = "%s: %s" %(feed.name, entry_data.get('title', 'Sin título'))
            content = entry_data.get('description',
                    entry_data.get('summary', 'Sin contenido'))
            guid = entry_data.get('guid', entry_data.link)
            link = entry_data.link
            print repr([date, title])
            entry = Entry.get_or_create(
                date = date,
                title = title,
                content = content,
                guid=guid,
                feed=feed,
                link=link,
            )
            entry.save()
    for feed in Feed.select():
        yield {
            'name': feed.name.encode('utf8'),
            'actions': [(update_feed,(feed,))],
            'uptodate': [timeout(datetime.timedelta(minutes=20))],
            }

def task_generate_posts():

    def generate_post(entry):
        meta_path = os.path.join('posts',str(entry.id)+'.meta')
        post_path = os.path.join('posts',str(entry.id)+'.txt')
        with codecs.open(meta_path, 'wb+', 'utf8') as fd:
            fd.write(u'%s\n' % entry.title.replace('\n', ' '))
            fd.write(u'%s\n' % entry.id)
            fd.write(u'%s\n' % entry.date.strftime('%Y/%m/%d %H:%M'))
            fd.write(u'\n')
            fd.write(u'%s\n' % entry.link)
        with codecs.open(post_path, 'wb+', 'utf8') as fd:
            fd.write(u'.. raw:: html\n\n')
            content = entry.content
            if not content:
                content = u'Sin contenido'
            for line in content.splitlines():
                fd.write(u'    %s\n' % line)

    for entry in Entry.select().order_by(('date', 'desc')):
        yield {
            'name': entry.id,
            'actions': [(generate_post, (entry,))],
            }

Nikola 2.1.1 + GitHub

By pop­u­lar re­quest, Niko­la now has its source code at GitHub.

Al­so, if you tried ver­sion 2.1 and it failed, try 2.1.1, be­cause I for­got to add a cou­ple of files in one of the themes in 2.1.

Dogfooding a new theme

I am do­ing a sec­ond theme for Niko­la, and what bet­ter way to test it than mi­grat­ing this site to it!

In the pro­cess, I stopped us­ing cus­tom tem­plates here, or­ga­nized how a theme is done, how trans­la­tions work, and oth­er stuff.

Com­ments about bro­ken / un­trans­lat­ed / miss­ing stuff are much ap­pre­ci­at­ed!