One of the nice things about work­ing at Canon­i­cal is that we pro­duce open source soft­ware. I, specif­i­cal­ly, work in the team that does the desk­top clients for Ubun­tu One which is a re­al­ly cool job, and a re­al­ly cool piece of soft­ware. How­ev­er, one thing not enough peo­ple know, is that we of­fer damn nice APIs for de­vel­op­er­s. We have to, since all our client code is open source, so we need those APIs for our­selves.

So, here is a small tu­to­ri­al about us­ing some of those APIs. I did it us­ing Python and PyQt for sev­er­al rea­son­s:

• Both are great tools for pro­­to­­typ­ing

• Both have good sup­­port for the re­quired stuff (D­Bus, HTTP, OAu­th)

• It's what I know and en­joy. Since I did this code on a sun­­day, I am not go­ing to use oth­­er things.

Hav­ing said that, there is noth­ing python-spe­cif­ic or Qt-spe­cif­ic in the code. Where I do a HTTP re­quest us­ing Qt­Net­work, you are free to use lib­soup, or what­ev­er.

So, on to the nuts and bolt­s. The main pieces of Ubun­tu One, from a in­fra­struc­ture per­spec­tive, are Ubun­tu SSO Clien­t, that han­dles us­er reg­is­tra­tion and login, and Sync­Dae­mon, which han­dles file syn­chro­niza­tion.

To in­ter­act with them, on Lin­ux, they of­fer DBus in­ter­faces. So, for ex­am­ple, this is a frag­ment of code show­ing a way to get the Ubun­tu One cre­den­tials (this would nor­mal­ly be part of an ob­jec­t's __init__):

# Get the session bus
bus = dbus.SessionBus()

:
:
:

# Get the credentials proxy and interface
self.creds_proxy = bus.get_object("com.ubuntuone.Credentials",
                        "/credentials",
                        follow_name_owner_changes=True)

# Connect to signals so you get a call when something
# credential-related happens
self.creds_iface = dbus.Interface(self.creds_proxy,
    "com.ubuntuone.CredentialsManagement")
self.creds_proxy.connect_to_signal('CredentialsFound',
    self.creds_found)
self.creds_proxy.connect_to_signal('CredentialsNotFound',
    self.creds_not_found)
self.creds_proxy.connect_to_signal('CredentialsError',
    self.creds_error)

# Call for credentials
self._credentials = None
self.get_credentials()

You may have no­ticed that get_­cre­den­tials does­n't ac­tu­al­ly re­turn the cre­den­tial­s. What it does is, it tells Sync­Dae­mon to fetch the cre­den­tial­s, and then, when/if they are there, one of the sig­nals will be emit­ted, and one of the con­nect­ed meth­ods will be called. This is nice, be­cause it means you don't have to wor­ry about your app block­ing while Sync­Dae­mon is do­ing all this.

But what's in those meth­ods we used? Not much, re­al­ly!

def get_credentials(self):
    # Do we have them already? If not, get'em
    if not self._credentials:
        self.creds_proxy.find_credentials()
    # Return what we've got, could be None
    return self._credentials

def creds_found(self, data):
    # Received credentials, save them.
    print "creds_found", data
    self._credentials = data
    # Don't worry about get_quota yet ;-)
    if not self._quota_info:
        self.get_quota()

def creds_not_found(self, data):
    # No credentials, remove old ones.
    print "creds_not_found", data
    self._credentials = None

def creds_error(self, data):
    # No credentials, remove old ones.
    print "creds_error", data
    self._credentials = None

So, ba­si­cal­ly, self­._­cre­den­tials will hold a set of cre­den­tial­s, or None. Con­grat­u­la­tion­s, we are now logged in­to Ubun­tu One, so to speak.

So, let's do some­thing use­ful! How about ask­ing for how much free space there is in the ac­coun­t? For that, we can't use the lo­cal APIs, we have to con­nect to the server­s, who are, af­ter al­l, the ones who de­cide if you are over quo­ta or not.

Ac­cess is con­trolled via OAuth. So, to ac­cess the API, we need to sign our re­quest­s. Here is how it's done. It's not par­tic­u­lar­ly en­light­en­ing, and I did not write it, I just use it:

def sign_uri(self, uri, parameters=None):
    # Without credentials, return unsigned URL
    if not self._credentials:
        return uri
    if isinstance(uri, unicode):
        uri = bytes(iri2uri(uri))
    print "uri:", uri
    method = "GET"
    credentials = self._credentials
    consumer = oauth.OAuthConsumer(credentials["consumer_key"],
                                   credentials["consumer_secret"])
    token = oauth.OAuthToken(credentials["token"],
                             credentials["token_secret"])
    if not parameters:
        _, _, _, _, query, _ = urlparse(uri)
        parameters = dict(cgi.parse_qsl(query))
    request = oauth.OAuthRequest.from_consumer_and_token(
                                        http_url=uri,
                                        http_method=method,
                                        parameters=parameters,
                                        oauth_consumer=consumer,
                                        token=token)
    sig_method = oauth.OAuthSignatureMethod_HMAC_SHA1()
    request.sign_request(sig_method, consumer, token)
    print "SIGNED:", repr(request.to_url())
    return request.to_url()

And how do we ask for the quo­ta us­age? By ac­cess­ing the http­s://one.ubun­tu.­com/api/quo­ta/ en­try point with the prop­er au­tho­riza­tion, we would get a JSON dic­tio­nary with to­tal and used space. So, here's a sim­ple way to do it:

    # This is on __init__
    self.nam = QtNetwork.QNetworkAccessManager(self,
        finished=self.reply_finished)

:
:
:

def get_quota(self):
    """Launch quota info request."""
    uri = self.sign_uri(QUOTA_API)
    url = QtCore.QUrl()
    url.setEncodedUrl(uri)
    self.nam.get(QtNetwork.QNetworkRequest(url))

Again, see how get_quo­ta does­n't re­turn the quo­ta? What hap­pens is that get_quo­ta will launch a HTTP re­quest to the Ubun­tu One server­s, which will, even­tu­al­ly, re­ply with the da­ta. You don't want your app to block while you do that. So, QNet­workAc­cess­Man­ag­er will call self­.re­ply_fin­ished when it gets the re­spon­se:

def reply_finished(self, reply):
    if unicode(reply.url().path()) == u'/api/quota/':
        # Handle quota responses
        self._quota_info = json.loads(unicode(reply.readAll()))
        print "Got quota: ", self._quota_info
        # Again, don't worry about update_menu yet ;-)
        self.update_menu()

What else would be nice to have? How about get­ting a call when­ev­er the sta­tus of sync­dae­mon changes? For ex­am­ple, when sync is up to date, or when you get dis­con­nect­ed? Again, those are DBus sig­nals we are con­nect­ing in our __init__:

self.status_proxy = bus.get_object(
    'com.ubuntuone.SyncDaemon', '/status')
self.status_iface = dbus.Interface(self.status_proxy,
    dbus_interface='com.ubuntuone.SyncDaemon.Status')
self.status_iface.connect_to_signal(
    'StatusChanged', self.status_changed)

# Get the status as of right now
self._last_status = self.process_status(
    self.status_proxy.current_status())

And what's sta­tus_changed?

def status_changed(self, status):
    print "New status:", status
    self._last_status = self.process_status(status)
    self.update_menu()

The pro­cess_s­ta­tus func­tion is bor­ing code to con­vert the in­fo from sync­dae­mon's sta­tus in­to a hu­man-read­able thing like "Sync is up­-­to-­date". So we store that in self­._last_s­ta­tus and up­date the menu.

What menu? Well, a QSys­tem­Tray­I­con's con­text menu! What you have read are the main pieces you need to cre­ate some­thing use­ful: a Ubun­tu One tray app you can use in KDE, XFCE or open­box. Or, if you are on uni­ty and in­stall sni-qt, a Ubun­tu One app in­di­ca­tor!

You can find the source code for the whole ex­am­ple app at my u1-­toys project in launch­pad and here is the full source code (miss­ing some icon re­sources, just get the re­po)

Com­ing soon(ish), more ex­am­ple app­s, and cool things to do with our APIs!

Turns out there was a ma­jor, show stop­per bug in rst2pdf 0.90: sphinx sup­port was ab­so­lute­ly bro­ken. Not bro­ken as in bug­gy, bro­ken as in it had syn­tax er­rors.

So, 0.91 is now re­leased, and on­ly bro­ken in the tra­di­tion­al sense. En­joy!

Yes, af­ter many moon­s, it's out. Here is the (as usu­al) in­com­plete changel­og:

• Added raw HTML sup­­port, by Dim­itri Christodoulou

• Fixed Is­­sue 422: Hav­ing no .afm files made font lookup slow.

• Fixed Is­­sue 411: Some­­times the win­­dows reg­istry has the font's ab­s­path.

• Fixed Is­­sue 430: Us­ing --­­con­­fig op­­tion caused oth­­er op­­tions to be ig­nored (by charles at cstan­hope dot com)

• Fixed Is­­sue 436: Add pdf_style_­­path to sphinx (by tyler@­­datas­­tax.­­com)

• Fixed Is­­sue 428: page num­bers logged as er­rors

• Added sup­­port for many pyg­­ments op­­tions in code-block (by Joaquin So­ri­anel­lo)

• Im­­ple­­men­t­ed Is­­sue 404: plan­­tuml sup­­port

• Is­­sue 399: sup­­port sphinx's tem­­plate path op­­tion

• Fixed Is­­sue 406: calls to the wrong log­ging func­­tion

• Im­­ple­­men­t­ed Is­­sue 391: New --­sec­­tion-­­head­­er-depth op­­tion.

• Fixed Is­­sue 390: the --­­con­­fig op­­tion was ig­nored.

• Added sup­­port for many pyg­­ments op­­tions in code-block (by Joaquin So­ri­anel­lo)

• Fixed Is­­sue 379: Wrong style ap­­plied to para­­graphs in de­f­i­ni­­tion­s.

• Fixed Is­­sue 378: Mul­ti­­line :ad­­dress: were shown col­lapsed.

• Im­­ple­­men­t­ed Is­­sue 11: Frame­Break (and con­di­­tion­al Frame­Break)

• The de­scrip­­tion of frames in page tem­­plates was just wrong.

• Fixed Is­­sue 374: in some cas­es, lit­er­al blocks were split in­­­side a page, or the page­break came too ear­­ly.

• Fixed Is­­sue 370: warn­ing about sphinx.addnodes.high­­­light­lang not be­ing han­­dled re­­moved.

• Fixed Is­­sue 369: crash in hy­phen­a­tor when spec­i­­fy­ing "en" as a lan­guage.

• Com­­pat­i­­bil­i­­ty fix to Sphinx 0.6.x (For python 2.7 doc­s)

This re­lease did not fo­cus on Sphinx bugs, so those are prob­a­bly still there. Hope­ful­ly the next round is at­tack­ing those.

So, you are writ­ing a PyQt ap­p, and you want it to sup­port com­mand line ar­gu­ments. So you do some­thing like this:

opt_parser = OptionParser()
opt_parser.add_option("-q", dest="quickly", action="store_true",
    help="Do it quickly (default=False)")
(options, args) = opt_parser.parse_args(sys.argv)
app = QApplication(sys.argv)
:
:
:

Or maybe even QAp­pli­ca­tion([]). Ok, you are doing it wrong. And this is wrong in most tutorials, too. Why? Because Qt (and thus PyQt) supports a bunch of useful command line options already. So if you do it like in the first listing, and pass "-style=oxygen" or whatever, one of the following will happen.

1. Op­t­­Pars­er is go­ing to tell you it's not a valid op­­tion and abort

2. You will ig­nore the op­­tion and not do any­thing use­­ful with it

3. You will have your own -style op­­tion and do two things with it

All three out­comes are less than ide­al.

The right way to do this is:

opt_parser = OptionParser()
opt_parser.add_option("-q", dest="quickly", action="store_true",
    help="Do it quickly (default=False)")
app = QApplication(sys.argv)
(options, args) = opt_parser.parse_args(app.arguments())
:
:
:

This way, you give PyQt a chance to process the options it recognizes, and, then, you get to handle the rest, because app.arguments() has all Qt options removed.

The bad side of this is, you will make --help slightly slower, since it will have to build a QApplication to do nothing, and you will have undocumented options. Solution for both problems left as an exercise.

I am in gen­er­al al­ler­gic to HTM­L, spe­cial­ly when it comes to pars­ing it. How­ev­er, ev­ery now and then some­thing comes up and it's fun to keep the mus­cles stretched.

So, con­sid­er the Ted Talks site. They have a re­al­ly nice ta­ble with in­for­ma­tion about their talk­s, just in case you want to do some­thing with them.

But how do you get that in­for­ma­tion? By scrap­ing it. And what's an easy way to do it? By us­ing Python and Beau­ti­ful­Soup:

from BeautifulSoup import BeautifulSoup
import urllib

# Read the whole page.
data = urllib.urlopen('http://www.ted.com/talks/quick-list').read()
# Parse it
soup = BeautifulSoup(data)

# Find the table with the data
table = soup.findAll('table', attrs= {"class": "downloads notranslate"})[0]
# Get the rows, skip the first one
rows = table.findAll('tr')[1:]

items = []
# For each row, get the data
# And store it somewhere
for row in rows:
    cells = row.findAll('td')
    item = {}
    item['date'] = cells[0].text
    item['event'] = cells[1].text
    item['title'] = cells[2].text
    item['duration'] = cells[3].text
    item['links'] = [a['href'] for a in cells[4].findAll('a')]
    items.append(item)

And that's it! Sur­pris­ing­ly pain-free!