Skip to main content

Ralsina.Me — Roberto Alsina's website

Ubuntu One APIs by Example (part 1)

One of the nice things about work­ing at Canon­i­cal is that we pro­duce open source soft­ware. I, specif­i­cal­ly, work in the team that does the desk­top clients for Ubun­tu One which is a re­al­ly cool job, and a re­al­ly cool piece of soft­ware. How­ev­er, one thing not enough peo­ple know, is that we of­fer damn nice APIs for de­vel­op­er­s. We have to, since all our client code is open source, so we need those APIs for our­selves.

So, here is a small tu­to­ri­al about us­ing some of those APIs. I did it us­ing Python and PyQt for sev­er­al rea­son­s:

  • Both are great tools for pro­­to­­typ­ing

  • Both have good sup­­port for the re­quired stuff (D­Bus, HTTP, OAu­th)

  • It's what I know and en­joy. Since I did this code on a sun­­day, I am not go­ing to use oth­­er things.

Hav­ing said that, there is noth­ing python-spe­cif­ic or Qt-spe­cif­ic in the code. Where I do a HTTP re­quest us­ing Qt­Net­work, you are free to use lib­soup, or what­ev­er.

So, on to the nuts and bolt­s. The main pieces of Ubun­tu One, from a in­fra­struc­ture per­spec­tive, are Ubun­tu SSO Clien­t, that han­dles us­er reg­is­tra­tion and login, and Sync­Dae­mon, which han­dles file syn­chro­niza­tion.

To in­ter­act with them, on Lin­ux, they of­fer DBus in­ter­faces. So, for ex­am­ple, this is a frag­ment of code show­ing a way to get the Ubun­tu One cre­den­tials (this would nor­mal­ly be part of an ob­jec­t's __init__):

# Get the session bus
bus = dbus.SessionBus()

:
:
:

# Get the credentials proxy and interface
self.creds_proxy = bus.get_object("com.ubuntuone.Credentials",
                        "/credentials",
                        follow_name_owner_changes=True)

# Connect to signals so you get a call when something
# credential-related happens
self.creds_iface = dbus.Interface(self.creds_proxy,
    "com.ubuntuone.CredentialsManagement")
self.creds_proxy.connect_to_signal('CredentialsFound',
    self.creds_found)
self.creds_proxy.connect_to_signal('CredentialsNotFound',
    self.creds_not_found)
self.creds_proxy.connect_to_signal('CredentialsError',
    self.creds_error)

# Call for credentials
self._credentials = None
self.get_credentials()

You may have no­ticed that get_­cre­den­tials does­n't ac­tu­al­ly re­turn the cre­den­tial­s. What it does is, it tells Sync­Dae­mon to fetch the cre­den­tial­s, and then, when/if they are there, one of the sig­nals will be emit­ted, and one of the con­nect­ed meth­ods will be called. This is nice, be­cause it means you don't have to wor­ry about your app block­ing while Sync­Dae­mon is do­ing all this.

But what's in those meth­ods we used? Not much, re­al­ly!

def get_credentials(self):
    # Do we have them already? If not, get'em
    if not self._credentials:
        self.creds_proxy.find_credentials()
    # Return what we've got, could be None
    return self._credentials

def creds_found(self, data):
    # Received credentials, save them.
    print "creds_found", data
    self._credentials = data
    # Don't worry about get_quota yet ;-)
    if not self._quota_info:
        self.get_quota()

def creds_not_found(self, data):
    # No credentials, remove old ones.
    print "creds_not_found", data
    self._credentials = None

def creds_error(self, data):
    # No credentials, remove old ones.
    print "creds_error", data
    self._credentials = None

So, ba­si­cal­ly, self­._­cre­den­tials will hold a set of cre­den­tial­s, or None. Con­grat­u­la­tion­s, we are now logged in­to Ubun­tu One, so to speak.

So, let's do some­thing use­ful! How about ask­ing for how much free space there is in the ac­coun­t? For that, we can't use the lo­cal APIs, we have to con­nect to the server­s, who are, af­ter al­l, the ones who de­cide if you are over quo­ta or not.

Ac­cess is con­trolled via OAuth. So, to ac­cess the API, we need to sign our re­quest­s. Here is how it's done. It's not par­tic­u­lar­ly en­light­en­ing, and I did not write it, I just use it:

def sign_uri(self, uri, parameters=None):
    # Without credentials, return unsigned URL
    if not self._credentials:
        return uri
    if isinstance(uri, unicode):
        uri = bytes(iri2uri(uri))
    print "uri:", uri
    method = "GET"
    credentials = self._credentials
    consumer = oauth.OAuthConsumer(credentials["consumer_key"],
                                   credentials["consumer_secret"])
    token = oauth.OAuthToken(credentials["token"],
                             credentials["token_secret"])
    if not parameters:
        _, _, _, _, query, _ = urlparse(uri)
        parameters = dict(cgi.parse_qsl(query))
    request = oauth.OAuthRequest.from_consumer_and_token(
                                        http_url=uri,
                                        http_method=method,
                                        parameters=parameters,
                                        oauth_consumer=consumer,
                                        token=token)
    sig_method = oauth.OAuthSignatureMethod_HMAC_SHA1()
    request.sign_request(sig_method, consumer, token)
    print "SIGNED:", repr(request.to_url())
    return request.to_url()

And how do we ask for the quo­ta us­age? By ac­cess­ing the http­s://one.ubun­tu.­com/api/quo­ta/ en­try point with the prop­er au­tho­riza­tion, we would get a JSON dic­tio­nary with to­tal and used space. So, here's a sim­ple way to do it:

    # This is on __init__
    self.nam = QtNetwork.QNetworkAccessManager(self,
        finished=self.reply_finished)

:
:
:

def get_quota(self):
    """Launch quota info request."""
    uri = self.sign_uri(QUOTA_API)
    url = QtCore.QUrl()
    url.setEncodedUrl(uri)
    self.nam.get(QtNetwork.QNetworkRequest(url))

Again, see how get_quo­ta does­n't re­turn the quo­ta? What hap­pens is that get_quo­ta will launch a HTTP re­quest to the Ubun­tu One server­s, which will, even­tu­al­ly, re­ply with the da­ta. You don't want your app to block while you do that. So, QNet­workAc­cess­Man­ag­er will call self­.re­ply_fin­ished when it gets the re­spon­se:

def reply_finished(self, reply):
    if unicode(reply.url().path()) == u'/api/quota/':
        # Handle quota responses
        self._quota_info = json.loads(unicode(reply.readAll()))
        print "Got quota: ", self._quota_info
        # Again, don't worry about update_menu yet ;-)
        self.update_menu()

What else would be nice to have? How about get­ting a call when­ev­er the sta­tus of sync­dae­mon changes? For ex­am­ple, when sync is up to date, or when you get dis­con­nect­ed? Again, those are DBus sig­nals we are con­nect­ing in our __init__:

self.status_proxy = bus.get_object(
    'com.ubuntuone.SyncDaemon', '/status')
self.status_iface = dbus.Interface(self.status_proxy,
    dbus_interface='com.ubuntuone.SyncDaemon.Status')
self.status_iface.connect_to_signal(
    'StatusChanged', self.status_changed)

# Get the status as of right now
self._last_status = self.process_status(
    self.status_proxy.current_status())

And what's sta­tus_changed?

def status_changed(self, status):
    print "New status:", status
    self._last_status = self.process_status(status)
    self.update_menu()

The pro­cess_s­ta­tus func­tion is bor­ing code to con­vert the in­fo from sync­dae­mon's sta­tus in­to a hu­man-read­able thing like "Sync is up­-­to-­date". So we store that in self­._last_s­ta­tus and up­date the menu.

What menu? Well, a QSys­tem­Tray­I­con's con­text menu! What you have read are the main pieces you need to cre­ate some­thing use­ful: a Ubun­tu One tray app you can use in KDE, XFCE or open­box. Or, if you are on uni­ty and in­stall sni-qt, a Ubun­tu One app in­di­ca­tor!

http://ubuntuone.com/7iXTbysoMM9PIUS9Ai4TNn

My Ubun­tu One in­di­ca­tor in ac­tion.

You can find the source code for the whole ex­am­ple app at my u1-­toys project in launch­pad and here is the full source code (miss­ing some icon re­sources, just get the re­po)

Com­ing soon(ish), more ex­am­ple app­s, and cool things to do with our APIs!

Matěj Cepl / 2012-03-13 11:03:

I usually try hard not to comment on products by companies which compete with my employer (I work at Red Hat), but sheer wrongness of the sentences "One of the nice things about working at Canonical is that we produce open source software.
I, specifically, work in the team that does the desktop clients for
Ubuntu One" compelled me to react. Do you intentionally ignore the fact that Ubuntu One (as a server) is completely proprietary or there is some reality distortion field in action here and you try to persuade yourself that you do something else than you do?

To make myself completely clear, I don't object to the fact that Canonical tries to make money for their open source products by selling proprietary service, that's their business decision, and if you make money with which you can provide Ubuntu, good for you. I am just offended by the depth of hypocrisy and misleading sleaze with which Canonical is not willing to admit it. It is not just about you, but many other cases (see infamous http://twit.tv/show/floss-w... which turned out to be an unpaid infomercial for the service).

Best,

Matěj Cepl (mcepl at redhat dot com ; sorry, Disqus is broken, so I was not able to login with my account)

Roberto Alsina / 2012-03-13 11:19:

I have no idea what that link you provide is (chromium won't let me open it, claims it's malware).

I will not try to answer to your comments in kind, because, really, I feel like you are trolling me, it's 8AM, and I am not interested. Sorry if you felt "compelled to react". That was hardly the point of the post. But really, hypocrisy and sleaze? That's what you get out of a 20 minute tutorial on accessing some APIs?

That the server side of Ubuntu One is not open source is known. Hey, if you ask in, say, askubuntu.com about how you can get your own Ubuntu One server someone (maybe even me) will explain to you that no, we are not sharing 
that, but there is a perfectly nice, clear and open protocol definition you can use if you want to.

But no, we are not sharing that, and yes, it's because Canonical hopes and wishes to make some money out of it. Does it mean I don't do free software? No, it doesn't. Does it mean I don't feel good writing it? No it doesn't. Did I say Canonical produces only open source software, no I didn't.

So, congratulations on taking a true statement, getting all worked up about it and spitting a little bile on the comment section of my personal blog. Hope that makes you feel all warm inside.

Matěj Cepl / 2012-03-13 12:15:

 > I have no idea what that link you provide is (chromium won't let me open it, claims it's malware).
An episode of FLOSS Weekly podcast where Jono Bacon and Stuart Langridge (both of Canonical) presented Ubuntu One ignoring to mention (notice once again the name of the show) that the server is proprietary. If that is labeled by Chromium as a malware, one more reason why not to use Chromium ;).

Matěj

Roberto Alsina / 2012-03-13 12:55:

I suggest you go troll an Evolution developer. I heard they support proprietary GMail, and never say that GMail is not Free Software, so they are not real free software people.

Shulai / 2012-03-13 22:21:

I almost asked you to post an OwnCloud version, but I guess you won't take it as a joke, but rather sending me to do a gynecological test to my sister. :-D

BTW, what happened with the "RMS hates me" t-shirts? (Sorry, I can't resist!)

Roberto Alsina / 2012-03-13 22:57:

I am thinking about doing a "Yes, I am a freedom-hating argentinian" ones. I would be happy to do a OwnCLoud version, but... well, I have no idea about their APIs. Have any pointers?

Shulai / 2012-03-15 00:51:

I just realized comments are shared between English and Spanish blog posts, so there is no real reason to write in English, painfully trying to put my ideas in the right words. What a relief!

La verdad, "la nube" no me impresiona gran cosa, y OwnCloud no fue la excepción, así que no llegué a ver sobre como instalarlo, y menos nada que tenga que ver con código.

PhoenixRevived / 2012-04-26 20:47:

Do you have a brief tutorial on how I can do this using Qt/C++ since I can't use python?

Roberto Alsina / 2012-04-26 20:50:

The only part that is not a straighforward translation is the OAUTH bits. Is there any C++ Oauth library you recommend? If so, I will be happy to rewrite this in C++.

PhoenixRevived / 2012-04-26 21:31:

I have found KQOauth to be ideal for oauth authentication from ubuntu (works for Twitter, etc.): 

http://www.johanpaul.com/bl...

Thank you for offering to rewrite the tutorial in C++/Qt. Greatly appreciated!

PhoenixRevived / 2012-04-26 23:40:

Hi Roberto,
I had a related question: In your example above, you are using an undocumented API called 'quota'. Is there someplace I can get a list of all the undocumented APIs? Basically, here is what I am trying to do in my c++ app:

1. Allow a user to log in to their u1 account
2. Get a list of folders that they have published to the cloud
3. For each folder, get a list of files they have published t0 the cloud
4. For each file, get the public-URL to read the file from the cloud

Since I will be running this on an embedded device with limited storage, I don't want to sync with the cloud. Instead, I want to directly read the file from the public-URL

I would greatly appreciate if your example tutorial in Qt/C++ could show me how to do this. Huge thanks for your help!!

Roberto Alsina / 2012-04-27 00:38:

We are not supposed to have undocumented APIs :-)

I will check tomorrow to see if it's documented, and if not, will get it documented. Your use case should be doable using only the REST API, since you will not be using SSO client or syncdaemon, so it's going to look quite different from this example.

PhoenixRevived / 2012-04-27 03:57:

Thank you so much! Looking forward to it!

sil / 2012-04-27 08:29:

Hey there! As ralsina says, all our public APIs should be documented (at https://one.ubuntu.com/deve... as you likely already know). The quota API isn't yet documented; good catch, there! I let that slip through my fingers :)

On doing what you want to do, you should be able to do that by getting an OAuth token (either by asking for username and password (https://one.ubuntu.com/deve... or by opening a browser and using the OAuth 1.0 dance (https://one.ubuntu.com/deve..., and then querying the Files API (https://one.ubuntu.com/deve... to get a list of synced folders and the folders and files within, or better still by hitting https://one.ubuntu.com/api/... to get the list of public files.

PhoenixRevived / 2012-04-27 15:18:

Thanks for the response - very quick! I will try these and get back to you.

PhoenixRevived / 2012-04-27 17:26:

Hi sil, I am afraid I was not able to get very far with the links you provided. Would it be possible for you to show some example REST requests for my use case? Your help is greatly appreciated!

John Lenton / 2012-04-27 09:07:

One nitpick: instead of doing `_, _, _, _, query, _ = urlparse(uri)`, just do `query = urlparse(uri).query` :)

PhoenixRevived / 2012-08-10 19:43:

I am scratching my head with the Ubuntu One files API.

1. I performed an oAuth authentication dance and got a token.2. I used the following URL with an authenticated GET to fetch the volumes list, which it did successfully: https://one.ubuntu.com/api/.... I got the following bacK: [{"resource_path": "/volumes/~/Ubuntu One", "when_created": "2012-06-18T19:05:13Z", "generation": 5, "path": "~/Ubuntu One", "content_path": "/content/~/Ubuntu One", "type": "root", "node_path": "/~/Ubuntu One"}]4. I then tried to get the next level from the root "https://one.ubuntu.com/api/... One"This returns nothing although there are folders below with files in them.
What am I doing wrong?