Ubuntu One APIs by Example (part 1)
One of the nice things about working at Canonical is that we produce open source software. I, specifically, work in the team that does the desktop clients for Ubuntu One which is a really cool job, and a really cool piece of software. However, one thing not enough people know, is that we offer damn nice APIs for developers. We have to, since all our client code is open source, so we need those APIs for ourselves.
So, here is a small tutorial about using some of those APIs. I did it using Python and PyQt for several reasons:
Both are great tools for prototyping
Both have good support for the required stuff (DBus, HTTP, OAuth)
It's what I know and enjoy. Since I did this code on a sunday, I am not going to use other things.
Having said that, there is nothing python-specific or Qt-specific in the code. Where I do a HTTP request using QtNetwork, you are free to use libsoup, or whatever.
So, on to the nuts and bolts. The main pieces of Ubuntu One, from a infrastructure perspective, are Ubuntu SSO Client, that handles user registration and login, and SyncDaemon, which handles file synchronization.
To interact with them, on Linux, they offer DBus interfaces. So, for example, this is a fragment of code showing a way to get the Ubuntu One credentials (this would normally be part of an object's __init__):
# Get the session bus bus = dbus.SessionBus() : : : # Get the credentials proxy and interface self.creds_proxy = bus.get_object("com.ubuntuone.Credentials", "/credentials", follow_name_owner_changes=True) # Connect to signals so you get a call when something # credential-related happens self.creds_iface = dbus.Interface(self.creds_proxy, "com.ubuntuone.CredentialsManagement") self.creds_proxy.connect_to_signal('CredentialsFound', self.creds_found) self.creds_proxy.connect_to_signal('CredentialsNotFound', self.creds_not_found) self.creds_proxy.connect_to_signal('CredentialsError', self.creds_error) # Call for credentials self._credentials = None self.get_credentials()
You may have noticed that get_credentials doesn't actually return the credentials. What it does is, it tells SyncDaemon to fetch the credentials, and then, when/if they are there, one of the signals will be emitted, and one of the connected methods will be called. This is nice, because it means you don't have to worry about your app blocking while SyncDaemon is doing all this.
But what's in those methods we used? Not much, really!
def get_credentials(self): # Do we have them already? If not, get'em if not self._credentials: self.creds_proxy.find_credentials() # Return what we've got, could be None return self._credentials def creds_found(self, data): # Received credentials, save them. print "creds_found", data self._credentials = data # Don't worry about get_quota yet ;-) if not self._quota_info: self.get_quota() def creds_not_found(self, data): # No credentials, remove old ones. print "creds_not_found", data self._credentials = None def creds_error(self, data): # No credentials, remove old ones. print "creds_error", data self._credentials = None
So, basically, self._credentials will hold a set of credentials, or None. Congratulations, we are now logged into Ubuntu One, so to speak.
So, let's do something useful! How about asking for how much free space there is in the account? For that, we can't use the local APIs, we have to connect to the servers, who are, after all, the ones who decide if you are over quota or not.
Access is controlled via OAuth. So, to access the API, we need to sign our requests. Here is how it's done. It's not particularly enlightening, and I did not write it, I just use it:
def sign_uri(self, uri, parameters=None): # Without credentials, return unsigned URL if not self._credentials: return uri if isinstance(uri, unicode): uri = bytes(iri2uri(uri)) print "uri:", uri method = "GET" credentials = self._credentials consumer = oauth.OAuthConsumer(credentials["consumer_key"], credentials["consumer_secret"]) token = oauth.OAuthToken(credentials["token"], credentials["token_secret"]) if not parameters: _, _, _, _, query, _ = urlparse(uri) parameters = dict(cgi.parse_qsl(query)) request = oauth.OAuthRequest.from_consumer_and_token( http_url=uri, http_method=method, parameters=parameters, oauth_consumer=consumer, token=token) sig_method = oauth.OAuthSignatureMethod_HMAC_SHA1() request.sign_request(sig_method, consumer, token) print "SIGNED:", repr(request.to_url()) return request.to_url()
And how do we ask for the quota usage? By accessing the https://one.ubuntu.com/api/quota/ entry point with the proper authorization, we would get a JSON dictionary with total and used space. So, here's a simple way to do it:
# This is on __init__ self.nam = QtNetwork.QNetworkAccessManager(self, finished=self.reply_finished) : : : def get_quota(self): """Launch quota info request.""" uri = self.sign_uri(QUOTA_API) url = QtCore.QUrl() url.setEncodedUrl(uri) self.nam.get(QtNetwork.QNetworkRequest(url))
Again, see how get_quota doesn't return the quota? What happens is that get_quota will launch a HTTP request to the Ubuntu One servers, which will, eventually, reply with the data. You don't want your app to block while you do that. So, QNetworkAccessManager will call self.reply_finished when it gets the response:
def reply_finished(self, reply): if unicode(reply.url().path()) == u'/api/quota/': # Handle quota responses self._quota_info = json.loads(unicode(reply.readAll())) print "Got quota: ", self._quota_info # Again, don't worry about update_menu yet ;-) self.update_menu()
What else would be nice to have? How about getting a call whenever the status of syncdaemon changes? For example, when sync is up to date, or when you get disconnected? Again, those are DBus signals we are connecting in our __init__:
self.status_proxy = bus.get_object( 'com.ubuntuone.SyncDaemon', '/status') self.status_iface = dbus.Interface(self.status_proxy, dbus_interface='com.ubuntuone.SyncDaemon.Status') self.status_iface.connect_to_signal( 'StatusChanged', self.status_changed) # Get the status as of right now self._last_status = self.process_status( self.status_proxy.current_status())
And what's status_changed?
def status_changed(self, status): print "New status:", status self._last_status = self.process_status(status) self.update_menu()
The process_status function is boring code to convert the info from syncdaemon's status into a human-readable thing like "Sync is up-to-date". So we store that in self._last_status and update the menu.
What menu? Well, a QSystemTrayIcon's context menu! What you have read are the main pieces you need to create something useful: a Ubuntu One tray app you can use in KDE, XFCE or openbox. Or, if you are on unity and install sni-qt, a Ubuntu One app indicator!
You can find the source code for the whole example app at my u1-toys project in launchpad and here is the full source code (missing some icon resources, just get the repo)
Coming soon(ish), more example apps, and cool things to do with our APIs!
I usually try hard not to comment on products by companies which compete with my employer (I work at Red Hat), but sheer wrongness of the sentences "One of the nice things about working at Canonical is that we produce open source software.
I, specifically, work in the team that does the desktop clients for
Ubuntu One" compelled me to react. Do you intentionally ignore the fact that Ubuntu One (as a server) is completely proprietary or there is some reality distortion field in action here and you try to persuade yourself that you do something else than you do?
To make myself completely clear, I don't object to the fact that Canonical tries to make money for their open source products by selling proprietary service, that's their business decision, and if you make money with which you can provide Ubuntu, good for you. I am just offended by the depth of hypocrisy and misleading sleaze with which Canonical is not willing to admit it. It is not just about you, but many other cases (see infamous http://twit.tv/show/floss-w... which turned out to be an unpaid infomercial for the service).
Best,
Matěj Cepl (mcepl at redhat dot com ; sorry, Disqus is broken, so I was not able to login with my account)
I have no idea what that link you provide is (chromium won't let me open it, claims it's malware).
I will not try to answer to your comments in kind, because, really, I feel like you are trolling me, it's 8AM, and I am not interested. Sorry if you felt "compelled to react". That was hardly the point of the post. But really, hypocrisy and sleaze? That's what you get out of a 20 minute tutorial on accessing some APIs?
That the server side of Ubuntu One is not open source is known. Hey, if you ask in, say, askubuntu.com about how you can get your own Ubuntu One server someone (maybe even me) will explain to you that no, we are not sharing
that, but there is a perfectly nice, clear and open protocol definition you can use if you want to.
But no, we are not sharing that, and yes, it's because Canonical hopes and wishes to make some money out of it. Does it mean I don't do free software? No, it doesn't. Does it mean I don't feel good writing it? No it doesn't. Did I say Canonical produces only open source software, no I didn't.
So, congratulations on taking a true statement, getting all worked up about it and spitting a little bile on the comment section of my personal blog. Hope that makes you feel all warm inside.
> I have no idea what that link you provide is (chromium won't let me open it, claims it's malware).
An episode of FLOSS Weekly podcast where Jono Bacon and Stuart Langridge (both of Canonical) presented Ubuntu One ignoring to mention (notice once again the name of the show) that the server is proprietary. If that is labeled by Chromium as a malware, one more reason why not to use Chromium ;).
Matěj
I suggest you go troll an Evolution developer. I heard they support proprietary GMail, and never say that GMail is not Free Software, so they are not real free software people.
I almost asked you to post an OwnCloud version, but I guess you won't take it as a joke, but rather sending me to do a gynecological test to my sister. :-D
BTW, what happened with the "RMS hates me" t-shirts? (Sorry, I can't resist!)
I am thinking about doing a "Yes, I am a freedom-hating argentinian" ones. I would be happy to do a OwnCLoud version, but... well, I have no idea about their APIs. Have any pointers?
I just realized comments are shared between English and Spanish blog posts, so there is no real reason to write in English, painfully trying to put my ideas in the right words. What a relief!
La verdad, "la nube" no me impresiona gran cosa, y OwnCloud no fue la excepción, así que no llegué a ver sobre como instalarlo, y menos nada que tenga que ver con código.
Do you have a brief tutorial on how I can do this using Qt/C++ since I can't use python?
The only part that is not a straighforward translation is the OAUTH bits. Is there any C++ Oauth library you recommend? If so, I will be happy to rewrite this in C++.
I have found KQOauth to be ideal for oauth authentication from ubuntu (works for Twitter, etc.):
http://www.johanpaul.com/bl...
Thank you for offering to rewrite the tutorial in C++/Qt. Greatly appreciated!
Hi Roberto,
I had a related question: In your example above, you are using an undocumented API called 'quota'. Is there someplace I can get a list of all the undocumented APIs? Basically, here is what I am trying to do in my c++ app:
1. Allow a user to log in to their u1 account
2. Get a list of folders that they have published to the cloud
3. For each folder, get a list of files they have published t0 the cloud
4. For each file, get the public-URL to read the file from the cloud
Since I will be running this on an embedded device with limited storage, I don't want to sync with the cloud. Instead, I want to directly read the file from the public-URL
I would greatly appreciate if your example tutorial in Qt/C++ could show me how to do this. Huge thanks for your help!!
We are not supposed to have undocumented APIs :-)
I will check tomorrow to see if it's documented, and if not, will get it documented. Your use case should be doable using only the REST API, since you will not be using SSO client or syncdaemon, so it's going to look quite different from this example.
Thank you so much! Looking forward to it!
Hey there! As ralsina says, all our public APIs should be documented (at https://one.ubuntu.com/deve... as you likely already know). The quota API isn't yet documented; good catch, there! I let that slip through my fingers :)
On doing what you want to do, you should be able to do that by getting an OAuth token (either by asking for username and password (https://one.ubuntu.com/deve... or by opening a browser and using the OAuth 1.0 dance (https://one.ubuntu.com/deve..., and then querying the Files API (https://one.ubuntu.com/deve... to get a list of synced folders and the folders and files within, or better still by hitting https://one.ubuntu.com/api/... to get the list of public files.
Thanks for the response - very quick! I will try these and get back to you.
Hi sil, I am afraid I was not able to get very far with the links you provided. Would it be possible for you to show some example REST requests for my use case? Your help is greatly appreciated!
One nitpick: instead of doing `_, _, _, _, query, _ = urlparse(uri)`, just do `query = urlparse(uri).query` :)
I am scratching my head with the Ubuntu One files API.
1. I performed an oAuth authentication dance and got a token.2. I used the following URL with an authenticated GET to fetch the volumes list, which it did successfully: https://one.ubuntu.com/api/.... I got the following bacK: [{"resource_path": "/volumes/~/Ubuntu One", "when_created": "2012-06-18T19:05:13Z", "generation": 5, "path": "~/Ubuntu One", "content_path": "/content/~/Ubuntu One", "type": "root", "node_path": "/~/Ubuntu One"}]4. I then tried to get the next level from the root "https://one.ubuntu.com/api/... One"This returns nothing although there are folders below with files in them.
What am I doing wrong?