This book is either the best idea of the year, but the author did not quite nail it, or the worst idea of the decade brilliantly written.
Not often does a book shock me. This one shocked me right at the beginning, and close to the end.
If you are reading this there is a good chance you are involved somehow in open source development, or software development in general. One thing lots of people ask me when they know I have lead this sort of projects for a long time is "how do you decide things?". To which I have all sorts of bad answers like:
"It's a consensus thing"
"It happens organically"
"Sometimes it just happens"
"You do what you do"
So, now here I have an AWESOME example of FLOSS decision making in action, which is ... all of the above.
Some context: Nikola is a static site generator, so it deals with reading and writing textual data from disk. It's also an internationalized project, which supports multilingual sites and translated data. It also runs un multiple platforms, like Windows, OSX, Linux, etc.
And to make that more fun, it also works on Python 2.7, and 3.3 or later. Which means it has to handle two
different models on how to work with unicode data, in the same codebase. And that's not fun. So, we have been
floating around the idea of deprecating python 2.7. And so, when
s2hc_johan walks in with a unicode
14:23:16 <s2hc_johan> I don't have a site with sections, but I tested it for the other case 14:35:42 <s2hc_johan> strange it worked for a while broken again, probably because I've got åäö in it now. 14:35:45 <s2hc_johan> https://github.com/getnikola/plugins/blob/master/v7/recent_posts_json/recent_posts_json.py#L134 14:36:17 <s2hc_johan> if you wrap data with unicode it works, but I'm not sure that works in python3 14:36:37 <ChrisWarrick> s2hc_johan: how do you wrap it with unicode? 14:36:48 <s2hc_johan> unicode(data) 14:37:05 <s2hc_johan> but is that valid in python3? 14:37:11 <ChrisWarrick> s2hc_johan: this is wrong on so many levels 14:37:16 <ChrisWarrick> s2hc_johan: please don’t do that, ever 14:37:48 <ChrisWarrick> s2hc_johan: This won’t work in Python 3 either. You must have an actual encoding, and use the decode method. try: foo = foo.decode('utf-8'); except AttributeError: foo = foo # python 3 14:38:02 <s2hc_johan> what do you mean, that is like my standard when I get strnage data in, undoce(data) data.encode(whatever) data.decode(whatever) :) 14:38:23 <s2hc_johan> one of them ussually work 14:39:22 <ChrisWarrick> s2hc_johan: unicode() assumes ASCII, it never works right 14:39:32 <s2hc_johan> true 14:39:40 <ChrisWarrick> s2hc_johan: encode/decode with a specified encoding is fine 14:40:00 <ChrisWarrick> s2hc_johan: but you might need a try/except for Python 3 if it could have Unicode data already 14:40:16 <s2hc_johan> I'm a bit confused in this case since the output comes from json.dumps 14:40:34 <s2hc_johan> thought that would produce a unicode object 14:40:51 <ChrisWarrick> s2hc_johan: not necessarily on python 2 14:41:05 <ralsina_> if isinstance(thing, utils.str_bytes): thing=thing.decode('utf8') 14:41:15 <ralsina_> that works in py2 and py3 14:42:12 <ChrisWarrick> easier to ask for forgiveness imo 14:43:07 <ralsina_> maybe we should have helpers in utils enforce_unicode and enforce_bytes 14:43:13 -GitHub[nikola]:#nikola- [nikola] Aeyoun pushed 1 new commit to feed-previewimage: http://git.io/vnqek 14:43:13 -GitHub[nikola]:#nikola- nikola/feed-previewimage 4b79e20 Daniel Aleksandersen: Deprecated RSS_READ_MORE_LINK and RSS_LINKS_APPEND_QUERY... 14:44:58 <Aeyoun> Or upgrade to Py3. 14:45:11 <ChrisWarrick> ++ 14:45:47 <Aeyoun> Unicode in Py27 is a nightmare. It tries as hard as it can to kill you at every turn. 14:48:09 -travis-ci:#nikola- getnikola/nikola#6426 (feed-previewimage - 4b79e20 : Daniel Aleksandersen): The build is still failing. 14:48:10 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/c4c69c02db34...4b79e20d1ebc 14:48:10 -travis-ci:#nikola- Build details: https://travis-ci.org/getnikola/nikola/builds/81026762 14:48:27 <ralsina_> ok, let's consider py3-only seriously. 14:48:40 <ralsina_> 1) Is there any distro commonly used with py3 < 3.3 ? 14:48:55 <ralsina_> 2) Do we just stop using py2, or we deprecate slowly? 14:49:15 <ralsina_> 3) Do we just start doing py3-only code, or we actively de-hack the codebase? 14:49:21 <ralsina_> That's my 3 questions :-) 14:50:13 <SteveDrees> Unicode is a nightmare 14:50:53 <SteveDrees> different python versions just changes where the pain point is 14:50:53 <s2hc_johan> which one is better isinstance... or hasattr('decode', ..) 14:51:02 <ralsina_> isinstance 14:51:08 <s2hc_johan> oki then 14:51:10 <ralsina_> hasattr is evil in itself 14:51:26 <s2hc_johan> just going to feed the kids then I'll make another pr 14:51:28 -GitHub[nikola]:#nikola- [nikola] Aeyoun pushed 1 new commit to feed-previewimage: http://git.io/vnqJ2 14:51:28 -GitHub[nikola]:#nikola- nikola/feed-previewimage 4c950ac Daniel Aleksandersen: flake8 14:52:13 <Aeyoun> ralsina_: user survey? pip download data? 14:52:33 <gour> ralsina_: create some poll at website/mailing-list about it? 14:53:18 <ralsina_> dude, I offered free shirts and I got only 10 requests ;-) 14:53:30 <ralsina_> so, how many answers do you expect about that sort of thing? 14:53:43 * gour thought shirts are jsut for devs :-( 14:53:47 <Aeyoun> ralsina_: release a unchanged version on pip that is flagged as py3 only. see how many downlaod it versus previous version in same amount of time. 14:53:51 <ralsina_> gour: go add yourself dude 14:54:18 <ralsina_> gour: TO THE SHIRT LIST! I just notced that sounded very rude :-) 14:54:43 <gour> ralsina_: where it is? 14:54:43 <Aeyoun> ralsina_: or one py27 version number and and one version py3 only version number at the same time. 14:55:17 <ralsina_> gour: https://docs.google.com/forms/d/18YFwdgukmpkjr5b8FGEKL0arxPePuLHNsuEa-Gl80D8/viewform?c=0&w=1 14:55:17 <gour> found it 14:56:00 <gour> ralsina_: wonder if xxl is too large or xl is enough 14:56:00 <Aeyoun> ralsina_: american or european sizes by the by? 14:56:03 <ralsina_> Aeyoun: that reflects how many people use py2.7 by reflex. I know *i* do because it's "python" and not "python3" 14:56:20 <ralsina_> Aeyoun: no idea about sizes to be honest... probably american 14:56:21 <Aeyoun> American sizes are … a big bigger. I’m probably a XS/S american but M european. :P 14:56:28 <Aeyoun> *bit bigger 14:56:39 <gour> ok 14:56:57 * gour submitted request 14:57:17 <ralsina_> So, what I would prefer to do is make people use py3 if they can. And it seems to me that pretty much everyone can, regardless of whether they still use py2 by defect. 14:57:26 <ralsina_> by default*, spanishism leaked there. 14:57:52 <ChrisWarrick> technically, using py2 is a defect 14:57:59 <ralsina_> So, if we all agree that most users *could* run nikola in py3... then let's do it. 14:58:02 <Aeyoun> Agreed. 14:58:15 <gour> sites won't stop working :-) 14:58:26 <Aeyoun> ralsina_: act on data not dev agreement? 14:58:42 <ChrisWarrick> guess we could change our docs/webiste to highlight 3.x 14:58:59 <ralsina_> Aeyoun: the only data we'd need is to know how many people have py2.7 and no py3.3 14:59:14 <ralsina_> not how many are *using* 2.7 instead of 3.3 14:59:38 <ChrisWarrick> micro-survey via ml? 14:59:39 <ralsina_> How about: let's announce that, unless lots of people complaint, we deprecate py2 by end of october 14:59:45 -travis-ci:#nikola- getnikola/nikola#6429 (feed-previewimage - 4c950ac : Daniel Aleksandersen): The build was fixed. 14:59:46 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/4b79e20d1ebc...4c950ac5e52e 14:59:46 -travis-ci:#nikola- Build details: https://travis-ci.org/getnikola/nikola/builds/81028389 14:59:47 <Aeyoun> Mac is shipping with Py2.7 and no Py3. BUT MacPorts and Homebrew offer painfree Py3 installs. 14:59:58 <ralsina_> ok, mac is a good point 15:00:25 <ChrisWarrick> it’s not like we have Homebrew/MacPorts/Fink-based install instructions for them… 15:00:27 <Aeyoun> ralsina_: we could add a deprecation message every time `nikola` is run and ask people to bitch in a bug? 15:00:32 <Aeyoun> ChrisWarrick: hehe. ;) 15:00:50 <ralsina_> "I see you have python3 installed but I am running on 2.7 ... dude, what's wrong with you?" 15:00:51 <Aeyoun> Or maybe once per 24 hour rather than every time its run. 15:01:00 <ralsina_> doit timed tasks :-) 15:01:12 <Aeyoun> ralsina_: "Don’t get in the way of progress! Upgrade to Py3 and save a developer’s mind today!" 15:01:32 <ralsina_> "niec unicode you have there, would be a shame something happened to it.. switch to python 3!" 15:01:39 <ChrisWarrick> ralsina_: hey, let’s start with a Google Docs survey on the ML. One question: what Python version and OS are you using for Nikola? 2.7/3.3/3.4/3.5; Windows/OS X/[other: linux/bsd distro] 15:01:57 <gour> "Free t-shirt foreveryone switching from py2.7 to py3.3" 15:01:58 <ChrisWarrick> ralsina_: Just don’t require a Google account like you did last time. 15:02:00 <ralsina_> Second question: "Do you have python 3.3 or later installed?" 15:02:03 <Aeyoun> How much code can be removed with dropping Py27? Lowers maintenance cost and increases performance. That is also an important datapoint. 15:02:11 <ralsina_> ChrisWarrick: I needed to know who was asking for the shirt :-) 15:02:21 <ChrisWarrick> ralsina_: good point 15:02:25 <ralsina_> Aeyoun: not all that much, really 15:02:47 <ChrisWarrick> Aeyoun: it would need to start with a huge rewrite to remove all of our pointers in nikola.utils 15:03:00 <ralsina_> Aeyoun: there are a number of tiny hacks, which were a pain to get right but they always amount to one if and/or one decode :-) 15:03:26 <ralsina_> We can just turn a bunch of helpers in utils into noops 15:04:52 <gour> py3-only nikola is going to become v8? 15:05:15 <Aeyoun> gour: seems like a likely outcome. you’re following the discussion live. 15:06:34 <ChrisWarrick> if we do v8, we’ll have to merge the early tasks garbage 15:07:03 <ralsina_> Is it technically backwards-incompatible if we just stop working on py2.7? 15:07:21 <ralsina_> gour: welcome to open source software: behind the code. 15:07:30 <gour> ralsina_: :-) 15:07:35 <Aeyoun> Someone call in a documentary crew! 15:07:43 <ralsina_> Aeyoun: we have logs! 15:07:51 <Aeyoun> Oh, wait. This is already logged for prosperity. 15:07:57 <ralsina_> I am totally posting this somewhere as "this is how decisions are made in FLOSS" 15:08:40 <ralsina_> Ok, who creates the poll and who posts it in the blog, and who makes sure it appears on planet, and who sends it to the list? 15:08:49 <ralsina_> I would do it but I have work to do :) 15:08:51 <ChrisWarrick> ralsina_: I’ll do it 15:08:57 <ralsina_> ChrisWarrick: you rock dude! 15:09:01 <ChrisWarrick> ralsina_: should be really simple 15:09:03 <ralsina_> Ok, we have a plan! 15:09:17 <ralsina_> Let's consider the poll results in ... a week? 15:09:25 <Aeyoun> Let the logs show we’re all in favor of this plan of action. ;-) 15:09:29 <ralsina_> aye 15:09:51 <ralsina_> Also: can I do the "shame on you" thing on nikola build? It sounds like fun :-) 15:10:27 <ChrisWarrick> ralsina_: for the python version question: radiobox vs checkbox? 15:10:28 <gour> ralsina_: you can mention that Nikola (Tesla) was always for innovation ;) 15:10:44 <Aeyoun> "You’re using FIVE YEAR OLD SOFTWARE. Update your system." 15:11:00 <ralsina_> Aeyoun: I am totally getting at least 5 different comments there 15:11:01 <Aeyoun> https://en.wikipedia.org/wiki/History_of_Python#Version_release_dates 15:11:05 <ralsina_> ChrisWarrick: checkbox... maybe 2? 15:11:23 <ralsina_> ChrisWarrick: one for python version, one for operating system 15:11:32 <ChrisWarrick> ralsina_: ? 15:11:38 <ralsina_> ChrisWarrick: two questions 15:11:54 <ChrisWarrick> ralsina_: there will even be three questions (py2/3 used, OS, has py3) 15:11:57 <ChrisWarrick> ralsina_: and checkboxes it is 15:12:02 <ralsina_> right 15:12:05 <ralsina_> awesome 15:14:44 <ralsina_> Copied / Pasted for posterity
There you go, half an hour later, we have a plan to (maybe) deprecate it.
Now go vote here: Should Nikola support python2.7? Gives us data to decide!