--- author: '' category: '' date: 2015-09-18 15:25:58 UTC description: '' link: '' priority: '' slug: floss-decision-making-in-action tags: python, programming title: FLOSS Decision Making in Action type: text updated: 2015-09-18 15:25:58 UTC url_type: '' --- If you are reading this there is a good chance you are involved somehow in open source development, or software development in general. One thing lots of people ask me when they know I have lead this sort of projects for a long time is "how do you decide things?". To which I have all sorts of bad answers like: * "It's a consensus thing" * "It happens organically" * "Sometimes it just happens" * "Anarchy!" * "You do what you do" So, now here I have an AWESOME example of FLOSS decision making in action, which is ... all of the above. Some context: Nikola is a static site generator, so it deals with reading and writing textual data from disk. It's also an internationalized project, which supports multilingual sites and translated data. It *also* runs un multiple platforms, like Windows, OSX, Linux, etc. And to make that more fun, it also works on Python 2.7, and 3.3 or later. Which means it has to handle two different models on how to work with unicode data, in the same codebase. And that's not fun. So, we have been floating around the idea of deprecating python 2.7. And so, when ``s2hc_johan`` walks in with a unicode problem... .. raw:: html
    14:23:16 <s2hc_johan> I don't have a site with sections, but I tested it for the other case
    14:35:42 <s2hc_johan> strange it worked for a while broken again, probably because I've got åäö in it now.
    14:35:45 <s2hc_johan> https://github.com/getnikola/plugins/blob/master/v7/recent_posts_json/recent_posts_json.py#L134
    14:36:17 <s2hc_johan> if you wrap data with unicode it works, but I'm not sure that works in python3
    14:36:37 <ChrisWarrick> s2hc_johan: how do you wrap it with unicode?
    14:36:48 <s2hc_johan> unicode(data)
    14:37:05 <s2hc_johan> but is that valid in  python3?
    14:37:11 <ChrisWarrick> s2hc_johan: this is wrong on so many levels
    14:37:16 <ChrisWarrick> s2hc_johan: please don’t do that, ever
    14:37:48 <ChrisWarrick> s2hc_johan: This won’t work in Python 3 either.  You must have an actual encoding, and use the decode method.   try: foo = foo.decode('utf-8'); except AttributeError: foo = foo  # python 3
    14:38:02 <s2hc_johan> what do you mean, that is like my standard when I get strnage data in, undoce(data) data.encode(whatever) data.decode(whatever) :)
    14:38:23 <s2hc_johan> one of them ussually work
    14:39:22 <ChrisWarrick> s2hc_johan: unicode() assumes ASCII, it never works right
    14:39:32 <s2hc_johan> true
    14:39:40 <ChrisWarrick> s2hc_johan: encode/decode with a specified encoding is fine
    14:40:00 <ChrisWarrick> s2hc_johan: but you might need a try/except for Python 3 if it could have Unicode data already
    14:40:16 <s2hc_johan> I'm a bit confused in this case since the output comes from json.dumps
    14:40:34 <s2hc_johan> thought that would produce a unicode object
    14:40:51 <ChrisWarrick> s2hc_johan: not necessarily on python 2
    14:41:05 <ralsina_> if isinstance(thing, utils.str_bytes): thing=thing.decode('utf8')
    14:41:15 <ralsina_> that works in py2 and py3
    14:42:12 <ChrisWarrick> easier to ask for forgiveness imo
    14:43:07 <ralsina_> maybe we should have helpers in utils enforce_unicode and enforce_bytes
    14:43:13 -GitHub[nikola]:#nikola- [nikola] Aeyoun pushed 1 new commit to feed-previewimage: http://git.io/vnqek
    14:43:13 -GitHub[nikola]:#nikola- nikola/feed-previewimage 4b79e20 Daniel Aleksandersen: Deprecated RSS_READ_MORE_LINK and RSS_LINKS_APPEND_QUERY...
    14:44:58 <Aeyoun> Or upgrade to Py3.
    14:45:11 <ChrisWarrick> ++
    14:45:47 <Aeyoun> Unicode in Py27 is a nightmare. It tries as hard as it can to kill you at every turn.
    14:48:09 -travis-ci:#nikola- getnikola/nikola#6426 (feed-previewimage - 4b79e20 : Daniel Aleksandersen): The build is still failing.
    14:48:10 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/c4c69c02db34...4b79e20d1ebc
    14:48:10 -travis-ci:#nikola- Build details: https://travis-ci.org/getnikola/nikola/builds/81026762
    14:48:27 <ralsina_> ok, let's consider py3-only seriously.
    14:48:40 <ralsina_> 1) Is there any distro commonly used with py3 < 3.3 ?
    14:48:55 <ralsina_> 2) Do we just stop using py2, or we deprecate slowly?
    14:49:15 <ralsina_> 3) Do we just start doing py3-only code, or we actively de-hack the codebase?
    14:49:21 <ralsina_> That's my 3 questions :-)
    14:50:13 <SteveDrees> Unicode is a nightmare
    14:50:53 <SteveDrees> different python versions just changes where the pain point is
    14:50:53 <s2hc_johan> which one is better isinstance... or hasattr('decode', ..)
    14:51:02 <ralsina_> isinstance
    14:51:08 <s2hc_johan> oki then
    14:51:10 <ralsina_> hasattr is evil in itself
    14:51:26 <s2hc_johan> just going to feed the kids then I'll make another pr
    14:51:28 -GitHub[nikola]:#nikola- [nikola] Aeyoun pushed 1 new commit to feed-previewimage: http://git.io/vnqJ2
    14:51:28 -GitHub[nikola]:#nikola- nikola/feed-previewimage 4c950ac Daniel Aleksandersen: flake8
    14:52:13 <Aeyoun> ralsina_: user survey? pip download data?
    14:52:33 <gour> ralsina_: create some poll at website/mailing-list about it?
    14:53:18 <ralsina_> dude, I offered free shirts and I got only 10 requests ;-)
    14:53:30 <ralsina_> so, how many answers do you expect about that sort of thing?
    14:53:43 * gour thought shirts are jsut for devs :-(
    14:53:47 <Aeyoun> ralsina_: release a unchanged version on pip that is flagged as py3 only. see how many downlaod it versus previous version in same amount of time.
    14:53:51 <ralsina_> gour: go add yourself dude
    14:54:18 <ralsina_> gour: TO THE SHIRT LIST! I just notced that sounded very rude :-)
    14:54:43 <gour> ralsina_: where it is?
    14:54:43 <Aeyoun> ralsina_: or one py27 version number and and one version py3 only version number at the same time.
    14:55:17 <ralsina_> gour: https://docs.google.com/forms/d/18YFwdgukmpkjr5b8FGEKL0arxPePuLHNsuEa-Gl80D8/viewform?c=0&w=1
    14:55:17 <gour> found it
    14:56:00 <gour> ralsina_: wonder if xxl is too large or xl is enough
    14:56:00 <Aeyoun> ralsina_: american or european sizes by the by?
    14:56:03 <ralsina_> Aeyoun: that reflects how many people use py2.7 by reflex. I know *i* do because it's "python" and not "python3"
    14:56:20 <ralsina_> Aeyoun: no idea about sizes to be honest... probably american
    14:56:21 <Aeyoun> American sizes are … a big bigger. I’m probably a XS/S american but M european. :P
    14:56:28 <Aeyoun> *bit bigger
    14:56:39 <gour> ok
    14:56:57 * gour submitted request
    14:57:17 <ralsina_> So, what I would prefer to do is make people use py3 if they can. And it seems to me that pretty much everyone can, regardless of whether they still use py2 by defect.
    14:57:26 <ralsina_> by default*, spanishism leaked there.
    14:57:52 <ChrisWarrick> technically, using py2 is a defect
    14:57:59 <ralsina_> So, if we all agree that most users *could* run nikola in py3... then let's do it.
    14:58:02 <Aeyoun> Agreed.
    14:58:15 <gour> sites won't stop working :-)
    14:58:26 <Aeyoun> ralsina_: act on data not dev agreement?
    14:58:42 <ChrisWarrick> guess we could change our docs/webiste to highlight 3.x
    14:58:59 <ralsina_> Aeyoun: the only data we'd need is to know how many people have py2.7 and no py3.3
    14:59:14 <ralsina_> not how many are *using* 2.7 instead of 3.3
    14:59:38 <ChrisWarrick> micro-survey via ml?
    14:59:39 <ralsina_> How about: let's announce that, unless lots of people complaint, we deprecate py2 by end of october
    14:59:45 -travis-ci:#nikola- getnikola/nikola#6429 (feed-previewimage - 4c950ac : Daniel Aleksandersen): The build was fixed.
    14:59:46 -travis-ci:#nikola- Change view: https://github.com/getnikola/nikola/compare/4b79e20d1ebc...4c950ac5e52e
    14:59:46 -travis-ci:#nikola- Build details: https://travis-ci.org/getnikola/nikola/builds/81028389
    14:59:47 <Aeyoun> Mac is shipping with Py2.7 and no Py3. BUT MacPorts and Homebrew offer painfree Py3 installs.
    14:59:58 <ralsina_> ok, mac is a good point
    15:00:25 <ChrisWarrick> it’s not like we have Homebrew/MacPorts/Fink-based install instructions for them…
    15:00:27 <Aeyoun> ralsina_: we could add a deprecation message every time `nikola` is run and ask people to bitch in a bug?
    15:00:32 <Aeyoun> ChrisWarrick: hehe. ;)
    15:00:50 <ralsina_> "I see you have python3 installed but I am running on 2.7 ... dude, what's wrong with you?"
    15:00:51 <Aeyoun> Or maybe once per 24 hour rather  than every time its run.
    15:01:00 <ralsina_> doit timed tasks :-)
    15:01:12 <Aeyoun> ralsina_: "Don’t get in the way of progress! Upgrade to Py3 and save a developer’s mind today!"
    15:01:32 <ralsina_> "niec unicode you have there, would be a shame something happened to it.. switch to python 3!"
    15:01:39 <ChrisWarrick> ralsina_: hey, let’s start with a Google Docs survey on the ML.  One question: what Python version and OS are you using for Nikola? 2.7/3.3/3.4/3.5; Windows/OS X/[other: linux/bsd distro]
    15:01:57 <gour> "Free t-shirt foreveryone switching from py2.7 to py3.3"
    15:01:58 <ChrisWarrick> ralsina_: Just don’t require a Google account like you did last time.
    15:02:00 <ralsina_> Second question: "Do you have python 3.3 or later installed?"
    15:02:03 <Aeyoun> How much code can be removed with dropping Py27? Lowers maintenance cost and increases performance. That is also an important datapoint.
    15:02:11 <ralsina_> ChrisWarrick: I needed to know who was asking for the shirt :-)
    15:02:21 <ChrisWarrick> ralsina_: good point
    15:02:25 <ralsina_> Aeyoun: not all that much, really
    15:02:47 <ChrisWarrick> Aeyoun: it would need to start with a huge rewrite to remove all of our pointers in nikola.utils
    15:03:00 <ralsina_> Aeyoun: there are a number of tiny hacks, which were a pain to get right but they always amount to one if and/or one decode :-)
    15:03:26 <ralsina_> We can just turn a bunch of helpers in utils into noops
    15:04:52 <gour> py3-only nikola is going to become v8?
    15:05:15 <Aeyoun> gour: seems like a likely outcome. you’re following the discussion live.
    15:06:34 <ChrisWarrick> if we do v8, we’ll have to merge the early tasks garbage
    15:07:03 <ralsina_> Is it technically backwards-incompatible if we just stop working on py2.7?
    15:07:21 <ralsina_> gour: welcome to open source software: behind the code.
    15:07:30 <gour> ralsina_: :-)
    15:07:35 <Aeyoun> Someone call in a documentary crew!
    15:07:43 <ralsina_> Aeyoun: we have logs!
    15:07:51 <Aeyoun> Oh, wait. This is already logged for prosperity.
    15:07:57 <ralsina_> I am totally posting this somewhere as "this is how decisions are made in FLOSS"
    15:08:40 <ralsina_> Ok, who creates the poll and who posts it in the blog, and who makes sure it appears on planet, and who sends it to the list?
    15:08:49 <ralsina_> I would do it but I have work to do :)
    15:08:51 <ChrisWarrick> ralsina_: I’ll do it
    15:08:57 <ralsina_> ChrisWarrick: you rock dude!
    15:09:01 <ChrisWarrick> ralsina_: should be really simple
    15:09:03 <ralsina_> Ok, we have a plan!
    15:09:17 <ralsina_> Let's consider the poll results in ... a week?
    15:09:25 <Aeyoun> Let the logs show we’re all in favor of this plan of action. ;-)
    15:09:29 <ralsina_> aye
    15:09:51 <ralsina_> Also: can I do the "shame on you" thing on nikola build? It sounds like fun :-)
    15:10:27 <ChrisWarrick> ralsina_: for the python version question: radiobox vs checkbox?
    15:10:28 <gour> ralsina_: you can mention that Nikola (Tesla) was always for innovation ;)
    15:10:44 <Aeyoun> "You’re using FIVE YEAR OLD SOFTWARE. Update your system."
    15:11:00 <ralsina_> Aeyoun: I am totally getting at least 5 different comments there
    15:11:01 <Aeyoun> https://en.wikipedia.org/wiki/History_of_Python#Version_release_dates
    15:11:05 <ralsina_> ChrisWarrick: checkbox... maybe 2?
    15:11:23 <ralsina_> ChrisWarrick: one for python version, one for operating system
    15:11:32 <ChrisWarrick> ralsina_: ?
    15:11:38 <ralsina_> ChrisWarrick: two questions
    15:11:54 <ChrisWarrick> ralsina_: there will even be three questions (py2/3 used, OS, has py3)
    15:11:57 <ChrisWarrick> ralsina_: and checkboxes it is
    15:12:02 <ralsina_> right
    15:12:05 <ralsina_> awesome
    15:14:44 <ralsina_> Copied / Pasted for posterity
    
There you go, half an hour later, we have a plan to (maybe) deprecate it. Now go vote here: `Should Nikola support python2.7? Gives us data to decide! `__