Skip to main content

Ralsina.Me — Roberto Alsina's website

Posts about python (old posts, page 97)

I am trying to write a Python book

Once up­on a time, I tried to write a book. It did not end well. I was try­ing to dump a whole lot of knowl­edge at once. Knowl­edge I did not re­al­ly have, to be hon­est. When I look at that book I see a failed thing.

So, of course, many years lat­er, I am try­ing again, but with the lessons learned in my mind.

  • It will be a smal­l­­er book.

  • I am not al­­so writ­ing a whole tool chain for it.

  • It will be about things I know.

So, what is it?

The tem­po­rary ti­tle, right now, is some­thing like "Box­es: your sec­ond Python book". It says your sec­ond Python book be­cause you do need a work­ing knowl­edge of Python syn­tax as pro­vid­ed by the of­fi­cial Python Tu­to­ri­al, but not much else. When there is a par­tic­u­lar­ly hairy piece of code it may link to the tu­to­ri­al or the ref­er­ence or some­thing.

The "idea" of the book is to bridge a gap that ex­ists be­tween know­ing the ba­sics of read­ing and writ­ing a lan­guage (spe­cial­ly if it's your first!) and be­ing able to ef­fec­tive­ly us­ing it to cre­ate a use­ful projec­t.

It fol­lows the growth of "Box­es", a sim­plis­tic text lay­out en­gine, from a vague idea to a ful­ly work­ing, use­ful, test­ed, and pub­lished piece of soft­ware.

It's not there yet, but it's about 25% of the way there.

You can read it here: http­s://ralsi­na.git­lab.io/box­es-­book/ and the sources are at http­s://git­lab.­com/ralsi­na/box­es-­book

Com­ments much ap­pre­ci­at­ed!

Lois Lane, Reporting

So, 9 years ago I wrote a post about how I would love a tool that took a JSON da­ta file, a Mako tem­plate, and gen­er­at­ed a re­port us­ing re­Struc­tured Tex­t.

If you don't like that, pre­tend it says YAM­L, Jin­ja2 and Mark­down. Any­way, same idea. Re­ports are not some crazy dif­fi­cult thing, un­less you have very de­mand­ing lay­out or need to add a ton of log­ic.

And hey, if you do need to add a ton of log­ic, you do know python, so how hard can it be to add the miss­ing bit­s?

Well, not very hard. So here it is, 9 years lat­er be­cause I am sit­ting at an au­di­to­ri­um and the guy giv­ing the talk is hav­ing com­put­er prob­lem­s.

Lois Lane Re­ports from PyP­I. and GitHub

New mini-project: Gyro

History

Facu­batis­ta: ralsi­na, yo, vos, cerveza, un lo­cal-wik­i-server-he­cho-en-un-­solo-.py-­con-in­ter­faz-web en tres ho­ras, pen­sa­lo

Facu­batis­ta: ralsi­na, you, me, beer, a lo­cal-wik­i-server-­done-in-one-.py-with­-we­b-in­ter­face in three hours, think about it

/images/gyro-1.thumbnail.png

The next day.

So, I could not get to­geth­er with Facu, but I did sort of write it, and it's Gy­ro. [1]

Technical Details

Gy­ro has two part­s: a very sim­ple back­end, im­ple­ment­ed us­ing San­ic [2] which does a few things:

  • Serve stat­ic files out of _stat­ic/

  • Serve tem­plat­ed mark­down out of pages/

  • Save mark­down to pages/

  • Keep an in­dex of file con­tents up­dat­ed in _stat­ic/in­dex.js

The oth­er part is a web­page, im­plem­nt­ed us­ing Boot­strap [3] and JQuery [4]. That page can:

  • Show mark­­down, us­ing Show­­down [5]

  • Ed­it mark­­down, us­ing Sim­­pleMDE [6]

  • Search in your pages us­ing Lunr [7]

And that's it. Open the site on any URL that doesn't start with _static and contains only letters and numbers:

  • http://lo­­cal­host:8000/My­­Page : GOOD

  • http://lo­­cal­host:8000/My­Dir/My­­Page: BAD

  • http://lo­­cal­host:8000/__­­foo­bar__: BAD

At first the page will be sort of empty, but if you edit it and save it, it won't be empty anymore. You can link to other pages (even ones you have not created) using the standard markdown syntax: [go to Foo­Bar](­Foo­Bar)

There is re­al­ly not much else to say about it, if you try it and find bugs, file an is­sue and as usu­al patch­es are wel­come.


How I Learned to Stop Worrying and Love JSON Schema

Intro

This post op­er­ates on a few shared as­sump­tion­s. So, we need to ex­plic­it­ly state them, or oth­er­wise you will read things that are more or less ra­tio­nal but they will ap­pear to be garbage.

  • APIs are good
  • Many APIs are web APIs
  • Many web APIs con­sume and pro­duce JSON
  • JSON is good
  • JSON is bet­ter if you know what will be in it

So, JSON Schema is a way to in­crease the num­ber of times in your life that JSON is bet­ter in that way, there­fore mak­ing you hap­pi­er.

So, let's do a quick in­tro on JSON Schema. You can al­ways read a much longer and sure­ly bet­ter one from which I stole most ex­am­ples at Un­der­stand­ing JSON Schema. lat­er (or right now, it's your time, la­dy, I am not the boss of you).

Schemas

So, a JSON Schema de­scribes your da­ta. Here is the sim­plest schema, that match­es any­thing:

{ }

Scary, uh? Here's a more re­stric­tive one:

{
  "type": "string"
}

That means "a thing, which is a string." So this is valid: "foo" and this isn't 42 Usually, on APIs you exchange JSON objects (dictionaries for you pythonistas), so this is more like you will see in real life:

{
  "type": "object",
  "properties": {
    "street_address": { "type": "string" },
    "city": { "type": "string" },
    "state": { "type": "string" }
  },
  "required": ["street_address", "city", "state"]
}

That means "it's an ob­jec­t", that has in­side it "street_ad­dress", "c­i­ty" and "s­tate", and they are all re­quired.

Let's sup­pose that's all we need to know about schemas. Again, be­fore you ac­tu­al­ly use them in anger you need to go and read Un­der­stand­ing JSON Schema. for now just as­sume there is a thing called a JSON Schema, and that it can be used to de­fine what your da­ta is sup­posed to look like, and that it's de­fined some­thing like we saw here, in JSON. Cool?

Using schemas

Of course schemas are use­less if you don't use them. You will use them as part of the "con­trac­t" your API prom­ises to ful­fil­l. So, now you need to val­i­date things against it. For that, in python, we can use json­schema

It's pret­ty sim­ple! Here is a "ful­l" ex­am­ple.

import jsonschema

schema = {
  "type": "object",
  "properties": {
    "street_address": {"type": "string"},
    "city": {"type": "string"},
    "state": {"type": "string"},
  },
  "required": ["street_address", "city", "state"]
}

jsonschema.validate({
    "street_address": "foo",
    "city": "bar",
    "state": "foobar"
}, schema)

If the da­ta does­n't val­i­date, jsonchema will raise an ex­cep­tion, like this:

>>> jsonschema.validate({
...     "street_address": "foo",
...     "city": "bar",
... }, schema)
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
  File "jsonschema/validators.py", line 541, in validate
    cls(schema, *args, **kwargs).validate(instance)
  File "jsonschema/validators.py", line 130, in validate
    raise error
jsonschema.exceptions.ValidationError: 'state' is a required property

Failed validating 'required' in schema:
    {'properties': {'city': {'type': 'string'},
                    'state': {'type': 'string'},
                    'street_address': {'type': 'string'}},
     'required': ['street_address', 'city', 'state'],
     'type': 'object'}

On instance:
    {'city': 'bar', 'street_address': 'foo'}

Hey, that is a pret­ty nice de­scrip­tion of what is wrong with that da­ta. That is how you use a JSON schema. Now, where would you use it?

Getting value out of schemas

Schemas are use­less if not used. They are worth­less if you don't get val­ue out of us­ing them.

These are some ways they add val­ue to your code:

  • You can use them in your web app end­point, to val­i­date things.
  • You can use them in your client code, to val­i­date you are not send­ing garbage.
  • You can use a fuzzer to feed da­ta that is tech­ni­cal­ly valid to your end­point, and make sure things don't ex­plode in in­ter­est­ing ways.

But here is the most val­ue you can ex­tract of JSON schemas:

You can dis­cuss the con­tract be­tween com­po­nents in un­am­bigu­ous terms and en­force the con­tract once it's in place.

We are de­vs. We dis­cuss via branch­es, and com­ments in code re­view. JSON Schema turns a vague ar­gu­ment about doc­u­men­ta­tion in­to a fac­t-based dis­cus­sion of da­ta. And we are much, much bet­ter at do­ing the lat­ter than we are at do­ing the for­mer. Dis­cuss the con­tract­s.

Since the doc­u­ment de­scrib­ing (this part of) the con­tract is ac­tu­al­ly used as part of the API def­i­ni­tions in the code, that means the doc­u­ment can nev­er be left be­hind. Ev­ery change in the code that changes the con­tract is ob­vi­ous and re­quires an ex­plic­it rene­go­ti­a­tion. You can't break API by ac­ci­den­t, and you can't break API and hope no­body will no­tice. En­force the con­tract­s.

Fi­nal­ly, you can ver­sion the con­trac­t. Use that along with API ver­sion­ing and voilá, you know how to man­age change! Ver­sion your con­tract­s.

  • Dis­cuss your con­tracts
  • En­force your con­tracts
  • Ver­sion your con­tracts

So now you can stop wor­ry­ing and love JSON Schema as well.

Creating Languages For Dummies

Intro

I don't have the usu­al pro­gram­mer's ed­u­ca­tion. I stud­ied math­s, and then dropped out of that, and am most­ly self­-­taugh­t. So, there are some parts of pro­gram­ming I al­ways saw weari­ly, think­ing to my­self that I re­al­ly should go to school to learn them. One re­mark­able such area is pars­ing and im­ple­ment­ing lan­guages.

Well... sure, school is al­ways a good idea, but this is not that hard. In this ar­ti­cle I will ex­plain how to go from noth­ing to a func­tion­ing, ex­ten­si­ble lan­guage, us­ing Python and Py­Pars­ing. If you are as scared of gram­mars, parsers and all that jazz as I used to be, come along, it's pret­ty sim­ple,

Read more…


Contents © 2000-2023 Roberto Alsina