I have a few small projects where I need a serv­er as back­end. That means I need to run a server, which usu­al­ly means I need to do a lot of stuff. BUT these are as­ton­ish­ing­ly sim­ple back­end­s. Usu­al­ly just one end­point, which does one thing.

For ex­am­ple, con­sid­er nom­bres a web­site where you can ex­am­ine his­tor­i­cal in­for­ma­tion about names in Ar­genti­na. Like, how has the pop­u­lar­i­ty of the name "Juan" changed over time?

Like this:

That is lit­er­al­ly one func­tion that takes as ar­gu­ment names, does a cou­ple of queries to a database, builds a chart and re­turns that. De­ploy­ing that should not re­quire me set­ting up in­fra­struc­ture spe­cial­ly be­cause I have like 5 or 10 of those and they are ac­cessed 10 times a day or so.

If I were to use "the cloud" the so­lu­tion would be to use AWS Lamb­da, or the sim­i­lar clones in Azure or Google Cloud. But I don't want to pay for things, so I looked for a way to do that in my own server, which I al­ready have and has more than enough pow­er to han­dle it.

So, I did it! I used faasd which lead to a whole slew of prob­lems that you can see in this post.

Ba­si­cal­ly faasd hates shar­ing the ma­chine it's in with any­thing else, so I seg­re­gat­ed it to a VM us­ing Ig­nite. Now, I am con­sid­er­ing mov­ing to a new­er, more pow­er­ful serv­er (A Radxa Rock 5C) and I look at set­ting up Ig­nite and ... it's ob­so­lete.

It says the re­place­ment is Flint­lock­... which is "on hold", which means aban­doned.

So I try set­ting up a VM us­ing lib­virt, but since this is ar­m64, things are a bit com­pli­cat­ed, so I say, damn, let's just use QE­mu, which works ... as long as I don't use KVM, be­cause it makes the VM su­per flaky.

So I can choose a su­per flaky VM, or a very slow VM, or use ob­so­lete soft­ware, all be­cause I want to run a func­tion that takes a few sec­onds to run and is ac­cessed 10 times a day and faasd is needy.

So, I say­d, what the heck, faasd can't be the on­ly thing. Let's look again!

• Open­Faas: by the same peo­ple as faas­d, needs ku­ber­netes, way overkill
• Fx: looks good and sim­ple, lit­er­al­ly does­n't work.

You don't be­lieve me? Here is what hap­pens when I fol­low the in­struc­tions to run fx, on a nor­mal x86 ma­chine:

> curl -o- https://raw.githubusercontent.com/metrue/fx/master/scripts/install.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1471  100  1471    0     0   4138      0 --:--:-- --:--:-- --:--:--  4143
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  8635  100  8635    0     0  17948      0 --:--:-- --:--:-- --:--:-- 17914
Downloading fx from https://github.com/metrue/fx/releases/download/0.9.48-alpha.d91a7a0/fx_0.9.48-alpha.d91a7a0_Tux_64-
Download complete, saved to /home/ralsina/fx/fx.tar.gz
Installing fx to /home/ralsina/fx
fx
fx installed successfully at /home/ralsina/fx
fx version 0.9.48
Cleaning up /home/ralsina/fx/fx.tar.gz

> cat > func.js
module.exports = (ctx) => {
  ctx.body = 'hello world'
}

> ./fx up -p 8080 func.js
2024/06/28 10:52:54  info provisioning localhost ...
*****************
exit status 125
*****************

I spent an hour or so on it, it just does­n't work.

What else is out there?

• Apache Open­Whisk looks promis­ing, I will try it out nex­t, need to see if it works in my re­source-lim­it­ed serv­er.

If it does­n't work out I swear I am rolling my own.

UP­DATE: Looks like I am rolling my own:

I have a server, her name is Pinky

Pinky does a lot of things but pinky has one prob­lem: Pinky is to­tal­ly hand-­made. Ev­ery­thing in it has been in­stalled by hand, con­fig­ured by hand, and main­tained by hand. This is ok.

I mean, it's ok, un­til it's not ok. It has back­ups and ev­ery­thing, but when a chance presents to, for ex­am­ple, move to a new server, be­cause I just got a nice new com­put­er ... I would need to do ev­ery­thing by hand again.

So, let's fix this us­ing tech­nol­o­gy. I have known about an­si­ble for a long time, I have used things like an­si­ble. I have used pack­er, and salt, and pup­pet, and (re­lat­ed) dock­er, and ku­ber­netes, and ter­rafor­m, and cloud­for­ma­tion, and chef, and ... you get the idea.

But I have nev­er used an­si­ble!

So, here's my plan:

• I will start do­ing an­si­ble play­books for pinky.
• Since an­si­ble is idem­po­ten­t, I can run the play­books on pinky and noth­ing should change.
• I can al­so run them on the new server, and ev­ery­thing should be set up.
• At some point the new serv­er will be suf­fi­cient­ly pinky-­like and I can switch.

So, what is ansible?

In non-tech­ni­cal terms: An­si­ble is a tool to change things on ma­chines. An­si­ble can:

• Set­up a us­er
• Copy a file
• In­stall a pack­age
• Con­fig­ure a thing
• En­able a ser­vice
• Run a com­mand

And so on.

Ad­di­tion­al­ly:

• It will on­ly do things that need to be done.
• It will do things in the re­quest­ed or­der.
• It will do things in mul­ti­ple ma­chines.

First: inventory

The first thing I need to do is to tell an­si­ble where to run things. This is done us­ing an in­ven­to­ry file. The in­ven­to­ry file is a list of ma­chi­nes, and groups of ma­chi­nes, that an­si­ble can run things on.

Mine is very sim­ple, a file called hosts in the same di­rec­to­ry as the play­book:

[servers]
pinky ansible_user=ralsina
rocky ansible_user=rock

[servers:vars]
ansible_connection=ssh 

This defines two machines, called pinky (current server) and rocky (new server). Since rocky is still in pretty much brand new shape it has only the default user it came with, called rock. I have logged into it and done some things ansible needs:

• En­abled ssh
• Made it so my per­son­al ma­chine where an­si­ble runs can log in with­out a pass­word
• In­stalled python
• Made rock a sudoer so it can run commands as root using sudo

So, I tell ansible I can log in as ralsina in pinky and as rock in rocky, in both cases using ssh.

First playbook

I want to be able to log into these machines using my user ralsina and my ssh key. So, I will create a playbook that does that. Additionally, I want my shell fish and my prompt starship to be installed and enabled.

A play­book is just a YAML file that lists tasks to be done. We start with some gener­ic stuff like "what ma­chines to run this on" and "how do I be­come root?"

# Setup my user with some QoL packages and settings
- name: Basic Setup
  hosts: servers
  become_method: ansible.builtin.sudo
  tasks:

And then guess what? Tasks. Each task is a thing to do. Here's the first one:

    - name: Install some packages
      become: true
      ansible.builtin.package:
        name:
          - git
          - vim
          - htop
          - fish
          - rsync
          - restic
          - vim
        state: present

There "an­si­ble.builtin.­pack­age" is a mod­ule that in­stalls pack­ages. An­si­ble has tons of mod­ules, and they are all doc­u­ment­ed in the an­si­ble doc­u­men­ta­tion.

Each task can take parameters, which depend on what the module does. In this case, as you can see there's a list of packages to install, and the state means I want them to be there.

BUT while rocky is a Debian, pinky is arch (btw), so there is at least one package I need to install only in rocky. That's the next task:

    - name: Install Debian-specific packages
      become: true
      when: ansible_os_family == 'Debian'
      ansible.builtin.apt:
        name:
          - ncurses-term
        state: present

Same thing, ex­cep­t:

• It uses a debian-specific package thing, called ansible.builtin.apt
• It has a when clause that only runs the task if the OS family is Debian.

What nex­t? Well, more tasks! Here they are, you can un­der­stand what each one does by look­ing up the docs for each an­si­ble mod­ule.

    - name: Add the user ralsina
      become: true
      ansible.builtin.user:
        name: ralsina
        create_home: true
        password_lock: true
        shell: /usr/bin/fish
    - name: Authorize ssh
      become: true
      ansible.posix.authorized_key:
        user: ralsina
        state: present
        key: "{{ lookup('file', '/home/ralsina/.ssh/id_rsa.pub') }}"
    - name: Make ralsina a sudoer
      become: true
      community.general.sudoers:
        name: ralsina
        user: ralsina
        state: present
        commands: ALL
        nopassword: true
    - name: Create fish config directory
      ansible.builtin.file:
        path: /home/ralsina/.config/fish/conf.d
        recurse: true
        state: directory
        mode: '0755'
    - name: Get starship installer
      ansible.builtin.get_url:
        url: https://starship.rs/install.sh
        dest: /tmp/starship.sh
        mode: '0755'
    - name: Install starship
      become: true
      ansible.builtin.command:
        cmd: sh /tmp/starship.sh -y
        creates: /usr/local/bin/starship
    - name: Enable starship
      ansible.builtin.copy:
        dest: /home/ralsina/.config/fish/conf.d/starship.fish
        mode: '0644'
        content: |
          starship init fish | source

And that's it! I can run this playbook using ansible-playbook -i hosts setup_user.yml and it will do all those things on both pinky and rocky, if needed:

> ansible-playbook -i hosts setup_user.yml

PLAY [Basic Setup] ******************************

TASK [Gathering Facts] **************************
ok: [rocky]
ok: [pinky]

TASK [Install some packages] ********************
ok: [rocky]
ok: [pinky]

TASK [Install Debian-specific packages] *********
skipping: [pinky]
ok: [rocky]

TASK [Add the user ralsina] *********************
ok: [rocky]
ok: [pinky]

TASK [Authorize ssh] ****************************
ok: [rocky]
ok: [pinky]

TASK [Make ralsina a sudoer] ********************
ok: [rocky]
ok: [pinky]

TASK [Create fish config directory] *************
changed: [rocky]
changed: [pinky]

TASK [Get starship installer] *******************
ok: [rocky]
ok: [pinky]

TASK [Install starship] *************************
ok: [rocky]
ok: [pinky]

TASK [Enable starship] **************************
changed: [rocky]
changed: [pinky]

PLAY RECAP **************************************
pinky : ok=9    changed=2    unreachable=0    failed=0    skipped=1 
        rescued=0    ignored=0
rocky : ok=10   changed=2    unreachable=0    failed=0    skipped=0 
        rescued=0    ignored=0

If you look care­ful­ly you can see rocky ran one more task, and pinky skipped one (the de­bian-spe­cif­ic pack­age in­stal­la­tion), and that on­ly two things got ac­tu­al­ly ex­e­cut­ed on each ma­chine.

I could run this a dozen times from now on, and it would not do any­thing.

Did it work?

Sure, I can ssh into rocky and everything is nice:

> ssh rocky
Linux rock-5c 5.10.110-37-rockchip #27a257394 SMP Thu May 23 02:38:59 UTC 2024 aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jun 26 15:32:33 2024 from 100.73.196.129
Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish

ralsina in 🌐 rock-5c in ~ 

There is a star­ship promp­t, and I can use fish. And I can su­do. Nice!

I can now change the inventory so rocky also uses the ralsina user and delete the rock user.

Next steps

There is a lot more to an­si­ble, specif­i­cal­ly roles but this is al­ready enough to get use­ful things done, and hope­ful­ly it will be use­ful to you too.

I run a very cheap home serv­er. How cheap? Very, very cheap. Sub Rasp­ber­ry Pi 4 cheap.

It runs a ton of ser­vices, and it al­so works as my "Func­tion­s" serv­er.

What is a func­tions server?

It's a cheap man's AWS Lamb­da which al­lows me to cre­ate small "ser­vices" like this, and de­ploy them au­to­mat­i­cal­ly. It's re­al­ly a game chang­er for sim­ple code avail­abil­i­ty, and it's my favourite way to share func­tion­al­i­ty with oth­er­s.

But some­times a ser­vice re­lies on a 3rd par­ty API, and it needs things like a to­ken to be avail­able. Faasd sup­ports this us­ing their se­cret API. You cre­ate a se­cret like this:

faas-cli secret create whatever

And when you declare in your functions.yml that your function needs a secret:

myfunc:
  lang: python3-fastapi
  handler: ./myfunc
  secrets:
  - whatever

Your code reads a file in /var/openfaas/secrets/whatever and that's all, there is a secret on the server, your app can see it, it's not in the app's code, all good.

Ex­cept ... what hap­pens if you need to re­de­ploy faas­d? You will need to cre­ate the se­cret again! So you need to keep the se­cret some­where.

So­lu­tion: pass

I already use pass to keep many passwords, it's easy to also put secrets there. It manages everything using a git repo, so it's a known factor. You can even do things like add them all inside a faasd/ folder and then recreate them using scripts, like this:

pass faasd/whatever | faas-cli secret create whatever

pass will ask for your mas­ter passphrase, se­cret cre­at­ed. You can even pub­lish your pass re­po since ev­ery­thing in it is en­crypt­ed with gpg, so no­body can re­al­ly read it (don't do that).

So, this so­lu­tion us­es:

• pass
• gpg
• git
• faasd
• shell
• what­ev­er lan­guage and frame­work you use in your code

And ev­ery­thing is seam­less!

I think this is a nice ex­am­ple of how ran­dom tools can con­nect with each oth­er be­cause they all fol­low the unix con­ven­tion about mov­ing things around as tex­t.

Just ran in­to this video where the host ex­plains how he's us­ing a home serv­er and how he made it use un­der 23W, and the steps he took to make it qui­et and ef­fi­cien­t.

So, know­ing that our servers are not com­pa­ra­ble I want­ed to check how much pow­er my serv­er used.

Yes, my serv­er is pret­ty un­usu­al, you can read about it here: 1 2 3 4 5

First let's look at MAX­I­MUM pow­er draw for all com­po­nents.

• Com­put­er (Radxa Ze­ro): 900 mA at 5V
• HD­D: 500 mA at 5V (x2)
• Net­work USB adapter: 500 mA at 5V

There is al­so a USB 3.0 hub there but the pow­er us­age is neg­li­gi­ble, prob­a­bly be­low 100 mA at 5V.

So, the max­i­mum pow­er us­age is ~2400 mA at 5V, which is about 12 watts.

When more or less idle, mea­sured with a USB thingie, the Radxa Ze­ro us­es ~200 mA and the Net­work adapter nev­er seems to go above 150 mA, so a low­er bound is maybe about ~1300 mA, or about 6 watts.

And then there's cost. My whole sys­tem (which does ev­ery­thing I want nice­ly AFAIC­S) costs un­der 100 dol­lars. And my pow­er bill from it is, if it runs full throt­tle ALL THE TIME (it does­n't) ... $87

Mind you, that's 87 ar­gen­tini­an pe­sos, or be­tween 50 and 25 USD cents, de­pend­ing on your ex­change rate.

I think I'll man­age.

Be­cause I want to be able to de­ploy ran­dom python code eas­i­ly to my own server, I have set­up a "Func­tion as a Ser­vice" thing, called faasd (think of it as poor peo­ple's AWS lamb­da). More de­tails on how, why and how it turned out will come in the fu­ture. BUT: this ex­plains how to fix the un­avoid­able headache CORS will give you.

Sit­u­a­tion:

• The Faasd serv­er runs in some ma­chine, which is prox­­ied by a Ng­inx serv­er avail­able at http­s://­­faas­d.ral­si­­na.me

• Apps are just HTML pages some­where in­­­side ei­ther http­s://ral­si­­na.me or some oth­­er sim­i­lar do­­main.

What will happen?

You will set­up your func­tion, test it out us­ing curl, be hap­py it work­s, then set it up in your web app and get an er­ror in the con­sole about how CORS is not al­low­ing the re­quest.

What is CORS and why is it annoying me?

CORS is a way for a ser­vice liv­ing in a cer­tain URL to say which oth­er URLs are al­lowed to call it. So, if the app are in, say, http­s://nom­bres.ralsi­na.me and the func­tion lives in http­s://­faas.ralsi­na.me then the ORI­GIN for the app is not the same as the ORI­GIN for the func­tion, so this is a "Cross Ori­gin Re­quest" and you are try­ing to do "Cross Ori­gin Re­source Shar­ing" (CORS) and the brows­er won't let you.

How do I fix it?

There are a num­ber of fix­es you can try, but they all come down to the same two ba­sic ap­proach­es:

Option 1

Make it so the re­quest is not cross-­source. To do that, move the func­tion some­how in­to the same URL as the page, and bob's your un­cle.

So, just change the proxy con­fig so nom­bres.ralsi­na.me/­func­tions is prox­ied to the faasd server's /func­tions and change the page to use a re­quest that is not cross-o­rig­in, and that's fixed.

I don't want to do this be­cause I don't want to have to set­up the proxy dif­fer­ent­ly for each ap­p.

Option 2

Have the func­tion re­turn a head­er that says "Ac­cess-­Con­trol-Al­low-O­rig­in: some­thing". That "some­thing" should be the ori­gin of the re­quest (in our ex­am­ple nom­bres.ralsi­na.me) or "*" to say "I don't care".

So, you may say "Fine, I'll just add that head­er in my re­sponse and it will work!". Oh sweet sum­mer child. That will NOT work (at least not in the case of Faas­d)

Why?

Be­cause web browsers don't just make the re­quest they want and then look at the head­er­s. They do a spe­cial pre­flight re­quest, which is some­thing like "Hey, server, if I were to ask you to give me /func­tion­s/what­ev­er from this orig­in, would you give me a CORS per­mis­sion or not?"

That re­quest is done us­ing the OP­TIONS HTTP method, and Faasd (and, to be hon­est, most web frame­work­s) will not process those by pass­ing them to your code.

So, even if your func­tion says CORS is al­lowed, you still will get CORS er­rors.

You can see this if you ex­am­ine your browser's HTTP traf­fic us­ing the de­vel­op­er tool­s. There will be an OP­TIONS pre­flight re­quest, and that one does­n't have the head­er.

So, the eas­i­est thing is to add those in the proxy.

So, in my case, in the prox­y's ng­inx.­con­f, I had to add this in "the right place":

  add_header 'Access-Control-Allow-Origin' '*';

What is the right place will vary de­pend­ing on how you have con­fig­ured things. But hey, there you go.