Be­cause yes, all self­-host­ed FaaS so­lu­tions suck this week­end I wrote the be­gin­nings of a new one, called Faa­SO.

Is it go­ing to be great? Prob­a­bly not, but it's go­ing to do ex­act­ly what I need it to do. Be­cause the best part of rein­vent­ing the wheel is that by the sec­ond left el­bow of Kali, this wheel is go­ing to be ex­act­ly the shape I like.

Faa­SO has very strict de­sign con­straints:

1. It needs to be easy to use. I need to be able to write a funko (func­tion in Faa­SO par­lance) in a minute and de­ploy it with one com­mand, and I won't have to con­fig­ure any­thing for that funko to work.
2. It will run in a sin­gle ma­chine, it will de­ploy in a min­ute, and it will be ready to take new de­ploy­ment re­quests right away.
3. It will have some sort of se­cret man­age­ment API
4. It will sup­port mul­ti­ple lan­guages, be­cause I want to use dif­fer­ent lan­guages.
5. It will have very lit­tle mag­ic. It will not lock you in­to need­ing it.
   • You should be able to take a funko and make it a sep­a­rate app in a minute
   • You should be able to con­trol what you are run­n­ing, and how it run­s, and mon­i­­tor it and so on with­­out go­ing through the tool if you wan­t.
6. It will be small. My cur­rent goal is un­der 1500 LOC.
7. It's aimed at de­ploy­ing one ten­ant. It will not pro­tect one funko from a hos­tile funko run­ning in the same sys­tem. It will not pro­tect you from your­self.
8. In the same way, it will be as se­cure as I can make it against ex­ter­nal threat­s, but it's not go­ing to pro­tect you from some­one with ac­cess to the same sys­tem.
9. It will be light. I am writ­ing it in Crys­tal so it's na­tive code and runs with very lim­it­ed de­pen­den­cies and lit­tle over­head.

Can I do all that? Maybe. The cur­rent pro­to­type does about half of what I wan­t, so there is on­ly an­oth­er 90% of the work left :-)

If you want to check the pro­to­type, it's here, I am not look­ing for con­trib­u­tors now be­cause I want a free hand on sud­den re­design.

There is some doc­u­men­ta­tion about how it works here and some brain­dump about the se­cret man­age­ment as well as the ini­tial brain­dump about de­sign

I have a few small projects where I need a serv­er as back­end. That means I need to run a server, which usu­al­ly means I need to do a lot of stuff. BUT these are as­ton­ish­ing­ly sim­ple back­end­s. Usu­al­ly just one end­point, which does one thing.

For ex­am­ple, con­sid­er nom­bres a web­site where you can ex­am­ine his­tor­i­cal in­for­ma­tion about names in Ar­genti­na. Like, how has the pop­u­lar­i­ty of the name "Juan" changed over time?

Like this:

That is lit­er­al­ly one func­tion that takes as ar­gu­ment names, does a cou­ple of queries to a database, builds a chart and re­turns that. De­ploy­ing that should not re­quire me set­ting up in­fra­struc­ture spe­cial­ly be­cause I have like 5 or 10 of those and they are ac­cessed 10 times a day or so.

If I were to use "the cloud" the so­lu­tion would be to use AWS Lamb­da, or the sim­i­lar clones in Azure or Google Cloud. But I don't want to pay for things, so I looked for a way to do that in my own server, which I al­ready have and has more than enough pow­er to han­dle it.

So, I did it! I used faasd which lead to a whole slew of prob­lems that you can see in this post.

Ba­si­cal­ly faasd hates shar­ing the ma­chine it's in with any­thing else, so I seg­re­gat­ed it to a VM us­ing Ig­nite. Now, I am con­sid­er­ing mov­ing to a new­er, more pow­er­ful serv­er (A Radxa Rock 5C) and I look at set­ting up Ig­nite and ... it's ob­so­lete.

It says the re­place­ment is Flint­lock­... which is "on hold", which means aban­doned.

So I try set­ting up a VM us­ing lib­virt, but since this is ar­m64, things are a bit com­pli­cat­ed, so I say, damn, let's just use QE­mu, which works ... as long as I don't use KVM, be­cause it makes the VM su­per flaky.

So I can choose a su­per flaky VM, or a very slow VM, or use ob­so­lete soft­ware, all be­cause I want to run a func­tion that takes a few sec­onds to run and is ac­cessed 10 times a day and faasd is needy.

So, I say­d, what the heck, faasd can't be the on­ly thing. Let's look again!

• Open­Faas: by the same peo­ple as faas­d, needs ku­ber­netes, way overkill
• Fx: looks good and sim­ple, lit­er­al­ly does­n't work.

You don't be­lieve me? Here is what hap­pens when I fol­low the in­struc­tions to run fx, on a nor­mal x86 ma­chine:

> curl -o- https://raw.githubusercontent.com/metrue/fx/master/scripts/install.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1471  100  1471    0     0   4138      0 --:--:-- --:--:-- --:--:--  4143
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  8635  100  8635    0     0  17948      0 --:--:-- --:--:-- --:--:-- 17914
Downloading fx from https://github.com/metrue/fx/releases/download/0.9.48-alpha.d91a7a0/fx_0.9.48-alpha.d91a7a0_Tux_64-
Download complete, saved to /home/ralsina/fx/fx.tar.gz
Installing fx to /home/ralsina/fx
fx
fx installed successfully at /home/ralsina/fx
fx version 0.9.48
Cleaning up /home/ralsina/fx/fx.tar.gz

> cat > func.js
module.exports = (ctx) => {
  ctx.body = 'hello world'
}

> ./fx up -p 8080 func.js
2024/06/28 10:52:54  info provisioning localhost ...
*****************
exit status 125
*****************

I spent an hour or so on it, it just does­n't work.

What else is out there?

• Apache Open­Whisk looks promis­ing, I will try it out nex­t, need to see if it works in my re­source-lim­it­ed serv­er.

If it does­n't work out I swear I am rolling my own.

UP­DATE: Looks like I am rolling my own:

I have an of­fice. It's like a home of­fice but it's in an­oth­er place. It's just mine, so I get the peace and qui­et of a home of­fice but al­so get to go out­side to get there. It's a good set­up.

BUT I have two com­put­ers there. Well, ac­tu­al­ly I have like 10, but I in­ter­act with two. One is my per­son­al desk­top com­put­er, the oth­er is my work lap­top. and I want to use the same pe­riph­er­als in the same way with both of them. That's why I have the world's most com­pli­cat­ed schizo desk­top set­up.

This se­ries of posts will doc­u­ment it, the why and how of it, and the var­i­ous things I've learned along the way, along with mak­ing you want to buy weird chi­nese gad­get­s.

Today: My USB Switch

What is it? A 4x2 switch. That means you can con­nect 4 USB de­vices to it, and it can be con­nect­ed to 2 com­put­er­s.

It's not an ex­pen­sive de­vice, it costs around 6 dol­lars!

How do you use it? It has a but­ton. When you click it, all the de­vices move from one com­put­er to the oth­er.

What's plugged in­to it? My we­b­cam, my mi­cro­phone (not re­al­ly, it's a bit more com­pli­cat­ed, but that's an­oth­er ar­ti­cle) and my head­phones.

The head­phones are ac­tu­al­ly con­nect­ed via a small USB au­dio card be­cause that gives me a vol­ume knob for them (a­gain, about 6 dol­lars).

Why these de­vices? Be­cause they are my "video cal­l" de­vices. If I am do­ing a video call for work, I use them from the work com­put­er, and when I want to do a per­son­al one, I use them from my per­son­al com­put­er.

They switch in about half a sec­ond, and they work fine.

I do not use this for key­board and mouse. I have oth­er, bet­ter so­lu­tions (a­gain, an­oth­er ar­ti­cle)

Phys­i­cal­ly how is it mount­ed? I at­tached to the bot­tom of the desk to my left, next to a hook for the head­set and the arm for the mi­cro­phone. When I want to switch, I just reach down and press the but­ton.

What can fail?

• If you up­date the ker­nel in one of the ma­chines and did­n't re­boot then hot­plug­ging the USB de­vices may fail. Don't do that :-)
• Some op­er­at­ing sys­tems or users may get con­fused by au­dio de­vices pop­ping in­to ex­is­tence and dis­ap­pear­ing.
• If you for­get to switch noth­ing works and you will be the per­son in the video call say­ing "can you hear me?"

I have a server, her name is Pinky

Pinky does a lot of things but pinky has one prob­lem: Pinky is to­tal­ly hand-­made. Ev­ery­thing in it has been in­stalled by hand, con­fig­ured by hand, and main­tained by hand. This is ok.

I mean, it's ok, un­til it's not ok. It has back­ups and ev­ery­thing, but when a chance presents to, for ex­am­ple, move to a new server, be­cause I just got a nice new com­put­er ... I would need to do ev­ery­thing by hand again.

So, let's fix this us­ing tech­nol­o­gy. I have known about an­si­ble for a long time, I have used things like an­si­ble. I have used pack­er, and salt, and pup­pet, and (re­lat­ed) dock­er, and ku­ber­netes, and ter­rafor­m, and cloud­for­ma­tion, and chef, and ... you get the idea.

But I have nev­er used an­si­ble!

So, here's my plan:

• I will start do­ing an­si­ble play­books for pinky.
• Since an­si­ble is idem­po­ten­t, I can run the play­books on pinky and noth­ing should change.
• I can al­so run them on the new server, and ev­ery­thing should be set up.
• At some point the new serv­er will be suf­fi­cient­ly pinky-­like and I can switch.

So, what is ansible?

In non-tech­ni­cal terms: An­si­ble is a tool to change things on ma­chines. An­si­ble can:

• Set­up a us­er
• Copy a file
• In­stall a pack­age
• Con­fig­ure a thing
• En­able a ser­vice
• Run a com­mand

And so on.

Ad­di­tion­al­ly:

• It will on­ly do things that need to be done.
• It will do things in the re­quest­ed or­der.
• It will do things in mul­ti­ple ma­chines.

First: inventory

The first thing I need to do is to tell an­si­ble where to run things. This is done us­ing an in­ven­to­ry file. The in­ven­to­ry file is a list of ma­chi­nes, and groups of ma­chi­nes, that an­si­ble can run things on.

Mine is very sim­ple, a file called hosts in the same di­rec­to­ry as the play­book:

[servers]
pinky ansible_user=ralsina
rocky ansible_user=rock

[servers:vars]
ansible_connection=ssh 

This defines two machines, called pinky (current server) and rocky (new server). Since rocky is still in pretty much brand new shape it has only the default user it came with, called rock. I have logged into it and done some things ansible needs:

• En­abled ssh
• Made it so my per­son­al ma­chine where an­si­ble runs can log in with­out a pass­word
• In­stalled python
• Made rock a sudoer so it can run commands as root using sudo

So, I tell ansible I can log in as ralsina in pinky and as rock in rocky, in both cases using ssh.

First playbook

I want to be able to log into these machines using my user ralsina and my ssh key. So, I will create a playbook that does that. Additionally, I want my shell fish and my prompt starship to be installed and enabled.

A play­book is just a YAML file that lists tasks to be done. We start with some gener­ic stuff like "what ma­chines to run this on" and "how do I be­come root?"

# Setup my user with some QoL packages and settings
- name: Basic Setup
  hosts: servers
  become_method: ansible.builtin.sudo
  tasks:

And then guess what? Tasks. Each task is a thing to do. Here's the first one:

    - name: Install some packages
      become: true
      ansible.builtin.package:
        name:
          - git
          - vim
          - htop
          - fish
          - rsync
          - restic
          - vim
        state: present

There "an­si­ble.builtin.­pack­age" is a mod­ule that in­stalls pack­ages. An­si­ble has tons of mod­ules, and they are all doc­u­ment­ed in the an­si­ble doc­u­men­ta­tion.

Each task can take parameters, which depend on what the module does. In this case, as you can see there's a list of packages to install, and the state means I want them to be there.

BUT while rocky is a Debian, pinky is arch (btw), so there is at least one package I need to install only in rocky. That's the next task:

    - name: Install Debian-specific packages
      become: true
      when: ansible_os_family == 'Debian'
      ansible.builtin.apt:
        name:
          - ncurses-term
        state: present

Same thing, ex­cep­t:

• It uses a debian-specific package thing, called ansible.builtin.apt
• It has a when clause that only runs the task if the OS family is Debian.

What nex­t? Well, more tasks! Here they are, you can un­der­stand what each one does by look­ing up the docs for each an­si­ble mod­ule.

    - name: Add the user ralsina
      become: true
      ansible.builtin.user:
        name: ralsina
        create_home: true
        password_lock: true
        shell: /usr/bin/fish
    - name: Authorize ssh
      become: true
      ansible.posix.authorized_key:
        user: ralsina
        state: present
        key: "{{ lookup('file', '/home/ralsina/.ssh/id_rsa.pub') }}"
    - name: Make ralsina a sudoer
      become: true
      community.general.sudoers:
        name: ralsina
        user: ralsina
        state: present
        commands: ALL
        nopassword: true
    - name: Create fish config directory
      ansible.builtin.file:
        path: /home/ralsina/.config/fish/conf.d
        recurse: true
        state: directory
        mode: '0755'
    - name: Get starship installer
      ansible.builtin.get_url:
        url: https://starship.rs/install.sh
        dest: /tmp/starship.sh
        mode: '0755'
    - name: Install starship
      become: true
      ansible.builtin.command:
        cmd: sh /tmp/starship.sh -y
        creates: /usr/local/bin/starship
    - name: Enable starship
      ansible.builtin.copy:
        dest: /home/ralsina/.config/fish/conf.d/starship.fish
        mode: '0644'
        content: |
          starship init fish | source

And that's it! I can run this playbook using ansible-playbook -i hosts setup_user.yml and it will do all those things on both pinky and rocky, if needed:

> ansible-playbook -i hosts setup_user.yml

PLAY [Basic Setup] ******************************

TASK [Gathering Facts] **************************
ok: [rocky]
ok: [pinky]

TASK [Install some packages] ********************
ok: [rocky]
ok: [pinky]

TASK [Install Debian-specific packages] *********
skipping: [pinky]
ok: [rocky]

TASK [Add the user ralsina] *********************
ok: [rocky]
ok: [pinky]

TASK [Authorize ssh] ****************************
ok: [rocky]
ok: [pinky]

TASK [Make ralsina a sudoer] ********************
ok: [rocky]
ok: [pinky]

TASK [Create fish config directory] *************
changed: [rocky]
changed: [pinky]

TASK [Get starship installer] *******************
ok: [rocky]
ok: [pinky]

TASK [Install starship] *************************
ok: [rocky]
ok: [pinky]

TASK [Enable starship] **************************
changed: [rocky]
changed: [pinky]

PLAY RECAP **************************************
pinky : ok=9    changed=2    unreachable=0    failed=0    skipped=1 
        rescued=0    ignored=0
rocky : ok=10   changed=2    unreachable=0    failed=0    skipped=0 
        rescued=0    ignored=0

If you look care­ful­ly you can see rocky ran one more task, and pinky skipped one (the de­bian-spe­cif­ic pack­age in­stal­la­tion), and that on­ly two things got ac­tu­al­ly ex­e­cut­ed on each ma­chine.

I could run this a dozen times from now on, and it would not do any­thing.

Did it work?

Sure, I can ssh into rocky and everything is nice:

> ssh rocky
Linux rock-5c 5.10.110-37-rockchip #27a257394 SMP Thu May 23 02:38:59 UTC 2024 aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jun 26 15:32:33 2024 from 100.73.196.129
Welcome to fish, the friendly interactive shell
Type `help` for instructions on how to use fish

ralsina in 🌐 rock-5c in ~ 

There is a star­ship promp­t, and I can use fish. And I can su­do. Nice!

I can now change the inventory so rocky also uses the ralsina user and delete the rock user.

Next steps

There is a lot more to an­si­ble, specif­i­cal­ly roles but this is al­ready enough to get use­ful things done, and hope­ful­ly it will be use­ful to you too.

Some­times you want to ask da­ta ques­tion­s. And of­ten that da­ta is in a CSV. Sure, you can write a quick Python script and use that to ex­tract the in­for­ma­tion you wan­t. Or you can im­port it in­to a data­base and use SQL.

But TIL the eas­i­est thing is to just ask the duck.

The duck is Duck­DB here.

Why? Be­cause you can use SQL queries di­rect­ly on CSV files.

For examples, let's use a random CSV called luarocks-packages.csv I have lying around:

It starts like this:

name,src,ref,server,version,luaversion,maintainers
alt-getopt,,,,,,arobyn
bit32,,,,5.3.0-1,5.1,lblasc
argparse,https://github.com/luarocks/argparse.git,,,,,
basexx,https://github.com/teto/basexx.git,,,,,
binaryheap,https://github.com/Tieske/binaryheap.lua,,,,,vcunat
busted,,,,,,
cassowary,,,,,,marsam alerque
cldr,,,,,,alerque
compat53,,,,0.7-1,,vcunat
cosmo,,,,,,marsam

And how do I query it? Well, sup­pose I want to find all pack­ages where alerque is one of the main­tain­er­s:

> duckdb
v1.0.0 1f98600c2c
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D select name from 'luarocks-packages.csv' where maintainers like '%alerque%';
┌───────────┐
│   name    │
│  varchar  │
├───────────┤
│ cassowary │
│ cldr      │
│ fluent    │
│ loadkit   │
│ penlight  │
└───────────┘

And boom! There you go. So, if you know even some very ba­sic SQL (and you should!) you can lever­age duck­db to ex­tract in­for­ma­tion from CSV files quick­ly, re­li­ably and in a re­peat­able man­ner.

Which is awe­some!