Ir al contenido principal

Ralsina.Me — El sitio web de Roberto Alsina

Publicaciones sobre ra-plugins

New qmail plugin idea: overload

But then you start seeing how your "not pre­pro­ce­ss­e­d" queue star­ts gro­win­g, and gro­win­g...

This can al­so mean things like cla­mav or spa­ma­ssas­si­n, whi­ch need to che­ck the mail be­fo­re it ge­ts queued are not kee­ping up wi­th the mail flo­w, or ma­y­be so­me IO per­for­ma­ce is­sue.

But what can you do ri­gh now to fix it?

We­ll, you can di­sa­ble spa­ma­ssas­si­n, or, in ex­tre­me ca­ses, shu­tdo­wn SM­TP so the sys­tem has a chan­ce to ca­tch its brea­th so to speak.

Of cour­se, clo­sing SM­TP means your own users can't send email ei­the­r, whi­ch su­cks.

Now the­re is a li­gh­ter al­ter­na­ti­ve: shu­tdo­wn SM­TP for tho­se who are not your user­s.

He­re's the tri­vial co­de, im­ple­men­ted as a SPP plu­gi­n:

#!/bin/dash

if [ -f /var/qmail/control/overloaded ]
then
      if [ -z "$SMTPAUTHUSER" ]
      then
              echo R451 Temporary Failure: Server overload
              echo overload: $PPID Temporary Failure: Server overload >&2
      fi
fi

And if you are da­ring and want to make your sys­tem se­l­f-­co­rrec­tin­g, ma­y­be you should cron so­me­thing like this:

* * * * * if [ `qmail-qstat  | tail -1 | cut -d: -f2` -gt 100 ];\
then touch /var/qmail/control/overloaded ;\
else rm -f /var/qmail/control/overloaded; fi

I wi­ll pro­ba­bly co­de it again in C and make it part of ra/­plu­gin­s.

Si usas qmail: lee esto

  • Mi plu­­gin más útil es pro­­­ba­­ble­­men­­te ip­­th­­ro­­­ttle, que pue­­des usar pa­­ra ha­­cer que IPs an­­sio­­­sos no se co­­­ne­c­­ten tan se­­gui­­do.

  • La ve­r­­sión ac­­tual en SVN pue­­de au­­to­­­blo­­­quear es­­tos IPs por un pe­­río­­­do co­n­­fi­­gu­­ra­­ble si usas ip­s­vd, que es co­­­mo tcp­­se­r­­ve­­r, pe­­ro mu­­cho me­­jo­­­r.

  • Rea­l­­men­­te ne­­ce­­si­­to ayu­­da pa­­ra pro­­­bar la ve­r­­sión en SVN, que de­­be­­­ría ser mui­­cho, mu­­cho, mu­­cho me­­jor que los re­­lea­­ses ac­­tua­­le­s.

  • El re­­po­­­si­­to­­­rio SVN es­­tá en google­­co­­­de

Rater progresses (slowly)

I am hack­ing a bit on rater my dae­mon/­client to see if things are hap­pen­ing more of­ten than they should (in oth­er word­s, gener­ic rate lim­it­ing).

I had to take a few days of­f, since my broth­er got mar­ried and we all went back to San­ta Fe for that and a week­end, and then ev­ery­one else has sore throats and I am the on­ly one healthy.

But hey, it works well enough al­ready:

  • The sim­­plis­tic pro­­to­­col is done

  • The serv­er works

    • It can take hours of gib­ber­ish with­­­out prob­lem­s.

    • It can take hours of valid in­­­put with­­­out prob­lem­s.

    • It does what it's sup­­­posed to do.

  • It's stay­ing be­low 300S­LOC, which was my goal.

Miss­ing stuff:

  • Val­­grind it.

  • Client li­brary.

  • Gen­er­ic CLI clien­t.

  • A qmail-spp plug­in that us­es it.

And then, I can for­get all about it.

Snow and rates

Mon­day was a very spe­cial day:

  • Hol­i­­day (In­de­pen­­dence day)

  • An­niver­sary (3 years as Rosar­i­o's boyfriend)

  • The first snow­­fall in Buenos Aires in 89 years.

Be­sides that, this week my broth­er is get­ting mar­ried so the whole fam­i­ly (in­clud­ing 2.5 mon­th-old JF) is leav­ing for my an­ces­tral lands to­mor­row.

And I start­ed a new small pro­jec­t, wh­cih should be fin­ished soon.

This is some­thing that seems use­ful to me in the con­text of mail server­s, but maybe it will al­so find its us­es else­where.

I call it rater, and it tells you if things are hap­pen­ing faster than a spe­cif­ic rate.

For ex­am­ple, I in­tend to use it to fig­ure out if a spe­cif­ic IP is con­nect­ing to a serv­er more than X times ev­ery Y sec­ond­s, or if a us­er is send­ing more than Z emails ev­ery T min­utes.

The on­ly thing I found for this is re­lay­d, which is old, un­main­tained and whose site has van­ished.

The con­fig file is some­thing like this (thanks to lib­con­fig):

limits : {
      user: (
                    ("rosario",90,20),
                    ("ralsina",90,10),
              ("*",2,10)
              );
      ip:   (
                    ("10.0.0.*",90 , 20),
                    ("10.0.1.*",90 , 20),
                    ("*",2 , 10)
              );

};

You can de­fine as many class­es of lim­its as you want (that would be ip and us­er in this ex­am­ple) and as many lim­it keys as you wan­t, that will be matched us­ing some­thing like fn­match.

I am us­ing an in­-mem­o­ry SQLite DB for the ac­count­ing, and an in­ter­est­ing li­brary called libut for the sock­et­s, log­ging, and event loop.

This li­brary has a very in­ter­est­ing fea­ture: your app gets an ad­min­is­tra­tive in­ter­face for free!

[ralsina@monty rater]$ telnet localhost 4445
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
libut control port interpreter
Type 'help' for command list.

help
command           description
----------------- -------------------------------------
* mem             - memory pool usage summary
* var             - Display or set config variables
* log             - Change log file or verbosity
  fds             - list selected file descriptors
  tmr             - show pending timers
  uptime          - show uptime
* prf             - Performance/profiling stats
* cops            - List coprocesses
  help            - view command help
  exit            - close connection

Commands preceded by * have detailed help. Use help <command>.

Ok
var
 name                 description                    value
--------------------- ------------------------------ --------------------
*ut_log_level         log level                      Debugk
*ut_log_file          log file                       /dev/stdout
*ut_jobname           job name                       job1
*ut_control_port      control port IP/port           127.0.0.1:4445
*ut_basedir           shl base directory             /mnt/centos/home/ralsina/Desktop/proyectos/rater

Variables prefixed with '*' can be changed.

Ok
var ut_log_level Debug

Ok
var
 name                 description                    value
--------------------- ------------------------------ --------------------
*ut_log_level         log level                      Debug
*ut_log_file          log file                       /dev/stdout
*ut_jobname           job name                       job1
*ut_control_port      control port IP/port           127.0.0.1:4445
*ut_basedir           shl base directory             /mnt/centos/home/ralsina/Desktop/proyectos/rater

Variables prefixed with '*' can be changed.

Ok

Pret­ty neat.

Be­yond this, there will be a small clien­t-­side li­brary that hides all the net­work stuff be­hind a cou­ple of block­ing calls (or you can do your own be­cause the pro­to­col is sil­ly sim­ple).

Playing with literate programming

I am us­ing ra-­plu­g­ins as a toy to do things I nev­er both­ered in oth­er project­s.

I am do­ing unit-test­ing. And now... some lit­er­ate pro­gram­ming!

Ok, not much, and not very well, but at least I am play­ing with Lp4all which is a nice, sim­ple tool to gen­er­ate nice HTML from slight­ly wik­i-­marked sources.

You can see some lit­tle things in my code here. My vere­dict so far? A nice way to keep the code doc­u­ment­ed in a fash­ion that ocasi­nal browsers can fol­low.

The main thing miss­ing is au­to­mat­ic cross-ref­er­enc­ing.

In gen­er­al, I am find­ing that this (and unit test­ing) helps me ex­press ex­plic­it­ly to my­self what the heck I am try­ing to do, and see if the code ac­tu­al­ly does it. Which is a re­al­ly good thing.


Contents © 2000-2020 Roberto Alsina