Introduction

I am a big fan of qmail. I hon­est­ly think it's the best MTA around, even if it has a few is­sues. Well, what does­n't?

Since I man­age a few small and large in­stal­la­tion­s, I think it's on­ly fair that I doc­u­ment what those is­sues seem to me, and what so­lu­tions I've found.

One thing about qmail is that its queue, for sake of ef­fi­cien­cy, is not as easy to ma­nip­u­late as, for ex­am­ple, send­mail's.

So, I've found some tool­s, and writ­ten some more, to help man­age, con­trol and ma­nip­u­late qmail's mes­sage queue.

But first:

The Qmail Queue

When you use one of the tools de­scribed be­low, or look in qmail's logs, you will find things like men­tions of "msg 457" be­ing queued. What is that?

Well, the num­ber is just an iden­ti­fier, and you will find files called 457 in qmail's queue di­rec­to­ries.

Qmail's queue con­sists of sev­er­al di­rec­to­ries. Some of them, which can be ex­pect­ed to hold many hun­dreds or thou­sands of files are split in­to num­bered sub­di­rec­to­ries. That is be­cause in Unix, nor­mal­ly find­ing a file in a di­rec­to­ry be­comes slow if that di­rec­to­ry con­tains many files.

A mes­sage go­ing through qmail will have re­lat­ed files in sev­er­al places in that di­rec­to­ry struc­ture.

The fol­low­ing is from qmail's IN­TER­NALS file, some of these files will ex­ist, some not:

The num­ber of the in­terne­di­ate di­rec­to­ry (20 in this ex­am­ple) is the re­main­der of di­vid­ing the mes­sage num­ber by 23... at least by de­fault ;-)

The num­bers used to iden­ti­fy the mes­sages are not ar­birary and you should nev­er change them un­less you re­al­ly know what you are do­ing. If you mess that up, maybe queue-re­name (see be­low) can fix it.

If you re­move a file, make sure you re­move all the re­lat­ed ones, too, or qmail will whine.

If you gonna be mess­in' with qmail's queue, make sure you shut qmail-send down first. Bad things hap­pen if you don't.

Tools that come with Qmail

qmail-qread

Lists the messages in the queue, it shows sender, recipient, date and message number.

[root@correo root]# qmail-qread
16 Sep 2003 21:13:33 GMT  #145337  2218  <joe@home.com>
        remote  shmoe@work.net

qmail-qstat

Simple tool: Reports the number of messages in qmail's queue, and of messages waiting to enter the queue.

[root@correo root]# qmail-qstat
messages in queue: 1025
messages in queue but not yet preprocessed: 2

Tools on the Net

QmHandle

Awesome tool. Combines the features of qmail-qstat and qmail-qread, adding functions to remove mails from the queue, display mails, dispatch queue immediately, treat local and remote queues separately, and more. A must have for any qmail admin.

queue-fix

A tool to automatically fix the problems explained above. You probably need to patch it to use big-todo (don't ask me)

queue-repair

Another tool with the same goal as queue-fix. Written in Python.

queue-rename

Renames the files in the queue structure so they have the proper names. If you move a qmail queue from one disk to another, you probably need to run this, or one of the queue-fixing tools.

Missing tool

qmail-cleaner

A tool to remove messages from the queue based on different criteria. qmHandle can do the hard work of actually removing, but which ones? Should be able to do it by sender, or by recipient, or by subject, or simply by regexp. Of course it is possible to do it using grep and shell, but that gets tricky when your queue exceeds a few hundred messages. Here's a first shot I made at writing such a tool.

Source code for qmail-cleaner.py

Im­por­tant notes: This will sim­ply print a list of the mes­sages it finds match­ing your cri­te­ri­a. Since reg­u­lar ex­pres­sions are easy to get wrong, you should be care­ful about what you do with that list, be­cause it could not be what you want­ed....

In any case, it is easy to mod­i­fy qmail-­clean­er to out­put a qmHan­dle com­mand line that would, for ex­am­ple, re­move those mes­sages from the queue. Such po­ten­tial­ly de­struc­tive ver­sion is left as an ex­er­cise to the read­er, though.

if you have any sug­ges­tions or prob­lems with the scrip­t, please post a com­men­t.

                  
#!/usr/bin/env python

import sys,os,re

#Option handling nice and easy through Optik, http://optik.sf.net

from optik import OptionParser
parser = OptionParser()

parser.add_option ("-d", "--queue-dir",action="store", type="string", dest="queuedir",
                   help="The qmail queue directory, default=/var/qmail/queue",
                   default="/var/qmail/queue")

parser.add_option ("-s", "--sender",action="store", type="string", dest="sender",
                   help="Find messages with a From: header matching this regexp",
                   default=None)

parser.add_option ("-t", "--to",action="store", type="string", dest="to",
                   help="Find messages with a To: header matching this regexp",
                   default=None)

parser.add_option ("--header",action="store", type="string", dest="header",
                   help="Find messages with any header matching this regexp",
                   default=None)

parser.add_option("-v", action="store_true", dest="verbose", help="Verbose",default=0)
parser.add_option("-q", action="store_false", dest="verbose", help="Quiet")

(options,args) = parser.parse_args (sys.argv)


#This is where the messages are
msgdir=options.queuedir+"/mess"

#Array of matching messages
matches=[]

#Create the regexp to be matched
regex=""
if options.header: #User gave us a full regexp
        regex=options.header
elif options.sender: #User asks for a From: header
        regex="^From: "+options.sender
elif options.to: #User asks for a To: header
        regex="^To: "+options.to

exp=re.compile(regex)

if options.verbose:
        print "Searching messages with headers matching: %s\n"%regex

#Function that checks if a message matches our criteria
def  check(message):
        global exp,matches
        for line in open(message):
                #Only check the headers, so stop on blank line
                if len(line)==1:
                        return

                #Check for match
                if exp.search(line,1):
                        matches.append(message.split("/")[-1])
                        return


#Get list of split directories (usually they are 23, but why bet)

try:
        dirs=os.listdir(msgdir)
except:
        sys.stdout.write("Error listing queue directories")
        sys.exit(1)


#Walk the tree and examine each file

for dir in dirs:
        files=os.listdir(msgdir+"/"+dir)
        for file in files:
                check (msgdir+"/"+dir+"/"+file)

#report files matching our criterion

for match in matches:
        print match

Sam­ple run of qmail-­clean­er.py:

[root@correo p]# ./qmail-cleaner.py --header=ralsina  -v
Searching messages with headers matching: ralsina

145443
145307
146746
144469

Important Links

• Qmail
• Qmail's In­ter­nals
• Life with qmail
• Python Reg­u­lar Ex­pres­sions (please read be­fore us­ing qmail-­clean­er)
• Qmail-­clean­er