Queue Management for Qmail
I am a big fan of qmail. I honestly think it's the best MTA around, even if it has a few issues. Well, what doesn't?
Since I manage a few small and large installations, I think it's only fair that I document what those issues seem to me, and what solutions I've found.
One thing about qmail is that its queue, for sake of efficiency, is not as easy to manipulate as, for example, sendmail's.
So, I've found some tools, and written some more, to help manage, control and manipulate qmail's message queue.
The Qmail Queue
When you use one of the tools described below, or look in qmail's logs, you will find things like mentions of "msg 457" being queued. What is that?
Well, the number is just an identifier, and you will find files called 457 in qmail's queue directories.
Qmail's queue consists of several directories. Some of them, which can be expected to hold many hundreds or thousands of files are split into numbered subdirectories. That is because in Unix, normally finding a file in a directory becomes slow if that directory contains many files.
A message going through qmail will have related files in several places in that directory structure.
The following is from qmail's INTERNALS file, some of these files will exist, some not:
- the message
- the envelope: where the message came from, where it's going
- the envelope, under construction by qmail-queue
- the envelope sender address, after preprocessing
- local envelope recipient addresses, after preprocessing
- remote envelope recipient addresses, after preprocessing
- permanent delivery errors
The number of the internediate directory (20 in this example) is the remainder of dividing the message number by 23... at least by default ;-)
The numbers used to identify the messages are not arbirary and you should never change them unless you really know what you are doing. If you mess that up, maybe queue-rename (see below) can fix it.
If you remove a file, make sure you remove all the related ones, too, or qmail will whine.
If you gonna be messin' with qmail's queue, make sure you shut qmail-send down first. Bad things happen if you don't.
Tools that come with Qmail
- Lists the messages in the queue, it shows sender, recipient, date and message number.
[root@correo root]# qmail-qread 16 Sep 2003 21:13:33 GMT #145337 2218 <email@example.com> remote firstname.lastname@example.org
- Simple tool: Reports the number of messages in qmail's queue, and of messages waiting to enter the queue.
[root@correo root]# qmail-qstat messages in queue: 1025 messages in queue but not yet preprocessed: 2
Tools on the Net
- Awesome tool. Combines the features of qmail-qstat and qmail-qread, adding functions to remove mails from the queue, display mails, dispatch queue immediately, treat local and remote queues separately, and more. A must have for any qmail admin.
- A tool to automatically fix the problems explained above. You probably need to patch it to use big-todo (don't ask me)
- Another tool with the same goal as queue-fix. Written in Python.
- Renames the files in the queue structure so they have the proper names. If you move a qmail queue from one disk to another, you probably need to run this, or one of the queue-fixing tools.
- A tool to remove messages from the queue based on different criteria. qmHandle can do the hard work of actually removing, but which ones? Should be able to do it by sender, or by recipient, or by subject, or simply by regexp. Of course it is possible to do it using grep and shell, but that gets tricky when your queue exceeds a few hundred messages. Here's a first shot I made at writing such a tool.
Source code for qmail-cleaner.py
Important notes: This will simply print a list of the messages it finds matching your criteria. Since regular expressions are easy to get wrong, you should be careful about what you do with that list, because it could not be what you wanted....
In any case, it is easy to modify qmail-cleaner to output a qmHandle command line that would, for example, remove those messages from the queue. Such potentially destructive version is left as an exercise to the reader, though.
if you have any suggestions or problems with the script, please post a comment.
Source code for qmail-cleaner.py
#!/usr/bin/env python import sys,os,re #Option handling nice and easy through Optik, http://optik.sf.net from optik import OptionParser parser = OptionParser() parser.add_option ("-d", "--queue-dir",action="store", type="string", dest="queuedir", help="The qmail queue directory, default=/var/qmail/queue", default="/var/qmail/queue") parser.add_option ("-s", "--sender",action="store", type="string", dest="sender", help="Find messages with a From: header matching this regexp", default=None) parser.add_option ("-t", "--to",action="store", type="string", dest="to", help="Find messages with a To: header matching this regexp", default=None) parser.add_option ("--header",action="store", type="string", dest="header", help="Find messages with any header matching this regexp", default=None) parser.add_option("-v", action="store_true", dest="verbose", help="Verbose",default=0) parser.add_option("-q", action="store_false", dest="verbose", help="Quiet") (options,args) = parser.parse_args (sys.argv) #This is where the messages are msgdir=options.queuedir+"/mess" #Array of matching messages matches= #Create the regexp to be matched regex="" if options.header: #User gave us a full regexp regex=options.header elif options.sender: #User asks for a From: header regex="^From: "+options.sender elif options.to: #User asks for a To: header regex="^To: "+options.to exp=re.compile(regex) if options.verbose: print "Searching messages with headers matching: %s\n"%regex #Function that checks if a message matches our criteria def check(message): global exp,matches for line in open(message): #Only check the headers, so stop on blank line if len(line)==1: return #Check for match if exp.search(line,1): matches.append(message.split("/")[-1]) return #Get list of split directories (usually they are 23, but why bet) try: dirs=os.listdir(msgdir) except: sys.stdout.write("Error listing queue directories") sys.exit(1) #Walk the tree and examine each file for dir in dirs: files=os.listdir(msgdir+"/"+dir) for file in files: check (msgdir+"/"+dir+"/"+file) #report files matching our criterion for match in matches: print match
Sample run of qmail-cleaner.py:
[root@correo p]# ./qmail-cleaner.py --header=ralsina -v Searching messages with headers matching: ralsina 145443 145307 146746 144469
- Qmail's Internals
- Life with qmail
- Python Regular Expressions (please read before using qmail-cleaner)
could i manage this add-ons by painel plesk?
i would like to use this on our server.