RE: OpenDKIM in Postfix multi-instance environment

From: Murray S. Kucherawy <msk_at_cloudmark.com>
Date: Fri, 21 Oct 2011 12:41:29 -0700

> -----Original Message-----
> From: opendkim-users-bounce_at_lists.opendkim.org [mailto:opendkim-users-bounce_at_lists.opendkim.org] On Behalf Of Fabio Sangiovanni
> Sent: Friday, October 21, 2011 2:57 AM
> To: opendkim-users_at_lists.opendkim.org
> Subject: OpenDKIM in Postfix multi-instance environment
>
> 1) in order to balance the workload, can I rely on OpenDKIM multithread
> capabilities or should I launch multiple instances of OpenDKIM (to match
> Postfix instances in a 1:1 fashion, with multiple unix sockets for
> communication)?

OpenDKIM's threading is driven mainly by libmilter, which by default creates a new worker thread corresponding to every SMTP connection reported by the MTA. It's possible to configure libmilter to have a fixed worker pool size, though I haven't tried this myself for reliability or performance. The sendmail open source documentation probably has more information about this.

If you find that it falls over, you might want to look into multiple instances, especially on different machines.

Since you said speed with verification is key, you should be aware that in most of the tests I've done the vast majority of the time spent verifying a single message is waiting for DNS answers. Caching helps, to be sure, but slow verification isn't necessarily a sign that configuration changes in the filter are needed. Verifying doesn't go to disk at all unless you have KeepTemporaryFiles set or use any of the reporting features that require a copy of the message to be temporarily retained, so it's all DNS and memory for verifying. The same goes for signing: The filter won't go to disk during signing except if you keep your keys in a Sleepycat DB.

> 2) are there particular settings that impact on signing performances,
> beside hash calculation?

If you're able to store your keys in a flat file or even just use a single key, signing performance is much faster because the keys are loaded into memory. If the filter has to go to an external database to get keys, there's some delay in signing because keys are not (currently) cached in memory. This might be a useful optimization, but it hasn't come up yet. Feel free to open a feature request on SourceForge if this might affect you.

> 3) is there a way to control OpenDKIM thread management? What is the
> default behavior about the number of children spawned by the main
> process?

It's all driven mainly by libmilter. The resolver library (if you're using libar or libunbound) might create a thread or two of its own, and the filter itself creates one whose task is to sit around waiting for signals (i.e., it's in a constant wait state), but they persist for the lifetime of the process.

Let me know if you need any other information.

-MSK
Received on Fri Oct 21 2011 - 19:41:37 PST

This archive was generated by hypermail 2.3.0 : Mon Oct 29 2012 - 23:20:20 PST