RE: multithreaded environment ?

From: Murray S. Kucherawy <msk_at_cloudmark.com>
Date: Wed, 30 Nov 2011 16:29:56 -0800

> -----Original Message-----
> From: opendkim-users-bounce_at_lists.opendkim.org [mailto:opendkim-users-bounce_at_lists.opendkim.org] On Behalf Of Frank Gadegast
> Sent: Wednesday, November 30, 2011 7:08 AM
> To: opendkim-users_at_lists.opendkim.org
> Subject: multithreaded environment ?
>
> We use opendkim for signing outgoing mails with sendmail.
> Works, but is quite slow.

Have you done any profiling of the executable to figure out why it's slow in your environment?

> I read about multithreaded milter aso, but this already seems to work
> with a spamassassin-milter and clamav-milter, both have several child
> processes running and are really quick.
> They are also running on multiple cores.

Thread creation inside opendkim is controlled by libmilter, not by the filter itself. The milter API creates one thread per connection by default, though you can rebuild it in a way that implements a fixed worker pool. The opendkim filter is driven by that library, so no changes are needed in the filter itself to adapt. You might find that having a fixed number of threads improves performance, especially if you usually have more than that number of concurrent signing operations going on, in which case your application might be spending a lot of time switching between threads.

> Just wonder why there is no option to specify the amount of workers
> (like there is for spamd).

spamd and spamass-milter are different things. Spamass-milter appears (on my system, at least) to be a single process that gets multiple threads, just like opendkim does. They all query spamd, which is the thing that has multiple processes to handle multiple queries.

The last time I profiled opendkim when signing, the biggest time sink was libcrypto. It doesn't go to disk with your configuration except at startup, so all the analysis and encryption is happening in-memory. In that case, running in multiple processes wouldn't help much.
Received on Thu Dec 01 2011 - 00:30:04 PST

This archive was generated by hypermail 2.3.0 : Mon Oct 29 2012 - 23:20:21 PST