RE: multithreaded environment ? (better now: multi-core environment)

> -----Original Message-----
> From: opendkim-users-bounce_at_lists.opendkim.org [mailto:opendkim-users-bounce_at_lists.opendkim.org] On Behalf Of Frank Gadegast
> Sent: Thursday, December 01, 2011 8:21 AM
> To: Todd Lyons; opendkim-users_at_lists.opendkim.org
> Subject: Re: multithreaded environment ? (better now: multi-core environment)
>
> Todd Lyons wrote:
> > On Thu, Dec 1, 2011 at 7:18 AM, Frank Gadegast<frank_at_powerweb.de> wrote:
> >> Specially when most other processes (sendmail or spamassassin) are
> >> running on all virtual CPUs ...
> >
> > both of which fork.
>
> and because its nearly impossible to have a libmilter that supports
> forking, why not having a opendkim-milter thats support forking to
> single opendkim-processes like spamd-milter does ?

For a milter application, libmilter creates the socket to which the MTA will connect, and listens on that socket. One connection to opendkim (or spamass-milter) is made for every MTA connection received. Inside libmilter, on receipt of a connection from an MTA, it creates a new thread there and begins making use of opendkim callbacks to process various SMTP steps. opendkim is not creating the threads, libmilter is.

So to get opendkim to operate via subprocesses rather than threads, you'd need to hack libmilter to call fork() instead of pthread_create(). It might not be too complicated to do this actually, but opendkim isn't where the work needs to be done.

Todd's description of the difference is correct. spamass-milter doesn't do any of the heavy work; it simply makes a connection to spamd, which is where the forking and heavy work happens, and that's why you see the better performance. To mirror that design, opendkim would have to have two processes, one that simply receives data from MTAs and hands it to the other one, and in that second one all the hashing and signing/verifying gets done, returning results to the first one.

But it doesn't sound to me that such a big architectural change is worthwhile. If all you need to do is have each connection handled by a subprocess instead of a thread, a fairly simple hack to libmilter might be all you need.

We don't maintain libmilter, sendmail does. If you come up with a patch, we could test it for you, but they're the ones that have to accept and incorporate it.

> Probably much easier to implement (if someone will copy the code from
> spamd-milter ;o)

We'll look forward to your patch. :-)

-MSK
Received on Thu Dec 01 2011 - 17:41:59 PST