RE: opendkim performance

From: Murray S. Kucherawy <msk_at_cloudmark.com>
Date: Wed, 31 Aug 2011 16:29:54 -0700

What's "active queue"? Are you talking about your MTA's on-disk queue?

If there's anyplace that could use some optimization, it's probably the canonicalization code. That's where I intend to focus some attention.

The "query cache" feature will only really help you for signing; it caches DNS keys so you don't have to do a round-trip to a nameserver.

From: Lukasz Ochoda [mailto:lochoda_at_endai.com]
Sent: Wednesday, August 31, 2011 3:23 PM
To: Murray S. Kucherawy
Cc: opendkim-users_at_lists.opendkim.org
Subject: Re: opendkim performance

I am using package for FreeBSD.

But, actually I think opendkim works perfectly. Correct me if I am wrong, but since opendkim is a milter, it works upfront of active queue, so my issue with growing active queue is not related at all to opendkim???

I will still do some tests with new servers and fixed-sized pool. Also I found that opendkim has some cache, that can be useful for me.

On Wed, Aug 31, 2011 at 5:13 PM, Murray S. Kucherawy <msk_at_cloudmark.com<mailto:msk_at_cloudmark.com>> wrote:
Do you build from source, or install from an RPM or ports tree?

From: opendkim-users-bounce_at_lists.opendkim.org<mailto:opendkim-users-bounce_at_lists.opendkim.org> [mailto:opendkim-users-bounce_at_lists.opendkim.org<mailto:opendkim-users-bounce_at_lists.opendkim.org>] On Behalf Of Lukasz Ochoda
Sent: Wednesday, August 31, 2011 11:29 AM

To: opendkim-users_at_lists.opendkim.org<mailto:opendkim-users_at_lists.opendkim.org>
Subject: Re: opendkim performance

Hi,

Thanks for you quick reply... I have not done anything yet... the problem is that I try to sign about 1000 messages per second... I can see big improvement after I switched from dkim proxy. OpenDKIM works great, but still can see some emails that are not signed on the fly... the active queue is still growing, but very slow which is great.

I will try to work on this fixed-sized pool, any improvement is good for me...

DNS is not an issue right now, because I do not verify signatures right now... just signing outgoing emails.

Thanks.

On Wed, Aug 31, 2011 at 2:22 PM, Murray S. Kucherawy <msk_at_cloudmark.com<mailto:msk_at_cloudmark.com>> wrote:
I should also point out that the vast majority of the latency in opendkim processing is very likely DNS delays. There's very little we can do in the source code to deal with that.

From: opendkim-users-bounce_at_lists.opendkim.org<mailto:opendkim-users-bounce_at_lists.opendkim.org> [mailto:opendkim-users-bounce_at_lists.opendkim.org<mailto:opendkim-users-bounce_at_lists.opendkim.org>] On Behalf Of Murray S. Kucherawy
Sent: Wednesday, August 31, 2011 11:14 AM

To: opendkim-users_at_lists.opendkim.org<mailto:opendkim-users_at_lists.opendkim.org>
Subject: RE: opendkim performance

Have you done any profiling to see where it's bottlenecking?

There are some libmilter build-time features that do things like restrict libmilter to a fixed-size worker pool rather than making one thread per client. I don't recall how much faster that gets it going, though.

There's an open item to improve the speed at which canonicalization occurs, which is where libopendkim spends most of its time. I'm hoping to do that for the next release.

From: opendkim-users-bounce_at_lists.opendkim.org<mailto:opendkim-users-bounce_at_lists.opendkim.org> [mailto:opendkim-users-bounce_at_lists.opendkim.org<mailto:opendkim-users-bounce_at_lists.opendkim.org>] On Behalf Of Lukasz Ochoda
Sent: Wednesday, August 31, 2011 11:01 AM
To: opendkim-users_at_lists.opendkim.org<mailto:opendkim-users_at_lists.opendkim.org>
Subject: opendkim performance

Hi All!

Is there any way to optimize opendkim to work with high performance systems/servers. For example, can I increase number of processes of opendkim or make it work faster and better?



--
Lukasz Ochoda
Endai Worldwide<http://www.endai.com/>
MarketTraq Developer
Tel: 212-430-0808x113<tel:212-430-0808x113>
--
Lukasz Ochoda
Endai Worldwide<http://www.endai.com/>
MarketTraq Developer
Tel: 212-430-0808x113
Received on Wed Aug 31 2011 - 23:30:05 PST

This archive was generated by hypermail 2.3.0 : Mon Oct 29 2012 - 23:20:19 PST