RE: async resolver

From: Murray S. Kucherawy <msk_at_cloudmark.com>
Date: Tue, 20 Oct 2009 09:54:40 -0700

> -----Original Message-----
> From: opendkim-dev-bounce_at_lists.opendkim.org [mailto:opendkim-dev-
> bounce_at_lists.opendkim.org] On Behalf Of Daniel Black
> Sent: Monday, October 19, 2009 8:32 PM
> To: opendkim-dev_at_lists.opendkim.org
> Subject: Re: async resolver
>
> To me, and I'm an expert on threading by any stretch of the
> imagination, this
> is still largely synchronous. The level of concurrency achieved seems
> to be
> limited to 1/2 a page of code on the same function between the
> ar_request and
> wait function.
>
> To my understanding async processing would be:
>
> 1. found DKIM-Signature - send of async request for DNS key with a
> callback to
> A)
> 2. see From header - send async ADSP request off with callback to B
> 3. continue buffering/hashing message.
> 4. eom milter request received - check we've got the answer that are
> pending
> and if not we can wait here as the MTA needs an answer as to what to
> do.
>
> A. DKIM call back received
> - if NX/tempfail/permail /timeout- set flag for main milter thread to
> stop
> processing the DKIM-signature and set AR results for it
> - if found put the key in the data-structure for the signature and
> allow the
> mail milter thread to hash the message buffer
>
> B - ADSP call back
> - set store ADSP response in call back

That's a pretty nice design.

The ADSP call could be launched upon seeing the From: header and only checked (or canceled) at EOM, I agree. That would be an improvement.

The bit about the signature is interesting, but we also have to consider that the entire signature validation might be completed (with failure detected) as early as the end of the headers; the "b=" only covers the signed headers, so it's possible to complete that piece of the DKIM evaluation much earlier. And if we can tell that the signature will not validate at that point because maybe some header got changed, then there's no need for the filter to accept what could be megabytes of body data it doesn't actually need. So there's possibly a substantial I/O benefit to completing the first step of the DKIM evaluation before the body begins to arrive. At sites that regularly mail around DKIM-signed ISOs, for example, this can be a big win.

It's hard to tell whether or not that scenario is really something about which we should be concerned, though, without more data. The only other concern I have is the amount of code that would have to change. Not a bad thing, just thinking out loud for planning purposes.

> > libopendkim implements the callback mechanism. The function
> > dkim_set_dns_callback()
> from a package maintenance this is an anomaly. This function only
> exists if
> enabled when libopendkim is compiled. The opendkim needs to be compiled
> with
> the same option to actually use it. This actually creates to ABI's
> depending
> on who it was compiled.
>
> For libopendkim recommend - making dkim-set_dns_callback always exist
> and
> return 'no supported' if not compiled right to the applications knows
> not to
> expect callbacks.
>
> For opendkim - the callback could be set and just ignore the result of
> dkim_set_dns_callback as we can't do anything with it anyway.

Yep, that's not a bad idea.
Received on Tue Oct 20 2009 - 16:54:58 PST