Re: LUA?

From: Murray S. Kucherawy <msk_at_blackops.org>
Date: Sat, 10 Oct 2009 17:32:05 -0700 (PDT)

On Sat, 10 Oct 2009, Daniel Black wrote:
> I've only considered verification policy in the below. I don't know if
> signing policy is really an issue.

It is; look at the "bodylengthdb" feature and the "DontSignMailTo"
config setting, for example.

> Is the plan to migrate the existing policy decisions to a reference
> script?

Ideally I would think we'd either drop the current database/file features,
or make them accessible in a more generic way via LUA. But this is all
conceptual right now; I'm open to suggestions from those with more
exposure.

> Some questions about the LUA choice.
>
> why lua?

LUA's the only one I've looked into at all. It's not a final choice by
any stretch. Are there others we should be considering?

> why tie into a language? perhaps a socket interface through unix/tcp
> sockets
>
> Something like:
> http://www.postfix.org/SMTPD_POLICY_README.html#protocol
> [...]

This is also workable, and means we're not dependent on an external system
for our policy engine. We would need it to be more than a single response
from the user-provided server though; it would need access to primitives
that are inside libopendkim.

The vision I have is that the policy script/engine/agent/whatever would
receive all of the details of the message before any signature evaluation
is done. The policy agent would then be able to decide which signatures,
if any, it wants to evaluate. If a message has several signatures, for
example, then maybe the verifier only cares about the one from the From:
domain (if there is one); for "n" signatures, this would save it having to
do as many as "2n" hashes, "n" DNS round-trips and "n" RSA signature
verifications. So the policy agent determines which ones are of interests
and requests evaluation of those only. Then it has to decide whether or
not it wants to perform the ADSAP stuff based on those results.
Basically, we need to provide detail about the message (including
signature data) before any DKIM work actually takes place, then let the
policy engine request signature verifications according to its policy, and
then do ADSP (or not) based on those results. There's a lot of
back-and-forth before a final choice can be made.

We could provide those values and export the validation/policy primitives
via LUA, or we could invent a mechanism by which a user-provided agent
receives a connection from the filter and there's an exchange of data and
commands until some final mechanism occurs.

It seems to me the amount of work involved in both cases is similar.
The advantage to the LUA (or alternative) approach is that everything is
inside one process; you'd never have a case where the connection to the
policy agent hangs and then you have to deal with it. You also don't
require your customers to learn perl. The disadvantage to the LUA
approach is that we become dependent on another external entity to provide
this service, and users have to learn the LUA syntax to write their policy
scripts. There may be others I've not yet considered.

> what platforms do people want to maintain support for?

We've never really discussed this. OpenDKIM's antecedent attempted to
support every kind of UNIX that open source sendmail supports, but really
all we had to go on there is what people actually tried. I suppose the
right answer is "as many as possible", but of course we're constrained by
the volunteer resources we have.

We have access to Solaris, several flavours of Linux and FreeBSD among the
current participants, so it gets tested there before releases happen. I
think that's a good start.

> what performance criteria is desirable?

Minimal impact (of course), but I think adding any kind of complexity is
understood to slow things down some amount. This is a good point about
the external method though: with LUA, there's no round-trip to another
server that could be slow to respond.

> what language features are required?

We don't have any specific requests so far. To date people have been fine
with flat files or Berkeley DB hash/btree databases.

> my answers to the prior 3 questions:
>[...]
> language features:
> good database support (sqlite, mysql, postgres, MySQL, MSQL)
> good client email support - generating user confirmation emails
> good DNS support - blacklists
> good client http/https/xmlrpc support - reputation services or other
> webservices
> good http server support - sending internal user's web links to whitelist
> specific email lists/third party senders
> async implementations on DNS/http information highly desirable.
> async database support desirable too

This does lean toward doing something external since there are already
Perl and C modules for doing all of these things.

In that sense, libopendkim does have the prescan and final callback
facilities that makes doing some (but not all) of what we're talking about
possible. It's just missing hooks for more direct control over ADSP. But
interfacing customer code to C hooks in the filter is non-trivial.

Thanks for pointing out the Postfix policy stuff. I'd not seen it yet.
Food for thought as we explore this further.

-MSK
Received on Sun Oct 11 2009 - 00:32:24 PST

This archive was generated by hypermail 2.3.0 : Mon Oct 29 2012 - 23:32:29 PST