Re: Scalability

From: SM <sm_at_resistor.net>
Date: Sat, 19 Dec 2009 08:24:48 -0800

Hi Murray,
At 14:10 18-12-2009, Murray S. Kucherawy wrote:
>First, adding LDAP to the opendkim database stuff will be a chore,
>not to mention the possible side effort of getting a schema
>published at the IETF. Documentation for OpenLDAP is reputed to be
>poor, and there's not much in the way of examples to work from
>either. But I don't expect this is the last time we'll get such a
>request, so it really should be on the roadmap.

There are two parts, implementing and publishing. We should separate the two.

>Second, it's a bit of an oversight for me not to have tied the
>KeyList into the new database code somehow. The catch is that the
>KeyList supports regular expressions, where querying a database
>can't do so; you'd have to give a user_at_host or just a host, and it
>comes back with a "no" or "yes, with this selector/key". It's hard
>to do that using a large-scale database like LDAP or SQL; you can't
>(as far as I know, but could be wrong) have an entry containing a
>wildcard, so that a query for "xyz" returns a stored value for
>"*y*". Obviously the opposite is true, but that doesn't help us
>here. So if we tie the KeyList to the new database code, we lose
>pattern matching. But maybe that's okay because nobody really needs it.

Regular expressions would not be optimal for the database
end. Whether it is LDAP or SQL, we still have to define the
"query". That could be user customizable. We define the results we
expect. If there are no results, then it is a "no".

We do not need pattern matching as that is done on the database
side. We pass user_at_host to the query where some processing is done
to determine whether to query for user_at_host or a wildcard
variant. The result returned is the selector and the key. We might
as well keep the key in the database as someone will ask for that.

>So I propose this: We leave the KeyList in there for now in case
>someone is using it for the pattern matching, but also add a new
>database called the KeyTable. The database thus referenced has to
>be able to do key-value queries (SQL or BDB, really), and the keys
>would be domains taken from From: header fields, and the values
>would be of the from "name:sigkey" where "name" is the name of the
>selector to be used for signing (the domain would be the From:
>domain) and "sigkey" is the text representation of the private key
>to be used to generate signatures. Both the BDB and SQL approaches
>lend themselves to changes to the underlying data without having to
>reconfigure or restart opendkim, and LDAP will as well once
>added. And if nobody's really using the KeyList feature, we
>eventually deprecate and remove it.

If we have KeyTable enabled, we should not have KeyList or else it
gets complicated. I am using the KeyList feature. :-)

The topic of this message could have been customization instead of
scalability as we can associate different values for the DKIM
signing. This "feature" may be useful for verification too. That
would be part of the policy stuff where we can customize the actions
to be performed.

Regards,
-sm
Received on Sat Dec 19 2009 - 16:25:31 PST