Re: Design ideas for stats and LDAP caching in 2.1.0

From: Daniel Black <daniel.subs_at_internode.on.net>
Date: Sun, 28 Mar 2010 01:55:15 +1100

On Saturday 27 March 2010 04:41:12 Murray S. Kucherawy wrote:
>
> Stats:
> - still use Sleepycat for persistent storage
ok
> - have a "sentinel" record, key of just "." or "_at_" or something, that
> contains version information so that future iterations know how to
> interpret the records (e.g. "opendkim-stats" can always read older
> databases)
ok
> - new version "2" starts to be used
> - new binaries won't write to old DBs (log an error and disable stats)
> - v2 stats keys are just domain names
> - v2 stats values are structs containing most or all of the big array of
> stats currently suggested in RFE #SF2964378
> - "opendkim-stats" would have a new flag that hashes domain names on
> output so that data submission is somewhat anonymized but can still be
> collated
>
> LDAP caching, to alleviate repetitive LDAP server load:
> - enabled by a new --enable flag; forces --with-db
ok
> - creates an in-core (NULL path) Sleepycat hash whose keys are LDAP
> queries and whose values are previously discovered results and TTLs
> (configurable)
ok
> - on a query, check that DB for a cached result; use it if it's there
> - if not, check a linked list for pending queries (pending queries have
> pthread conditions associated with them)
> -- if there's a pending query for the same thing, go to sleep waiting on
> the associated condition
> -- if there's no pending query, create a new linked list node with
> associated pthread condition and start an LDAP query; go to sleep waiting
> for the result
async call back instead?

> -- when the reply comes in, put it in the cache DB and signal
> corresponding pthread condition; all blocked query threads wake up and get
> the answer from the DB cache
Received on Sat Mar 27 2010 - 14:57:26 PST