More on databases

From: Murray S. Kucherawy <msk_at_cloudmark.com>
Date: Thu, 7 Jan 2010 10:53:36 -0800

The idea of adding LDAP support will likely make our DB layer a lot more complicated. I have some ideas of how to tackle this but maybe there are some better ones out there.

The problem: So far we've gotten away with our database formats being text records with field delimiters, like the KeyList whose values are of the form domain:selector:key-path. Although we could do this with LDAP and have with SQL, those two mechanisms are really designed to have different fields in different columns (SQL) or attributes (LDAP). This would mean, in order to accommodate the more advance databases, there has to be some way to make a query in to a DB handle and request specific attributes back out. We would also have to maintain downward compatibility with the existing simpler mechanisms so that we don't need to split up the API or overhaul configuration.

I'm thinking about a "get" operation that, rather than providing a buffer to receive the query, provides an array of buffers to retrieve specific attributes. So one would make a call that asks for record "r" and attributes "attr1", "attr2", and so on, providing a destination buffer and length for each. On return, the lengths would be set to the size of the returned data or to -1 if there was no such attribute. For flat files, regex files and hash/btree files for which the value is of the old "x:y:z:..." form, the attributes would just be filled in left-to-right without regard for the names of the attributes, returning -1 lengths for attributes if there are more requested than were present in the record. For SQL and LDAP, returned values would line up with their specific columns (SQL) or attributes (LDAP).

Make sense? Anyone have another idea?

The next trick will be to figure out a way to specify, within the existing framework, all the parameters we need to configure an LDAP connection. It's complicated.

-MSK
Received on Thu Jan 07 2010 - 18:53:46 PST

This archive was generated by hypermail 2.3.0 : Mon Oct 29 2012 - 23:32:52 PST