RE: converting all newlines to CRLF

From: Murray S. Kucherawy <msk_at_blackops.org>
Date: Sat, 6 Apr 2013 00:13:11 -0700 (PDT)

On Sat, 6 Apr 2013, Vinubalaji Gopal wrote:
> I believe I get the message that way, but I can try to do more checks to
> verify that. Basically we have a program which gets all the email (a lot
> of spam messages) we receive and writes the message as it receives. I am
> just using those messages to pass through opendkim using the dkim-chunk
> api/opendkim-testmsg. If i use the opendkim binary it does the
> conversion properly and the dkim verification passes. I am interested
> to know why the following code in dkim_header of libopendkim/dkim.c
> doesn't handle the case:
>
> if (*p == '\n' && prev != '\r') /* bare LF */
> {
> dkim_dstring_catn(tmphdr, CRLF, 2);
> }

It's guarded inside this:

   if ((dkim->dkim_libhandle->dkiml_flags & DKIM_LIBFLAGS_FIXCRLF) != 0)

The opendkim filter itself doesn't default that flag on, so the cited
conversion doesn't normally get executed.

> The patch I created pretty much does the same thing but in dkim_chunk
> instead of dkim_header.

There's a long-standing goal here not to do those kinds of conversions
unless specifically requested. There might be some applications where
it's important to preserve whatever line termination was there in the
first place, and so the library and the filter both default to that
assumption. opendkim-testmsg forces that correction flag on, but it's a
test tool and not meant for production use.

I think you could argue that dkim_chunk() shouldn't be adaptive based on
the first newline found and instead should evaluate each one
independently, but that would only change the handling of things like your
special case where the line termination varies from one part of the
message to the next. That means the caller is either receiving or
producing some pretty broken input. I'm not so sure we're fixing your
problem in the right place here.

> [...]
> And I notice that it does convert the folded headers with bare newlines
> to folded headers with carriage returns in both the cases (one with the
> patch and one without the patch). I tried to debug and see how doing it
> in dkim_chunk make the verification pass and doing the same thing in
> dkim_header doesn't pass. But it got complicated with the rsa signatures
> and I couldn't easily debug after that.

If you hit the RSA code, you've gone too far.

Bare newline fixing in dkim_header() is only done if the FIXCRLF flag is
enabled. They're preserved otherwise. It's different in mlfi_header()
(which is in the filter, not the library) because that function is meant
to interface to milter clients which do pass bare linefeeds when header
fields are folded, but no line termination at the end. That's an artifact
of the milter protocol, and one of the reasons why the two interfaces are
slightly different (and, hence, why opendkim and opendkim-testmsg can
produce differing results in certain cases, as you observed).

-MSK
Received on Sat Apr 06 2013 - 07:13:43 PST

This archive was generated by hypermail 2.3.0 : Sat Apr 06 2013 - 07:18:01 PST