Re: test 70 error on 1.2.0

On Thursday 10 December 2009 06:06:23 Murray S. Kucherawy wrote:
> On Wed, 9 Dec 2009, Daniel Black wrote:
> > with diffheaders (tre version - 0.7.5)
> >
> > still looking...
>
> Sorry, missed the attachment:
> > Breakpoint 1, dkim_eom (dkim=0x8a8cd0, testkey=0x0) at dkim.c:3646
> > 3646 else if (sig->sig_error == DKIM_SIGERROR_KEYFAIL)
> > (gdb) p sig->sig_error
> > No symbol "sig" in current context.
> > (gdb) p dkim->dkim_siglist[0]->sig_error
> > $1 = 24
>
> Yep, that's DKIM_SIGERROR_KEYFAIL.
>
> Did you run this test specifically (e.g. "./t-test70" from the shell
> prompt)? If so, the keys it needs to verify probably aren't available.

quite right - this was the case.

> This is a holdover from the old "devtools" test environment. Maybe the
> right thing to do now is to have that file always present in the "tests"
> directory, and we can drop t-setup and t-cleanup.

sound good.

now that I'm looking at the right error is seems to be an environment specific problem that results in a tre regex structure initialisation problem.

using tre-0.7.5 with --enable-system-abi

in gentoo compile environment where it fails:

/var/tmp/portage/mail-filter/opendkim-1.2.0/work/opendkim-1.2.0/libopendkim/tests $ gdb ./.libs/t-test70
GNU gdb 6.8
This GDB was configured as "x86_64-pc-linux-gnu"...
(gdb) run
Starting program: /var/tmp/portage/mail-filter/opendkim-1.2.0/work/opendkim-1.2.0/libopendkim/tests/.libs/t-test70
[Thread debugging using libthread_db enabled]
[New Thread 0x7f2cc5c56710 (LWP 5372)]
[New Thread 0x7f2cc3050950 (LWP 5375)]
*** relaxed/simple rsa-sha256 verifying with header diffing

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f2cc5c56710 (LWP 5372)]
0x00007f2cc3f6b9a6 in tre_tnfa_run_approx (tnfa=0x1f52930, string=0x1f4d240, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff2fa99fb0,
default_params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>, match_end_ofs=0x7fff2fa9968c) at tre-match-approx.c:353
353 tre-match-approx.c: No such file or directory.
        in tre-match-approx.c
(gdb) bt
#0 0x00007f2cc3f6b9a6 in tre_tnfa_run_approx (tnfa=0x1f52930, string=0x1f4d240, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff2fa99fb0,
    default_params={cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>, match_end_ofs=0x7fff2fa9968c) at tre-match-approx.c:353
#1 0x00007f2cc3f6b3a0 in tre_match_approx (tnfa=0x1f52930, string=0x1f52930, len=32821825, type=2147483647, match=0x7fff2fa99fb0, params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>) at regexec.c:297
#2 0x00007f2cc586ef9e in dkim_diffheaders (dkim=0x1f4ccd0, maxcost=<value optimized out>, ohdrs=0x7fff2fa9b050, nohdrs=8,
out=0x7fff2fa9b258,
    nout=0x7fff2fa9b274) at dkim.c:5400
#3 0x00000000004011fb in main (argc=<value optimized out>, argv=<value optimized out>) at t-test70.c:183
(gdb) list
348 been found. */
349 if (match_costs[TRE_M_COST] > 0)
350 {
351 tre_tnfa_transition_t *trans;
352 DPRINT((" init"));
353 for (trans = tnfa->initial; trans->state; trans++)
354 {
355 int id = trans->state_id;
356
357 /* If this state is not currently in `reach_next', add it
(gdb) p trans
$1 = (tre_tnfa_transition_t *) 0x319
(gdb) p trans->state
Cannot access memory at address 0x321
(gdb) p match_costs
$2 = {2147483647, 2147483647, 2147483647, 2147483647, 2147483647}
(gdb) p tnfa
$3 = (const tre_tnfa_t *) 0x1f52930
(gdb) p *tnfa
$4 = {transitions = 0x1f5b430, num_transitions = 804, initial = 0x319, final = 0x1f7b6e0, submatch_data = 0x0, firstpos_chars = 0x1f7d020 "\001",
  first_char = 33030768, num_submatches = 0, tag_directions = 0x0, minimal_tags = 0x1f5e680, num_tags = 33068592, num_minimals = 0, end_tag =
33068976,
  num_states = 0, cflags = 33069360, have_backrefs = 0, have_approx = 33069760, params_depth = 0}

Seen here the tnfa->initial isn't a memory location allocated to the program and first_char and other data elements substantially differ from the working
structure below:

/tmp/opendkim-1.2.0/libopendkim/tests $ gdb .libs/t-test70
GNU gdb 6.8
This GDB was configured as "x86_64-pc-linux-gnu"...
(gdb) run
Starting program: /tmp/opendkim-1.2.0/libopendkim/tests/.libs/t-test70
[Thread debugging using libthread_db enabled]
*** relaxed/simple rsa-sha256 verifying with header diffing

Program exited normally.
(gdb) break tre_tnfa_run_approx
Breakpoint 1 at 0x7fc2bb5346a4: file tre-match-approx.c, line 222. (2 locations)
(gdb) run
Starting program: /tmp/opendkim-1.2.0/libopendkim/tests/.libs/t-test70
[Thread debugging using libthread_db enabled]
*** relaxed/simple rsa-sha256 verifying with header diffing
[New Thread 0x7f4663eb8710 (LWP 11851)]
[Switching to Thread 0x7f4663eb8710 (LWP 11851)]

Breakpoint 1, tre_tnfa_run_approx (tnfa=0x15d0510, string=0x15a1aa0, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff8eabf200,
default_params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=0,
    match_end_ofs=0x7fff8eabe90c) at tre-match-approx.c:222
222 {
(gdb) bt
#0 tre_tnfa_run_approx (tnfa=0x15d0510, string=0x15a1aa0, len=-1, type=STR_BYTE, match_tags=0x0, match=0x7fff8eabf200, default_params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5}, eflags=0,
    match_end_ofs=0x7fff8eabe90c) at tre-match-approx.c:222
#1 0x00007f4662ca43a0 in tre_match_approx (tnfa=0x15d0510, string=0x15a1aa0, len=18446744073709551615, type=STR_BYTE,
match=0x7fff8eabf200, params=
      {cost_ins = 1, cost_del = 1, cost_subst = 2, max_cost = 5, max_ins = 4096, max_del = 4096, max_subst = 4096, max_err = 5},
    eflags=<value optimized out>) at regexec.c:297
#2 0x00007f4663ace2fc in dkim_diffheaders (dkim=0x15a1510, maxcost=5, ohdrs=0x7fff8eabf2d0, nohdrs=8, out=0x7fff8eac04e0,
nout=0x7fff8eac04f4)
    at dkim.c:5400
#3 0x00000000004015b4 in main (argc=1, argv=0x7fff8eac05e8) at t-test70.c:183
(gdb) p *tnfa
$1 = {transitions = 0x160ab30, num_transitions = 1573, initial = 0x16234c0, final = 0x1623430, submatch_data = 0x0, firstpos_chars = 0x0, first_char
= 68,
  num_submatches = 1, tag_directions = 0x0, minimal_tags = 0x0, num_tags = 0, num_minimals = 0, end_tag = 0, num_states = 787, cflags = 8,
  have_backrefs = 0, have_approx = 0, params_depth = 0}
(gdb) p regcomp
$2 = {int (regex_t *, const char *, int)} 0x7f4662ca3f00 <regcomp>
(gdb) p regaexec
$3 = {int (const regex_t *, const char *, regamatch_t *, regaparams_t, int)} 0x7f4662ca4530 <regaexec>

trying tre-0.8.0 results in the same SEGV only in the gentoo build environment

tre-0.7.5 compiled without --enable-system-abi wouldn't compile with opendkim

tre-0.8.0 compiled without --enable-system-abi passes tests in both gentoo build environment and outside.

I'm therefore assuming that this is a odd library intialision problem where the regex structure is intialised by glibc and then tested against libtre.

Murray - think you were 100% right when you posted "Concern about libtre and feature conflict" on 2009-11-23.

options:
1. check for this type of failure in configure.ac
2. not support system-abi (NetBSD and maybe others will start using this as the system default - http://laurikari.net/tre/netbsd-takes-tre-in-the-c-
library/#comments which would be convenient )
3. lots more ugly hacks

I'll ask the gentoo maintainer to disabling --system-abi in future

debian seems to have gone the same approach 0.8.0-1 http://packages.debian.org/changelogs/pool/main/t/tre/tre_0.8.0-2/changelog
Received on Mon Dec 14 2009 - 01:37:36 PST