Quantcast

Body flag doesn't look into attachments in 2.7

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Body flag doesn't look into attachments in 2.7

Davide Davini
On: FreeBSD gino.bramieri.com 8.4-RELEASE-p12 FreeBSD 8.4-RELEASE-p12
#14: Thu Jun 19 15:45:31 CEST 2014

Mail drop port version: maildrop-2.7.1_1

Since [yesterday] I upgraded to 2.7[1] my scripts that use :b flag don't
seem to look into the attachment anymore. Curses.

Example:
if (/pippo/:b && /^To:.*[hidden email]/)
{
        <do stuff>
}

It works fine when "pippo" is in the, let's say, main body of the email
but it fails miserably when "pippo" is in the attachment part of the
body. It was working like a charm until the upgrade.

Did something that I do not know of change? Is it a bug? Workarounds?

Any advice is welcome.

Ciao,
Daivde

[1] from 2.6, if I recall correctly.

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Sam Varshavchik
Davide Davini writes:

> On: FreeBSD gino.bramieri.com 8.4-RELEASE-p12 FreeBSD 8.4-RELEASE-p12
> #14: Thu Jun 19 15:45:31 CEST 2014
>
> Mail drop port version: maildrop-2.7.1_1
>
> Since [yesterday] I upgraded to 2.7[1] my scripts that use :b flag don't
> seem to look into the attachment anymore. Curses.
>
> Example:
> if (/pippo/:b && /^To:.*[hidden email]/)
> {
> <do stuff>
> }
>
> It works fine when "pippo" is in the, let's say, main body of the email
> but it fails miserably when "pippo" is in the attachment part of the
> body. It was working like a charm until the upgrade.
>
> Did something that I do not know of change? Is it a bug? Workarounds?
:b should look in attachments if they are text attachments. It will no  
longer look inside binary attachments.

Starting with 2.7.0, maildrop's text search is fully UTF-8-compliant. The  
search strings must be specified in the UTF-8 codeset; and maildrop will, if  
necessary, transcode each MIME section into UTF-8 text, for searching.

This means that non-textual binary attachments are no longer searched; only  
text attachments get searched.


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Davide Davini
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sam Varshavchik wrote:
> :b should look in attachments if they are text attachments. It
> will no longer look inside binary attachments.

We are talking about text attachments, I'm processing delivery failure
notifications. If I find the IP address of my client server I forward
the undeliverable DFN to them.

> Starting with 2.7.0, maildrop's text search is fully
> UTF-8-compliant. The search strings must be specified in the UTF-8
> codeset; and maildrop will, if necessary, transcode each MIME
> section into UTF-8 text, for searching.

I mean, it works if the same text is in the "main body" so I would be
surprised if that is the problem, but I don't know really.

> This means that non-textual binary attachments are no longer
> searched; only text attachments get searched.

This is an example of the emails I'm talking about where
999.999.999.999 is the IP address I'm looking for:

Return-Path: <MAILER-DAEMON>
Received: from pino.bramieri.com (pino.bramieri.com [***.***.***.***])
        by gino.bramieri.com (8.14.7/8.14.7) with ESMTP id s5N8pGTI018808
        (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL)
        for <[hidden email]>; Mon, 23 Jun 2014 10:51:16 +0200 (CEST)
Received: from localhost (localhost)
        by pino.bramieri.com (8.14.7/8.14.7) id s5N8pG3Y095815;
        Mon, 23 Jun 2014 10:51:16 +0200 (CEST)
        (envelope-from MAILER-DAEMON)
Date: Mon, 23 Jun 2014 10:51:16 +0200 (CEST)
From: Mail Delivery Subsystem <[hidden email]>
Message-Id: <[hidden email]>
To: [hidden email]
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
        boundary="s5N8pG3Y095815.1403513476/pino.bramieri.com"
Subject: Postmaster notify: see transcript for details
Auto-Submitted: auto-generated (postmaster-notification)

This is a MIME-encapsulated message

- --s5N8pG3Y095815.1403513476/pino.bramieri.com

The original message was received at Mon, 23 Jun 2014 10:51:16 +0200
(CEST)
from localhost
with id s5N8pG3X095815

   ----- The following addresses had permanent fatal errors -----
<[hidden email]>
    (reason: 550 #5.1.1 Rejected User unknown.)

   ----- Transcript of session follows -----
... while talking to ***.***.***.***.com.:
>>> RCPT To:<[hidden email]>
<<< 550 5.1.1 <[hidden email]>... User unknown
>>> DATA
<<< 503 #5.5.1 RCPT first

- --s5N8pG3Y095815.1403513476/pino.bramieri.com
Content-Type: message/delivery-status

Reporting-MTA: dns; pino.bramieri.com
Received-From-MTA: DNS; [999.999.999.999]
Arrival-Date: Mon, 23 Jun 2014 10:51:16 +0200 (CEST)

Final-Recipient: RFC822; [hidden email]
Action: failed
Status: 5.1.1
Remote-MTA: DNS; ***.***.***.***.com
Diagnostic-Code: SMTP; 550 #5.1.0 Rejected by bounce verification.
Last-Attempt-Date: Mon, 23 Jun 2014 10:51:16 +0200 (CEST)

- --s5N8pG3Y095815.1403513476/pino.bramieri.com
Content-Type: text/rfc822-headers

Return-Path: <MAILER-DAEMON>
Received: from localhost (localhost)
        by pino.bramieri.com (8.14.7/8.14.7) id s5N8pG3X095815;
        Mon, 23 Jun 2014 10:51:16 +0200 (CEST)
        (envelope-from MAILER-DAEMON)
Date: Mon, 23 Jun 2014 10:51:16 +0200 (CEST)
From: Mail Delivery Subsystem <MAILER-DAEMON>
Message-Id: <[hidden email]>
To: <[hidden email]>
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
        boundary="s5N8pG3X095815.1403513476/pino.bramieri.com"
Subject: Returned mail: see transcript for details
Auto-Submitted: auto-generated (failure)

- --s5N8pG3Y095815.1403513476/pino.bramieri.com--



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlOoBC4ACgkQYqpk3E5VqXGIGgCZAStK5L8SfKg9wJZzF9X/PK91
U+4AniIsDuxnmHcCyVrr6WFXRDdrWqY3
=MH2v
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Sam Varshavchik
Davide Davini writes:

> > This means that non-textual binary attachments are no longer
> > searched; only text attachments get searched.
>
> This is an example of the emails I'm talking about where
> 999.999.999.999 is the IP address I'm looking for:

I see. What you're looking for has

Content-Type: message/<subtype>

and maildrop is looking only inside MIME text content type. This will need  
to be changed; I don't see a problem with doing a text search inside the  
"message" MIME content type.



------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Davide Davini
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sam Varshavchik wrote:

> Davide Davini writes:
>
>>> This means that non-textual binary attachments are no longer
>>> searched; only text attachments get searched.
>>
>> This is an example of the emails I'm talking about where
>> 999.999.999.999 is the IP address I'm looking for:
>
> I see. What you're looking for has
>
> Content-Type: message/<subtype>
>
> and maildrop is looking only inside MIME text content type. This
> will need to be changed; I don't see a problem with doing a text
> search inside the "message" MIME content type.

If I'm understanding it correctly you are saying that maildrop finds
"Content-Type: message/delivery-status" and not the content type it
aspects to find [Content-Type: text/plain or whatever] and that is why
it doesn't get parsed.

That's a bummer because those messages are generated automatically by
the MTA and I can't change them. As far as I know anyway.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlOoHjYACgkQYqpk3E5VqXFKtgCaAo6GwNTIKQiRum3QYdwnY9/N
yPQAoOLd+K4+Ylm+4mxy6fkkDSeWLfy9
=mUed
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Sam Varshavchik
Davide Davini writes:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Sam Varshavchik wrote:
> > Davide Davini writes:
> >
> >>> This means that non-textual binary attachments are no longer
> >>> searched; only text attachments get searched.
> >>
> >> This is an example of the emails I'm talking about where
> >> 999.999.999.999 is the IP address I'm looking for:
> >
> > I see. What you're looking for has
> >
> > Content-Type: message/<subtype>
> >
> > and maildrop is looking only inside MIME text content type. This
> > will need to be changed; I don't see a problem with doing a text
> > search inside the "message" MIME content type.
>
> If I'm understanding it correctly you are saying that maildrop finds
> "Content-Type: message/delivery-status" and not the content type it
> aspects to find [Content-Type: text/plain or whatever] and that is why
> it doesn't get parsed.
>
> That's a bummer because those messages are generated automatically by
> the MTA and I can't change them. As far as I know anyway.
Correct. maildrop will need to be changed, so that it can search inside the  
message/delivery-status MIME content.

As a workaround in the meantime, you can have maildrop use fgrep:

FOUND=`fgrep -q 999.999.999.999; echo \$?`

Something like that. Might need experimenting, to get the syntax right.


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Davide Davini

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
Sam Varshavchik wrote:
>
> As a workaround in the meantime, you can have maildrop use fgrep:
>
> FOUND=`fgrep -q 999.999.999.999; echo \$?`
>
> Something like that. Might need experimenting, to get the syntax right.

Yep, I did this:
FOUND=`fgrep -m1 999.999.999.999`
if ($FOUND == /999.999.999.999/ && /^To:.*[hidden email]/)
{
    <do stuff>
}

So far so good. Thanks a bunch!

Ciao,
Davide
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
 
iEYEARECAAYFAlOoOgkACgkQYqpk3E5VqXFEHACg8VhBbxh3kRdTBz+nxGJAe1Ad
mqQAnjF20F7+maxrzNAjoF/tm1VauJGH
=cEz9
-----END PGP SIGNATURE-----


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Body flag doesn't look into attachments in 2.7

Davide Davini
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 23/06/2014 16:30, Davide Davini wrote:
> Yep, I did this: FOUND=`fgrep -m1 999.999.999.999` if ($FOUND ==
> /999.999.999.999/ && /^To:.*[hidden email]/) { <do stuff> }

I was checking old emails when I noticed this post of mine.

It works much better like this.

if (/^To:.*[hidden email]/)
{
        FOUND=`fgrep -m1 -o 999.999.999.999`
        if ( $FOUND == 999.999.999.999)
        {
                <do stuff>
        }
}

Bye,
Davide
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (MingW32)

iEYEARECAAYFAlRmKK0ACgkQYqpk3E5VqXF4sACffq/qR21IBLbMT+uw7jeMw91V
85AAoJXoTjWyx/ifKTbQmOhxr2Bl2z+p
=giXt
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
Courier-maildrop mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/courier-maildrop
Loading...