[nmh-workers] mhstore and encoded filename | mhfixmsg

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
nmh
Reply | Threaded
Open this post in threaded view
|

[nmh-workers] mhstore and encoded filename | mhfixmsg

nmh
Consider:

    $ mhlist 3677
     msg part  type/subtype              size description
    3677       multipart/mixed            34K
             boundary="----=_Part_985234_1081815005.1565970594687"
         1     multipart/alternative      598
             boundary="----=_Part_985233_613472701.1565970594686"
         1.1   text/plain                  61
             charset="UTF-8"
         1.2   text/html                  256
             charset="UTF-8"
         2     application/msword         24K
             disposition "attachment"
               filename="=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?="

header snip from msg 3677:

    ------=_Part_985234_1081815005.1565970594687
    Content-Type: application/msword
    Content-Transfer-Encoding: base64
    Content-Disposition: attachment;
     filename="=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?="
    Content-ID: <[hidden email]>

mhstore is not decoding the filename:

    $ mhstore -auto -part 2
    storing message 3677 part 2 as file [...]/=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?=

I was about to post for help, then I tried:

    $ mhfixmsg -outfile - 3677 | mhlist -file -
    mhfixmsg: did not find semicolon in Content-Type: application/msword


     msg part  type/subtype              size description
       0       multipart/mixed            34K
                 boundary="----=_Part_985234_1081815005.1565970594687"
         1     multipart/alternative      599
                 boundary="----=_Part_985233_613472701.1565970594686"
         1.1   text/plain                  61
                 charset="UTF-8"
         1.2   text/html                  257
                 charset="UTF-8"
         2     application/msword         24K
                 disposition "attachment"
                   filename="gmgp cochair duties.doc"

So maybe this post will help someone else.

"mhfixmsg rewrites MIME messages, applying specific transformations such as
decoding of MIME-encoded message parts and repairing invalid MIME headers" It
would be interesting to hear some background on why these repairs are needed.
msg 3677 is from yahoo email.  I assume some email clients are worse than
others / which ones?

I'm not running mhfixmsg on each message, so I have to remember this workaround.

--
thanks!,
Tom


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: mhstore and encoded filename | mhfixmsg

Ralph Corderoy
Hi Tom,

>     Content-Type: application/msword
>     Content-Transfer-Encoding: base64
>     Content-Disposition: attachment;
>      filename="=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?="
>     Content-ID: <[hidden email]>

I put this into a test email.

    $ cat `mhpath .`
    To: ralph
    From: ralph
    MIME-Version: 1.0
    Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
    Content-ID: <[hidden email]>
    Date: Sat, 17 Aug 2019 13:18:14 +0100
    Message-Id: <[hidden email]>

    ------- =_aaaaaaaaaa0
    Content-Type: application/msword
    Content-Transfer-Encoding: base64
    Content-Disposition: attachment;
     filename="=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?="
    Content-ID: <[hidden email]>

    YmFyCg==

    ------- =_aaaaaaaaaa0--
    $

>     $ mhfixmsg -outfile - 3677 | mhlist -file -
>     mhfixmsg: did not find semicolon in Content-Type: application/msword
>
>

I wonder why mhfixmsg produces those blank stderr lines?  Ah, it uses
inform() and that adds one itself,

    /* inform calls advertise() with no what and no tail.
     * Thus the simple "[invo_name: ]fmt\n" results. */
    void
    inform(char *fmt, ...)

but the value for fmt that's passed in has one,

    inform("did not find semicolon in %s:%s\n", hf->name, hf->value);

and my guess is the third is in hf->value interpolated for the `%s'
before `\n'.

Anyway, popping out of that rabbit hole...

>      msg part  type/subtype              size description
...
>          2     application/msword         24K
>                  disposition "attachment"
>                    filename="gmgp cochair duties.doc"

Note though that it doesn't add in a semicolon.

    $ mhfixmsg -outfile - . |
    > diff `mhpath .` -
    mhfixmsg: did not find semicolon in Content-Type: application/msword


    12,13c12
    < Content-Disposition: attachment;
    <  filename="=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?="
    ---
    > Content-Disposition: attachment; filename="gmgp cochair duties.doc"
    $

And it only noticed the problem because it was altering the
Content-Disposition field; once that's fixed it doesn't grumble.

    $ mhfixmsg -outfile - . |
    > mhfixmsg -file - -outfile /dev/null 2> >(wc -c)
    mhfixmsg: did not find semicolon in Content-Type: application/msword


    0
    $

The second mhfixmsg's stderr is empty despite being fed the same field.

--
Cheers, Ralph.

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: mhstore and encoded filename | mhfixmsg

Paul Fox-3
unrelated to current topic...

ralph wrote:
 >     $ cat `mhpath .`

I can't believe I've been using MH for almost 30 years and only just
now found out that '.' is a synonym for 'cur'.

paul
=----------------------
paul fox, [hidden email] (arlington, ma, where it's 65.8 degrees)


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: mhstore and encoded filename | mhfixmsg

Ken Hornstein-2
In reply to this post by nmh
>mhstore is not decoding the filename:
>
>    $ mhstore -auto -part 2 storing message 3677 part 2 as file
>    [...]/=?UTF-8?b?Z21ncCBjb2NoYWlyIGR1dGllcy5kb2M=?=

Unfortunately, RFC 2047 encoding is explicitly forbidden to be used for
MIME parameter values.  There is a different standard for encoding MIME
parameter values (RFC 2231) that nmh supports.  But it seems that a lot
of MUAs still get this wrong.  I'd rather leave it that mhfixmsg will
correct this instead of natively decoding it.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: mhstore and encoded filename | mhfixmsg

David Levine-3
In reply to this post by nmh
Tom writes:

> It would be interesting to hear some background on why these repairs are needed.

Ken already mentioned the RFCs, but let me point out:

  mhfixmsg applies two transformations unconditionally.  The
  first removes an extraneous trailing semicolon from the
  parameter lists of MIME header field values.  The second
  replaces RFC 2047 encoding with RFC 2231 encoding of name and
  filename parameters in Content-Type and Content-Disposition
  header field values, respectively.

> msg 3677 is from yahoo email.  I assume some email clients are worse than
> others / which ones?

I usually don't identify specific email clients:  the fixes apply to a message from any client.

Ralph, I'll fix the extra newlines, thanks.  As far as the warning about the missing semicolon goes, I don't think it's necessary or even correct. If I'm reading the RFCs correctly now, parameters are optional for Content-Type and Content-Disposition.

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers