[nmh-workers] Multiple identities and envelope-from

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
36 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[nmh-workers] Multiple identities and envelope-from

Conrad Hughes
Quick question: any suggestion where I should start looking for stuff to
do with envelope-from in my outgoing emails?  Does this entry in the
very first Received: header in my outgoing emails come from NMH or exim4
(which provides my sendmail)?

  Received: from conrad by sendinghost with local (Exim 4.89)
          (envelope-from <[hidden email]>)
          id blahblahblah
          for [hidden email]; Sat, 06 Jul 2019 16:55:48 +0100

I've been given a third party email account to use, with an appalling
webmail interface, and while I can fetchmail from it using IMAP and get
exim4 to route via its SMTP host, I'm not masquerading perfectly when I
edit a "From: [hidden email]" into my outgoing email in NMM: the
emails that go out still have an "envelope-from: [hidden email]"
in them.  Looks as if it's added as they go from NMH to exim4/sendmail.

I saw some discussion in the archives of an Envelope-from header, but it
appears to have no effect.

Anybody have something like this working better?

C.

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Multiple identities and envelope-from

Conrad Hughes
Tried grep'ing all the NMH docs installed on my machine, and indeed news
from 1.5 (I'm on 1.6-16) documents post's support of both Envelope-From:
and Sender:, but both of these simply end up in the outgoing email and
have no effect on the (envelope-from my-private-address) in the first
Received: header.  Have tried bare email address as well as "my name
<[hidden email]>" forms.  Any suggestion as to what I'm doing
wrong?

Conrad

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Multiple identities and envelope-from

Ken Hornstein-2
In reply to this post by Conrad Hughes
>Quick question: any suggestion where I should start looking for stuff to
>do with envelope-from in my outgoing emails?  Does this entry in the
>very first Received: header in my outgoing emails come from NMH or exim4
>(which provides my sendmail)?

How exactly are you submitting email?  smtp, sendmail/smtp, or sendmail/pipe?

If it's the last one, then that interface has reduced functionality and
things like Envelope-From won't work (and you only need that in very
unusual circumstances and this doesn't seem like one of them); you'll
have to look at your exim docs.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Multiple identities and envelope-from

Ken Hornstein-2
In reply to this post by Conrad Hughes
>Anybody have something like this working better?

To expand on this a bit ... yes, I believe I do.

I have configured post(8) to submit mail via SMTP directly to the
appropriate server based on my From line (I do this for my work email
and my personal email).  I do this via a custom postproc, but this is
not required; you can also use the "sendfrom" profile entries.
In this mode the default SMTP envelope address is based on your From
header and everything works as expected (I don't remember if sendfrom
is in 1.6; it certainly is in 1.7).

See send(1) for the appropriate switches for using SMTP and the details
of sendfrom (hm, it may not have existed directly in send for 1.6, but
it looks like there was contrib/sendfrom).

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Multiple identities and envelope-from

Conrad Hughes
In reply to this post by Ken Hornstein-2
> How exactly are you submitting email?  smtp, sendmail/smtp, or sendmail/pipe?

sendmail/pipe, the reduced functionality one.  But if I telnet 25 it
seems to invoke exim directly, and changing my mts.conf to 'smtp'
(server was already localhost) accordingly seems to work.

.. and the envelope-from is now correctly '[hidden email]', so all
appears to be fine from just making that switch to smtp.

Is there anything potentially problematic I should be aware of about
invoking via smtp like this, instead of pipe?

Otherwise thanks very much for a very quick solution!

C.

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Multiple identities and envelope-from

Ken Hornstein-2
>> How exactly are you submitting email? smtp, sendmail/smtp, or
>> sendmail/pipe?
>
>sendmail/pipe, the reduced functionality one.  But if I telnet 25 it
>seems to invoke exim directly, and changing my mts.conf to 'smtp'
>(server was already localhost) accordingly seems to work.
>
>.. and the envelope-from is now correctly '[hidden email]', so all
>appears to be fine from just making that switch to smtp.
>
>Is there anything potentially problematic I should be aware of about
>invoking via smtp like this, instead of pipe?

I can't think of anything, except that you need to make sure exim is
configured correctly (it sounds like that has already been done).  And
as a warning, future versions of nmh default to the SMTP submission port
so maybe you want to add a "-port 25" under send in your profile, or
make sure exim is also listening on the submission port (587).

I personally want as few moving parts as possible so that's why I
configure nmh to submit directly to my provider's SMTP servers, but
either way should work fine.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

[nmh-workers] logging outgoing messages

Steven Winikoff-2
In reply to this post by Ken Hornstein-2
I recently modified my configuration for nmh-1.7.1 to connect directly to
my ISP's sendmail, rather than going through sendmail on my desktop Linux
system.

This works perfectly, but as a side effect I lost all logging of outgoing
messages.  This isn't the end of the world, but it's a pain because there
are times when outgoing messages may need to be traced, and in cases like
that it's important to be able to quote the ISP's assigned QID.

...so I hacked the appended patch together, and it seems to work, but I'm
sure there must be a better way to do this.  Problems with my code include
(but are almost certainly not limited to) the following:

    - It doesn't check to see whether it's connecting to a local sendmail
      (if so, each message will be logged twice).

    - It logs successful delivery, but not delivery failures (no matter
      the reason).

    - It seems to break two of the tests in make check:

         # make check
         [...]
         PASS: test/post/test-messageid
         send: message not delivered to anyone
         FAIL: test/post/test-mts
         send: message not delivered to anyone
         FAIL: test/post/test-post-aliases
         PASS: test/post/test-post-basic
         [...]
         =======================================
         2 of 112 tests failed
         Please report to [hidden email]
         =======================================
         Makefile:4763: recipe for target 'check-TESTS' failed
         make[1]: *** [check-TESTS] Error 1
         make[1]: Leaving directory '/big/local/pkg/nmh/nmh-1.7.1'
         Makefile:5019: recipe for target 'check-am' failed

Is there any interest in adding an improved version of this to the code
base?

     - Steven


8<-----------------------------   cut here   ---------------------------->8
--- h/mts.h.original 2017-11-22 09:37:56.000000000 -0500
+++ h/mts.h 2019-07-09 16:51:34.260314328 -0400
@@ -57,3 +57,15 @@
  * Global MailDelivery File
  */
 extern char *maildelivery;
+
+#ifndef NOSYSLOG
+#  define SYSLOG_FIELD_SIZE 1024
+#  define SYSLOG_FIELD_LAST 1023
+
+   char syslog_from  [SYSLOG_FIELD_SIZE];
+   char syslog_to    [SYSLOG_FIELD_SIZE];
+   char syslog_msgid [SYSLOG_FIELD_SIZE];
+   char syslog_server[SYSLOG_FIELD_SIZE];
+   char syslog_port  [SYSLOG_FIELD_SIZE];
+   char syslog_qid   [SYSLOG_FIELD_SIZE];
+#endif
--- mts/smtp/smtp.c.original 2018-03-06 14:05:55.000000000 -0500
+++ mts/smtp/smtp.c 2019-07-09 17:17:34.382593987 -0400
@@ -13,6 +13,9 @@
 #include <h/netsec.h>
 
 #include <sys/socket.h>
+#ifndef NOSYSLOG
+   #include <syslog.h>
+#endif
 
 /*
  * This module implements an interface to SendMail very similar
@@ -74,6 +77,24 @@
          int debug, int sasl, const char *saslmech, const char *user,
          const char *oauth_svc, int tls)
 {
+#ifndef NOSYSLOG
+    /**  ensure fields are properly initialized to empty strings:  **/
+
+    syslog_from [0] = '\0';
+    syslog_to   [0] = '\0';
+    syslog_msgid[0] = '\0';
+    syslog_qid  [0] = '\0';
+
+
+    /**  ...except server and port, which can take their real values:  **/
+
+    (void)strncpy(syslog_server, server, SYSLOG_FIELD_SIZE);
+    syslog_server[SYSLOG_FIELD_LAST] = '\0';
+
+    (void)strncpy(syslog_port, port, SYSLOG_FIELD_SIZE);
+    syslog_port[SYSLOG_FIELD_LAST] = '\0';
+#endif
+
     if (sm_mts == MTS_SMTP)
  return smtp_init (client, server, port, watch, verbose,
   debug, sasl, saslmech, user, oauth_svc, tls);
@@ -454,6 +475,13 @@
         }
     }
 
+#ifndef NOSYSLOG
+    /**  record from field for syslog:  **/
+
+    (void)strncpy(syslog_from, from, SYSLOG_FIELD_SIZE);
+    syslog_from[SYSLOG_FIELD_LAST] = '\0';
+#endif
+
     switch (smtalk (SM_MAIL, "MAIL FROM:<%s>%s", from, mail_parameters)) {
  case 250:
     sm_addrs = 0;
@@ -473,6 +501,37 @@
 int
 sm_wadr (char *mbox, char *host, char *path)
 {
+#ifndef NOSYSLOG
+    {
+       /**  record recipient address for syslog:  **/
+
+       char address[SYSLOG_FIELD_SIZE];
+       int  length, used, remaining;
+
+       if (host && *host)
+          (void)snprintf(address, SYSLOG_FIELD_SIZE, "<%s%s@%s>",
+                                  FENDNULL(path), mbox, host);
+       else
+          (void)snprintf(address, SYSLOG_FIELD_SIZE, "<%s%s>",
+                                  FENDNULL(path), mbox);
+
+       length    = strlen(address);
+       used      = strlen(syslog_to);
+       remaining = (SYSLOG_FIELD_SIZE - used) - 1;
+
+       if ((used == 0) && (length < remaining))
+       {
+          (void)strcat(syslog_to, address);
+       }
+       else if ((length + 1) < remaining)
+       {
+          (void)strcat(syslog_to, " ");
+          (void)strcat(syslog_to, address);
+       }
+
+       syslog_to[SYSLOG_FIELD_LAST] = '\0';
+    }
+#endif
     switch (smtalk (SM_RCPT, host && *host ? "RCPT TO:<%s%s@%s>"
    : "RCPT TO:<%s%s>",
      FENDNULL(path), mbox, host)) {
@@ -717,6 +776,19 @@
     }
 
     for (bp = buffer; bp && len > 0; bp++, len--) {
+#ifndef NOSYSLOG
+        if (strncmp(bp, "Message-ID: ", 12) == 0)
+        {
+           int i;
+
+           (void)strncpy(syslog_msgid, bp + 12, SYSLOG_FIELD_SIZE);
+           for (i=0; i<SYSLOG_FIELD_LAST; i++)
+              if (syslog_msgid[i] == '\012')
+                 syslog_msgid[i] = '\0';
+
+           syslog_msgid[SYSLOG_FIELD_LAST] = '\0';
+        }
+#endif
  switch (*bp) {
     case '\n':
  sm_nl = TRUE;
@@ -858,6 +930,15 @@
 
  sm_reply.length = rp - sm_reply.text;
  sm_reply.text[sm_reply.length] = 0;
+#ifndef NOSYSLOG
+        if (strncmp(sm_reply.text, "OK id=", 6) == 0)
+        {
+           (void)snprintf(syslog_qid, SYSLOG_FIELD_SIZE,
+                          "Sent (%s Message accepted for delivery)",
+                          sm_reply.text+6);
+           syslog_qid[SYSLOG_FIELD_LAST] = '\0';
+        }
+#endif
  return sm_reply.code;
     }
 
--- uip/post.c.original 2018-03-06 14:05:56.000000000 -0500
+++ uip/post.c 2019-07-09 17:18:58.345425453 -0400
@@ -23,6 +23,10 @@
 #endif
 #include <time.h>
 
+#ifndef NOSYSLOG
+   #include <syslog.h>
+#endif
+
 #include <mts/smtp/smtp.h>
 
 #ifndef CYRUS_SASL
@@ -1760,6 +1764,15 @@
         }
 
         fflush (stdout);
+
+#ifndef NOSYSLOG
+        openlog("nmh_smtp", LOG_PID, LOG_MAIL);
+        syslog(LOG_NOTICE,
+               "from=%s, to=%s, msgid=%s, relay=%s, port=%s, stat=%s",
+               syslog_from, syslog_to, syslog_msgid, syslog_server,
+               syslog_port, syslog_qid);
+        closelog();
+#endif
     }
 }
 
8<-----------------------------   cut here   ---------------------------->8
--
___________________________________________________________________________
Steven Winikoff                |
Montreal, QC, Canada           | "I'd love to go out with you, but I want
[hidden email]               |  to spend more time with my blender."
http://smwonline.ca            |
                               |                            - fortune(6)

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Ken Hornstein-2
>Is there any interest in adding an improved version of this to the code
>base?

So ... maybe?  But, some thoughts.

- We don't, in general, want to have any more #ifdefs in the code unless
  they are completely unavoidable (e.g., operating system differences or
  optional third-party libraries like OpenSSL).  So this would require
  some run-time configuration.

- It is not clear to me that you can state with certainly that the
  250 response code will contain the queue identifier (that is, in
  fact, not a concept that appears anywhere that I can find in the SMTP
  RFCs).  As a practical matter I've never had to give anyone the queue
  identifier of a message (because it's not normally logged on the
  client; really, most people are happy with recipients and a time, and
  they are really happy if you have a message-id).  But ... fine.  I can
  imagine cases where it would be helpful.  But there's no way we could
  accept this patch as-is, because it depends on the specific format of
  your ISP's response message.  I tested out two email providers that I
  use and their formats differ, specifically:

  250 2.0.0 Ok: queued as QUEUE-ID
  250 2.0.0 QUEUE-ID Message accepted for delivery

  It looks like your ISP's format is "250 id=QUEUE-ID".  So that's 3
  servers and 3 different formats.

I think this should be a lot more generic.  So ... an alternate proposal.

Right now we have the hooks interface (see doc/nmh/README-HOOKS).  It
should be relatively straightforward to create a "post hook" that could
be invoked when email is submitted to a mail server.  One of the arguments
to the post hook could be the response message from the SMTP server.
Another one of the arguments to the post hook could be the message
draft so you could interrogate it with scan(1) to extract out anything
useful you might want like the message-id.  You could then use your favorite
shell tools to process the SMTP server response and then send it to syslog
with logger(1).  I realize that would be a lot more work than the code
you submitted, and we'd have to think about the arguments to the post
hook, but I don't see any huge blockers other than WRITING the code.

I am neutral about this being made to work with sendmail/pipe; it would
actually be a lot of work to do that.  We could just accept that it is
one more thing that doesn't work with sendmail/pipe.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Valdis Klētnieks
In reply to this post by Steven Winikoff-2
On Tue, 09 Jul 2019 17:43:06 -0400, Steven Winikoff said:

>   sm_reply.length = rp - sm_reply.text;
>   sm_reply.text[sm_reply.length] = 0;
> +#ifndef NOSYSLOG
> +        if (strncmp(sm_reply.text, "OK id=", 6) == 0)
> +        {

This is highly dependent on the remote MTA.
Google, for instance, returns:

250 2.0.0 OK  1562718444 r17sm180161qtf.26 - gsmtp

The 250 is required.  2.0.0 (or similar) if the remote server does extended
return codes.  The rest is up for grabs.

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

attachment0 (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Michael Richardson-5
In reply to this post by Ken Hornstein-2

Ken Hornstein <[hidden email]> wrote:
    > - It is not clear to me that you can state with certainly that the
    > 250 response code will contain the queue identifier (that is, in
    > fact, not a concept that appears anywhere that I can find in the SMTP
    > RFCs).  As a practical matter I've never had to give anyone the queue
    > identifier of a message (because it's not normally logged on the

I have both asked for it and provided it when debugging bizarre situations.

Could we log the entire result, and let the post hook take care of the
various queue formats?

    > I am neutral about this being made to work with sendmail/pipe; it would
    > actually be a lot of work to do that.  We could just accept that it is
    > one more thing that doesn't work with sendmail/pipe.

slowly, that interface is dying. I have mixed feelings about that.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [hidden email]  http://www.sandelman.ca/        |   ruby on rails    [


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Ken Hornstein-2
>Could we log the entire result, and let the post hook take care of the
>various queue formats?

That was what I suggested.  Clearly nmh shouldn't be in the business of
figuring out what (if any) the queue identifier is based on the SMTP
DATA response message.

>    > I am neutral about this being made to work with sendmail/pipe; it would
>    > actually be a lot of work to do that.  We could just accept that it is
>    > one more thing that doesn't work with sendmail/pipe.
>
>slowly, that interface is dying. I have mixed feelings about that.

If I could make sendmail/pipe punch the user in the face every time a
message was sent using it, I would.  Sadly, that does not seem possible
given the current constraints of POSIX.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Robert Elz
In reply to this post by Ken Hornstein-2
    Date:        Tue, 09 Jul 2019 19:39:08 -0400
    From:        Ken Hornstein <[hidden email]>
    Message-ID:  <[hidden email]>

  | - We don't, in general, want to have any more #ifdefs in the code unless
  |   they are completely unavoidable

I agree with that, and even when ifdef's are added, they should be
positive, not double negative, so
        #ifndef NOSYSLOG
is just perferse, where
        #ifdef USE_SYSLOG
would work just as well (it does mean the name needs to be explicitly
defined to get the new code, but that's the way it should be, rather
than requiring a new name to be defined to retain the previous behaviour).

  | - It is not clear to me that you can state with certainly that the
  |   250 response code will contain the queue identifier

No, you can't, but these days it almost always does.

  |   As a practical matter I've never had to give anyone the queue
  |   identifier of a message

Then you have been lucky, are are relatively new to e-mail tracking.
The need is less common today, than it once was, since more and more
e-mail is direct from sender's MTA to recipient's - but back when
more mail relaying was done (when there was more than just "the internet")
the queue-id along with the transfer timestamp was the info that you
could send to a relay postmaster and actually expect them to look and
see what happened to the message (because that generally makes it trivial
for them to do ... send the message-id instead, or worse, just addresses,
and it was typically a lot more work, which resulted in requests for
tracking sometimes simply being ignored).

  | (because it's not normally logged on the client;

Any rational (MTA) client does.   MUA's typically don't, but those should
not really ever be talking to anything but their local MTA.   What is
different now than what used to be true, is what people regard as their
local MTA, which in the past would normally have been either on the same
host, or at absolute worst, in the same organisation as the sender (and
usually same department of the organisation when it is big enough to have
such things).   (Organisation can mean household for personal e-mail).
Way too many people today are relying upon "free" giant e-mail providers
to do everything for them.

  | really, most people are happy with recipients and a time, and
  | they are really happy if you have a message-id).

time yes, recipients, no, that's close to useless (though it can work if
the recipient receives very little e-mail and the site postmaster is in a
helpful mood at the time a request is made).   The message-id is better, but
the queue-id (along with time) is perfect.   (With a message-id (with or
without the time) the first thing that normally needs doing to track a message
is to convert that into the relevant local queue-id - avoiding the need for
that step makes it just that much more likely that a message can, and will,
be traced.)

  |   But there's no way we could
  |   accept this patch as-is, because it depends on the specific format of
  |   your ISP's response message.

Yes, that's no good - the only rational solution is to log all of the
250 response (after the 250 itself).   It is not only the queue-id in
it which can be helpful, sometimes the wording of the response can give
more info to the site which issued it (when you ask them to see what
happened) - it can differ for mail which was queued there, immediately
forwarded before being ack'd, or which was delivered locally.

  |   It looks like your ISP's format is "250 id=QUEUE-ID".  So that's 3
  |   servers and 3 different formats.

I used to have my mailer say something like
        250 OK Receipt: queue-id
to make it clear that that was the info that I wanted sent to me if
you wanted me to track your e-mail.

Not sure if I still do that or not, it has been a while since munnari
did any significant e-mail relaying (it used to connect 5 or 6 different
e-mail environments).

  | I think this should be a lot more generic.  So ... an alternate proposal.

Personally, I'd just suggest keeping the local MTA, having post deliver
to that, and let it do the logging - it will also do queueing in case
your local ISP link is down (like you want to send e-mail while in an
airplane...).  That is, I wouldn't bother with this at all - there is
an alternative (and better) solution already available.

kre


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Ronald F. Guilmette-2
In reply to this post by Ken Hornstein-2
In message <[hidden email]>,
Ken Hornstein <[hidden email]> wrote:

>If I could make sendmail/pipe punch the user in the face every time a
>message was sent using it...

Please don't.  I'm using it.

It is MUCH faster than trying to feed the message to Postfix
(aka Sendmail) via SMTP/587 because in the case of just piping the
message, Postfix doesn't make me wait until it has done the
DNS lookups it thinks it needs to do in order to process
the message.


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Michael Richardson-5
Ronald F. Guilmette <[hidden email]> wrote:
    >> If I could make sendmail/pipe punch the user in the face every time a
    >> message was sent using it...

    > Please don't.  I'm using it.

    > It is MUCH faster than trying to feed the message to Postfix
    > (aka Sendmail) via SMTP/587 because in the case of just piping the
    > message, Postfix doesn't make me wait until it has done the
    > DNS lookups it thinks it needs to do in order to process
    > the message.

Then you are missing an entry in /etc/hosts for 127.0.0.1 and ::1, or you are
missing your hostname in that file.


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Bakul Shah
In reply to this post by Ronald F. Guilmette-2
On Jul 9, 2019, at 5:56 PM, Ronald F. Guilmette <[hidden email]> wrote:

>
> In message <[hidden email]>,
> Ken Hornstein <[hidden email]> wrote:
>
>> If I could make sendmail/pipe punch the user in the face every time a
>> message was sent using it...
>
> Please don't.  I'm using it.
>
> It is MUCH faster than trying to feed the message to Postfix
> (aka Sendmail) via SMTP/587 because in the case of just piping the
> message, Postfix doesn't make me wait until it has done the
> DNS lookups it thinks it needs to do in order to process
> the message.

I've just added

send: -push

to ~/.mh_profile.
--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Steven Winikoff-2
In reply to this post by Ken Hornstein-2
>>Is there any interest in adding an improved version of this to the code
>>base?
>
>So ... maybe?  But, some thoughts.

Thank you (and everyone else!) for taking the time to reply to this.

Before I say anything else, I never meant to ask for my patch to be
incorporated as-is -- I know there are many ways in which it would
need to be improved for production use.

I sent it mostly as a proof of concept (it's currently just barely
good enough to do what I personally need :-/), and partly in hopes
it might help anyone else if something like it isn't added to nmh
itself.


>- We don't, in general, want to have any more #ifdefs in the code unless
>  they are completely unavoidable (e.g., operating system differences or
>  optional third-party libraries like OpenSSL).  So this would require
>  some run-time configuration.

Understood, of course.  I used those mostly as an easy way to mark the
code I added -- and for those wondering why I chose to write them in
the negative, that was purely out of laziness (so that I didn't have to
add -DSYSLOG to the configure process).

Again, this was never intended for production use, and I apologize if I
didn't make that clear originally.


>- It is not clear to me that you can state with certainly that the
>  250 response code will contain the queue identifier (that is, in
>  fact, not a concept that appears anywhere that I can find in the SMTP
>  RFCs).

That's unfortunate.  I've mostly worked with sendmail, and I've never
seen a case where the QID wasn't sent back to the originating MTA, so
I wasn't aware that the RFCs don't require that behaviour.


>  As a practical matter I've never had to give anyone the queue
>  identifier of a message (because it's not normally logged on the
>  client; really, most people are happy with recipients and a time, and
>  they are really happy if you have a message-id).

That doesn't match my experience.


>I think this should be a lot more generic.  So ... an alternate proposal.
>
> [ details snipped for brevity, but the summary is be to create a
>   "post hook" and use that instead ]

I'd have no problem with that as long as the post hook provides the same
information gathered in my patch (i.e., sender and recipient addresses,
message ID, relay server and port, and resulting status and QID).

     - Steven
--
___________________________________________________________________________
Steven Winikoff                | "...and every single one of them wanted
Montreal, QC, Canada           |  to be involved in the decision-making
[hidden email]               |  process without necessarily going
http://smwonline.ca            |  through the intelligence-using process
                               |  first."              - Terry Pratchett

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Steven Winikoff-2
In reply to this post by Robert Elz
>I agree with that, and even when ifdef's are added, they should be
>positive, not double negative, so
> #ifndef NOSYSLOG
>is just perferse,

Of course it is.  As I mentioned in my previous message...


> #ifdef USE_SYSLOG
>would work just as well (it does mean the name needs to be explicitly
>defined to get the new code,

...I was just too lazy to do that for a proof of concept.  There's no
question that you're right if such a patch were to be added in production
while using #ifdef


>  | - It is not clear to me that you can state with certainly that the
>  |   250 response code will contain the queue identifier
>
>No, you can't, but these days it almost always does.

That matches my experience.


>Personally, I'd just suggest keeping the local MTA, having post deliver
>to that, and let it do the logging

That's exactly what I've always done, from time immemorial until just
about two weeks ago.

Ironically enough I actually prefer to do it this way, but I was under the
impression that this is deprecated in modern configurations.  I'd be happy
to be wrong about that.

     - Steven
--
___________________________________________________________________________
Steven Winikoff                |
Montreal, QC, Canada           |  Don't use no double negatives.
[hidden email]               |
http://smwonline.ca            |

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Ralph Corderoy
In reply to this post by Ronald F. Guilmette-2
Hi Ronald,

> It is MUCH faster than trying to feed the message to Postfix (aka
> Sendmail) via SMTP/587 because in the case of just piping the message,
> Postfix doesn't make me wait until it has done the DNS lookups it
> thinks it needs to do in order to process the message.

I have send(1) submit to the local Postfix and it appears instant.
Bakul pointed out send's -push, but I don't use that and it shouldn't be
necessary.  Michael's probably on the right track; check your local
Postfix log for the length of the delay and what occurred around it.

--
Cheers, Ralph.

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Bakul Shah
I had greylisting turned on and my postfix wasn’t setup quite right so
when there was a long cc list, there was a long wait. -push was the
easiest way to fix the delay problem!

> On Jul 10, 2019, at 1:42 AM, Ralph Corderoy <[hidden email]> wrote:
>
> Hi Ronald,
>
>> It is MUCH faster than trying to feed the message to Postfix (aka
>> Sendmail) via SMTP/587 because in the case of just piping the message,
>> Postfix doesn't make me wait until it has done the DNS lookups it
>> thinks it needs to do in order to process the message.
>
> I have send(1) submit to the local Postfix and it appears instant.
> Bakul pointed out send's -push, but I don't use that and it shouldn't be
> necessary.  Michael's probably on the right track; check your local
> Postfix log for the length of the delay and what occurred around it.
>
> --
> Cheers, Ralph.
>
> --
> nmh-workers
> https://lists.nongnu.org/mailman/listinfo/nmh-workers


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: logging outgoing messages

Ralph Corderoy
In reply to this post by Robert Elz
Hi kre,

> The need is less common today, than it once was, since more and more
> e-mail is direct from sender's MTA to recipient's - but back when more
> mail relaying was done (when there was more than just "the internet")
> the queue-id along with the transfer timestamp

I grovel around MTA logs on a machine hosting Mailman for all the UK LUG
lists and the queue ID is the key thing.  It's the first thing I have to
work out if it's not provided.

The Message-ID field isn't necessarily a one-to-one mapping to QID, as
the same email with the same Message-ID can arrive and be queued more
than once.

> Any rational (MTA) client does.   MUA's typically don't, but those
> should not really ever be talking to anything but their local MTA.

Here, here.

> the only rational solution is to log all of the 250 response (after
> the 250 itself).

I'd keep the 250 as I expect we'd be doing this for 25x and its value is
significant.  Also, there might be multiple lines of 25x.

Is it just 25x that should trigger this hook, or 4xx, etc., too?

> Personally, I'd just suggest keeping the local MTA, having post
> deliver to that, and let it do the logging - it will also do queueing
> in case your local ISP link is down (like you want to send e-mail
> while in an airplane...).  That is, I wouldn't bother with this at all
> - there is an alternative (and better) solution already available.

I agree.  Something specialising in queueing locally and then forwarding
on will do a better job and offer more options than extending nmh to
include those options over time.  I see that nmh has to talk TLS SMTP to
submit an email locally, e.g. same host or company, and that can be used
to also submit across the Internet, but that doesn't mean it's a good
idea...

I run a local Postfix so it has all the configuration about the next
smarthost based on the hat I'm wearing, not nmh which, IMO, is the wrong
place.  Thus mail(1) benefits too.  And I'm aware of mhmail(1), but it's
not the same.  crontab(5)'s MAILTO, at(1)'s LOGNAME, ~/.forward files,
... all benefit from central configuration rather than inside nmh.

Ditto fetching email.  I don't fetch large emails by default because
they're typically spam; the features of the specialist program for
fetching emails already provide for that, and lots more; no patching of
inc(1) required.  :-)

--
Cheers, Ralph.

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
12