[nmh-workers] Need help with sending mail

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[nmh-workers] Need help with sending mail

Jay Dresser-3
I'm hoping someone can point me to the documentation (or give me the answer) regarding configuration for outgoing mail. 
I am using MH-E and I am able to receive mail fine but get this error when sending:

post: problem initializing server; [RPLY] 530 Authentication required
send: message not delivered to anyone

Adding -verbose to send or post does not provide any more detail.
I'm not sure what exactly is missing. I think I've set everything correctly but can't find explicit instructions in the docs, and I've been googling for over a week.
The password is set correctly in ~/.netrc and ~/.authinfo and I don't see anywhere else it is supposed to be.

My ~/.mh_profile is:

MH-Profile-Version: 1.0
Path: /home/jay/Mail
Editor: emacsclient
credentials: file:netrc
post: -mts smtp -server smtp.1and1.com -port 587 -initialtls -user "*@jaydresser.us"

(with or without quotes around user does not seem to matter)
I think -initialtls is what I need but -tls does not work either.
I get the same error when using mh from the command line.
Notmuch and mu4e work fine inside Emacs. I like MH-E and would like to use it.

I hope someone can advise me.
Thank you.

-- Jay Dresser, nmh[hidden email]

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ralph Corderoy
Hi Jay,

> Adding -verbose to send or post does not provide any more detail.

Try send(1)'s `-snoop'.

> I've been googling for over a week.

Feel free to ask these kind of user questions here sooner than that.
:-)  For clarity, the output of `mhparam version' is handy to us.

> post: -mts smtp -server smtp.1and1.com -port 587 -initialtls -user "*@jaydresser.us"

The `*' in that username looks a little odd.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ken Hornstein-2
In reply to this post by Jay Dresser-3
First off ... please don't construe anything I say as an attack on you.
I would honestly like some feedback about how we could improve our
documentation.  If you could let us know what we need to improve, that
would be great; I view the failure here of you not figuring this out
as a problem with our documentation.  I think this information is mostly
all there, but not organized well so it's hard to put it all together;
I've been in the middle of this for so long it's hard to remember how
this appears to someone who comes into it cold.

Ralph gave you the right starting advice, but ...

>credentials: file:netrc

You have a file called "netrc" in your Mail directory?  If you want
to use a file called ".netrc" in your HOME directory, maybe you want
to use "legacy" in the credentials entry?  (The mh-profile man page says
the home directory is searched for the credentials file, but when I look at
that code I am not sure that is true).  I thought reading the man
page that it was obvious that when we say "file:netrc" you are supposed
to put the name of YOUR credentials file where "netrc" is, but maybe
it isn't.

So, a few other problems I see:

- post(8) doesn't read the profile, so putting an entry under "post" in there
  won't do anything.  You probably want to put that all under "send".
- You don't specify the -sasl flag, which is really what kicks off all
  authentication nowadays (with the exception of the old-style POP).  Reading
  the man page for send(1) and post(8) I thought that it was clear that
  the authentication only happens under -sasl (it's the only place where
  -user is mentioned), but again, I would appreciate some feedback as to
  how we could improve this.

What I would do is try running send(1) testing out a few flags on the
command line until you get something that works.  What I think you
should start with is:

        -snoop
        -sasl
        -mts smtp
        -server smtp.land1.com
        -tls

(Port 587 is the default so you shouldn't need that, and if you are using
that port you probably are NOT using -initialtls).

If authentication starts properly, you should be prompted for a username
and password; once you get that part working you can then put stuff in
your profile and start using your .netrc.  If that doesn't work then
please post the output from -snoop.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ralph Corderoy
Hi Jay,

Ken wrote:
> If that doesn't work then please post the output from -snoop.

Look out:

    (Beware that the SMTP transaction may contain authentication
    information either in plaintext or easily decoded base64.)
        ― send(1).

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ken Hornstein-2
>Ken wrote:
>> If that doesn't work then please post the output from -snoop.
>
>Look out:
>
>    (Beware that the SMTP transaction may contain authentication
>    information either in plaintext or easily decoded base64.)
>        ― send(1).

Oh, um, yes.  I forgot to mention that; thanks!  If you are running
1.7 or newer, -snoop will automatically decode the base64 for you if
it can, so your username and/or password will be easily visible.  Which
makes debugging easier and it more obvious where your password is.  I
guess what I'm trying to say is, "look at the output carefully before
you send it to a public mailing list".  Although I THINK if you get to
the point where the password is sent, it is probably working and you won't
need to send it.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
In reply to this post by Ken Hornstein-2
Ken wrote:

> - post(8) doesn't read the profile, so putting an entry under "post" in there
>   won't do anything.

Should we add a warning to the profile parser for post, whom, et al., entries to note that they're ignored?

I suppose someone could write a postproc that does obey a profile entry.  Add a suppression for the warning?

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ken Hornstein-2
>> - post(8) doesn't read the profile, so putting an entry under "post" in there
>>   won't do anything.
>
>Should we add a warning to the profile parser for post, whom, et al.,
>entries to note that they're ignored?

Sigh.  I get where you're coming from, but ... which program would do
that?  If it is post(8), then that means post would have to read the
profile so it would know that there is something that it should warn
about, and somehow that strikes me as ridiculous ... although there are
plenty of other ridiculous things in the nmh source code :-)

If it's any program that pareses the config file, that seems lousy.
If it's a program like mhparam, I'd be fine with that.

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
Ken wrote:

> >> - post(8) doesn't read the profile, so putting an entry under "post" in there
> >>   won't do anything.
> >
> >Should we add a warning to the profile parser for post, whom, et al.,
> >entries to note that they're ignored?
>
> Sigh.  I get where you're coming from, but ... which program would do
> that?

All nmh programs.

> If it is post(8), then that means post would have to read the profile

If anyone calls post directly, they're likely very aware of that.

> If it's any program that pareses the config file, that seems lousy.

Why?  This is so subtle that the more warning, the better.  Why not add
the warning?

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
I wrote:

> All nmh programs.

I should have said:  All nmh programs that read the user's profile.

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ken Hornstein-2
In reply to this post by David Levine-3
>> If it's any program that pareses the config file, that seems lousy.
>
>Why?  This is so subtle that the more warning, the better.  Why not add
>the warning?

It's absolutely true that this is a subtle point, and I guess I cannot
come up with any reason to not generate such a warning for every nmh
program; the argument can be made that this is a configuration error
and the user should be notified immediately.  My reasons against this
change are kinda vague and all fall under "it just sort of SEEMS inelegant"
and I can't really rationally defend them.

I think we need some way to suppress this warning, just in case (I
cannot imagine any reason why you would need to, but nmh users surprise
me all of the time :-) ).

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Robert Elz
    Date:        Fri, 26 Oct 2018 07:15:21 -0400
    From:        Ken Hornstein <[hidden email]>
    Message-ID:  <[hidden email]>

  | >Why?  This is so subtle that the more warning, the better.  Why not add
  | >the warning?
  |
  | It's absolutely true that this is a subtle point,

Actually, subtle it is not - it is quite blatant, and is explicit in the post
man page.

To use options (to any nmh program) one really needs to either know them
well (which would mean having read the man page previously) or to read
the man page.  We should not need more than that, though the message
could be moved (or duplicated) earlier in the man page to make it even
harder to miss.

If we are to add warnings, why stop at post?   Why not warn about other
entries in the profile that aren't used?   But how does anything know?
I have a "spin" entry in my profile, used by some script that someone
posted (probably on this list) years ago (I no longer have the script in
my bin directory, though I could probably find it).

  | I think we need some way to suppress this warning, just in case (I
  | cannot imagine any reason why you would need to, but nmh users surprise
  | me all of the time :-) ).

I think we should not have a warning at all.

I have an entry for "post" in my profile (have done for a long time).
I don't expect it to be used, but it allows me to do
        mhparam post
to see what default options get used (the actual entries are also in
the profile entry for send).

I also think that MH users are (by and large) able to deal with issues
like this, and preventing the occasional question to the list isn't worth
the effort of extra code, of making the profile parser bigger and even
messier (and also forcing a full read every time, which one day might
be good to avoid - just reading enoughto get the entry needed) and
just generally being obnoxious.

kre


--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
kre wrote:

> Actually, subtle it is not - it is quite blatant, and is explicit in the
> post man page.

It has subtle effects.  This went undetected in the mh-profile(5) man page
until late 2011:

    https://lists.gnu.org/archive/html/nmh-workers/2011-12/msg00088.html

I (quickly) counted that this (post doesn't read the profile) has come up
on nmh-workers twice a year, on average, since 2011.

> If we are to add warnings, why stop at post?   Why not warn about other
> entries in the profile that aren't used?

Because we're not sliding down that slope.  This is a specific issue
that can be easily addressed.

> I have an entry for "post" in my profile (have done for a long time).
> I don't expect it to be used, but it allows me to do
> mhparam post
> to see what default options get used (the actual entries are also in
> the profile entry for send).

I prefer to view the actual entries.  And not have to keep two sets
in sync.

> of making the profile parser bigger and even messier
> (and also forcing a full read every time, which one day might
> be good to avoid - just reading enoughto get the entry needed)

I cannot imagine that profile parser size is a real concern, or that
the difference between a full and partial read is measurable on
modern platforms.

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ralph Corderoy
In reply to this post by David Levine-3
Hi David,

> Should we add a warning to the profile parser for post, whom, et al.,
> entries to note that they're ignored?

I don't think whom(1)'s entry is ignored?  Maybe I'm interpreting that
wrongly.

    $ cat >.mh_profile
    path: .
    whom: -vote-for-ken
    ^D
    $ HOME=$PWD mhparam whom
    -vote-for-ken
    $ HOME=$PWD whom
    whom: -vote-for-ken unknown
    $

mh-profile(5) does mention `post'.

    locale: locale
        Set the locale for all nmh programs except post, install-mh, and
        slocal.  See the LC_ALL, LC_CTYPE, and LANG variables in the
        "ENVIRONMENT" section below for a reference on how the locale is
        set if this profile component is not used.

It also has

    program: default switches
        Sets default switches to be used whenever the mh program program
        is invoked.  For example, one could override the “Editor:”
        profile component when replying to messages by adding a
        component such as:

            repl: -editor /bin/ed

        (profile, no defaults)

Perhaps that should mention programs that don't read their .mh_profile
entry?  Or say how to check if a program does using the program's own
man page?

My fallible manual scan of man pages suggests these have `None' or
similar under the `PROFILE COMPONENTS' section.

    ap
    dp
    fmttest
    mh-mkstemp
    msgchk
    post

These pages don't have a `PROFILE COMPONENTS' section.

    mhical
    mhmail
    mhn (deprecated)
    rcvdist
    rcvpack
    rcvtty
    slocal

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ralph Corderoy
In reply to this post by Robert Elz
Hi kre,

> We should not need more than that, though the message could be moved
> (or duplicated) earlier in the man page to make it even harder to
> miss.

Given I'm one who consistently misses the existing documention in
post(8) when I reply to users having problems with post, due to not
reading to the end of the page, I think duplicating the warning might
help.  Something like

    post will not normally be called directly by the user
  + and so does not consult the user's .mh_profile.
  - but
  + But it
    can be replaced by the user with a postproc profile component that
    will have file as its final argument.  See mh-profile(5) for more
    information on postproc.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ken Hornstein-2
In reply to this post by Robert Elz
Robert has managed to articulate what I only had as vague feelings, and of
course we were presented with someone who actually had profile entries
under 'post' and had a legitimate (if surprising) reason to do so.

Thinking about this some more ... while it WOULD be weird, certainly a
user could symlink "show" to "post", and then a profile entry for post
would make perfect sense.  I cannot think of any reason why you would do
that, but it would be legitimate (and as we've seen, certainly people
do plenty of things where I've said, "I can't think of any reason why
you would that")

>I also think that MH users are (by and large) able to deal with issues
>like this, and preventing the occasional question to the list isn't worth
>the effort of extra code, of making the profile parser bigger and even
>messier (and also forcing a full read every time, which one day might
>be good to avoid - just reading enoughto get the entry needed) and
>just generally being obnoxious.

Weeeelll .... the whole reason this started was because people aren't
reading the documentation :-)  Even Ralph (RALPH!) admitted he doesn't
always read to the end of the post(8) man page all of the time :-)

David brings up the point that on average, it's come up twice a year on
the mailing list ...  and that seems a lot to me.  That suggests to me
at the very least our documentation needs improving.  I think I am leaning
back to NOT having a warning, but I don't feel super strong about that.

So, I went back and looked AGAIN at this.  Currently, only "post",
"install-mh", and "slocal" call context_foil() (the function that
disables the reading of the profile).  And even though David added a
comment about this in sbr/mts.c, whom DOES read the profile (and as far
as I can tell, it always did).

Back in MH 6.8.5, the following programs called m_foil() (the precursor
to context_foil()):

        conflict
        mhmail
        mhn
        post
        rmail
        sbboards
        slocal
        spop
        vmhtest

This ... is an odd list, but there are some things in common.  Most of
these programs are designed be run by OTHER programs; that's true of
post & mhn (other MH programs typically ran them, but not always true
for mhn). rmail, sbboards, slocal & spop were designed to be run as
local delivery agents.  I can't really explain conflict; except maybe it
was normally run via cron?  mhmail was designed to be a replacement for
Berkeley mail, so maybe it was intended that nothing in the MH profile
would affect it.  I have no idea what vmhtest was supposted to do, so
don't ask me about it :-)  To muddle the waters even further, spost
DID read the profile explicitly.  install-mh isn't on this list, but it
did not use the profile to add to it's arguments.

Along the way from MH to nmh, context_foil was dropped from mhn & added
to spost.  That left the other programs, but most of them (conflict,
rmail, sbboards, spop, vmhtest) have been removed as obsolete.  When
David created nmh_init(), he changed install-mh to call context_foil()
(which makes sense to me).  mhmail has been changed to a shell script,
which obviously doesn't check the profile either.

What does this all mean?  Well, people are still confused about it,
which isn't great.  I think improving our documentation would be good.

Did Jay Dresser ever get email working?  That WAS the original point of
this thread, after all :-)

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
In reply to this post by Ralph Corderoy
Ralph wrote:

> I don't think whom(1)'s entry is ignored?

You're right.

The only programs that don't read the profile are install-mh(1), post(8),
slocal(1), and mhmail(1):

    $ grep 'nmh_init.*false[^)]' uip/*.c
    uip/install-mh.c:    if (nmh_init(argv[0], false, false)) { return 1; }
    uip/post.c:    if (nmh_init(argv[0], false, false)) { return 1; }
    uip/slocal.c:    if (nmh_init(argv[0], false, false)) { return 1; }

If the second argument to nmh_init() is false, the context and
profile are not read.  mhmail doesn't show up here because it
doesn't have a uip/mhmail.c.

> Perhaps that should mention programs that don't read their .mh_profile
> entry?  Or say how to check if a program does using the program's own
> man page?
>
> My fallible manual scan of man pages suggests these have `None' or
> similar under the `PROFILE COMPONENTS' section.
>
>     ap
>     dp
>     fmttest
>     mh-mkstemp
>     msgchk
>     post
>
> These pages don't have a `PROFILE COMPONENTS' section.
>
>     mhical
>     mhmail
>     mhn (deprecated)
>     rcvdist
>     rcvpack
>     rcvtty
>     slocal

I'm all for updating the man pages, but don't have the time now.  I
will add a PROFILE COMPONENTS section to the mhmail page and correct
its CONTEXT section.

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ralph Corderoy
In reply to this post by Ken Hornstein-2
Hi Ken,

> Even Ralph (RALPH!) admitted he doesn't always read to the end of the
> post(8) man page all of the time :-)

I think that's because I wade through the DESCRIPTION, searching for an
option or two amongst the verbose prose, and then don't skip to the end
unless I think SEE ALSO might be useful to continue the search.  This
means I rarely consult FILES, PROFILE COMPONENTS, DEFAULTS, or CONTEXT.
They're too late to the party.

One of the many things on my to-do list was to re-write a typical nmh
man page so that it had enough of a DESCRIPTION to understanding its
purpose, and then a new OPTIONS would list them, probably ordered
lexically, and using `.TP' to give the convention hanging-tag
presentation.  So post might say

    -msgid  Add a ‘Message-ID’ or ‘Resent-Message-ID’ header.
            Default: -nomsgid.

    -messageid hostsrc
            The host part of any added message or content ID is
            the hostname if hostsrc is ‘localname’, or a random sequence
            of characters if hostsrc is ‘random’.  Default: -messageid
            localname.

If options couldn't be given in a profile because the command doesn't
read it then this would be prominently after the OPTIONS section header,
before the options.

DEFAULTS isn't needed because each default is listed with the option.
This also lessens the chance the code and the man page drifts.

Feedback on the one example man page would help decide whether to
convert the others, and also provide a bit of a template.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
In reply to this post by Ken Hornstein-2
Ken wrote:

> Weeeelll .... the whole reason this started was because people aren't
> reading the documentation :-)

Documentation is not the whole reason.  The fact that post(8) behaves
differently from nearly all other MH/nmh programs requires an exception
when thinking about it.  (SUBTLE!)  That is validated by the fact that
three of us looked at the user's profile excerpt and two missed the use
of a post: profile component.  (And I was one of those two and am very
well aware of this exception.)  And that the documentation itself had
missed this until late 2011.

Is there any reason that we should NOT provide a warning to help users?
Especially first-time users who haven't read all 19K lines of man pages?
Aren't we trying to minimize configuration impediments?

Of course, there would be a mechanism to disable the warning for
those who want to have a post component in their profile.

> And even though David added a
> comment about this in sbr/mts.c, whom DOES read the profile (and as far
> as I can tell, it always did).

Fixed.

> install-mh isn't on this list, but it
> did not use the profile to add to it's arguments.

And I don't expect that many users run install-mh more than once.  Until
someone asks for install: -check to be supported in their profile, I
think that we can ignore it.  And similarly for slocal.

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

Ken Hornstein-2
>Ken wrote:
>
>> Weeeelll .... the whole reason this started was because people aren't
>> reading the documentation :-)
>
>Documentation is not the whole reason.  The fact that post(8) behaves
>differently from nearly all other MH/nmh programs requires an exception
>when thinking about it.  (SUBTLE!)  That is validated by the fact that
>three of us looked at the user's profile excerpt and two missed the use
>of a post: profile component.  (And I was one of those two and am very
>well aware of this exception.)  And that the documentation itself had
>missed this until late 2011.
>
>Is there any reason that we should NOT provide a warning to help users?
>Especially first-time users who haven't read all 19K lines of man pages?
>Aren't we trying to minimize configuration impediments?

I've thought about this a bit.  Here's the best answer I can give.

If you have a profile entry under "post", it really isn't for the
program post(8); they're for a program who's argv[0] is "post".  I know
that seems like nitpicking, and maybe it is ... but I think it's an
important point; it's a commonly-used idiom to create a different set of
options to commands by symlinking your custom name to the original nmh
utility and putting the options under your custom name.

I'm fine with a warning for a misconfiguration, but ... given the way
the profile entries work, I'm not sure it IS a misconfiguration.  Now,
you're going to raise the entirely-reasonable point that "hey, 9 times
out of 10 it's not going to do anything when the user thinks it will,
and that is a misconfiguration."  I get where you're coming from.  It's
just ... we have a situation where a program that is NOT misconfigured
is warning about ANOTHER program that may or may not be misconfigured.
This is of course unavoidable if you want to generate a warning because
all nmh programs are independent of each other, and the program you
really want to generate a warning can't, because it doesn't read the
profile.

But ... if you're not convinced, how about a compromise?  send(1) is
the normal front-end to post(8); how about have it check and issue a
warning, rather than all programs?

--Ken

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
Reply | Threaded
Open this post in threaded view
|

Re: Need help with sending mail

David Levine-3
Ken wrote:

> If you have a profile entry under "post", it really isn't for the
> program post(8); they're for a program who's argv[0] is "post".

OK.  Would this always be through a postproc entry?  See below for
why I ask.

A user could replace the installed post with their own program, but
I'd argue that they should use postproc.

> But ... if you're not convinced, how about a compromise?  send(1) is
> the normal front-end to post(8); how about have it check and issue a
> warning, rather than all programs?

Well, that would miss mhmail, whom, rcvdist, and viamail.  And
whatnow push, though I don't think there's a need to help diagnose
use of a post profile entry with that.

And I would still prefer to notify the user as soon as possible,
so prefer that all programs check.

How about suppressing the warning if the user has set postproc in
their profile?

David

--
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers
12