Monit start delay only on boot

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Monit start delay only on boot

David Jones
Loving monit.  Just started using it a few months ago to deploy Letsencrypt.org certs from a central server to about 800 servers/VMs.  Monit will restart/reload each service (Apache, Postfix, Haproxy, Nginx, etc.) that references an LE cert when it's renewed every 60 days.  This is working great.

I am remotely monitoring Monit using Icinga/Nagios and they are working very well together.  I am moving many of my Icinga checks into Monit using scripts to generate /etc/monit.d/*.cfg files custom for each server.  Then a single Icinga/Nagios check using check_monit.py will be customized to each box.

The only minor issue is the delay start that is needed for bootup but causes "connection refused" in the monit web interface used by check_monit.py when monit restarts.

Would it be possible to have an option added to only delay XX seconds on a fresh boot where the uptime is less than a few minutes?  Or allow the monit web interface to immediately accept connections if a recent statefile exists?

Thanks,
Dave

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general
Reply | Threaded
Open this post in threaded view
|

Re: Monit start delay only on boot

martinp@tildeslash.com
Hi Dave,

if monit depends on some services on your system, it'll be maybe better to set the dependencies and drop the start delay option from the configuration file.

Best regards,
Martin


On 1 Jun 2019, at 21:31, David Jones <[hidden email]> wrote:

Loving monit.  Just started using it a few months ago to deploy Letsencrypt.org certs from a central server to about 800 servers/VMs.  Monit will restart/reload each service (Apache, Postfix, Haproxy, Nginx, etc.) that references an LE cert when it's renewed every 60 days.  This is working great.

I am remotely monitoring Monit using Icinga/Nagios and they are working very well together.  I am moving many of my Icinga checks into Monit using scripts to generate /etc/monit.d/*.cfg files custom for each server.  Then a single Icinga/Nagios check using check_monit.py will be customized to each box.

The only minor issue is the delay start that is needed for bootup but causes "connection refused" in the monit web interface used by check_monit.py when monit restarts.

Would it be possible to have an option added to only delay XX seconds on a fresh boot where the uptime is less than a few minutes?  Or allow the monit web interface to immediately accept connections if a recent statefile exists?

Thanks,
Dave


--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general
Reply | Threaded
Open this post in threaded view
|

Re: Monit start delay only on boot

David Jones
Correct, I want to drop the start delay down to 30 seconds from my current setting of 120 so service restarts will have less lag.  However, there could be some things like JAVA-based applications that need more time to startup completely after a server reboot.

This would be best solved if a new option was added to monit that could check the uptime (/proc/uptime) and allow for a special delay at startup separate from the current "with start delay."

Maybe something like this:

/etc/monitrc
set daemon 30
    with start delay 30
    and bootup start delay of 120

Then another setting like:

set bootup start delay period 600

so the first 10 minutes of the server booting up would have an extra 120 seconds of delay.

Thoughts?

I looked at doing something custom with systemd and it's going to be difficult/impossible to do a long delay reliably.


From: monit-general <monit-general-bounces+djones=[hidden email]> on behalf of [hidden email] <[hidden email]>
Sent: Monday, June 3, 2019 2:53 PM
To: This is the general mailing list for monit
Subject: Re: Monit start delay only on boot
 
Hi Dave,

if monit depends on some services on your system, it'll be maybe better to set the dependencies and drop the start delay option from the configuration file.

Best regards,
Martin


On 1 Jun 2019, at 21:31, David Jones <[hidden email]> wrote:

Loving monit.  Just started using it a few months ago to deploy Letsencrypt.org certs from a central server to about 800 servers/VMs.  Monit will restart/reload each service (Apache, Postfix, Haproxy, Nginx, etc.) that references an LE cert when it's renewed every 60 days.  This is working great.

I am remotely monitoring Monit using Icinga/Nagios and they are working very well together.  I am moving many of my Icinga checks into Monit using scripts to generate /etc/monit.d/*.cfg files custom for each server.  Then a single Icinga/Nagios check using check_monit.py will be customized to each box.

The only minor issue is the delay start that is needed for bootup but causes "connection refused" in the monit web interface used by check_monit.py when monit restarts.

Would it be possible to have an option added to only delay XX seconds on a fresh boot where the uptime is less than a few minutes?  Or allow the monit web interface to immediately accept connections if a recent statefile exists?

Thanks,
Dave


--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general
Reply | Threaded
Open this post in threaded view
|

Re: Monit start delay only on boot

Lutz Mader
Hello David,
like suggested by Martin, I use a script to defere the monit start after
a system reboot (AIX and Linux), on the one hand.

> Correct, I want to drop the start delay down to 30 seconds from my current setting of 120 so service restarts will have less lag.  However, there could be some things like JAVA-based applications that need more time to startup completely after a server reboot.
On the other hand I use "depends on" to some services/resources to
defere the start of the JVMs (WebSphere and/or Liberty Profile) for some
systems.

Back to your suggestion,
an additional monit start option to give the deplay is more fexible than
a additional "monitrc" statement to do this, I think.

With regards,
Lutz

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general
Reply | Threaded
Open this post in threaded view
|

Re: Monit start delay only on boot

David Jones
On 6/7/19 3:16 AM, Lutz Mader wrote:
> Hello David,
> like suggested by Martin, I use a script to defere the monit start after
> a system reboot (AIX and Linux), on the one hand.

I want to continue using the standard init.d and systemd startup scripts
built into the RPMs to start monit on boot.  Are you suggesting to
disable monit on boot and start it with another method like an
/etc/cron.d file with @reboot and sleep?  Or a script called by
/etc/rc.local?

This would work but it seems like a hack for each sysadmin to do when we
all need the same functionality and it should only be a few lines of
code to implement for everyone with a reasonable default value.

>> Correct, I want to drop the start delay down to 30 seconds from my current setting of 120 so service restarts will have less lag.  However, there could be some things like JAVA-based applications that need more time to startup completely after a server reboot.
> On the other hand I use "depends on" to some services/resources to
> defere the start of the JVMs (WebSphere and/or Liberty Profile) for some
> systems.
>
> Back to your suggestion,
> an additional monit start option to give the deplay is more fexible than
> a additional "monitrc" statement to do this, I think.

I have about 1,000 CentOS 5, 6, and 7 servers/VMs all running 5.25.3
with the same monitrc.  It would be much easier to update the monitrc
then change the startup scripts that are very different across these
servers and built into the RPMs.  Updating monit next time by the RPMs
could overwrite the systemd and init.d scripts.  It is generally not a
good idea to modify the RPM-managed startup scripts.  Also, the
different packagers of the RPMs aren't consistent with an
/etc/sysconfig/monit file for options like this.

---

Dave

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general