Multiple 'every' clauses not working as expected

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple 'every' clauses not working as expected

Aaron Lindsay
Hello and Happy New Year!

I am attempting to use multiple 'every' clauses to simultaneously control how frequent a 'program'-style check is made *and* what times of day it is run. Based on a previous mailing list discussion [0], I thought that using both a 'poll cycle multiple' and a 'negative cron-style' scheduling clause for a given service check would work. However, I'm finding (on monit 5.27.1) that it appears that when two 'every' clauses appear, only the later is effective. For example, the following check runs every 2nd cycle as intended, but does so even during hours 22-23,0-6 when it shouldn't:

check program test with path /path/to/monit_test.sh
    not every "* 22-23,0-6 * * *"
    every 2 cycles
    if status != 0 then alert

And the following doesn't run during 22-23,0-6, but runs *every* cycle otherwise:

check program test with path /path/to/monit_test.sh
    every 2 cycles
    not every "* 22-23,0-6 * * *"
    if status != 0 then alert

I've looked in the manual and searched the mailing list, but haven't been able to find a definitive answer to whether using duplicate 'every' clauses is intended to be supported in this way. Have I found a bug, or am I simply using this a way that it isn't intended?

(And if I'm just Doing It Wrong, is there a good way to both reduce how frequently a check is run (in terms of cycles) *and* limit it to certain times of day?)

Thanks for any help!

-Aaron

Reply | Threaded
Open this post in threaded view
|

Re: Multiple 'every' clauses not working as expected

Lutz Mader
Hello Aaron,
you are using a variant of a statement multiple times, the last one will
be used therefor. See
https://mmonit.com/monit/documentation/monit.html#SERVICE-POLL-TIME for
some more information.

But, if you use "every 2 cycles" in a service, all services depends to
this service are checked every two cycles too.

But you can use "every" ony once in a service definition.

With regards,
Lutz

Reply | Threaded
Open this post in threaded view
|

Re: Multiple 'every' clauses not working as expected

Aaron Lindsay
On Fri, Jan 1, 2021 at 5:48 PM Lutz Mader <[hidden email]> wrote:
Hello Aaron,
you are using a variant of a statement multiple times, the last one will
be used therefor. See
https://mmonit.com/monit/documentation/monit.html#SERVICE-POLL-TIME for
some more information.

But, if you use "every 2 cycles" in a service, all services depends to
this service are checked every two cycles too.

But you can use "every" ony once in a service definition.

I'm familiar with that section of the documentation, but I don't see where it says that 'every' may only be supplied once in a service definition. And, in fact, a previous email on this list (see https://lists.nongnu.org/archive/html/monit-general/2019-06/msg00022.html) suggests exactly the opposite of what you have said! (I'm not saying you're wrong, but explaining my confusion)

The more I think about this, the more I think I might consider it a bug one way or the other: Maybe I'm missing something, but it seems to me like monit should either raise an error when you supply multiple 'every' statements (at least for `monit -t`), *or* use the combination of all the 'every' statements. Allowing multiple 'every' statements but ignoring some of them seems it's a trap for people to fall into, especially since this behavior isn't mentioned in the documentation.

-Aaron
Reply | Threaded
Open this post in threaded view
|

Re: Multiple 'every' clauses not working as expected

Lutz Mader
Hello Aaron,
you see what monit will use by using "monit -v".

> The more I think about this, the more I think I might consider it a bug one
> way or the other: Maybe I'm missing something, but it seems to me like
> monit should either raise an error when you supply multiple 'every'
> statements (at least for `monit -t`), *or* use the combination of all the
> 'every' statements. Allowing multiple 'every' statements but ignoring some
> of them seems it's a trap for people to fall into, especially since this
> behavior isn't mentioned in the documentation.

You are right, sometimes some more error/warning or info messages are
helpful. But in general the config parser works well from my point of view.

With regards,
Lutz

p.s.
A snippet from the Monit documentation.
> We will address this limitation in a future release and convert the
> scheduler from ...

Feel free to add a ticket (see
https://bitbucket.org/tildeslash/monit/issues), suggestions are welcome
in general.

Reply | Threaded
Open this post in threaded view
|

Re: Multiple 'every' clauses not working as expected

Aaron Lindsay
On Sat, Jan 2, 2021 at 7:39 AM Lutz Mader <[hidden email]> wrote:
Hello Aaron,
you see what monit will use by using "monit -v".

> The more I think about this, the more I think I might consider it a bug one
> way or the other: Maybe I'm missing something, but it seems to me like
> monit should either raise an error when you supply multiple 'every'
> statements (at least for `monit -t`), *or* use the combination of all the
> 'every' statements. Allowing multiple 'every' statements but ignoring some
> of them seems it's a trap for people to fall into, especially since this
> behavior isn't mentioned in the documentation.

You are right, sometimes some more error/warning or info messages are
helpful. But in general the config parser works well from my point of view.

A snippet from the Monit documentation.
> We will address this limitation in a future release and convert the
> scheduler from ...

Ah, I interpreted that section to be referring to "The cron specification does not guarantee when exactly the test will run" in the previous paragraph as the limitation.

Feel free to add a ticket (see
https://bitbucket.org/tildeslash/monit/issues), suggestions are welcome
in general.

Okay, I can certainly add an issue. Is it realistic to put in an issue for allowing/interpreting multiple 'every' statements, or would this issue be for a documentation/`monit -t` change?

It sounds like there isn't currently a way for me to control both the 'cron' and cycle-based timing of a service, is that correct?

-Aaron
Reply | Threaded
Open this post in threaded view
|

Re: Multiple 'every' clauses not working as expected

Lutz Mader
Hello Aaron,
yes, at the time you can not use "every" multiple times in a service
configuration.

> It sounds like there isn't currently a way for me to control both the
> 'cron' and cycle-based timing of a service, is that correct?

Implicitly you can use a second service configuration to depend to with
a larger cycle interval. This is the only way I know.

A suggestion only,
with regards,
Lutz

Reply | Threaded
Open this post in threaded view
|

Re: Multiple 'every' clauses not working as expected

Lutz Mader
In reply to this post by Aaron Lindsay
Hello Aaron,
based on your ticket/issue sample, this looks similar to the my test case.

check program test-program path /Users/lutz/monit/scripts/test.sh
  timeout 10 seconds
  every 20 cycles
  depends on test-file
  if status !=0 then exec "/Users/lutz/monit/scripts/env.sh"
    else if succeeded then exec "/Users/lutz/monit/scripts/env.sh"

check file test-file path /Users/lutz/monit/test
  not every "* 0-6,22-23 * * *"
  if timestamp is older than 1 day then alert

If you start monit with option "-v" (and "-I") you will see something
like this in the monit log file (and the terminal).

The "test" file is not available and the "test-program can not started
therefore.

'test-file' file doesn't exist
'test-file' trying to restart
'test-program' stop skipped -- method not defined
'test-file' stop skipped -- method not defined
'test-file' start method not defined
'test-file' monitoring enabled
'test-file' start method not defined
'test-file' monitoring enabled
'test-file' file doesn't exist
'test-program' failed to start -- could not start required services:
'test-file'
'test-program' test skipped as required service 'test-file' has errors
:
'test-file' file doesn't exist
'test-file' trying to restart
'test-program' stop skipped -- method not defined
'test-file' stop skipped -- method not defined
'test-file' start method not defined
'test-file' monitoring enabled
'test-file' start method not defined
'test-file' monitoring enabled
'test-file' file doesn't exist
'test-program' failed to start -- could not start required services:
'test-file'
'test-program' test skipped as current cycle (1) < every cycle (20)

You will see the cycle count you used with "every" for "test-program".
As long this counter is not "20" the program will not start.

'test-file' file exists
'test-file' is a regular file
'test-file' modify/change time test succeeded for /Users/lutz/monit/test
[current modify/change time is Tue, 05 Jan 2021 12:18:03]
'test-program' status succeeded (0) -- Test Di 5 Jan 2021 12:28:38 CET
'test-program' test skipped as current cycle (1) < every cycle (20)

But after a while, the counter will be reached and the script will be
executed and the status will used by monit.

You should wait for 20 cycles (or less) until to get the status.
The first cycle start with the monit start or reload, but not at the
first time the "test-file" became available.

I think the answer fits to that, what you see.
With regards,
Lutz