Elastix reboot: cron job?
-
What are the risks of setting a cron job to restart the Elastix system?
-
To just reboot, very low as long as you have an accepted green zone from the client / business. You would ideally want this done in the middle of a weekend night or something of that nature and even more ideally have some automated checks associated and if at all possible, test calls made. But in most cases, just standard monitoring (is the system up) with a scheduled reboot is fine.
Doing a cronjob based reboot is very standard.
-
Yeah, the risk of a reboot is pretty much nothing as long as you know that no one is right in the middle of changing anything.
Biggest thing to think about on the scheduling is not to put it in the middle of other common maintenance windows.
Say you normally check backups and other stuff on Saturday nights, do not schedule the automated reboot for the same window. Do it on Sunday night or Friday night instead.
-
It's good to build in checks to make sure there are no current connections (or in this case calls). We do this because we have so many people working after hours. But if they can't access a resource when they try, well that's part of working after hours. If they get disconnected or loss work because of that well then that is a problem.
I personally would use your IT management server if you have one to restart the server remotely rather than on the server itself. Then you can configure checks to make sure it's back up within X mins and send an email alert if not.
-
@thecreativeone91 said:
It's good to build in checks to make sure there are no current connections (or in this case calls). We do this because we have so many people working after hours. But if they can't access a resource when they try, well that's part of working after hours. If they get disconnected or loss work because of that well then that is a problem.
I personally would use your IT management server if you have one to restart the server remotely rather than on the server itself. Then you can configure checks to make sure it's back up within X mins and send an email alert if not.
That would be my next question; can you configure the cron to do notification when it was about to restart, and once it came up.
-
@g.jacobse said:
That would be my next question; can you configure the cron to do notification when it was about to restart, and once it came up.
A notification like an email? Just put an email command in the cron job like so...
email start; command to do something; email end -
@thecreativeone91 said:
I personally would use your IT management server if you have one to restart the server remotely rather than on the server itself. Then you can configure checks to make sure it's back up within X mins and send an email alert if not.
I've worked in big environments that did this then moved off of it. It creates dependencies that need not exist. It is "easier to manage" in some ways but if the issue is resolving an SSH issue, for example, one can fix it automatically and the other just fails.
We found that moving from the jump servers doing it to the systems doing it themselves was a huge win. We had more control over the timing, far more reliable reboots (nothing missed because of a network blip), no extraneous dependencies, deals with SSH failures, etc.
-
@scottalanmiller said:
@thecreativeone91 said:
I personally would use your IT management server if you have one to restart the server remotely rather than on the server itself. Then you can configure checks to make sure it's back up within X mins and send an email alert if not.
I've worked in big environments that did this then moved off of it. It creates dependencies that need not exist. It is "easier to manage" in some ways but if the issue is resolving an SSH issue, for example, one can fix it automatically and the other just fails.
We found that moving from the jump servers doing it to the systems doing it themselves was a huge win. We had more control over the timing, far more reliable reboots (nothing missed because of a network blip), no extraneous dependencies, deals with SSH failures, etc.
Monitoring it is much easier when centralized. We'll never move to doing tasks on Servers individually.
-
@g.jacobse said:
@thecreativeone91 said:
It's good to build in checks to make sure there are no current connections (or in this case calls). We do this because we have so many people working after hours. But if they can't access a resource when they try, well that's part of working after hours. If they get disconnected or loss work because of that well then that is a problem.
I personally would use your IT management server if you have one to restart the server remotely rather than on the server itself. Then you can configure checks to make sure it's back up within X mins and send an email alert if not.
That would be my next question; can you configure the cron to do notification when it was about to restart, and once it came up.
I wouldn't send an email unless it fails others you're just turning alerts into noise.
Also relying on itself to tel you if it's successful or failed isn't that great, you really need another machine they connects to it to make sure it's up and then sends an email if it's not.
-
@thecreativeone91 said:
Monitoring it is much easier when centralized. We'll never move to doing tasks on Servers individually.
Monitoring should be centralized, agreed. But the actual maintenance?
-
@thecreativeone91 said:
Also relying on itself to tel you if it's successful or failed isn't that great, you really need another machine they connects to it to make sure it's up and then sends an email if it's not.
That's why you use the monitoring system for that, rather than the maintenance system. Let it handle its own maintenance and the centralized monitoring that should exist anyway can handle the monitoring.
-
@scottalanmiller said:
@thecreativeone91 said:
Monitoring it is much easier when centralized. We'll never move to doing tasks on Servers individually.
Monitoring should be centralized, agreed. But the actual maintenance?
Yes, By saying it's not you are suggestion Group Policy and WSUS are bad.
-
@thecreativeone91 said:
@scottalanmiller said:
@thecreativeone91 said:
Monitoring it is much easier when centralized. We'll never move to doing tasks on Servers individually.
Monitoring should be centralized, agreed. But the actual maintenance?
Yes, By saying it's not you are suggestion Group Policy and WSUS are bad.
Group Policy pushes out the changes, but runs them locally. It would be more like having a centralized script that set the crontabs to always be the same (or in a pattern or whatever.) I think that GPO is more as I have described, local maintenance jobs.
WSUS is patching which is a little different. I would prefer WSUS use localized patching and centralized reporting and filtering.
-
And don't translate "sub-optimal" as bad. But don't think of GPO as centralized pushes of management either.
I'm totally for the cronjobs being pushed out centrally, that makes sense.
-
@thecreativeone91 said:
Yes, By saying it's not you are suggestion Group Policy and WSUS are bad.
If you feel that that was what I was saying, are you implying that you feel that Chef, Puppet, cfEngine and other DevOps tools are bad? (They run locally and only pull centralized configurations.)
-
Thanks for the additional tags
As for e mail, it would be nice to get a health report on successful boot.
No report, something is wrong....
-
@g.jacobse said:
Thanks for the additional tags
As for e mail, it would be nice to get a health report on successful boot.
No report, something is wrong....
It sounds nicer than it is. But in reality you don't want emails saying that things are okay, that's fundamentally wrong. For a couple of reasons:
- At scale, you will be inundated with "everything is fine" emails. It just doesn't scale.
- It trains you to ignore your email and messages from the server. It's bad conditioning.
- You have to notice the absence of an alert to realize that something is wrong rather than the presence of it. You have no way to verify failure, you have to hope that you remember when success was supposed to, hope that you paid attention, etc.
- This doesn't give you a means of passing off to another person or connecting to other automation.
- You have to maintain a list of people to be alerted of "regular stuff."
- You are attempting to use your email as a logging system rather than as an alerting mechanism.
-
If you want this kind of information consider something like...
- Using proper logging and looking in the logs for this information rather than in email.
- Posting to a status feed (e.g. Twitter) with status info that anyone can look up if they care to.
-
@g.jacobse said:
Thanks for the additional tags
As for e mail, it would be nice to get a health report on successful boot.
No report, something is wrong....
That's backwards you want emails when something fails. I'd get way too many if I got once when things worked I easily get 200+ per day already.
-
@thecreativeone91 said:
@g.jacobse said:
Thanks for the additional tags
As for e mail, it would be nice to get a health report on successful boot.
No report, something is wrong....
That's backwards you want emails when something fails. I'd get way too many if I got once when things worked I easily get 200+ per day already.
I rant that over in my mind, and it seemed that if the reboot failed, you would not get any type of email.
