Centrally Controlled Local Backup System Options
-
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
-
I use Borg for my laptop and it's been running forever. It's just a systemd timer, but you could easily have that script customized for each site and run the exact same job at each site, just the script would determine what happens.
-
@stacksofplates said in Centrally Controlled Local Backup System Options:
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
Hmm, Jenkins running Onprem or remotely from the MSP?
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@stacksofplates said in Centrally Controlled Local Backup System Options:
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
Hmm, Jenkins running Onprem or remotely from the MSP?
Jenkins master running wherever for the MSP, build boxes on prem at customer sites to actually do the work.
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Tiny customer with a single server, no other hardware. They need to take a backup of their data and be able to restore it in a reasonable amount of time for most problems, primarily hard drive failure.
This is where built-in OS backup, scripts, and email come in real handy... for businesses that have data they didn't plan for, and can't afford to support. They can easily schedule a backup script to back up to local device, send email alerts, even have some free serverless app in Azure or AWS watch for things and also send out alerts if something fails.
I haven't seen what OSs need backed up, but I think it doesn't matter.
Right, but the email system means it doesn't do what's needed.
What's that? A fancy management portal some non-IT person can take care of? That's just too bad then. Windows and Ubuntu make it easy enough to backup a system.
No, something that shows failures, not just success. An email system can alert only when the agent is healthy, but the backup has failed. but if the agents fails, no email gets sent. So for emails to do the job of alerting, you need a central console to consume them. Otherwise, you don't get alerted to failures. It's not about being easy, it's about knowing when an "email didn't arrive."
-
@DustinB3403 said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Tiny customer with a single server, no other hardware. They need to take a backup of their data and be able to restore it in a reasonable amount of time for most problems, primarily hard drive failure.
This is where built-in OS backup, scripts, and email come in real handy... for businesses that have data they didn't plan for, and can't afford to support. They can easily schedule a backup script to back up to local device, send email alerts, even have some free serverless app in Azure or AWS watch for things and also send out alerts if something fails.
I haven't seen what OSs need backed up, but I think it doesn't matter.
Right, but the email system means it doesn't do what's needed.
What's that? A fancy management portal some non-IT person can take care of? That's just too bad then. Windows and Ubuntu make it easy enough to backup a system.
Exactly, this is the part that I'm not getting that scott is saying he wants.
What it sounds like he wants is a vector for a hacker to get to every MSP's client and ransom their data.
Not sure how "knowing a backup succeeded" gives people access to data. Seems like quite the stretch. If that were the case, just this conversation would expose the data, right?
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@DustinB3403 said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Tiny customer with a single server, no other hardware. They need to take a backup of their data and be able to restore it in a reasonable amount of time for most problems, primarily hard drive failure.
This is where built-in OS backup, scripts, and email come in real handy... for businesses that have data they didn't plan for, and can't afford to support. They can easily schedule a backup script to back up to local device, send email alerts, even have some free serverless app in Azure or AWS watch for things and also send out alerts if something fails.
I haven't seen what OSs need backed up, but I think it doesn't matter.
Right, but the email system means it doesn't do what's needed.
What's that? A fancy management portal some non-IT person can take care of? That's just too bad then. Windows and Ubuntu make it easy enough to backup a system.
Exactly, this is the part that I'm not getting that scott is saying he wants.
What it sounds like he wants is a vector for a hacker to get to every MSP's client and ransom their data.
If an MSP is managing it, then the need that isn't there I would think, as the MSP should be IT oriented enough to manage that per client basis.
MSPs can do that, but it's not cost effective for the client. Nor is it particularly robust. Trying to make a proper system rather than half assing it.
Like anything we do in IT, trying to move beyond "purely repetitive labour" to "automation".
-
@DustinB3403 said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@DustinB3403 said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Tiny customer with a single server, no other hardware. They need to take a backup of their data and be able to restore it in a reasonable amount of time for most problems, primarily hard drive failure.
This is where built-in OS backup, scripts, and email come in real handy... for businesses that have data they didn't plan for, and can't afford to support. They can easily schedule a backup script to back up to local device, send email alerts, even have some free serverless app in Azure or AWS watch for things and also send out alerts if something fails.
I haven't seen what OSs need backed up, but I think it doesn't matter.
Right, but the email system means it doesn't do what's needed.
What's that? A fancy management portal some non-IT person can take care of? That's just too bad then. Windows and Ubuntu make it easy enough to backup a system.
Exactly, this is the part that I'm not getting that scott is saying he wants.
What it sounds like he wants is a vector for a hacker to get to every MSP's client and ransom their data.
If an MSP is managing it, then the need that isn't there I would think, as the MSP should be IT oriented enough to manage that per client basis.
Which is where I would think, as an MSP each should have their own environment. Not one central (global for the MSP), because that would be one massive target for any attacker.
While that's a nice theory, it's not practical. For an ITSP like NTG, that's how we work, and it's not cost effective for small clients. MSPs all do this already, for everything, so your theory on exposure is moot. All of the exposure you are worried about is a 100% foregone conclusion from so many other vectors. This isn't additional exposure, it's just additional protection.
You need to stop harping on about factors that we've shown aren't real. There's no new risk and no additional risk. Only taking manual processes and making them more robust.
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller, is that what this is? You (MSP) wanting some broad scoped on-prem local backup solution that ties together all the clients that can be monitored via a single portal? Is that the missing piece that is preventing this from moving forward?
Or what exactly is the case here, rather than the hypothetical general SMB need of whatever it is you are looking for?
Yes, it's about being able to see successes (and failures and schedules) of backups for multiple places that backup on premises and know that the backups have (or have not) succeeded. Then, because it is centralized, that central solution could send out email alerts because you only need to know if a single system is working, not hundreds. If one system stops working, you are likely to know.
It's not just for MSPs, businesses with many sites of their own would use this technology the same. Don't assume the sites are different companies or customers, it would be for multiple sites of the same company. We have customers in the ITSP world that would use this internally, rather than shared like an MSP.
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
I imagine they have zero IT to monitor backups, and need an MSP to take care of it, but said MSP wants to do it as efficiently as possible, preferably via some portal to monitor the backup status of clients. But let's see what he says, I feel i'm still missing info.
Yes, the most simple and obvious use case. I can't think of what other purpose there would be (the MSP part is irrelevant, but the central monitoring and control is the point.)
The goal is... central monitoring / reporting of individual sites. Think like what an RMM does, but I don't know any RMM with this functionality. But it is just standard RMM-like features, but for backup and not just the "normal" stuff that RMM covers.
-
@Obsolesce said in Centrally Controlled Local Backup System Options:
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Basically we need to treat the backups at the customer the same way that we typically do RMM, AntiVirus, Remote Access and other functions. All of the risk already exists from those covering the same bases. They just seem to all lack the ability to kick off and monitor a backup.
This you can do for free:
Have the built-in backup software do a backup, or even the free version of Veeam. You could use a PowerShell script to run a Veeam backup, verify success, send a webhook to an Azure function or Runbook, or an email, to do whatever you want. This specific part of the how isn't the point, as there's tons of ways to do it to fit any requirement.
But after that part, you can schedule an Azure Runbook to run daily or whenever, to check for X, if not X, then it alerts. If X, then sends success message if you prefer. This way, no matter what, even if the on-prem solution totally fails and MS Azure or AWS is still up, you'll get whatever you want to happen still happen.
Example:
- Built-in backup runs via powershell or bash script.
- Backup success or fail webhook or email is sent via powershell or bash, or backup software if supported.
- Depending on method, you have a free serverless script take care of back end processing.
- Back-end processing can take care of whether or not something happened when it was supposed to, and if so, if it was a success for failure, and respond appropriately, by doing anything you can imagine... send email, write to database that your fancy web GUI can read, write to cloud storage, basically anything.
There you'll have a way to verify backups whether success, failure, or complete failure. You can even automate, via scripting, backup restore testing as well.
If Windows backup, I know you can back up to VHD(x), auto mount, attempt to restore a known file to somewhere, test for that, respond appropriately.
Yeah, that will work, for sure. Basically it's building the solution that we want, which I know we can do. I'm trying to determine if really no one has made this. Everyone makes it where this happens, but the data also goes external. But keeping the data on premises with the customer seems to make this just not exist in the real world. The tech is obviously all there, every backup vendor has all the pieces built, they just don't allow the local target.
-
@stacksofplates said in Centrally Controlled Local Backup System Options:
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
Not specifically looking to trigger the backups remotely. Would like them to trigger locally and just report because if the Internet goes down, you don't want things to not keep going.
Salt or Ansible should be able to do this too. And you could script the responses.
-
This just seems like such a simple thing. And I'm not the one that came up with the need, I was just the one asked to research it for another company. But I talked to companies that make this stuff as part of the process and it seems like there is just so much money in the cloud storage portion, that no one is willing to make local targets overly accessible or attractive.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
@stacksofplates said in Centrally Controlled Local Backup System Options:
I mean an outside the box scenario is you could use Jenkins to schedule all of this. Then just use something like Restic or Borg to do the backups either locally or to a central location.
Then you can have Jenkins send messages/emails on a failure. You'd just need a Jenkins slave at every site (can be whatever you are running the backup on).
Not specifically looking to trigger the backups remotely. Would like them to trigger locally and just report because if the Internet goes down, you don't want things to not keep going.
Salt or Ansible should be able to do this too. And you could script the responses.
I assumed that Jenkins would have the Ansible plugin (or Salt if there is one, not sure) so the slave could do whatever it needed with that tool. You could always deploy script and then just have a Jenkins job to scrape the data.
Probably more of a realistic scenario is to just have the scripts at each site deploy to the location (remote or local) and then have Prometheus monitor that. You could pretty easily have an exporter report any failures on the systemd unit (or whatever service you want).
-
Its been a long while but didn't BackupExec used to have a central management console? I haven't looked at or used their product in 5-7 years now but might be worth checking into
-
@jt1001001 said in Centrally Controlled Local Backup System Options:
Its been a long while but didn't BackupExec used to have a central management console? I haven't looked at or used their product in 5-7 years now but might be worth checking into
OMG, do they still exist? LMAO
-
Yes they were re-spun off of Symatec a couple years back.
-
@jt1001001 said in Centrally Controlled Local Backup System Options:
Yes they were re-spun off of Symatec a couple years back.
You'd think that I'd remember them. I worked for them both pre- and during-Symantec (I was there during the buyout.) But I never worked on the BE side, only on the Netbackup side.
-
Metallic looks like it might do what I want here.
-
@scottalanmiller said in Centrally Controlled Local Backup System Options:
Metallic looks like it might do what I want here.
Looks great, but I don't see anything to suggest it will centrally manage backups to locally attached USB devices.
