I don't really get the point of SAN snapshots
-
Does anyone else have a SAN and use snapshots? We have a 24TB SAN with a single 10TB volume which is simply storage space for our virtual servers in our VMware environment. I have snapshots enabled on that one volume and they run once a night at 12AM. I understand that snapshots are not backups and I get that they are a form of data protection at the SAN level. However, I can't see any reason for them, at least not in our situation.
Reasons:
-
If I restored a volume from snapshots, ALL virtual machines would be reverted to a 12 hour-previous state and that would cause so many problems. I would rather restore machines from backup as needed as our backups run incrementally every 4 hours.
-
If I increase the frequency of snapshots, that would create much more storage overhead.
-
Storage snapshots of the volume are stored on that volume (I think?) or at least the SAN unit itself. This eats up space and snapshots would be no good if the volume were somehow deleted.
-
I would rather take snapshots at the VM level
I can't think of anything else right now, but the point is, I don't really see the use case for snapshots at the storage controller level, unless it's a situation where I've lost all backups and some or all VM's have been corrupted and my only option is to restore the volume from a snapshot - such as in a ransomware situation.
-
-
Snapshots at the volume level don't make sense with how you are using your SAN. I agree.
They could maybe make sense if you had multiple SANs and that any given SAN was a master for a pool of SANs so if you lost 1 SAN the others would be able to be restored from that snapshot.
But in this case, it makes no sense.
-
A more obvious way to think about this is "I want to snapshot my hypervisor every 12 hours, and keep those snapshots on the hypervisor itself".
It's absurd to think of, and more so to do. You're gaining nothing (snapshots aren't backups) and if the hypervisor died, you'd still not be able to restore from that massive snapshot.
-
@DustinB3403 said in I don't really get the point of SAN snapshots:
Snapshots at the volume level don't make sense with how you are using your SAN. I agree.
They could maybe make sense if you had multiple SANs and that any given SAN was a master for a pool of SANs so if you lost 1 SAN the others would be able to be restored from that snapshot.
But in this case, it makes no sense.
Thanks for confirming my suspicions.
-
@dave247 you can use storage snapshot for fast and very efficient backups. Your backup software should be connected to the SAN of course… trigger a snapshot after hypervisor quiescence, and then just retrieve the backup data from the snapshot. At the end pf the backup job, the snapshot will just be discarded. That’s what any tier-1 software does today in dense and high-performance environments.
-
@Francesco-Provino said in I don't really get the point of SAN snapshots:
@dave247 you can use storage snapshot for fast and very efficient backups. Your backup software should be connected to the SAN of course… trigger a snapshot after hypervisor quiescence, and then just retrieve the backup data from the snapshot. At the end pf the backup job, the snapshot will just be discarded. That’s what any tier-1 software does today in dense and high-performance environments.
This isn't using the SAN as a backup repository, but using the SAN as production storage for his hypervisor. Likely providing iSCSI targets for his VM's to mount.
-
@DustinB3403 said in I don't really get the point of SAN snapshots:
@Francesco-Provino said in I don't really get the point of SAN snapshots:
@dave247 you can use storage snapshot for fast and very efficient backups. Your backup software should be connected to the SAN of course… trigger a snapshot after hypervisor quiescence, and then just retrieve the backup data from the snapshot. At the end pf the backup job, the snapshot will just be discarded. That’s what any tier-1 software does today in dense and high-performance environments.
This isn't using the SAN as a backup repository, but using the SAN as production storage for his hypervisor. Likely providing iSCSI targets for his VM's to mount.
Right. What @Francesco-Provino is talking about, I believe, is backup software that talks to the SAN, sees the snapshot and makes a backup of the snapshot from the SAN and stores it some place else. Then the backup software deletes the snapshot on the SAN.
-
@dave247 said in I don't really get the point of SAN snapshots:
I can't think of anything else right now, but the point is, I don't really see the use case for snapshots at the storage controller level, unless it's a situation where I've lost all backups and some or all VM's have been corrupted and my only option is to restore the volume from a snapshot - such as in a ransomware situation.
You are seeing the underlying fallacies that also make SAN in general not make as much sense as it seems at first glance. How silly it is to take a snapshot of an entire storage infrastructure, blindly, is just one of many reasons why SAN rarely makes any sense. It has its place, as do SAN snapshots, but they are rare and almost never where people think that they should be used and possibly least of all in a virtual infrastructure - that's nearly the worst place for a SAN or SAN snaps.
-
@dafyre said in I don't really get the point of SAN snapshots:
@DustinB3403 said in I don't really get the point of SAN snapshots:
@Francesco-Provino said in I don't really get the point of SAN snapshots:
@dave247 you can use storage snapshot for fast and very efficient backups. Your backup software should be connected to the SAN of course… trigger a snapshot after hypervisor quiescence, and then just retrieve the backup data from the snapshot. At the end pf the backup job, the snapshot will just be discarded. That’s what any tier-1 software does today in dense and high-performance environments.
This isn't using the SAN as a backup repository, but using the SAN as production storage for his hypervisor. Likely providing iSCSI targets for his VM's to mount.
Right. What @Francesco-Provino is talking about, I believe, is backup software that talks to the SAN, sees the snapshot and makes a backup of the snapshot from the SAN and stores it some place else. Then the backup software deletes the snapshot on the SAN.
Yes, some software can do that. It's layer upon layer of coordination between the backup tools, the SAN, the SAN snap utility, the hypervisors that talk to the SAN, etc. It can be done, but gets extremely complex.
-
@Francesco-Provino said in I don't really get the point of SAN snapshots:
@dave247 you can use storage snapshot for fast and very efficient backups. Your backup software should be connected to the SAN of course… trigger a snapshot after hypervisor quiescence, and then just retrieve the backup data from the snapshot. At the end pf the backup job, the snapshot will just be discarded. That’s what any tier-1 software does today in dense and high-performance environments.
Dense, but not high performance. But yes, this is the standard for super high density SAN setups where performance and reliability take a back seat to cost savings (which is a reasonably thing, but only works at massive scale.) In these cases, the SAN snaps are just an "under the hood" component of the backup mechanism.
Same as snapshots at the hypervisor level. Those aren't as silly as SAN snaps on their own, but are still a bit odd until you understand them to mostly be a component of another mechanism and rarely something you would interact with directly.
-
@scottalanmiller said in I don't really get the point of SAN snapshots:
@dave247 said in I don't really get the point of SAN snapshots:
I can't think of anything else right now, but the point is, I don't really see the use case for snapshots at the storage controller level, unless it's a situation where I've lost all backups and some or all VM's have been corrupted and my only option is to restore the volume from a snapshot - such as in a ransomware situation.
You are seeing the underlying fallacies that also make SAN in general not make as much sense as it seems at first glance. How silly it is to take a snapshot of an entire storage infrastructure, blindly, is just one of many reasons why SAN rarely makes any sense. It has its place, as do SAN snapshots, but they are rare and almost never where people think that they should be used and possibly least of all in a virtual infrastructure - that's nearly the worst place for a SAN or SAN snaps.
I mean, I get VM snapshots - if you somehow muck up a server you can quickly roll back from snap vs restore from backups. That is much more granular. If you muck up an entire datastore/volume on your SAN, you can restore from snap too, but that's going to be a much larger disruption and you probably shouldn't be allowed to touch the storage controller..
I know in the past, you've recommended vSAN but what would another solution be besides that? The only thing I can think of is having multiple storage servers connected to the compute hosts via iSCSI - basically taking the place of the SAN. I don't plan to replace our storage controller anytime soon, I just wanted some food for thought.
-
@dave247 if you're looking to replace what you have today, let's pretend it's an Inverted Pyramid, you'd likely want to reuse as much of the existing equipment as possible.
Pretending you have 1-2 hypervisors attached to a Dell SAN.
The hypervisors have maybe 1TB of storage each, and 128GB of RAM with a blazing fast CPU, and 6-12 available 2.5 HHD bays.
The simple answer would be, fill either of these servers with enough disks to match or exceed what the SAN provides and move everything onto the hypervisor. Use the second host as a backup target, migration target or a lab. Running on a single host is simple enough and it un-complicates your setup.
This 1 server runs everything, the VMs (and data on them) are backed up to here (maybe the SAN has NAS functionality), and from there to offsite.
-
@dave247 said in I don't really get the point of SAN snapshots:
I mean, I get VM snapshots - if you somehow muck up a server you can quickly roll back from snap vs restore from backups.
Yeah, it's the same concept but more granular, but still not super granular. Same basic concept, though.
-
@scottalanmiller said in I don't really get the point of SAN snapshots:
@dave247 said in I don't really get the point of SAN snapshots:
I mean, I get VM snapshots - if you somehow muck up a server you can quickly roll back from snap vs restore from backups.
Yeah, it's the same concept but more granular, but still not super granular. Same basic concept, though.
If you want to go the scenic route. . . you need a client on the VM that can watch everything.
-
@dave247 said in I don't really get the point of SAN snapshots:
I know in the past, you've recommended vSAN but what would another solution be besides that? The only thing I can think of is having multiple storage servers connected to the compute hosts via iSCSI - basically taking the place of the SAN.
vSAN is the direct replacement for legacy SAN - but vSAN just means a SAN that is virtualized (basically treating the SAN like a production workload.) So it is part of a solution, but not quite the solution on its own. I'll come back to that.
Doing multiple SANs as you describe is actually the only way that a SAN is considered production ready. That's the basic deployment methodology for SAN normally. Lots of SMBs skimp on that, but from a storage engineering perspective, that's not what "deploying a SAN" means. When enterprises deploy SANs, it is just an assumption that there are multiple ones. That SMBs sometimes deploy just one SAN is a different problem from what we are discussing here (the purpose to SANs.)
Likewise, when people say vSAN, they also mean multiple ones.
So what is the alternative? The problem with a SAN conceptually is that it is external and a new, unnecessary point of failure. SAN's problems come from the mixture of being too complex, expensive, and risky... without providing a benefit over simpler approaches.
vSAN is actually a way to replace SAN without changing technologies. On its own, it's a bad idea. Overly complex, but still way better than regular SAN. Not because it is virtual (that's a nice bonus), but from the implication that it will be used hyperconverged. That's the real problem with physical SAN - it has to be separate hardware, extra servers. vSAN we assume is not (but it could be, and that would be the same problem.)
Going past vSAN, we have technology like Gluster, CEPH, DRBD, HAST, and SCRIBE that do local storage replication and do it without needing the SAN layer. Not all products can talk to these, but those that do get big advantages by removing the complexity of iSCSI.
-
Too hard to type, made a video. It is uploading.
-
@scottalanmiller said in I don't really get the point of SAN snapshots:
Too hard to type, made a video. It is uploading.
Lazy. . Bast. . .
-
-
@scottalanmiller said in I don't really get the point of SAN snapshots:
Thanks for doing another video in response to one of my posts! Very cool of you and what a wealth of information.
I lost it at 0:41 with the pause/eye-roll
I had been getting the feeling that the term "SAN" was being used incorrectly becuase as you stated, it refers to a "storage area network" which would be the storage network portion, not the storage controller itself.
I get what you mean now about a single storage controller unit not being redundant. And in our case, ours is not as we only have the one. You probably know this by now about my setup, that we have the "inverted pyramid of doom" going on.
And thanks for the additional clarification. Very good stuff.
-
@dave247 said in I don't really get the point of SAN snapshots:
I get what you mean now about a single storage controller unit not being redundant. And in our case, ours is not as we only have the one. You probably know this by now about my setup, that we have the "inverted pyramid of doom" going on.
Essentially anyone with a SAN has one. Not exactly guaranteed, but almost. It's the key reason SANs are deployed.
Many enterprises knowingly run IPODs because at huge scale, they can be cheap. And while IPODs cannot be as fast or as reliable as alternatives, enterprises also know that performance and reliability are not the end all, be all and that "cheaper" and "fast enough" and "safe enough" can be the right choice.
SMBs tend to copy this, but forget that enterprises make the decision based on scale, and assuming that there must be value, just project that the idea must be fast and reliable. So with the lack of scale, they end up spending a fortune to make their system not work as well. So there are good reasons that it all exists, but they don't apply well to the SMB market.
